Horizontally partitionable reference counting system

Abstract

This application relates to systems and methods for efficiently managing data storage. A database can be a distributed database in which objects include references to multiple distributed data blocks. These data blocks can be referenced by multiple objects, and the system is configured to efficiently identify when any object within the system no longer references one or more data blocks. The system can be configured to have a two-tiered architecture in which reverse references to the data blocks are collected and a total count of reverse references is determined. The two tiers can be identified based on an increment to the reverse references, and each tier can be horizontally partitioned.

Description

Background Technology

[0001] Data storage systems can be configured as distributed data systems, where objects are divided into sets of smaller data slices that can be used by multiple objects. Each data slice within the system should correspond to one or more active objects. However, as data storage systems grow in size, maintaining these data slices efficiently and accurately becomes increasingly difficult as objects within the system are created, deleted, or otherwise modified. Summary of the Invention

[0002] A scalable data storage system is needed, configured to efficiently manage distributed data within the system even when objects are created, deleted, or otherwise modified. This application relates to systems and methods for managing distributed databases, where objects include references to multiple distributed data blocks. The system is configured to efficiently identify when any object within the system no longer references one or more data blocks. The system can be configured with a two-tier architecture, where backreferences from data blocks to objects are collected and the total count of backreferences is determined. These two tiers can be based on incremental identification of backreferences, and each tier can be horizontally partitioned.

[0003] According to aspects of this disclosure, a system for database maintenance may include: a memory having metadata for a plurality of objects, each object referencing one or more data blocks within the database; and one or more processors configured to: identify one or more incremental changes to references to one or more identified data blocks within the metadata; add or delete entries in a block reference table based on the identified incremental changes; change the reference count of the one or more identified data blocks in a reference count table based on the block reference table; and identify whether any entry in the reference count table has a reference count of zero.

[0004] According to other aspects of this disclosure, the metadata for multiple objects may further include multiple metadata inputs, and one or more processors may be further configured to identify one or more incremental changes to references to one or more identified data blocks regarding each of the multiple metadata inputs.

[0005] In other aspects of this disclosure, the block reference table may further include multiple block reference tables, and each of the multiple block reference tables may contain an entry associated with one of the multiple metadata inputs.

[0006] In other aspects of this disclosure, the reference count table may further include a plurality of reference count tables, and each of the reference count tables may correspond to a predetermined range of entries within each of the plurality of block reference tables. Additionally, the predetermined range of entries may be based on a block ID associated with each of the data blocks.

[0007] In other aspects of this disclosure, data blocks may be aggregated into data groups, which may be referred to as fragments, and each data block may be assigned a block ID based at least in part on the data group in which the data blocks are aggregated.

[0008] In other aspects of this disclosure, one or more processors may be further configured to provide commands to delete one or more identified data blocks from one or more databases based on determining that one or more identified data blocks have a reference count of zero.

[0009] In other aspects of this disclosure, the one or more incremental changes referenced may be based at least in part on timestamped changes to one or more objects.

[0010] In other aspects of this disclosure, metadata may include change logs for multiple objects.

[0011] In other aspects of this disclosure, one or more processors may be further configured to update one or more data groups for one or more data blocks in the data blocks, at least in part based on reference counts of one or more identified data blocks.

[0012] In other aspects of this disclosure, a method for database maintenance may include: one or more processors accessing a memory having metadata for a plurality of objects, each object referencing one or more data blocks within the database; the one or more processors identifying one or more incremental changes to references to the one or more identified data blocks within the metadata; the one or more processors adding or deleting entries in a block reference table based on the identified one or more incremental changes; the one or more processors changing the reference count of the one or more identified data blocks in a reference count table based on the block reference table; and the one or more processors identifying whether any entry in the reference count table has a reference count of zero.

[0013] In other aspects of this disclosure, the metadata for multiple objects further includes multiple metadata inputs, and wherein identifying one or more incremental changes referenced further includes identifying one or more incremental changes with respect to each of the multiple metadata inputs to one or more identified data blocks. Attached Figure Description

[0014] Figure 1This is a block diagram of a database for object metadata based on various aspects of this disclosure.

[0015] Figure 2 It is a diagram of the modified database based on various aspects of this disclosure.

[0016] Figure 3 This is a block diagram of data citation collection and data citation counting based on various aspects of this disclosure.

[0017] Figure 4 It is a block diagram of partitioned data reference collection and data reference counting according to various aspects of this disclosure. Detailed Implementation

[0018] Data storage systems can be designed to distribute stored data in a way that efficiently manages and stores it. For example, a globally distributed binary large object (BLOB) storage system is a distributed storage system where uploaded data content or objects are divided into slices or "blocks," which can then be collected to reconstruct the object. An object can be any data file or collection of data files, such as images, videos, audio, or user-facing data content. Data blocks can be stored within the system in a way that allows for efficient access and maintenance of the data. For example, multiple objects within the system can reference the same data block.

[0019] Data blocks can also be aggregated into groups or "shards" based on their attributes. For example, a block set can be aggregated into a shard based on whether the blocks in the set are referenced by the same object or by one or more different objects that share the same placement strategy within the system. Figure 1 The diagram 100 is part of a data storage system, which includes data blocks 130-134 stored in database 101. These blocks 130-134 are aggregated into two shards 140 and 141, where blocks 130-132 are aggregated into a portion of shard 140 and blocks 133 and 134 are aggregated into a portion of shard 141.

[0020] Objects can be retrieved by accessing the appropriate data blocks within database 101. Therefore, an object can exist within the storage system as a collection of references to data blocks within database 101. For example, a table or database 102 contains metadata for multiple objects, including objects 110 and 111. The metadata for each object 110, 111 contains multiple references to the data blocks that constitute objects 110, 111. For example, the metadata for object 110 in database 102 includes references 120a-e, each referring to a specific data block stored within database 102, and object 110 can be retrieved by accessing the block referenced by references 120a-e. References 120a-e can take the form of a unique block ID, which can be used to uniquely identify the data block within database 101.

[0021] When blocks are aggregated into shards, the unique block ID can take the form of a unique shard ID and identify the blocks within that shard. For example, reference 120a of object 110 refers to "s1c1," which refers to the first shard and the first block within that shard (i.e., block 130 of shard 140), while reference 120b refers to the second shard and the first shard within that shard (i.e., block 133 of shard 141). An object can contain more than one reference to a specific block, and different objects can each reference the same block. For example, the metadata for object 110 contains a reference to block 131, and the metadata for object 111 also contains a reference to that block. Therefore, block 131 within database 101 contains at least two references within the metadata of database 102.

[0022] According to various aspects of this disclosure, the data storage system can be configured to maintain data blocks within database 101 in a manner that allows for efficient access to and storage of data. This may include deleting a data block from the database when no object references it anymore. Therefore, sharding can be modified by removing any data blocks that have been identified as no longer referenced by objects within the system.

[0023] In some cases, blocks may remain immutable to a given shard to which they are aggregated. However, in other cases, the system may repackage the set of blocks into different shard sets while maintaining the same block ID. This repackaging may be done to reduce block fragmentation within the system. Furthermore, this repackaging may include identifying and deleting data blocks that are no longer referenced by any object. However, even if the same set of data blocks is repackaged into different shard sets, the object metadata 110, 111 within database 102 will continue to reference these blocks.

[0024] Figure 2This is a block diagram 200 of database 201, which contains at least two shards of data blocks. Specifically, database 201 contains shard 1 with blocks 1, 4, 5, 7, and 8, and database 201 contains shard 2 with blocks 2, 3, 6, 9, and 10. As shown in diagram 200, objects A, B, and C each reference blocks within shards 1 and 2. For example, object A contains a reference to block 1 located within shard 1, and references to blocks 2 and 3 located within shard 2. However, block 10 of shard 2 within database 201 is not referenced by any object. Therefore, block 10 can be deleted from database 201 because it is not needed by any existing or living object.

[0025] A block currently referenced by at least one object can be called a live block, while a block not referenced by any active object can be called a dead block. According to various aspects of this disclosure, a data storage system can be configured to identify dead blocks by recognizing instances within the database where one or more blocks are no longer referenced by any existing objects. Those identified blocks can be deleted from the database. In Figure 200, block 10 is identified as a dead block and removed from database 201, as indicated by the absence of the block in the updated database 201'.

[0026] As database 201 and related object metadata grow, specific blocks within database 201 will be referenced by an increasing number of objects within the system, making it potentially more difficult to identify when a particular block is dead. Object metadata provides unidirectional references from objects to data blocks, but to determine if a specific block is dead, it's necessary to confirm that no object is referencing that particular block. However, scanning the entire metadata source to identify dead blocks is inefficient, especially if the system is configured to allow unrestricted sharing of block references.

[0027] According to various aspects of this disclosure, data storage systems can be configured to efficiently identify when active objects no longer reference blocks. For example, the systems and methods disclosed herein allow for dead block identification within partitionable layers, thereby allowing incremental identification of backreferences to a set of objects for any given data block.

[0028] Figure 3This is block diagram 300, which uses a two-layer architecture to identify dead blocks within a system. The boxes within diagram 300 can include operations performed by one or more computing devices (such as servers, databases, or a series of networked devices). Box 302 represents object metadata, which includes identifiers of object references to a set of blocks. Blocks can be referenced based on unique block IDs. For example, the object's metadata can be in the form of a table that includes a list of block IDs referenced by the object. In box 303, references to blocks within the object's metadata are collected, and these references are identified in a block reference table (box 304). The block reference table can include entries identifying the block ID for each reference, and a backreference key that identifies the location of the reference to that block ID within the object's metadata. Therefore, each entry in the block reference table (box 304) can identify a backreference for a block ID, which is an identifier of the object's reference to the block ID based on the object's metadata within box 302.

[0029] Box 305 describes a block reference counting operation, where the number of references to a specific block ID is determined based on the contents of a block reference table. The total number of references to a block can be maintained in a block count table (box 306), which includes an entry for each block ID and an associated total count representing the current total number of references to that specific block ID. The block reference counting operation (box 305) identifies changes to the number of references to a specific block ID and can increase or decrease the total count in the block count table based on the identified changes.

[0030] The block count table in box 306 can be accessed by one or more subsystems, such as the block service subsystem (box 307) or the compaction updater subsystem (box 308). These subsystems can be configured to modify one or more databases based on changes to the block count table in box 306. For example, returning to... Figure 2 There are no object references to block 10 in database 201. Therefore, the block count table will indicate a zero total count for block 10. Based on this zero total count in the block count table, block 10 can be deleted from database 201, as indicated in database 201'.

[0031] In many cases, object metadata will reside in multiple different databases and tables. According to various aspects of this disclosure, block reference collection and block reference counting operations can be horizontally partitioned while still allowing for total reference counting for each unique block ID. For example, Figure 4This is a block diagram 400 showing that object metadata is stored in multiple databases 402a-c. In blocks 403a-c, block reference collection is performed for each database 402a-c, and each database 402a-c produces a separate block reference table 404a-c. For example, the block reference collection operation in block 403a identifies all block references within the object metadata used for database 402a, and block reference table 404a contains entries for block IDs identified by the block reference collection operation in block 403a. Similar block reference tables 404b and 404c are maintained based on the block reference collection operations (blocks 403b and 403c) performed on the object metadata databases 402b and 403c, respectively.

[0032] Each block reference table 404a-c can be arranged and divided into block ID ranges 414a-c. For example, block reference table 404a has been arranged into three ranges 414a. These ranges may be based on the arrangement of entries within block reference table 404a, which itself is based on the block ID of each entry. In boxes 405a-c, reference counting operations can be performed. As shown in Figure 400, a separate reference count can be performed for each range 414a-c of each block reference table 404a-c. Thus, the reference count for box 405a is based on the block ID range [1-10] from each of the block reference tables 404a-c, while the reference count for box 405b is based on the block ID range [11-20] from each of the block reference tables 404a-c. The reference counting operations 405a-c can then update the corresponding block count tables 406a-c, each based on the identifier of the total block references for each block ID on all block reference tables 404a-c. Entries in block reference tables 404a-c can be categorized by block ID to allow for efficient identification of block backreferences within partitioned datasets.

[0033] Block count tables 406a-c can then be used to identify any block IDs that have not yet been referenced. These block IDs can be identified as dead blocks and deleted or otherwise removed from one or more storage system databases. Although Figure 400 shows three metadata databases 402a-c, any number of metadata sources can be used according to this disclosure. Furthermore, any number of ranges 414a-c can be used to partition block reference tables 404a-c. For example, each block reference table 404a-c can be divided into four or more ranges by classifying the block IDs of each table 404a-c into four or more separate groups. Additionally, block reference tables 404a-c can be combined into a single reference table that can be classified by block ID to allow for efficient identification of the total reference count for each block ID.

[0034] Ranges 414a-c can also be based on the fragment range used for blocks within block reference tables 404a-c. For example, as mentioned above, the block ID can include the fragment ID of the block. Therefore, ranges 414a-c can be based on the range of fragment IDs of the referenced blocks.

[0035] According to various aspects of this disclosure, the block reference collection layer and the block reference counting layer can be executed incrementally within the data storage system. For example, they can be executed on identified changes within the metadata dataset. Figure 4 Instead of performing these operations on the entire metadata dataset, the block reference collection operation 403a-c can identify changes to metadata based on timestamps provided within the metadata database 402a-c. In this case, the block reference collection operation 403a-c can identify timestamped metadata within the database 402a-c to determine whether any changes have been made to the objects since the last block reference collection operation was performed.

[0036] This timestamped metadata can identify whether any objects have been created, deleted, or otherwise modified since the last block reference collection operation, and the block reference table 404a-c and block count table 406a-c can be updated based on these incremental changes to the metadata. Therefore, the block reference collection operation 404a-c can be configured so that for each block ID entry in the block reference table 404a-c, the entry is generated when the reference to the block ID is created, and the entry persists until it is determined that the reference no longer exists.

[0037] The object metadata provided for block reference collection can also be in the form of a change log, which identifies incremental events for all objects within a given database and provides information on which incremental events are used for block reference collection operations to create or delete backreferences for specific block ID entries based on these incremental events. These incremental events can be provided as events occur, allowing the system to operate as a streaming system, thereby identifying changes in reference counts as object data changes.

[0038] Inaccurate reference counting can cause live blocks to be incorrectly identified as dead blocks and thus deleted. Therefore, a verification system can be implemented using the system and method described herein. This verification system can be divided into two layers to identify errors in reference collection or reference counting operations. Furthermore, the verification system can be horizontally partitioned and configured to perform a full scan of the input to confirm the current reverse reference count.

[0039] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but can be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be presented in an illustrative rather than restrictive manner. Furthermore, the examples described herein and the provision of phrases such as "such as," "comprising," etc., should not be construed as limiting the subject matter of the claims to the specific examples; rather, these examples are intended to illustrate only one of many possible embodiments. Moreover, the same reference numerals in different figures can identify the same or similar elements.

Claims

1. A system for database maintenance, characterized in that, include: A memory having metadata for multiple objects, each object referencing one or more data blocks within a database; One or more processors, said one or more processors being configured to: Identify one or more incremental changes to references to one or more identified data blocks within the metadata; Based on one or more identified incremental changes, add or delete entries in the block reference table; Based on the block reference table, change the reference count of the one or more identified data blocks in the reference count table; as well as Identify whether any entry in the reference count table has a reference count of zero.

2. The system as described in claim 1, characterized in that, The metadata for the plurality of objects further includes a plurality of metadata inputs, and wherein the one or more processors are further configured to identify one or more incremental changes to references to one or more identified data blocks with respect to each of the plurality of metadata inputs.

3. The system as described in claim 2, characterized in that, The block reference table further includes multiple block reference tables, and each of the multiple block reference tables contains an entry associated with one of the multiple metadata inputs.

4. The system as described in claim 3, characterized in that, The reference count table further includes a plurality of reference count tables, and each of the reference count tables corresponds to a predetermined range of entries within each of the plurality of block reference tables.

5. The system as described in claim 4, characterized in that, The predetermined entry range is based on the block ID associated with each of the data blocks.

6. The system as described in claim 1, characterized in that, The data blocks are aggregated into data groups, and each data block is assigned a block ID based at least in part on the data groups in which the data blocks are aggregated.

7. The system as described in claim 1, characterized in that, The one or more processors are further configured to provide commands to delete the one or more identified data blocks from one or more databases based on determining that the one or more identified data blocks have a reference count of zero.

8. The system as described in claim 1, characterized in that, The one or more incremental changes mentioned herein are at least in part based on timestamped changes to one or more objects.

9. The system as described in claim 1, characterized in that, The metadata mentioned therein includes change logs for the plurality of objects.

10. The system as claimed in claim 1, characterized in that, The one or more processors are further configured to update one or more data groups of one or more data blocks based at least in part on the reference count of the one or more identified data blocks.

11. A method for database maintenance, characterized in that, include: The memory is accessed by one or more processors and has metadata for multiple objects, each object referencing one or more data blocks within a database. The one or more processors identify one or more incremental changes to references to one or more identified data blocks within the metadata; The one or more processors add or delete entries in the block reference table based on one or more identified incremental changes; The one or more processors change the reference count of the one or more identified data blocks in the reference count table based on the block reference table; as well as The one or more processors identify whether any entry in the reference count table has a reference count of zero.

12. The method as described in claim 11, characterized in that, The metadata for the plurality of objects further includes a plurality of metadata inputs, and the identification of the one or more incremental changes referenced further includes identifying one or more incremental changes for one or more identified data blocks with respect to each of the plurality of metadata inputs.

13. The method as described in claim 12, characterized in that, The block reference table further includes multiple block reference tables, and each of the multiple block reference tables contains an entry associated with one of the multiple metadata inputs.

14. The method as described in claim 13, characterized in that, The reference count table further includes a plurality of reference count tables, and each of the reference count tables corresponds to a predetermined range of entries within each of the plurality of block reference tables.

15. The method as described in claim 14, characterized in that, The predetermined entry range is based on the block ID associated with each of the data blocks.

16. The method as described in claim 11, characterized in that, The data blocks are aggregated into data groups, and each data block is assigned a block ID based at least in part on the data groups in which the data blocks are aggregated.

17. The method as described in claim 11, characterized in that, The method further includes providing commands by the one or more processors to delete the one or more identified data blocks from one or more databases based on determining that the one or more identified data blocks have a reference count of zero.

18. The method as described in claim 11, characterized in that, The one or more incremental changes mentioned herein are at least in part based on timestamped changes to one or more objects.

19. The method as described in claim 11, characterized in that, The metadata mentioned therein includes change logs for the plurality of objects.

20. The method as described in claim 11, characterized in that, It further includes updating one or more data groups of the one or more data blocks based at least in part on the reference count of the one or more identified data blocks.

Images