Cloud storage distributed file system

Abstract

Methods, systems, and computer programs for providing file system functions on a cloud storage system based on blob storage are proposed. The cloud storage system includes a storage cluster, and each storage cluster includes: a plurality of storage nodes (each storage node includes a partitioning layer that stores data in blobs organized in containers, and includes a streaming layer for streaming the blob data); a hierarchical namespace service for implementing file system functions for data stored in the blobs; and a front-end layer for processing user requests for accessing and managing data stored on the blobs. The file system functions include a flat namespace function and a hierarchical namespace function. In addition, the front-end layer accesses the partitioning layer to access the blobs based on the addresses of the blobs, and accesses the hierarchical namespace service to identify the blobs.

Description

Technical Field

[0001] The subject matter disclosed herein generally relates to methods, systems, and programs for performing distributed transactions in a cloud storage system. Background Art

[0002] Cloud storage is a model of data storage in which digital data is stored in a logical storage pool embodied in a physical storage device hosted by a cloud storage provider. A cloud storage system may include a collection of networked computing resources (including storage devices, servers, routers, etc.) that are configurable, shareable, provide data security, and provide access to cloud storage to user devices over the Internet. Cloud storage systems enable users to store large amounts of data for essentially any duration. Cloud storage system customers can access their data anytime, anywhere, and pay for what they use and store. Data stored in cloud storage can be persisted using both local and geographic replication to facilitate disaster recovery.

[0003] Some storage systems provide a flat storage structure where simple objects are stored in containers. These objects are sometimes called blobs (originally derived from Binary Large Object), which are blocks of data stored in a database, and can include images, frames of video, videos, readable documents, etc. To access these objects, the user provides an account, the name of the container, and a blob identifier. However, this simple structure may not be useful for some users who want to organize their objects based on their applications.

[0004] Users can create their own organization schemes, such as by organizing blobs by their names and then performing filtering based on the names. However, such schemes are often insufficient for file system workloads, and simple operations such as renaming an object may require thousands or even millions of operations to rename an object.

[0005] Traditional big data and analytics applications have forced customers to create many data silos for different use cases and protocol usages of their data, which not only requires managing a large number of data silos, but also requires copying data between them and managing all these copies, which greatly increases the cost and complexity of customer implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Each of the drawings illustrates merely example embodiments of the present disclosure and are not to be considered limiting of its scope.

[0007] Figure 1 is a diagram of an architecture for a multi-protocol cloud storage system, according to some example embodiments.

[0008] Figure 2is a block diagram of a storage system including a namespace service for mapping file system commands from a hierarchical namespace to a flat namespace according to an example embodiment.

[0009] Figure 3 A structure for storing streams according to some example embodiments is illustrated.

[0010] Figure 4 Different tiers for a flat storage system and a tiered storage system are illustrated according to some example embodiments.

[0011] Figure 5 A front-end component stack is illustrated according to some example embodiments.

[0012] Figure 6 Processing of requests by a storage system according to some example embodiments is illustrated.

[0013] Figure 7 Examples of paths and files in a hierarchical directory structure are shown according to some example embodiments.

[0014] Figure 8 A hierarchical namespace topology corresponding to a hierarchical directory structure is shown according to some example embodiments.

[0015] Fig. 9 A hierarchical namespace topology is shown with paths and file names overlaid according to some example embodiments.

[0016] Fig.10 An example architecture for implementing a hierarchical namespace service is shown in accordance with some example embodiments.

[0017] Fig.11 An example main directory block table is shown according to some example embodiments.

[0018] Fig.12 A block diagram of a hierarchical namespace service including physical nodes and virtual nodes is shown according to some example embodiments.

[0019] Fig.13 A block diagram of a virtual node is shown according to some example embodiments.

[0020] Fig.14 A block diagram of a hierarchical namespace service is shown in which a virtual node forwards a command to a virtual node that manages an entity block identified in the command, according to some example embodiments.

[0021] Fig.15 FIG. 1 illustrates the state of entries in a memory table according to some example embodiments

[0022] Fig.16A snapshot read operation according to some example embodiments is illustrated.

[0023] Fig.17 A read-only local phase operation is illustrated according to some example embodiments.

[0024] Fig.18 The operations of preparing a row during Phase 1 are illustrated according to some example embodiments.

[0025] Fig.19 A snapshot write operation according to some example embodiments is illustrated.

[0026] Fig. 20 Phase 2 for committing a transaction record is illustrated according to some example embodiments.

[0027] Fig.21 Illustrated is a submission notification according to some example embodiments.

[0028] Fig. 22 File system operations for a blob FS (file system) API are shown according to some example embodiments.

[0029] Fig.23 File and directory operations for a blob FS API are shown, according to some example embodiments.

[0030] Fig.24 A sample blob storage command is shown according to some example embodiments.

[0031] Fig.25 A sample blob FS command is shown according to some example embodiments.

[0032] Fig.26 is a flow chart of a method for performing distributed transactions in a cloud storage system having a hierarchical namespace, according to some example embodiments.

[0033] Fig. 27 is a flow chart of a method for providing multi-protocol access to a cloud storage system, according to some example embodiments.

[0034] Fig.28 A system for providing file system functionality on a cloud storage system based on blob storage according to some example embodiments.

[0035] Fig.29 is a block diagram illustrating an example of a machine on or by which one or more of the example process embodiments described herein may be implemented or controlled. DETAILED DESCRIPTION

[0036] Example methods, systems, and computer programs relate to performing distributed transactions in a cloud storage system with a hierarchical namespace. The examples represent possible variations only. Unless otherwise explicitly stated, components and functions are optional and may be combined or subdivided, and operations may vary in order or be combined or subdivided. In the following description, for the purpose of explanation, many specific details are set forth to provide a thorough understanding of the example embodiments. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details.

[0037] Some cloud storage systems store file system objects in a flat global namespace. However, many big data and data analytics applications are designed to store data in a hierarchical namespace. For example, many big data and data analytics applications are configured to work with Apache TM The HDFS is used together with the HDFS distributed file system. The HDFS design is based on the requirements of the POSIX file system, but in some key areas, POSIX semantics have been sacrificed to improve data throughput. The POSIX namespace is a hierarchical namespace with unlimited depth of nesting and atomic operations on it.

[0038] In order to implement the use of a hierarchical directory structure in a cloud storage system using a flat namespace, the embodiments disclosed herein provide a hierarchical namespace service to provide file system operations for a hierarchical structure and for a flat object storage structure, while using the same underlying object storage layer for both types of services. This implements a multi-mode cloud storage service that can be accessed under different interfaces, but multiple interfaces access the same underlying data.

[0039] Both flat namespaces (FN) and hierarchical namespaces (HN) are useful. In some scenarios, FN has cost and performance advantages, while HN has functional, semantic, and different performance advantages. Providing both FN and HN provides customers with the flexibility to choose the best solution for their workloads and applications.

[0040] The embodiments presented herein implement the hierarchical namespace feature of the cloud storage system, which is referred to herein as "blob FS". Blob FS coexists with the blob storage system, which is referred to herein as "blob SS". Both blob SS and blob FS access the same underlying blob objects. The hierarchical namespace adds both single-node partitioned transaction support and multi-node partitioned distributed transaction support, and the distributed transactions are fully flexible to combine any type of namespace operations together.

[0041] Unlike some types of cloud storage (e.g., blob storage), hierarchical namespaces traditionally do not scale horizontally, e.g., they only scale upward to a certain limit. Traditionally, this is due to the need to keep the namespace data structures in memory, and the inability to scale the computational and transactional aspects of the namespace. This is true of typical file systems.

[0042] BlobFS is designed to scale horizontally for file system solutions, just like users are used to blob storage systems. Single-node partition transactions are used for basic operations. In order to enable horizontal scaling for a single storage account or even a single directory within a storage account, operations (such as "rename directory", "delete directory", etc.) can be performed in a distributed manner across nodes and partitions. Strongly consistent distributed transactions are used to support these types of file system operations while providing true horizontal scalability.

[0043] Embodiments provide both blob SS and blob FS interfaces on the same data set, eliminating the need to copy data and create data warehouses. Given that file systems and object storage are two common interfaces for big data and analytics on cloud storage, supporting both interfaces enables most scenarios to take advantage of this flexibility.

[0044] In one embodiment, a method is provided. The method includes receiving a request from a user device, the request including an operation to be performed by a cloud storage system, the request for atomic execution of the operation. The method also includes identifying a node for performing the operation. Each node has a corresponding clock, controls the state of execution of at least one operation, and has at least a portion of a memory table for controlling updates to an entity in the cloud storage system (the entity is one of a file or a directory). The method also includes processing the corresponding operation by each of the nodes. Processing an operation also includes: if the node is the first node in the process, setting a commit timestamp (CS) to the value of the clock in the node; and if the node is not the first node and the value of the clock in the node is less than the current value of CS, waiting for the value of the clock in the node to reach the current value of CS; and updating the current value of CS based on the value of the clock in the node. The method also includes: updating the memory table based on the current value of CS; using the final value of CS to submit the atomic execution of the request in the cloud storage system; and returning the status of the request to the user device.

[0045] In another embodiment, a system includes a memory having instructions and one or more computer processors. The instructions, when executed by the one or more computer processors, cause the one or more computer processors to perform a plurality of actions, the actions including: receiving a request from a user device, the request including an operation to be performed by a cloud storage system, the request for atomic execution of the operation; identifying nodes for performing the operation, each node having a corresponding clock, each node controlling a state of execution of at least one operation, and each node having at least a portion of a memory table for controlling updates to entities in the cloud storage system, the entity being one of a file or a directory; processing the corresponding operation by each of the nodes, wherein processing an operation further includes: setting a commit timestamp (CS) to a value of a clock in the node if the node is the first node in processing; and waiting for the value of the clock in the node to reach the current value of CS if the node is not the first node and the value of the clock in the node is less than a current value of CS; and updating the current value of CS based on the value of the clock in the node; updating the memory table based on the current value of CS; and submitting the atomic execution of the request in the cloud storage system using the final value of CS; and returning the status of the request to the user device.

[0046] In another embodiment, a machine-readable storage medium (e.g., a non-transitory storage medium) includes instructions that, when executed by a machine, cause the machine to perform the following operations: receive a request from a user device, the request including an operation to be performed by a cloud storage system, the request being for atomic execution of the operation; identify nodes for performing the operation, each node having a corresponding clock, each node controlling a state of execution of at least one operation, and each node having at least a portion of a memory table for controlling updates to an entity in the cloud storage system, the entity being one of a file or a directory; process the corresponding operation by each of the nodes, wherein processing an operation further includes: if the node is the first node in processing, setting a commit timestamp (CS) to the value of a clock in the node; and if the node is not the first node and the value of the clock in the node is less than a current value of CS, waiting for the value of the clock in the node to reach the current value of CS; and updating the current value of CS based on the value of the clock in the node; updating the memory table based on the current value of CS; and submitting the atomic execution of the request in the cloud storage system using the final value of CS; and returning the status of the request to the user device.

[0047] In another embodiment, a method is provided. The method includes providing a first application programming interface (API) by a cloud storage system, which stores data in blobs organized in containers. In addition, the method includes providing a second API by the cloud storage system, the first API and the second API provide operations for a flat namespace and for a hierarchical namespace for accessing data, the flat namespace is used to access data based on accounts, containers, and blob identifiers, and the hierarchical namespace is used to access data based on accounts, directories, and files, wherein the directory is configured to include files and other directories. The first API uses the flat namespace by default, and the second API uses the hierarchical namespace by default. The method also includes receiving a command from a user device. When a command for a flat namespace is received, the command is executed using the container and blob identifier included in the command. When a command is received via a hierarchical namespace, a hierarchical namespace service is accessed to convert the directory and file included in the command into one or more blob identifiers, and the command is executed using the one or more blob identifiers generated by the conversion. In addition, after executing the command, a response is sent to the user device.

[0048] In another embodiment, a cloud storage system includes a storage cluster. Each storage cluster includes: a plurality of storage nodes; a hierarchical namespace service for implementing a file system function for data stored in a blob; and a front-end layer for processing user requests for accessing and managing data stored in a blob. Each storage node includes a partitioning layer that stores data in blobs organized within containers, and includes a streaming layer for streaming blob data. The file system function includes a directory function and a file function, each directory is configured to include files and other directories requested via the file system function, and each file is stored in a blob. The front-end layer accesses the partitioning layer to access the blob based on the address of the blob, and the front-end layer accesses the hierarchical namespace service to identify the blob.

[0049] Figure 1 is a diagram of an architecture for a multi-protocol cloud storage system according to some example embodiments. In some example embodiments, the storage system includes a blob storage system ("blob SS") 106 and a blob file storage system 102 ("blob FS"). Different end-user applications 112, backup applications 114, analysis applications 116, etc. can access the storage system via blob SS 106 or blob FS 102.

[0050] The blob storage layer 108 is accessible by both storage systems, and includes a plurality of blobs 110. The blob SS 106 and the blob FS perform direct access to the blob storage layer 108, or they can utilize the hierarchical namespace service 104 to access the blobs.

[0051] According to an example embodiment, the hierarchical namespace service 104 is used to map file system object paths in the hierarchical namespace to file system object identifiers in the flat namespace. The hierarchical namespace service 104 is configured to map commands for the hierarchical file system to flat file system commands for accessing blobs 110. In other words, commands may define hierarchical path names for file system objects even if the file system objects are stored in a flat file system. The hierarchical namespace service 104 is configured to map hierarchical paths for blob system objects (e.g., files) to blob 110 identifiers corresponding to the blob storage 108.

[0052] As used herein, a flat file system is a storage system that stores objects (e.g., clumps) in a container, and accessing objects in a flat file system requires specifying identifiers for containers and identifiers for clumps in the container. In some example embodiments, a hierarchical namespace service 104 maintains a mapping between a hierarchical namespace identifier (or path) and a flat namespace identifier of a file system object stored in a flat namespace clump SS106. In an example embodiment, the mapping is maintained in a namespace table. By maintaining this mapping, a hierarchical namespace service 104 can execute file system commands such as "move files" or "move directories" in a flat namespace storage system without physically moving files or moving folders (and all of its contents, which may be tens, hundreds, thousands, millions, billions, or even more files). On the contrary, in each case, one or more mapping entries can be modified, rather than physically moving file system objects (which means that expensive file system operations will be performed). By changing mapping entries rather than performing expensive file system object operations, embodiments achieve a significant reduction in processor operations and loads.

[0053] Blob FS handles Blob objects internally, but Blob FS abstracts the concepts of Blobs and Blob Types from clients. Furthermore, Blob FS optimizes the use of Blob Types for better performance and lower cost in Big Data and Analytics workloads. The internal format of data chunks and Blob metadata are shared between Blob SS and Blob FS, so they are fully interoperable. Ensuring this compatibility and sharing is one of the challenges and technical advances of the implementation presented in this paper, as support is provided for both interfaces while allowing each to be optimized for its own use case.

[0054] In some example embodiments, there are three types of blobs: block blobs, append blobs, and page blobs. Block blobs store text and binary data and consist of blocks of data that can be managed individually. Append blobs consist of blocks similar to block blobs, but are optimized for append operations. Append blobs are ideal in some scenarios, such as logging data. In addition, page blobs store random access files.

[0055] Blob FS leverages the underlying Blob SS utilities, such as compression and encryption. Since the same internal Blob is used internally, many Blob features, such as "soft delete", "change notification", "change feed", etc. are shared. Since Blob SS and Blob FS are interoperable, all Big Data and Analytics used with Blob FS can automatically leverage the entire ecosystem.

[0056] When a user wants to read from a file via the HN-enabled blob FS, the hierarchical namespace service 104 is used to identify where the data resides (eg, the requested blob), and the blob FS can then access the data directly from the blob.

[0057] Figure 2 2 is a block diagram of a storage system including a namespace service for mapping file system commands from a hierarchical namespace to a flat namespace according to an example embodiment. Storage system 202 stores file system objects for user devices, such as user devices 218, which may number tens, hundreds, thousands, millions, or even more. In some example embodiments, storage system 202 includes location service 204, domain name system (DNS) 206, first storage cluster 208A, and second storage cluster 208B.

[0058] The first storage cluster 208A includes a front-end tier 210A, a partitioning tier 212A, a hierarchical namespace service 104A, a partitioning tier 212A, and a stream tier 214A. The second storage cluster 208B includes a front-end tier 210B, a hierarchical namespace service 104B, a partitioning tier 212B, and a stream tier 214B. The storage system 202 may include any number of storage clusters implemented similarly to the first and second storage clusters 208A and 208B, including tens, hundreds, thousands, millions, or even more.

[0059] Storage clusters 206A and 206B have racks of physical storage servers, where each rack is built as an independent failure domain with redundant network and power. Each of storage clusters 206A and 206B may include any number of racks, each with any number of storage nodes.

[0060] Location service 204 may be implemented in one or more servers and is configured to manage storage clusters 206A and 206B. Location service 204 is also responsible for managing account namespace metadata across all storage clusters. Location service 204 assigns accounts to storage clusters 206A and 206B and manages accounts across storage clusters 206A and 206B for disaster recovery and load balancing.

[0061] In some example embodiments, storage system 202 includes storage in multiple locations in multiple geographic regions (e.g., North America, Europe, Asia, etc.). Each location may include a data center containing multiple storage clusters. To provide additional capacity, location service 204 may add new regions, add new locations to regions, or add new storage clusters to locations.

[0062] The location service 204 tracks the resources used by each storage cluster (including storage clusters 206A and 206B) across all locations. In some example embodiments, when an application requests a new account for storing data, the application specifies a location neighbor (e.g., the North United States) for the storage. The location service 204 uses heuristics to select a storage cluster within the location as the primary storage cluster for the account based on load information across all storage clusters (taking into account the integrity of the storage clusters as well as other metrics such as network and transaction utilization). In addition, the location service 204 stores account metadata information in the selected storage cluster, which tells the storage cluster to begin processing traffic for the assigned account. The location service 204 updates the DNS 206 to allow requests to be routed from the URI to the virtual IP (VIP) address of the storage cluster (the IP address that the storage cluster exposes to external traffic).

[0063] The front-end (FE) layers 210A and 210B each include a set of stateless servers that receive incoming requests from applications in user devices such as user device 218. Upon receiving a request, the corresponding FE 210 looks up the account name, resolves the request, authenticates and authorizes the request, and then routes the request to a partition server in the corresponding partition layer 212 based on the partition key. The partition layer 212 maintains a partition map that tracks the partition key ranges and which partition server is serving which partition key for the storage cluster. Partition keys are also referred to as partition names in this document. The FE servers store the partition map in a cache and use the partition map to determine which partition server to forward each request to. The FE servers also stream large objects directly from the corresponding streaming layer 214 and store frequently accessed data in a cache for efficiency. The hierarchical namespace services 104A and 104B are located between the front-end layer 210A and the partition layer 212A and between the front-end layer 210B and the partition layer 212B, respectively. The hierarchical namespace services 104A and 104B are both Figure 1An example of a hierarchical namespace service 104 is provided and is configured to transform requests that refer to file system objects using hierarchical namespace identifiers into requests that point to file system objects in the flat namespaces of storage clusters 208A and 208B.

[0064] Partitioning layers 212A and 212B are each configured to: manage and understand higher-level data abstractions (e.g., chunks, tables, queues), provide transaction ordering and strong consistency for objects, store object data on top of corresponding stream layers, and store object data in cache to reduce disk input / output (I / O).

[0065] In addition, the partitioning layers 212A and 212B each achieve scalability by partitioning data objects within the corresponding storage cluster 208. As previously described, data objects have partition keys. Data objects can be decomposed into unconnected ranges based on partition key values ​​and served by different partition servers. The partitioning layers 212A and 212B each manage which partition server is serving what partition key range of a data object (e.g., a blob, table, and queue). In addition, the partitioning layer 212 provides automatic load balancing of partition keys across partition servers to meet the traffic requirements of data objects.

[0066] The stream layer 214 stores data on physical storage (e.g., hard disks, solid-state storage, etc.) and is responsible for distributing and replicating data across many servers so that the data is persistent within the corresponding storage cluster. The stream layer 214 can be considered as a distributed file system layer within a storage cluster. The stream layer handles files (files are called "streams", which are ordered lists of large storage blocks called "extents"), how to store them, how to replicate them, etc., but the stream layer does not handle higher-level object construction or their semantics. Data is stored in the stream layer 214 and is accessible from the partition layer 212 and the front-end layer 210. In some cases, the front-end layer 210 obtains data pointers from the partition layer 212, and the front-end layer 210 reads data directly from the stream layer 214.

[0067] Note that data can be stored in any form, including file system objects such as files and folders, blobs 110 (e.g., user files), tables (structured storage), and queues (message delivery). The stream layer 214 accesses data in the form of large files called streams, and enables the corresponding partition layer to open, close, delete, rename, read, append, join streams, etc.

[0068] In traditional Hadoop implementations, data size is limited to a modest petabyte or a few hundred million files. Beyond this, the naming service cannot be scaled because the naming service must be kept in memory, similar to traditional operating systems and big data file systems. By implementing a distributed hierarchical namespace service 104, blob SS and blob FS can be scaled horizontally without the memory constraints of a single machine or a small number of machines.

[0069] Figure 3 A structure for storing a stream 302 is illustrated in accordance with some example embodiments. In some example embodiments, a stream is an ordered list of pointers 304 to intervals, where an interval is a sequence of appended blocks 306. For example, a stream " / / foo" 302 may contain pointers 304 to several intervals (e.g., E1, E2, E3, and E4). Each interval contains a set 306 of previously appended blocks. In one example, E1, E2, and E3 may be closed intervals, meaning that they may no longer be appended to, and E4 is not closed, meaning that new blocks may be appended to the interval. If an application reads the data of the stream " / / foo" from beginning to end, the application receives the block contents of the intervals in the order of E1, E2, E3, and E4.

[0070] A block is a unit of data for writing and reading. In one embodiment, a block can be up to N bytes (e.g., 4MB-100MB). Data is written (e.g., appended) to an interval as one or more concatenated blocks, where the blocks do not have to be of the same size. Append items can be specified based on blocks and the size of each block. A read gives a stream or interval offset, and the stream layer reads the required number of blocks at that offset to satisfy the length of the read. When performing a read, the entire contents of the block are read. For example, the corresponding stream layer can store its checksum verification at the block level, with one checksum per block. The entire block is read to perform the checksum verification, and it can be checked for each block read. All blocks can be periodically verified against their checksums to check for data integrity issues.

[0071] An interval is the unit of replication in the stream layer, and an example default replication policy is to maintain three copies of an interval within the storage cluster. The target interval size used by the partition layer can be, for example, 1GB, but other values ​​are possible. To store small objects, the partition layer appends many of them to the same interval, and possibly even in the same block. To store large objects (e.g., terabyte-sized objects, which may include several clumps), the partition layer can break the object into many intervals. The partition layer keeps track of which streams, intervals, and offsets the object is stored in, which are stored as part of the object's index.

[0072] Each stream has a name in the stream layer, and the stream appears as a large file to the partition layer. Streams can be appended to, and can be read randomly. When intervals are concatenated together, the intervals represent a complete contiguous address space, where the stream can be read in the order in which the intervals were added to the stream. New streams can be constructed by concatenating intervals from existing streams, which is a fast operation because the list of pointers can be updated. Only the last interval in a stream can be appended, and all previous intervals in a stream are immutable.

[0073] Figure 4 Different object tiering for a flat storage system and a tiered storage system are illustrated according to some example embodiments.The storage system 202 provides a global namespace that allows clients to address within the storage and allows clients to scale to any amount of desired storage over time.

[0074] In some example embodiments, the storage system 202 includes two different types of tiers: a tier of blob SS 106 and a tier of blob FS 102. These two tiers can coexist within the storage system 202 and allow users to access the storage in two different ways: using a flat namespace or using a hierarchical namespace. Objects in the blob storage can be accessed from anywhere via HTTP or HTTPS and can be addressed as blobs or files.

[0075] The blob SS 106 hierarchy includes accounts 401, which include blob containers 402, and which include blobs 110. The blob FS hierarchy 102 includes accounts 404, which include file systems 406, which include directories 408, and which include files 410.

[0076] Files 410 and directories 408 are interoperable with blobs 110. In some example embodiments, when FN is used, directory 408 uses an empty blob as a placeholder because HN has a first class directory entry in the directory block storage. Files 410 utilize blobs for storage. In some example embodiments, account 401 for blob SS and account for blob FS are the same account 404.

[0077] The blob container 402 is compatible with the blob FS, which treats the blob container 402 as a file system with default settings. In addition, the blob file system 406 created by the blob FS works with the blob SS, which treats the blob file system 406 as the blob container 402 and ignores the attached metadata. The service metadata used by the blob FS is kept separate from the user metadata.

[0078] A blob container 402 organizes a group of blobs 110, and all blobs 110 reside within a corresponding blob container 402. In some example embodiments, a storage account may contain an unlimited number of blob containers 402, and a container may store an unlimited number of blobs 110. Furthermore, a FS may contain an unlimited number of directories, and a directory may contain an unlimited number of directories and files.

[0079] With respect to accessing files via blob FS, data I / O operations use offset and length parameters. In addition, blob FS uses append commands to add data, and flush commands to commit data to disk. Append file operations are used to actually write data, and append operations write data to disk; the data is retained, but not yet readable by read file operations. Flush file operations commit the requested unflushed length of the latest append so that it can be read by read file operations; this is not a data write, just an index write operation. In addition, blob FS supports parallel reading and writing of the same file. Additionally, blob SS operations are compatible with blobs created as files using blob FS. In addition, flush commands are optimized to allow concurrent data appending while flushing, which greatly improves the performance of applications that write data sequentially (such as transaction logs).

[0080] In blob SS 106, the storage namespace is defined in three parts: account name, container name, and object name. As a result, data can be accessed in the storage system 20 via a URI of the following form:

[0081] http: / / sAccountName / ContainerName / BlockName

[0082] An example of where janeaccount.blob.core.windows.net is the account name, mycontainer is the container name, and files / myblob.txt is the object name is as follows: http: / / janeaccount.tuankuai .core.windows.net / mycontainer / files / myblob.txt

[0083] Account names are assigned to customers for accessing storage and are managed by DNS host names. DNS 206 performs DNS translation for account names to locate storage clusters 208 and data centers where data is stored. Applications can use multiple account names to store data across different locations. Once a request arrives at the storage cluster, the partition key locates the data. The partition key is used to expand access to data across storage nodes based on traffic demand. When a partition key holds many objects, the object name identifies each object within the partition. The storage system 202 supports atomic transactions across objects with the same partition key value. Atomic transactions include multiple operations to be performed by the cloud storage system, so that all operations succeed, or no operations are performed and the atomic transaction is terminated. This naming method enables the storage system to flexibly support multiple data abstractions. For example, for a blob, the full blob name is the partition key. A blob can have a snapshot. In addition, the row key (which identifies the object within the partition key) is AccountName; ContainerName; ObjectName; SnapshotVersion, so the system can perform transaction operations on snapshots of the same file / object.

[0084] The object name may be optional because for some types of data, the partition key uniquely identifies the object within the account. This naming method enables the storage system 202 to flexibly support multiple data abstractions. For example, for a blob, the full blob name is the partition key, which is in the format account; container; blob. In the above example, the partition key is janeaccount; mycontainer; files / myblob.txt.

[0085] The format of the partition key for HS operations is account; filesystem; directory; file. Data can be accessed in the storage system 202 via a URI of the following format:

[0086] http(s): / / Accountname / FileSystem / Directory / File

[0087] For example, an example where janeaccount.dfs.core.windows.net is the account name, myfs is the file system, files is the directory, and myblob.txt is the file name is as follows:

[0088] http: / / janeaccount.dfs.core.windows.net / myfs / files / myblob.txt

[0089] In some example embodiments, blob FS 102 provides an API, such as a REST API, for accessing data.Blob FS 102 is built on top of blob 110; therefore, blob FS 102 provides the features that users are accustomed to from blob SS, meaning that no benefits are lost due to the implementation of blob FS.

[0090] Furthermore, when using blob FS 102, the blob abstraction is hidden from the user by providing access to files. The user does not need to configure access permissions for the use of different types of blobs, such as block blobs, append blobs, and page blobs. With blob FS, the user does not have to configure or select a blob type.

[0091] Blob SS and Blob FS share internal data formats, compression and encryption mechanisms, and both can write different, unique metadata, but also shared metadata for the same account / container / blob. In addition, both interfaces use an internal blob abstract model, which is implemented by internal tables maintained by the partitioning layer.

[0092] By building the clique FS with clique services, clique SS and clique FS can share features such as encryption, object-level tiering, soft deletion, change notification, virtual network, geo-replication, object versioning, archival storage, lifecycle policy management, zone redundant storage, etc.

[0093] Figure 5 A front-end component stack according to some example embodiments is illustrated. The storage system supports multiple protocols and extensible metadata support. The blob SS API and the blob FS REST API are interoperable, and data and metadata are stored in the same object.

[0094] The blob FS frontend has an extensible architecture that supports multiple protocols. Blob SS and Blob FS have separate endpoints in storage, and for each storage account, the endpoints are addressed by different DNS records. <account>.blob.core.windows.net addressing, and blob FS by <account>.dfs.core.windows.net addressing.

[0095] For each storage cluster, there is a VIP for each endpoint, so blob SS and blob FS have different VIPs for the cluster. Therefore, when a request is received, it is known which frontend (blob SS or blob FS) received the request without having to perform any parsing.

[0096] The network load balancing layer 504 selects which FE instance corresponds to the VIP of the FE type and selects which instance / server will service the request.At the cluster level, endpoint load balancing is performed across all frontends for that endpoint in the cluster.

[0097] Furthermore, each instance of the front end in the cluster includes layers 506, 508, 510, and 512. The network protocol layer 506 is the transport of the request, and the authentication and authorization layer performs parsing and authentication of the request.

[0098] The corresponding REST API 510 processes the request, and the internal service API 512 then converts the command into internal operations performed by the storage system and acts as a gateway to the partition layer.

[0099] In some example embodiments, at the storage account level, users have FN by default, but in other embodiments, HN can be the default. When a storage account is created, the user can choose to enable HN. If the storage account uses FN, both blob SS and blob FS use FN. If the storage account uses HN, both blob SS and blob FS use HN. The front end for blob SS and blob FS understands FN and HN. If the front end is servicing a request for a storage account with FN, the front end talks directly to the blob internal service. If the front end is servicing a request for a storage account with HN, the front end first talks to the hierarchical namespace, which traverses the namespace to access the file, and ultimately returns a pointer to the blob in the "blob internal service", which the front end will then access.

[0100] Although blob SS and blob FS APIs support both FN and HN, blob SS and blob FS APIs do not provide exactly the same functionality. Blob and file system functions are different and provide different functionality. Blob SS provides pure key-value storage style access to blobs, without directory or file system functionality. Blob FS provides file system functionality with directories and provides operations such as rename directory, delete directory, etc. In addition, with the blob FS API, HN can enable or improve certain features such as atomic directory operations, directory operations, listing with specific directory efficiency, etc.

[0101] Figure 6 The processing of requests by a storage system according to some example embodiments is illustrated. Traditionally, cloud storage has provided general object storage features, but these systems have not provided utilities that are available in traditional file systems hosted on a single machine, a small number of machines, or in traditional distributed big data file systems (such as first-class directory support), including reliable, efficient, and consistent directory operations. For example, to rename a blob, some cloud object storage systems will create a new blob with the new name, copy the data in some manner (deep or shallow copy), and delete the old blob. If the user uses the names of the blobs to organize data (e.g., to simulate directories or categories with the same name prefix), then to rename the simulated directory, the cloud storage will have to copy all blobs (which may be millions or more).

[0102] Some cloud storage systems use a simple programming model for distributed processing of large data sets across clusters of computers, leveraging the Apache Hadoop software library. Apache Hadoop is designed to scale from a single server to thousands of machines or more, with each machine providing local computation and storage.

[0103] Most users want or expect their cloud object storage to provide the same functionality as the file system they use in a single device. In addition, users want cloud storage systems to scale to virtually unlimited amounts of storage and not be limited by the specific implementation of the cloud storage. Blob FS is able to provide these services and is able to provide the ability to scale without limits.

[0104] In addition, the services provided by cloud storage must provide consistency and robustness. Blob FS provides support for highly consistent atomic operations. For example, if a directory is renamed, the user expects to see all the files in that directory immediately. If the system only uses blob names to simulate directories, renaming a directory requires renaming millions of blobs.

[0105] Some operations must be atomic because, for example, they are used to implement mutually exclusive access between processes in the cluster. These operations include at least create file, delete file, rename file, rename directory, create directory, and delete directory.

[0106] Figure 6 The processing of a user request (eg, reading a file) is illustrated. The storage system 202 provides a blob SS API 508 and a blob FS API 510, which include a set of operations for accessing files using a flat namespace and a hierarchical namespace.

[0107] Thus, the blob SS API 508 provides a feature for reading a given blob within a given container, while the blob FS API 510 provides a feature for reading a given file provided with a path including one or more directories.

[0108] In the illustrated example, application 112 in computing device 218 sends request 604 to storage system via network 602. If the request is for blob SS, blob SS API 508 parses 612 the request. If the request is for blob FS, blob FS API 510 parses 614 the request.

[0109] After parsing, the request is sent to the front-end layer 210. As described above, the front-end layer 210 interacts with the partitioning and streaming layer 618 to access data from the blob store 108 (eg, read the requested blob 110).

[0110] If the request is for accessing a file, the front-end layer 210 interacts with the hierarchical namespace service 104 to determine the blob address involved in the request (e.g., the address of the blob storing the file for the read request). The hierarchical namespace server 104 ensures atomic execution of the request to ensure that the request is either successful or unsuccessful and that the consistency of the storage system 202 is always maintained.

[0111] After hierarchical namespace service 104 provides the address to front-end tier 210 , front-end tier 210 can access partition and stream layer 618 to fulfill the request (eg, read data from a file). After fulfilling the request, storage system 202 sends response 606 to application 112 .

[0112] Note that both the blob SS API 508 and the blob FS API 510 can simultaneously access the same underlying blob storage 108. By leveraging the existing blob storage 108, implementing the new blob FS API 510 leverages the existing infrastructure to access blobs while adding new features (such as the use of directories) to the interface. The storage system 202 is considered multi-protocol because the storage system 202 can support more than one access protocol simultaneously. For example, if a blob has certain properties that have been defined via the blob SS API 508, the blob FS API 510 can read those properties.

[0113] In the previous implementation, multi-protocol support requires having different copies of the data, each copy being structured for a different interface. However, this requires having multiple copies (which means additional storage is required), and the copies must be synchronized (which means increased computing resources and potential synchronization issues). On the other hand, since multiple protocols access the same underlying data, the multi-protocol support provided by the storage system 202 does not require multiple copies of the data.

[0114] In addition, the blob FS API 510 hides the complexity of having to select a blob type (e.g., block, page, append). In some example embodiments, the blob FS monitors how users write and read data, and based on this monitoring, the blob FS selects the type of blob to store files. For example, if the user typically writes data sequentially (e.g., a log file), the blob FS selects an append blob. If the user typically accesses files by reading and writing at different locations, the blob FS will select a page blob.

[0115] Figure 7 An example of paths and files in a hierarchical directory structure 700 is shown according to some example embodiments. The hierarchical directory structure 700 may include directories and files within the directories.

[0116] In this example, the hierarchical namespace may include a root " / ", directories / path1 and / path2 below the root, files file1 and file2 in directory / path2, directory path3 in directory / path2, and file file3 in directory / path2 / path3.

[0117] Hierarchical namespaces are used to provide an abstraction with directories and paths, leveraging the underlying blob storage. For example, to rename a directory, the blob FS changes the table that maps directories to directory names, without having to rename all blobs in the directory. One of the goals of the hierarchical namespace service is to provide fast and atomic operations on directories. The hierarchical namespace service provides atomic transactions, which allow multiple operations on blobs to be performed atomically: either all operations are performed successfully, or none of them are performed.

[0118] Figure 8 A hierarchical namespace topology 800 corresponding to the hierarchical directory structure 700 is shown according to some example embodiments. Topologically, the hierarchical namespace topology 800 is a tree, where the tree is formed by nodes and relationships between nodes. A node can be a directory or a file (e.g., an object). Except for the root, each node has a parent node, and each node has a set of attributes, one of which is the name of the node. The name of the node is unique to nodes with the same parent node.

[0119] The name can be changed without affecting the topology, and the change of the topology does not affect the name or properties. In the hierarchical namespace topology 800, a non-volatile globally unique identifier (GUID) that uniquely identifies the node is assigned to each node. Therefore, each file and directory has a unique GUID in the hierarchical namespace. In this article, a unique non-volatile identifier is often referred to as a GUID, although this reference is only for illustrative purposes. Embodiments are suitable for using any type of unique non-volatile identifier (such as, GUID, multiple GUIDs, GUID timestamping, or any other type of identifier that does not change and is unique within the relevant scope of the storage system) to identify file system objects such as files and folders.

[0120] In the illustrated example, GUID1-GUID4 are unique identifiers corresponding to / , / path1, / path2, and / path3. GUID1 is an identifier for the root directory, GUID2 is an identifier for the / path1 directory under the root directory, GUID3 is an identifier for the / path2 directory under the root directory, and GUID4 is an identifier for the / path3 directory under the / path2 directory.

[0121] Fig. 9 A hierarchical namespace topology 900 is shown with paths and file names overlaid according to some example embodiments. Figure 8 Comparing hierarchical namespace topology 800 with hierarchical namespace topology 900, it can be observed that GUID1 is the GUID of the root, GUID2 is the GUID of / path1, and so on.

[0122] Fig.10 An example architecture for implementing a hierarchical namespace service according to some example embodiments is shown. In some example embodiments, the namespace management architecture 1000 is configured to present a storage system namespace, including mapping hierarchical file system commands to a flat file system. The namespace management architecture 1000 includes a partition layer 1002, a hierarchical namespace service 104, physical nodes 1004, virtual nodes 1006, directory blocks 1008, entity blocks 1010, one or more file versions 1012, and one or more directory versions 1014.

[0123] The partitioning layer 1002 is configured to manage data structures that manage various aspects of the storage service, such as data objects (e.g., blobs, files, directories, etc.), queues, etc. For example, the data structure may be in the form of a table and may track objects in storage, such as by including an identifier for the object, an indication (e.g., indicated by a partition key) of where the object is stored (e.g., a partition), a storage timestamp of the object, etc. In some example embodiments, each row of the table may have a schema and may be accessed by a partition key and a row key, referred to as the primary key of the row. The partitioning layer 1002 maintains a namespace table (also referred to herein as a "master directory block map") as a persistent store of namespace state and managed partitions of the storage cluster. The master directory block map may maintain a mapping between hierarchical namespace identifiers (e.g., path names) of file system objects and flat namespace identifiers (e.g., GUIDs) of these file system objects, as well as an indication of parent-child relationships between file system objects.

[0124] As described above, hierarchical namespace service 104 is a service namespace that receives file system commands that reference file system objects using hierarchical namespace identifiers, maps the hierarchical namespace identifiers to flat namespace identifiers, and then applies commands to the file system objects in the flat namespace using the flat namespace identifiers. In one embodiment, hierarchical namespace service 104 includes and manages physical nodes 1004, which manage virtual nodes 1006 that perform namespace mapping.

[0125] In some example embodiments, each physical node of physical nodes 1004 may be implemented as a physical machine. For example, a physical node may be implemented as a physical server. A physical server may execute and / or implement one or more virtual nodes 1006, for example by executing a hypervisor that presents a virtual operating platform, and the virtual node may run on the physical node in the form of a virtual machine. There may be many physical nodes in a storage cluster, such as one thousand or other number of nodes.

[0126] The number of virtual nodes 1006 managed by physical node 1004 can be scalable, or can be a predefined static number. Virtual nodes 1006 can be moved between physical nodes 1004. For example, if the first virtual node is too busy (e.g., operating above a processor utilization level threshold) and the second virtual node is also busy, and they are all managed by the same physical node (e.g., running on it), then one of the virtual nodes can be transferred to another physical node that is available and has sufficient resources. In this way, load balancing can be performed by moving resources between physical nodes and virtual nodes. Virtual nodes 1006 can all maintain their states in persistent storage so that at any time, virtual nodes can be moved and / or restarted on different physical nodes 1004. In one embodiment, different identifiers (e.g., digital identifiers (IDs)) are associated with each virtual node in virtual node 1006, and at any given time, only one instance of a virtual node with a given identifier runs.

[0127] Directory block 1008 corresponds to a hierarchical namespace directory. Typically, a single directory block corresponds to a single directory. When a directory is created, a GUID is generated and assigned to the directory to become the permanent name of the directory. In one embodiment, a hash function is performed on the GUID to generate a hash result. The hash result is used to determine the permanent location of the directory block for the directory. In particular, in some example embodiments, the directory is assigned to a virtual node with a digital ID that matches the hash result, and the assignment does not change unless load balancing occurs. The directory is permanently managed by the virtual node via the GUID.

[0128] Directory blocks 1008 are managed by corresponding virtual nodes 1006, where each directory block corresponds to a directory (root directory or subdirectory) or a portion of a directory in the hierarchical namespace. Inside the directory block 1008 are entity blocks 1010, each entity block being a file or folder inside the directory. Note that any number of directory blocks 1008 and entity blocks 1010 can be managed by the hierarchical namespace service 104, including billions or more.

[0129] Each entity block 1010 can have multiple versions, each version associated with a different time period. The file entity block has one or more versions indicated as file version 1012, and the directory entity block has one or more versions indicated as directory version 1014. There can be any number of versions of directory blocks 1008 and entity blocks 1010, including hundreds or even thousands of versions. The version of the entity block is included after a specific name. For example, if the attribute of a file named "foo" is changed, a new version of "foo" is generated, and all versions of "foo" use the same name. Entity block versions enable the use of multi-version concurrency control (MVCC). According to MVCC, the namespace can not only execute transactions for entity blocks at the current moment, but also execute transactions for entity blocks at past time points by executing transactions for earlier versions of the entity block (the current time when the transaction is received (for example, as verified by comparing timestamps)).

[0130] Fig.11 An example master directory block table according to some example embodiments is shown. As described above, partition layer 1002 can manage a data structure that maps file system objects (such as folders and files) in a hierarchical namespace to file system object identifiers in a flat namespace and indicates parent-child relationships between file system objects. Such a data structure for mapping can have any form, such as a table. For example, Fig.11 An example home directory block table 1100 is shown according to an example embodiment.

[0131] The master directory block table 1100 is an example of a data structure that can be used to map hierarchical namespace identifiers of file system objects to flat namespace identifiers, and can be used to identify parent-child relationships between file system objects. The master directory block table 1100 identifies the objects contained in each directory. The master directory block table 1100 includes a directory block identifier (DBID) column 1102, a name column 1104, a commit time (CT) column 1106, and an entity block identifier (EBID) column 1108, and may optionally include additional columns, such as a deleted indication column ("delete flag"), a file indication column, and any other additional columns that may be required.

[0132] The main directory block table 1100 can be represented by Figure 6 The partition layer 1002 of the hierarchical namespace service 104 can manage versions of the main directory block table 1100 distributed in many locations. For example, each virtual node can maintain and manage a corresponding part of the main directory block table 1100, which is called a directory block map. For example, the main directory block table 1100 is shown as being divided into four parts 1110A-1110D. The four parts 1110A-1110D correspond to a specific set of one or more directory blocks and entity blocks stored in the form of one or more rows. In addition, each of the four parts 1110A-1110D can be managed by a corresponding virtual node, but one virtual node can also manage several parts. For example, the first virtual node can maintain the first part 1110A, the second virtual node can maintain the second part 1110B, the third virtual node can maintain the third part 1110C, and the fourth virtual node can maintain the fourth part 1110D. By maintaining the master directory block table 1100 distributed across virtual nodes in this manner, the performance of the hierarchical namespace service 104 is improved, and the ability to grow the master table is not limited because additional nodes can be added as storage needs grow.

[0133] Directory block identifier (DBID) column 1102 stores an identifier for each directory block (e.g., each directory) in the form of a DBID. A DBID is a unique identifier that never changes for a particular directory block. In one embodiment, a DBID is a 128-bit value generated for each new directory block.

[0134] The entity block identifier (EBID) column 1108 stores an identifier for each entity in the form of an EBID. When the entity is a directory, the EBID of the directory is also the DBID of the directory. When the entity is a file, the EBID is the GUID of the file, and the EBID never changes for the file. If the entity block has multiple versions, these versions are listed in the corresponding row of the directory block map. Different versions represent different states of the entity block file or directory at different time intervals.

[0135] exist Fig.11 In the example of , three versions of the same entity GUID-FILE4 (for file1) are listed in row entries 6, 7, and 8. Rows 6, 7, and 8 list the same DBID and EBID, but have different values ​​in the commit time (CT) column 1106, which indicates the time when the corresponding version of the entity block was committed to storage. Thus, a version of an entity block is read valid only when the transaction read timestamp (RT) of a command pointing to the entity block has a value between the commit time of the version and the commit time of the next newer version of the entity block, unless the version corresponds to the latest commit time, in which case the latest version is valid for all RTs after the latest commit time. In this way, a command can act on the version of the entity block that is valid when the command is issued, rather than on the latest version of the entity block.

[0136] Thus, in the master directory block table 1100, each row represents a version of a physical block. The primary key (PK) for the master directory block table 1100 is the DBID. The row key (RK) is the name (in the name column 1104) and the commit time for the physical block. Table 1 below shows example types and descriptions of various columns that may exist in the master directory block table 1100, including Fig.11 Columns shown in .

[0137] Table 1

[0138]

[0139]

[0140] It should be noted that the types of variables in the table can have variables of different sizes. In one embodiment, the data in the main directory block table 1100 is made immutable. When a directory is deleted, a new row is added and the value of the Del flag is set to "yes". In addition, when a file is deleted, the value of the Del flag is set to "yes", and the file will not be visible at any time after the CT of the file delete operation.

[0141] To rename a file or directory, the current entry in the table is deleted (Del flag set to "yes"), and a new entry with the new name is added. Create, update, and delete commands add new rows in the table. Garbage collection (GC) can be implemented to delete old rows from the table at predetermined intervals or configured in other ways.

[0142] When a client (for example, Figure 2 When an application 112) changes the properties of a file or directory, the change is indicated in the directory block of the parent directory of the file or directory. For example, the result of changing the access attribute of a file is to insert a row in the main directory block table 1100, which has the DBID of the parent directory, the name column 1104 of the file, the EBID equal to the GUID of the file, and the new value of the attribute.

[0143] However, there are directory properties that can be changed indirectly by the client. For example, when a client creates a new file in a directory, the last write timestamp for that directory can change. In these cases, each directory can have a special file called a "dot file" (e.g., with an internal name of "."), in which directory attributes that may change (such as internally maintained properties and properties set by the client) are maintained. At predefined intervals, some properties from the dot file are copied to the properties in the directory blocks of the parent directory, where the client can view them. For example, the last read time of NTFS can be propagated to the client part once every hour. Propagation can be performed on a more complex schedule. For example, when a directory timestamp changes, the updated time will be propagated to the parent directory immediately, and then propagated after a predefined period of time.

[0144] As described above, the directory block map is a distributed portion of the main directory block table 1100 that is associated with a virtual node. Each virtual node maintains a corresponding portion of the main directory block table 1100 as a directory block map. In some example embodiments, the directory block map is maintained in the main memory, and the directory block map can maintain data such as directory block and entity block identifiers in the form of hash tables and entries. The memory representation is optimized for supporting real-time transactions and retaining a small portion of hot data.

[0145] For each entity block listed in the directory block map of the virtual node, a row of the last few versions will be maintained, which has information about the transaction status and a flag that indicates whether there is an earlier (older) version in the main directory block table 1100. The versions of the entity block are a list of entity blocks, and the list grows from the beginning. If there are too many entries in the list, the oldest entries can be discarded, and an indication can be made at the end of the list that more versions are stored in the main directory block table 1100 and / or one or more name resolution caches. In memory, the directory block provides fast access to the mapping between the name specified by the client (name column 1104) and the EBID (EBID column 1108).

[0146] In one embodiment, an additional data structure is provided as another distributed form of the main directory block table 1100. The additional data structure is called a name resolution cache and forms a distributed cache service. Each virtual node can have an associated name resolution cache. The name resolution cache is used by a virtual node configured to perform a name resolution function, which maps a hierarchical namespace directory name (or path) to a DBID. This name resolution node can obtain the necessary name to DBID mappings from other virtual nodes (referred to as storage nodes in this article), but then, these name to DBID mappings are stored in a local name resolution cache so that subsequent name resolution operations can be performed more efficiently. Each virtual node can have an associated name resolution cache. The nature of the service provided by the name resolution cache is key-value storage. A key is a sequence of bytes. The name resolution cache supports sparse streams as values.

[0147] For example, a value may have 3 versions: v1, v2, and v3. At offset timestamp 1 (TS1) to offset timestamp 2 (TS2), v1 may be written, at offset TS2 to offset timestamp 3 (TS3), value v2 may be written, and from offset TS3 until the end of the name resolution cache (e.g., offset 2^64), value v3 may be written. Subsequently, the name resolution cache may be read at any offset in the stream, and the name resolution cache will return the correct value.

[0148] A virtual node can cache entity blocks and directory blocks in a name resolution cache. For entity blocks, the name resolution cache key is DBID+EBID. A stream can be written to a version of an entity block between the offset of the committed transaction that introduced the data and the offset of the committed transaction of the next version. The read of an entity block is performed by a specific timestamp, which is used as an offset in the stream. The name resolution cache returns the data stored in the range that the offset falls into. When a newer value is stored, the previously stored value is overwritten. For directory blocks, the name resolution cache key is DBID+EBID. The timestamp of the previous request is written to the stream list of the EBID.

[0149] During a read operation, if the name resolution cache returns data, the data is considered correct until a certain point in time. This point in time can be the current time or a time in the past. Namespace data is immutable, so there is no problem that may cause invalid data to be read, but there may be a newer version of the data that has not yet been uploaded to the name resolution cache. In this case, the results from the local memory (directory block mapping) and the name resolution cache will be combined. Because the name resolution cache is updated each time the main directory block table 1100 is written, if there is no newer version in the directory block mapping, the name resolution cache version is the latest.

[0150] Thus, at multiple levels (including the main directory block table 1100, the directory block map at the virtual node (distributed), and the name resolution cache at the virtual node (distributed)), an embodiment of the hierarchical namespace service implements mappings for mapping between namespace identifiers of file system objects and flat namespace identifiers of these file system objects, thereby providing redundancy, greater efficiency (e.g., reduced traffic in requesting mapping information), and faster operation (e.g., via faster access to nearby mapping information, storing hash values ​​of identifiers for faster comparison, etc.).

[0151] Fig.12 A block diagram of a hierarchical namespace service including physical nodes and virtual nodes according to some example embodiments is illustrated. The hierarchical namespace service 1200 can be implemented in various ways in physical nodes and virtual nodes to process file system transaction requests using mapping information. The hierarchical namespace service 1200 includes a plurality of physical nodes (PNs) 1202A-1202E and a plurality of virtual nodes (VNs) 1208 (represented as being located on a dashed circle).

[0152] Virtual nodes 1208 include a first virtual node set 1204A, a second virtual node set 1204B, and a third virtual node set 1204E. The first virtual node set 1204A includes a first virtual node 1208A and one or more optional additional virtual nodes. The second virtual node set 1204 includes a second virtual node 1208B and one or more optional additional virtual nodes. The third virtual node set 1204E includes a third virtual node 1208E and an optional additional virtual node.

[0153] In one embodiment, physical nodes 1202A-1202E are executed as physical machines (not shown) in one or more servers. There may be any number of physical nodes, including tens, hundreds, thousands, or even larger numbers of physical nodes. Each physical node is independent and communicates with other physical nodes and virtual nodes.

[0154] There may be any number of virtual nodes, including tens or hundreds of thousands or even more virtual nodes. Virtual nodes may be implemented in a variety of ways, including as virtual machines running on physical nodes.

[0155] Each virtual node set is managed (eg, hosted / run) by a corresponding physical node. For example, physical node 1208A manages a first virtual node set 1204A, physical node 1202B manages a second virtual node set 1204B, and physical node 1202E manages a third virtual node set 1204E.

[0156] Each virtual node manages a corresponding set of directory blocks, where each directory block is a corresponding partition of the storage namespace. The namespace table maintains a mapping between hierarchical namespace identifiers (e.g., path names) of file system objects and flat namespace identifiers (e.g., GUIDs) of these file system objects, as well as an indication of parent-child relationships between file system objects. In one embodiment, each virtual node in virtual node sets 1204A, 1204B, and 1204E can manage a corresponding set of one or more directory blocks, but at a particular time some virtual nodes can manage zero directory blocks.

[0157] Thus, the hierarchical namespace service 1200 provides management of a namespace for addressing file system objects in storage and may be implemented in the form of physical nodes and virtual nodes.

[0158] In some example embodiments, each virtual node has a write-ahead log, and updates occurring in the virtual node first enter the write-ahead log. The write-ahead log is stored in a persistent storage, so if a virtual node fails, the virtual node can be restarted on the same or different physical node and continue the operation that was suspended when the virtual node crashed by checking the write-ahead log.

[0159] Fig.13 A block diagram of a virtual node 1208 is illustrated according to some example embodiments. The virtual node 1208 includes a command forwarder 1304, a path resolver 1306, and a mapping manager 1308. The command forwarder 1304 includes a hash generator 1310. In addition, the virtual node 1208 maintains a directory block map 130R, a name resolution cache 1312, and a name resolution node registry 1314.

[0160] The command forwarder 1304 of the virtual node 1208 is configured to receive a command containing a file system operation for a physical block stored in a storage cluster managed by a hierarchical namespace service. When a client sends a command for a file system physical block, the client uses the name of a file or directory (e.g., a hierarchical namespace identifier of a file system object), which includes a path name to the file or directory. In one embodiment, when the virtual node 1208 is not a virtual node that manages the identified file or directory, the virtual node 1208 acts as a "query node" to forward the received command for processing. In this case, the command forwarder 1304 determines another virtual node (referred to as a "name resolution node") to process and identify another virtual node (referred to as a "storage node" in this article), which manages the file or directory and is therefore configured to execute the command. In one embodiment, the hash generator 1310 of the command forwarder 1304 can perform a hash function (such as a CRC64 algorithm or other suitable hash function) on the path name to generate a hash result. The hash result identifies the name resolution node. The command forwarder 1304 forwards the received command to the identified name resolution node.

[0161] The path resolver 1306 of the virtual node 1208 is configured to receive commands from the query node, resolve the directory name (or path) included in these commands into a flat namespace identifier, and use the flat namespace identifier to identify the virtual node of the storage node for the directory or file for which the command is directed. In particular, for a given command, the path resolver 1306 resolves the directory name or path included in the command into a DBID. Name resolution is performed efficiently and in a manner that is strictly consistent with all transactions. Note that the mapping between the path name and the DBID can change (such as when renaming a directory), and this may affect the name resolution cache for all children of the renamed directory (including the indirect children of the directory). In some cases, the name resolution cache 1312 of the virtual node 1208 can store the mapping of the path name to the DBID of the file or directory in the command. In this case, the path resolver 1306 can forward the command to the storage node of the DBID. Otherwise, the path resolver 1306 resolves the storage node by processing the path name.

[0162] In particular, the path resolver 1306 may process the path name in the command starting at the root and process path by path portion to determine the relevant storage node for executing the command. In particular, for the path name / path1 / path2 / , the path resolver 1306 may communicate with the virtual node that manages the root partition to determine the DBID for the first directory in the path name (e.g., / path1). The virtual node that manages the root partition finds the first directory in its directory block map 130R and provides the DBID back to the path resolver 1306. The path resolver 1306 may then communicate with the virtual node that manages the directory block for the DBID to determine the DBID for the second directory in the path name (e.g., / path2). The virtual node that manages / path1 finds the second directory in its directory block map 130R and provides the DBID back to the path resolver 1306. This process may be repeated for any additional path name portions. Ultimately, the path resolver 1306 determines the DBID of the directory block containing the entity block targeted by the command and transmits the command to the storage node that manages the directory block.

[0163] Using the path resolver 1306, any virtual node can find and return the DBID for the path at a specific RT (read timestamp). In the process of finding the mapping, the path resolver virtual node 1208 registers with all DBID owners of the path in its corresponding name resolution node registry 1314 for a period of time (such as 1 hour) for notification. Before the time expires, if any part of the path name changes, the corresponding manager of the path name part notifies each registered virtual node of the changed path name, and the entry for the path name in the name resolution cache 1312 for the registered virtual node is invalidated.

[0164] Note that the registration at the storage node may have a predetermined expiration time (lease), such as 1 minute. If in the next time period (e.g., 45 seconds), the name resolution node receives a new request for the same entity block, the name resolution node resolves the name using its name resolution cache 1312 without sending a new registration message to the DBID owner. Thereafter, if the name resolution node receives a new request, the name resolution node will register with the DBID owner again.

[0165] The mapping manager 1308 of the virtual node 1208 is configured to process commands for storage nodes received from the name resolution node. For example, the mapping manager 1308 can apply the name and timestamp in the received command as a key to the directory block map 130R of the virtual node 1208 to determine the entity block for which the command is directed. The command can then be executed by the virtual node 1208.

[0166] Fig.14 A block diagram of a hierarchical namespace service according to some example embodiments is shown, wherein a virtual node forwards a command to a virtual node that manages a physical block identified in the command. Virtual nodes 1208 communicate with each other. When a first virtual node wants to communicate with a second virtual node, the first virtual node requires a first physical node that hosts the first virtual node to connect to a second physical node that hosts the second virtual node to send a message. Each virtual node is capable of receiving requests from a client.

[0167] In the illustrated example, the hierarchical namespace service includes virtual nodes 1208A, 1208B, 1208E, and 1208R. A directory block map 1310B and a name resolution node registry 1314B are shown for virtual node 1208B, a name resolution cache 1312C is shown for virtual node 1208E, and a directory block map 1310R is shown for virtual node 1208R. For ease of illustration, other physical nodes, virtual nodes, directory block maps, name resolution node registries, and name resolution caches are not shown. Fig.14 Note that the communications shown in the form of arrows are shown as occurring between virtual nodes, but in reality the managed physical nodes are communicating between virtual nodes.

[0168] In the illustrated example, the virtual node 1208 receives a command 1222 (e.g., create / path1 / file) for a first file system object, the command indicating a path (e.g., / path1 / file) or a directory name in a hierarchical namespace associated with the first file system object. The command 1222 may include a file system operation for a physical block that is stored in a storage cluster having a namespace mapping managed by the hierarchical namespace service 1200. As such, the command 1222 may have been forwarded to the hierarchical namespace service 1200.

[0169] Command 1222 refers to a target entity, a hierarchical path name to the entity, and an operation to be performed on the target entity, and may also include an associated timestamp indicating the time of receipt.

[0170] A hash function is performed on the path or directory name to generate a first node identifier of a name resolution node for a virtual node. In one embodiment, the hash generator 1310 of the command forwarder 1304 can perform a hash function (such as, CRC64 algorithm or other suitable hash function) on the path name or directory name of the command 1222 to generate a hash result. The hash result identifies the name resolution node. The motivation for doing this is to send all requests for resolving the same path name to the same virtual node. If the same virtual node resolves the path name to a DBID, fewer virtual nodes will register for notification. In addition, the method improves the efficiency of the name resolution cache because it increases the possibility of a cache hit.

[0171] The command is forwarded 1424 to a name resolution node (in this example, virtual node 1208E) to determine a storage node for processing the command. The command forwarder 1304 in virtual node 1208A is configured to forward the received command 1222 to the identified name resolution node. In this example, virtual node 1208A forwards 1424 the command 1222 to virtual node 1208E. Based on the hash result, virtual node 1208A identifies virtual node 1208E as a name resolution node.

[0172] In one embodiment, virtual node 1208A selects another virtual node in hierarchical namespace service 1200 to handle name resolution for command 1222. The virtual node may be selected in any manner, including by selecting the next virtual node in a sequence of virtual nodes (e.g., by a virtual node identifier), by randomly selecting the next virtual node, or selecting the next virtual node in another manner.

[0173] The command is forwarded to the next virtual node to determine the storage node. The command forwarder 1304 is configured to forward the received command 1222 to the next identified name resolution node as a forward command 1424. In this way, the name resolution request can be distributed across multiple virtual nodes. For example, this allows the load to be distributed across multiple virtual nodes and handles the situation when a busy virtual node manages a directory storing billions of files.

[0174] Note that in one embodiment, the command forwarder 1304 of the query node can be implemented as a client library. When the client library sends a command to the name resolution node, the response to the command can be returned directly to the query node from the storage node that executes the command. This can minimize the number of messages sent back and forth. The query node can be synchronized with the rest of the virtual node clocks of the storage cluster, or it can not be synchronized with it. In this way, the client library is configured to manage transactions, but does not execute commands.

[0175] In addition, a storage node corresponding to the path is determined. In one embodiment, the path resolver 1306 of the virtual node 1208E can determine the virtual node 1208R of the DBID corresponding to the path name in the management forwarding command 1424. In one embodiment of selecting the next virtual node in the sequence, this can be implemented by adding a predefined number (e.g., 1) to the identifier of the previously selected virtual node to obtain the identifier of the next selected virtual node. The method of always selecting the same next virtual node will improve the benefits of the name resolution cache by increasing the probability of a cache hit.

[0176] Virtual node 1208E may communicate 1436 with virtual node 1208R that manages the root directory via node resolution communication. Virtual node 1208R accesses its directory block map 1310R, which maps the path portion in the path name of command 1222 to a GUID and returns the GUID to virtual node 1208E. Path resolver 1306 at virtual node 1208E continues to process the path name portion by portion, communicating with the virtual node that manages each path portion to determine the corresponding DBID until the entire path name is traversed and the storage node is determined. In one embodiment, path resolver 1306 communicates with the physical node that manages virtual node 1208 to determine the virtual node that owns the root directory and each determined GUID. The physical node may access master directory block table 1100, which includes mappings of all path portions to GUIDs (including the root), and may therefore find each virtual node that path resolver 1306 needs to communicate with based on the GUID determined from the previous virtual node.

[0177] The command is forwarded 1432 to the determined storage node (virtual node 1208B), which has a directory block map containing entries that map the first file system object to an entity block identifier in a flat namespace. The path resolver 1306 is configured to forward the command to the storage node. Virtual node 1208B is identified as a storage node by virtual node 1208E due to the DBID of the command path name.

[0178] The entity block identifier and timestamp are registered in a cache associated with the virtual node. In one embodiment, the path resolver 1306 is configured to store the entity block identifier determined for the command in an entry in the name resolution cache 1312 together with the timestamp and the path name of the command. In this way, when future commands including the path name are received, the path resolver 1306 can determine the storage node by only referring to the entry in the name resolution cache 1312. In one embodiment, the path resolver 1306 can receive the entity block identifier from the storage node in the notification message, and in response, can store the entry in the name resolution cache 1312.

[0179] It is noted that after a predetermined amount of time (such as one minute, one hour, or other time period) has passed since the timestamp value, the entry in the name resolution cache 1312 may time out and thus become invalid. In addition, the path resolver 1306 may receive an invalid cache entry notification for an entry in the name resolution cache 1312 from a storage node, and in response, the path resolver 1306 may indicate the entry as invalid. Similarly, when any of the other virtual nodes determine that the path portion they resolved has become invalid (e.g., due to a directory name change, etc.), the path resolver 1306 may receive an invalid cache entry notification from the other virtual nodes with which the path resolver 1306 communicates to resolve the DBID for the path portion of the path name in the command.

[0180] In one embodiment, if the transaction read sequence (RS) number Transaction.RS is equal to 0, the name resolution node may determine Transaction.RS as the current time minus a predetermined increment value. In addition, a search is performed in the local name resolution cache to search for a DBID for the path name, such as, if the file path is / path1 / path2 / / name, then / path1 / path2 / and Transaction.RS. If / path1 / path2 / is not in the local name resolution cache, the name resolution cache is checked for the presence of a leading path portion of the path name (e.g., / path1, etc.), which can be used to determine a DBID for at least the leading portion of the path name. In addition, when the mapping between the path and the DBID is not in the local name resolution cache, a path resolution request is sent to a virtual node that manages the first portion of the path (e.g., a root virtual node or a virtual node that manages a DBID determined for the leading portion of the path). The path resolution node returns a DBID for Transaction.RS, which is valid for RS plus a predetermined amount of time. When the owner virtual node of the DBID changes the mapping, the owner virtual node notifies all nodes that have cached the latest value in its name resolution path and the cached value has not expired. In addition, a command is sent to the determined storage node.

[0181] A command regarding a first file system object is received 1432 by the virtual node 1208B from the name resolution node 1208E, the command indicating a name and a directory block identifier associated with the first file system object.

[0182] In accordance with the command, an entry corresponding to the name and the directory block identifier is identified in a directory block map associated with the virtual node. In some example embodiments, the mapping manager 1308 may maintain a directory block map 130R (which may be a table or may have other forms) having entries (e.g., rows) corresponding to the directory blocks. For example, the directory block map 130R may include rows configured similarly to the rows of the main directory block table 700. The name entry and directory block identifier (e.g., a GUID for the last path portion determined by the name resolution node) in the forwarded command 1432 may be used by the mapping manager 1308 as a row key of the directory block map 130R to determine the row having the entity block identifier for which the command is directed.

[0183] Depending on the type of command, the mapping manager 1308 may interact with the determined entity block entry in any manner. For example, for a get-attributes command, the mapping manager 1308 may determine one or more attributes in the entry requested by the command. For commands such as one of set-attributes, create-file, delete-file, create-directory, and delete-directory, the mapping manager 1308 may create a new entry in the directory block map 130R for the new version of the entity block, the new entry having attributes copied from the previous entry for the entity block identifier, but with a new commit time, and some attributes may have been modified (e.g., modifying any attributes specified in a set-attributes command, setting a delete flag for a delete-file command, etc.).

[0184] It should be noted that some commands such as move file and move directory can be executed as multiple commands. For example, the move file command can be implemented by a create-file command and a delete-file command, wherein the create-file command creates a new entry for the physical block identifier in the directory block map 130R (same or different virtual node, depending on the move destination) for the new directory block, and the delete-file command creates a new physical block identifier in the directory block map 130R (for the same virtual node) with a delete flag set. In this case, the query node can issue more than two commands to name the resolution nodes in sequence to cause one or more storage nodes to execute the commands to execute the overall command.

[0185] In addition, the name resolution node and the path name indicated in the command are registered in a registry associated with the virtual node. As described above, in one embodiment, the mapping manager 1308 is configured to: create an entry in the name resolution node registry 1314, which associates the name resolution node (which forwards the command to the storage node) with the path name in the forwarded command. The entry can also include a timestamp of the creation of the entry. The entry in the name resolution node registry 1314 enables the name resolution node to be notified in the event of a path name change, and therefore the name resolution node should invalidate all entries associated with the path name (full path name or any portion thereof) in its name resolution cache 1312C.

[0186] Regarding the command, the query node in the virtual node is responded. In one embodiment, after the storage node 1208B executes the command in the forwarded command 1432, the storage node provides an indication 1430 of the command completion to the query node 1208A. By responding directly to the query node, rather than responding to the query node through the name resolution node, the number of communication links is reduced. Fig.14 In the example, virtual node 1208B sends a command completion indication 1430 to virtual node 1208A.

[0187] It is noted that the storage node may determine the query node to which the command completion indication 1430 is to be sent in a variety of ways. In one embodiment, the command forwarder 1304 inserts an identifier for the query node when forwarding the command to the name resolution node, and the name resolution node forwards the query node identifier to the storage node. In another embodiment, the command forwarder 1304 publishes the identifier for the query node in a data structure associated with the command (such as a memory location, file, cache, etc.). The data structure may be accessed by the storage node to determine the query node associated with the command. In other embodiments, the storage node may determine the query node in other ways.

[0188] The path name may change due to operations such as moving a directory, renaming a directory, etc., which change any path part in the path name. Such a path name change may adversely affect the entries in the name resolution cache for the path name, rendering them invalid. Thus, the storage node monitors commands containing operations that change the path name, which may cause the mapping manager 1308 to modify the entry about the path name in the directory block map 130R of its virtual node, and notify the name resolution node registered for the changed path name (in the name resolution node registry 1314).

[0189] Therefore, embodiments provide many advantages, including enabling file system operations to be performed on entity blocks by manipulating data in directory block mappings, rather than having to operate on the stored file system objects themselves. For example, as described above, file system objects in a hierarchical namespace can be identified at least by path and name. The path can be changed for file system objects in a hierarchical namespace. In an embodiment, the path change causes a modification of an entry in at least one directory block mapping, wherein the directory block mapping is used for an entity block identifier of an entity block corresponding to a file system object. The change in the directory block mapping illustrates the path change, and therefore, the change does not cause the file or folder corresponding to the entity block to be moved in the storage of the storage system. Compared to actually moving files and / or folders in storage, changing entries in the directory block mapping is a much less expensive operation. This is especially true when the entity block is a folder containing many files. If the folder is moved, this will result in many move file operations for the contained files (each move means a delete file and create file operation). In contrast, embodiments avoid moving stored file system objects by changing data block mappings.

[0190] Fig.15 The states of entries in the memory table according to some example embodiments are illustrated. Each row in the memory table corresponds to an EBID, and the row includes a list of transactions for the EBID 1108 with corresponding states and CTs. Each transaction has a unique transaction identifier.

[0191] In some example embodiments, the state of a transaction may be one of "active", "committed", or "terminated". The active state means that the transaction is still in progress and has not yet been completed. The committed state means that the transaction has succeeded. The terminated state means that the transaction cannot reach the committed state due to some reasons (such as conflict with another transaction or timeout). The client can set a maximum execution time for the transaction. If the transaction cannot be completed within that time, the transaction is automatically terminated.

[0192] The client can choose the number of retries before aborting the transaction. The storage service will retry the transaction multiple times at different points in time until it succeeds or it exhausts the number of retries.

[0193] In some example embodiments, there is a memory table (served by a partitioning layer) distributed across virtual nodes. In addition, the memory table has its own partitioning and load balancing.

[0194] Fig.16 A snapshot read operation according to some example embodiments is illustrated. The memory table 1600 holds information about the contents of the directory, and the structure of the memory table 1600 allows for efficient execution of read and write transactions. The memory table 1600 is logically organized as one row per entity, and each row contains a linked list of entries organized by the current timestamp CS, with newer entries at the head of the list.

[0195] exist Fig.16 In the example illustrated in , memory table 1600 includes data for three entities X, A, and Y. For entity X, there is a head entry 1602 and an older entry 1612. For entity A, there are three linked entries, starting with the newer entry 1604, followed by entry 1608, and then entry 1610. Entry 1604 has a committed status and a timestamp of 100. Entry 1608 has a committed status and a timestamp of 50, and entry 1610 has a committed status and a timestamp of 10. Entity Y has a head entry 1606, which is followed by two other entries.

[0196] To perform a snapshot read operation for a given time, the memory table is searched to identify the entry submitted for the given time (e.g., the latest entry). The search starts from the head entry and proceeds until an entry with a timestamp less than the time in the snapshot read request is found.

[0197] exist Fig.16 In the example illustrated in , a snapshot read is requested for time 70. Since the head entry has a timestamp of 100, this means that the head entry is after the request time, and therefore, the head entry is invalid for the snapshot read. The next entry 1608 has a timestamp of 50, which is less than the requested timestamp of 70; therefore, entry 1608 is selected as the entry with the correct information to satisfy the read request for time 70.

[0198] In some example embodiments, older entries may be discarded from the tail of the list based on a maximum number of entries or some time threshold.

[0199] Note that a transaction is atomic if it is either fully completed or not completed at all. All transactions in a partition are atomic, regardless of whether the transaction affects a single row or multiple rows. In some example embodiments, there are three possible results from a transaction request: success, failure, or timeout. Success means that the transaction succeeded, failure means that the transaction failed, and timeout means that the transaction may have failed.

[0200] Partitions are always consistent, and consistency ensures that any transaction executed on a partition will move the partition from one consistent state to another. For example, after a write transaction completes, any subsequent read transaction will observe the completed write transaction.

[0201] Transactions in a partition are isolated because each transaction appears to complete independently of all other concurrent transactions. Isolation ensures that concurrent execution of transactions produces the partition state that would be obtained if the transactions were executed sequentially. In addition, reads have transaction timestamps and cannot see updates committed after their timestamps or uncommitted updates. Writes can see their own updates and use locking to ensure that only one write is executed at a time. In some example embodiments, using more advanced locking techniques, special logic is applied to allow certain types of writes to occur in parallel. Multiple writes can be executed in parallel for multiple entities, but for a particular entity, one write is executed at a time.

[0202] Snapshot isolation allows read transactions to execute without waiting for write transactions. Each update introduces an updated version of the data, and each version has a defined start and end timestamp (which can be infinity for the latest entry).

[0203] Fig.17 A read-only local phase operation according to some example embodiments is illustrated. A transaction coordinator is a virtual node that coordinates distributed transactions that require operations of multiple virtual nodes. In some example embodiments, the first node to receive a transaction request is the transaction coordinator.

[0204] The transaction coordinator performs multiple operations, such as starting a transaction by inserting a row in a memory table, executing commands one by one in the correct nodes, sending prepare messages to all nodes, writing the result of the transaction to the memory table, and notifying all nodes of the result of the transaction.

[0205] In the illustrative example, the transaction coordinator has received a command 1702 that includes multiple transactions. The command 1702 includes the following transactions:

[0206]

[0207] Thus, command 1702 includes two read transactions and two write transactions to be executed together. A first read transaction is sent 1706 to virtual node 1704A, which is the node containing A. The first read transaction does not include a read sequence RS (eg, a timestamp).

[0208] Virtual node 1704A performs snapshot read, as shown in the above reference Fig.16 As described, it includes operations 1714, 1716, and 1718. At operation 1714, since the first read request does not include RS, the RS is assigned the value of the local clock.

[0209] At operation 1716, a snapshot read operation is performed using RS as the time of the snapshot read. At operation 1718, the read value of A is returned together with the assigned RS. Virtual node 1704A returns the values ​​of A and RS of 1708 to the transaction coordinator.

[0210] The second read request is sent 1710 to virtual node 1704B, which is the node containing B. The second read request includes a timestamp RS, and virtual node 1704B performs a snapshot read using the given RS. Virtual node 1704B returns 1712 the value of B to the transaction coordinator.

[0211] In some example embodiments, the virtual nodes have loosely synchronized clocks, which means that the clocks in the virtual nodes can have a certain degree of variation. For example, the clocks in the virtual nodes may vary within a range of 1 / 10 second, 1 second, or a few seconds. The coordination of operations between virtual nodes does not require the clocks in the virtual nodes to be synchronized. Note that any type of method can be used to implement the clock function. For example, the clock can be a logical clock, a physical clock, a strictly synchronized clock, a loosely synchronized clock, etc., and the embodiments proposed herein can be implemented with any type of clock.

[0212] By enabling loosely synchronized clocks, the transaction coordinator does not have to communicate with all virtual nodes involved to ensure clock synchronization, and the clock coordinator is not required to supervise all virtual nodes to ensure their clocks are synchronized.

[0213] In some example embodiments, the time synchronization beacon sends periodic time updates to the virtual nodes via UDP packets containing the current time. Periodic time updates can be sent periodically, where the period can range from 50ms to 10 seconds or longer. In this way, the clocks remain loosely synchronized, but it is not required that the clocks in all virtual nodes are exactly the same.

[0214] In some example embodiments, when a first virtual node sends a message (e.g., a request) to a second virtual node, the message includes the time of the first virtual node. If the time in the second virtual node is after the time received from the first virtual node, the second virtual node waits until the local time in the second virtual node reaches the time received from the first virtual node. After waiting, the second virtual node processes the request. On the other hand, if the time received from the first virtual node is after the time at the second virtual node, the second virtual node does not wait and continues to process the request.

[0215] In some example embodiments, the transaction coordinator selects the commit time of the transaction by selecting the maximum time of all nodes participating in the transaction.

[0216] In some example embodiments, a delta value may be used, and the allocated RS is set to the local time minus the delta value in operation 1714. The delta value may have a value of 0, and then the allocated RS is equal to the local clock time.

[0217] Fig.18 The operations for preparing a row during phase 1 according to some example embodiments are illustrated. To prepare for a write operation, the transaction coordinator sends 1806 a prepare-C request to virtual node 1704D, the prepare-C request including the RS obtained previously. Then, virtual node 1704 performs operations 1814 to 1819.

[0218] At operation 1814, as long as the current value of the local clock is greater than RS, virtual node 1704D assigns the current value of the local clock to optimistic concurrent timestamp 1 (OCS1). If not, virtual node 1704D waits until the local clock reaches RS and then assigns the current value of the local clock to OCS1.

[0219] At operation 1815, virtual node 1704D checks for any conflicts with other ongoing transactions. Assuming there is no conflict, the method continues to operation 1816, otherwise the transaction is terminated. If the transaction is terminated, the transaction coordinator sends a termination notification to the virtual nodes involved in the process.

[0220] At operation 1816, a new entry is inserted into the head of the row for entity C in the memory table 1600. The new entry is inserted with a state of "active" and a timestamp of OCS1.

[0221] At operation 1817, the server assigns the value of the current clock to OCS2. In addition, at operation 1818, the row for entity C from the memory table is written to the write-ahead log. At operation 1819, virtual node 1704D returns 1808 a status that the new entry has been initialized, and also returns the value of OCS2.

[0222] Similarly, sequentially or in parallel, the transaction coordinator sends 1810 a request to prepare entity D to virtual node 1704E, where the request includes the value of RS. Virtual node 1704E performs the same operations as virtual node 1704D, but for entity D. Virtual node 1704E then returns 1812 a status that a new entry for D has been initialized, and returns the OCS3 value obtained from reading the local clock. In some example embodiments, the prepare-D operation is performed after the prepare-C operation, and OCS1 is sent to virtual node 1704E instead of RS.

[0223] Fig.19 Illustrated is a diagram of a Figure 8 1816. A new entry 1902 is added to the memory table. The virtual node traverses the different rows until it finds the row for entity C, with the head at entry 1908. A new entry 1902 is added pointing to entry 1908, and the new entry 1902 becomes the head of the list of rows corresponding to entity C. As described above, the new entry 1902 is created with an "active" state and a timestamp equal to OCS1.

[0224] Fig. 20 Phase 2 for committing a transaction record according to some example embodiments is illustrated. To perform the commit transaction record, the transaction coordinator sends a request to commit a new row to virtual node 1704C at 2006, which is the virtual node that manages the transaction record partition. The request includes values ​​for OCS2 and OCS3.

[0225] Virtual node 1704C performs operations 2014-2017. At operation 2014, for command 1702, virtual node 1704C assigns CS as the maximum value of OCS2 and OCS3, and assigns the value of the current clock at virtual node 1704C, which means that CS is selected as the latest time of the three times. If the current value of the clock is greater than any one of OCS2 and OCS3, virtual node 1704C waits until the value of the current clock is equal to the maximum value of OCS2 and OCS3.

[0226] This means that CS is chosen as the maximum of the different times obtained at the virtual nodes involved in the transaction. By choosing the maximum of the different clock values, the virtual nodes are able to operate independently without having to rely on precise clock synchronization across all virtual nodes. This allows the storage system to scale easily due to the independent clock operation of the virtual nodes. This can reduce bottlenecks during execution that could slow down the system.

[0227] If the clocks happen to be close to each other, execution will be fast. If there is any slight clock discrepancy, the transaction may still succeed, but latency may increase and the system may slow down slightly (still fast, but slightly slower than the ideal case of perfect clock synchronization). However, even though the clocks may lack perfect synchronization, there will never be a false execution.

[0228] At operation 2015, virtual node 1704C writes the commit record to the update log, and at operation 2016, the role in the transaction record partition is updated to the committed state and the timestamp of the calculated CS. At operation 2017, virtual node 1704C returns 2008 a completion state with an assigned CS.

[0229] Fig.21 Commit notification according to some example embodiments is illustrated.After the transaction record has been committed, virtual nodes 1704D and 1704E are notified of the commit and are requested to commit the corresponding writes at time CS.

[0230] At operation 2114, virtual node 1704D writes a commit record to update the write-ahead log. At operation 2116, virtual node 1704D changes the state of entry 1090 for entity C in the memory table to "committed" and changes the timestamp to the calculated CS. Similarly, virtual node 1704E changes the entry for entity D to "committed" with a CS.

[0231] Fig. 22 File system operations for the blob FS API according to some example embodiments are shown. The blob FS API leverages mechanisms already implemented for blob SS, such as transaction management, indexing, and persistent storage, which provide strong consistency and can automatically scale horizontally without client or operator interaction.

[0232] In addition, the Blob FS API supports operations such as "Copy File" and "Join" (e.g., "Concat"), which are operations associated with metadata and indexes and do not require copying data as in traditional cloud object storage, which is enabled by the ability of Blob SS to already support efficient replication of Blob operations within the same storage account.

[0233] Table 2200 shows some file system operations in the file system operations of the blob FS API. These operations include creating a file system, deleting a file system, getting file system properties, setting file system properties, and listing paths.

[0234] The format of the create file system operation is the command "Put" and the name of the file system. In addition, the version of the API is provided. The format of the delete file system operation is the same, except the command is "Delete".

[0235] The following is an example of the syntax for the create file system operation:

[0236]

[0237] Fig.23 Table 2300 shows some file and directory operations for the blob FS API according to some example embodiments, including: create file or directory, delete file or directory, rename file or directory, append data, flush data, read data, get lease (including duration) for a path, renew lease, and release lease.

[0238] Thus, the blob FS API provides commands for adding, deleting, and renaming files or directories. In addition, the properties of files and directories can be set and read.

[0239] For example, the delete path command has the following request format:

[0240] DELETE{filesystem} / {path}? resource={ile|directory}&

[0241] recursive={true|false}&continuation{token}

[0242] x-ms-lease-id: {id}

[0243] If-Match: {eTag}

[0244] If-None-Match: {eTag}

[0245] If-Modified-Since: {date}

[0246] If-Unmodified-Since: {date}

[0247] AdditionalHeaders

[0248] Resource is a required value that indicates the type: file or directory. recursive is an optional parameter that is required and effective if the directory is being researched. If recursive is true, all paths under the directory will be deleted. If recursive is false and the directory is empty, the directory will be deleted, otherwise an error is returned.

[0249] continuation is an optional parameter. When deleting a directory, there is a limit on the number of paths deleted per call. If the number of paths to be deleted exceeds that limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, it must be specified in subsequent calls to the delete operation to continue deleting the directory. Note that for f or rename directory and delete directory operations, the continuation parameter is returned in the response to FN; based on the number of files in the directory, FN may need to continue. For HN, it is always a single atomic operation and there will be no continuation parameter.

[0250] The format of the response to the delete path command is:

[0251] HTTP / 1.1 200OK

[0252] Content-Length: 0

[0253] x-ms-continuation:token

[0254] AdditionalHeaders

[0255] \r\n\r\n

[0256] The rename path command is similar to the delete path command in its implementation, for example, the rename path command deletes a directory if it is empty. The rename path command will overwrite the target if it exists, but will fail if the resource type (file or directory) of the existing blob does not match the source blob. In addition, the rename path command also supports conditional constraints and lease constraints on the source.

[0257] The rename path command has the following request format:

[0258] PUT{filesystem} / {path}? resource={file|directory}&continuation={token}

[0259] x-ms-lease-id: {id}

[0260] If-Match: {eTag}

[0261] If-None-Match: {eTag}

[0262] If-Modified-Since: {date}

[0263] If-Unmodified-Since: {date}

[0264] x-ms-rename-source:{sourceUri}

[0265] x-ms-source-lease-id: {id}

[0266] x-ms-source-lease-action:{acquire}

[0267] x-ms-source-if-match:{eTag}

[0268] x-ms-source-if-none-match: {eTag}

[0269] x-ms-source-if-modified-since: {date}

[0270] x-ms-source-if-unmodified-since: {date}

[0271] AdditionalHeaders

[0272] Here, resource is a required value, which can be either a file or a directory to indicate the resource type. continuation is an optional parameter. When renaming a directory, the number of paths renamed per call is limited. If the number of paths to be renamed exceeds the limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, the continuation token must be specified in subsequent calls to the rename operation to continue renaming the directory.

[0273] The format of the response to the rename-path command is:

[0274] HTTP / 1.1 201 Created

[0275] Content-Length: 0

[0276] x-ms-continuation:token

[0277] AdditionalHeaders

[0278] \r\n\r\n

[0279] The list paths command uses continuation when it cannot be completed in a single call. The list paths command has the following request format:

[0280] GET / {filesystem}? resource=filesystem&recursive={true|false}&

[0281] directory={path}&continuation={token}

[0282] Headers

[0283] Here, resource is a required parameter indicating the file system. recursive is a required parameter and if the value is "true", all paths are listed; if the value is "false", only paths at the root of the file system are listed. If a directory is specified, the returned list includes only the paths within the directory and the directory itself. directory is an optional parameter used to filter the results for paths within the specified directory. If the directory does not exist, an error occurs.

[0284] Additionally, continuation is an optional parameter. If the number of paths to be returned exceeds the limit, a continuation token is returned in the response header x-ms-continuation. When a continuation token is returned in the response, the continuation token must be specified in subsequent calls to the list operation to continue listing paths.

[0285] The format of the response to the list paths command is:

[0286]

[0287] Note that the API details provided herein are examples, and the API may include additional commands and parameters. In addition, the API is versioned, and behavior changes are accomplished with new API versions, so new versions are backward compatible. This allows the storage system to be both compatible and scalable.

[0288] Fig.24 A sample blob storage command is shown according to some example embodiments. The sample command 2402 is as follows:

[0289] Create Account janeaccount

[0290] Create Container janeaccount;mycontainer

[0291] PUT http: / / janeaccount.blob.core.windows.net / mycontainer / files / myblob.txt

[0292] Partition Key=janeaccount;mycontainer;files / myblob.txt

[0293] Put Block(10MB, B1) http: / / janeaccount.blob.core.windows.net /

[0294] mycontainer / files / myblob.txt

[0295] Put Block(10MB, B2) http: / / janeaccount.blob.core.windows.net /

[0296] mycontainer / files / myblob.txt

[0297] Put Block List(B1, B2) http: / / janeaccount.blob.core.windows.net /

[0298] mycontainer / files / myblob.txt

[0299] Get Blob(2MB-4MB) http: / / janeaccount.blob.core.w indows.net /

[0300] mycontainer / files / myblob.txt

[0301] List Blobs delimeter " / ", prefix "files / "

[0302] The first two commands create the account janeaccount and the container mycontainer within janeaccount. The PUT operation indicates the URI for the blob files / myblob.txt, where the URI is:

[0303] http: / / janeaccount.tuankuai.core.windows.net / mycontainer / files / mytuankuai.txt

[0304] In addition, the partition key (a combination of the account name, container name, and blob name) is used to access the blob and uniquely identify the blob. In this example, the partition key is janeaccount;mycontainer;files / myblob.txt. The partition key is part of the command shown and has been omitted to simplify the description.

[0305] Note that for FN, the name of the blob includes the " / " character, but this character does not imply the presence of a directory structure in the blob store. The user uses this delimiter as an indication of a simulated directory. Thus, a user can request a list of blobs beginning with files / , and the system will return blobs based on the name of the blob rather than based on a search in a given directory.

[0306] The list blob s operation returns blobs with names beginning with files / . As mentioned above, this may be a simplified way to simulate a directory, but the directory feature is not implemented in blob SS. For example, a user creates a million image files and simulates a directory by making the million image files have the same name beginning (e.g., "securityvideo / "). If the user wants to rename the simulated directory using FN, the system must rename a million blobs.

[0307] Fig.25 A sample blob FS command according to some example embodiments is shown. Example commands include the following:

[0308] Create Account janeaccount

[0309] Create File System janeaccount;myfs

[0310] Create Directory janeaccount;myfs;files

[0311] Create File http: / / janeaccount.dfs.core.windows.net / myfs / files / myblob.txt

[0312] Append File(10MB) http: / / janeaccount.dfs.core.windows.net / myfs / files / myblob.xt

[0313] Append File(10MB) http: / / janeaccount.dfs.core.windows.net / myfs / files / myblob.txt

[0314] Flush File(20MB) http: / / janeaccount.dfs.core.windows.net / myfs / files / myblob.txt

[0315] The blob FS command includes the creation of the account janeaccount and the file system janeaccount;myfs. In addition, the blob FS command includes the creation of the directory janeaccount;myfs;files in the above directory, account and file system and the creation of the file myblob.txt. The URI for this file is:

[0316] http: / / janeaccount.dfs.core.windows.net / myfs / files / myclump.txt.

[0317] The blob FS API provides a command for creating directories, unlike the blob SS API where the create directory command is not available. In some example embodiments, the DNS domain for blob FS is different from the DNS domain for blob SS because the two interfaces utilize different access points.

[0318] To add data to a file, two append commands are used to append the data. In addition, a flush command is implemented to flush the data to disk. Note that appending is the actual data I / O, while flushing is an index update to make the append operation readable by the read file operation.

[0319] Fig.26 26 is a flowchart of a method 2600 for performing distributed transactions in a cloud storage system with a hierarchical namespace according to some example embodiments. Although the various operations in the flowchart are presented and described sequentially, those skilled in the art will appreciate that some or all of the operations may be performed in a different order, combined or omitted, or performed in parallel.

[0320] Operation 2602 is for receiving a request including a plurality of operations to be performed by a cloud storage system from a user device, the request being for atomic operations of the plurality of operations.

[0321] From operation 2602, the method proceeds to operation 2604 for identifying a plurality of nodes for performing a plurality of operations. Each node has a corresponding clock and controls the execution state of at least one operation. In addition, each node has at least a portion of a memory table for controlling updates to entities in the cloud storage system. The entity is one of a file or a directory.

[0322] From operation 2604, the method proceeds to operation 2606 for processing a corresponding operation by each of the plurality of nodes. Processing an operation includes performing method operations 2612-2617.

[0323] At operation 2612, a check is made to determine if the node is the first node in the process. If the node is the first node, the method proceeds to operation 2613, and if the node is not the first node, the method proceeds to operation 2614. Note that if the node receiving the request

[0251] is not the first node in the process, the first node acts as a coordinator to coordinate the execution of the various operations.

[0324] At operation 2613, a commit timestamp (CS) is set to the value of the clock in the node. For a read transaction, if the user has not provided a timestamp for the read, the transaction manager sets the read timestamp. For a write transaction, the transaction manager selects the maximum value of the commit CS returned from multiple nodes. At operation 2614, a check is made to determine if the value of the clock in the node is less than the current value of CS, and if the check is positive, the method proceeds to operation 2615, otherwise to operation 2616.

[0325] Operation 2615 is used to wait for the value of the clock in the node to reach the current value of CS. In addition, operation 2616 is used to update the current value of CS based on the value of the clock in the node.

[0326] From operation 2616, the method proceeds to operation 2617 for updating the memory table based on the current value of CS.

[0327] At operation 2608, the atomic execution of the request is submitted in the cloud storage system using the final value of CS. From operation 2608, the method proceeds to operation 2610, where the status of the request is returned to the user device.

[0328] In one example, the memory table includes a linked list of entries for each entity, each entry corresponding to a write operation and having a status and a CS.

[0329] In one example, the state is one of active, committed, or aborted, where active indicates that the transaction is still in progress and committed indicates that the transaction was successful.

[0330] In one example, each clock is one of a loosely synchronized clock, a strictly synchronized clock, a logical clock, or a physical clock.

[0331] In one example, a first operation from the plurality of operations is a read operation, wherein processing the read operation further includes performing a snapshot read based on a current value of CS, and a second operation from the plurality of operations is a write operation, wherein processing the write operation further includes adding an entry in a memory table for an entity associated with the write operation, the entry including the current value of CS.

[0332] In one example, atomic execution of the commit request also includes changing the state of an entry created in a memory table of the node during processing from active to committed with the final value of CS.

[0333] In one example, the node is a virtual node that executes on multiple physical nodes in the storage layer of the cloud storage system.

[0334] In one example, the atomic execution includes executing all of the plurality of operations such that all operations succeed, or no operations are executed and the atomic execution terminates.

[0335] In one example, a cloud storage system includes a hierarchical namespace service that maps entities to blobs in a storage tier.

[0336] In one example, atomic execution of the commit request also includes notifying the plurality of nodes of a final value of the CS.

[0337] Fig. 27 2700 is a flowchart of a method 2700 for providing multi-protocol access to a cloud storage system according to some example embodiments. Although the various operations in the flowchart are presented and described sequentially, those skilled in the art will appreciate that some or all of the operations may be performed in a different order, combined or omitted, or performed in parallel.

[0338] Operation 2702 is for providing a first API through a cloud storage system that stores data in blobs organized within containers.

[0339] From operation 2702, the method proceeds to operation 2704 for providing a second API by the cloud storage system, the first API and the second API providing operations for a flat namespace and a hierarchical namespace for accessing data. The flat namespace is used to access data based on accounts, containers, and blob identifiers, and the hierarchical namespace is used to access data based on accounts, directories, and files. The directory is configured to include files and other directories, wherein the first API utilizes the flat namespace by default, and the second API utilizes the hierarchical namespace by default.

[0340] From operation 2704, the method proceeds to operation 2708, where a command from a user device is received by the cloud storage system.

[0341] At operation 2710, a check is made to determine whether the command is received via a flat namespace or a hierarchical namespace. If the command is received via a flat namespace, the method proceeds to operation 2712, and if the command is received via a hierarchical namespace, the method proceeds to operation 2714.

[0342] At operation 2716, the command is executed using the container and blob identifiers included in the command. Additionally, at operation 2714, a hierarchical namespace service is accessed to convert the directories and files included in the command into one or more blob identifiers.

[0343] From operation 2714, the method proceeds to operation 2716 to execute the command using the one or more blob identifiers resulting from the conversion. At operation 2718, a response is sent to the user device after executing the command.

[0344] In one example, the hierarchical namespace service assigns a unique identifier to each file and each directory that is distinct from the name of the file or directory.

[0345] In one example, the hierarchical namespace service includes a primary directory block for identifying files or directories contained in each directory.

[0346] In one example, the second API includes the following commands: create file system, delete file system, list path of file system, create directory, rename directory, delete directory, append file, flush file, and read file.

[0347] In one example, renaming a directory does not require renaming any blobs.

[0348] In one example, data is accessed via a first API using an address having a user's account name, a container identifier, and a blob identifier.

[0349] In one example, data is accessed via the second API using an address having an account name of a user, a file system identifier, a directory identifier, and a file identifier.

[0350] In one example, files are stored in blobs.

[0351] In one example, the hierarchical namespace service assigns an empty blob to hold information for a directory.

[0352] In one example, the cloud storage system supports at least three blob types: block blob, append blob, and page blob.

[0353] Fig.28 A system 2800 for providing file system functionality on a cloud storage system based on blob storage according to some example embodiments. The system 2800 includes a plurality of storage clusters 208 within the cloud storage system. Each storage cluster includes: a plurality of storage nodes 2802, a hierarchical namespace service 104, and a front-end layer 210. Each storage node 2802 includes a partitioning layer that stores data in blobs 110 within containers 2804, and includes a streaming layer 214 for streaming blob data.

[0354] The hierarchical namespace service 104 is used to implement a file system function for the data stored in the blob 110. The file system function includes a directory function and a file function. Each directory is configured to include files and other directories requested via the file system function, and each file is stored in a blob. The front-end layer 210 is used to process user requests for accessing and managing data stored on the blob 110. The front-end layer 210 accesses the partition layer 212 to access the blob 110 based on the address of the blob 110. The front-end layer 210 accesses the hierarchical namespace service 104 to identify the blob 110.

[0355] In one example, the partitioning layer is configured to identify a partition key for flat namespace operations, the partition key comprising an account name, a container name, and a blob name.

[0356] In one example, the partitioning layer is configured to identify a partition key for hierarchical namespace operations, the partition key comprising an account name, a file system name, a directory name, and a file name.

[0357] In one example, a hierarchical namespace service includes one or more physical nodes having multiple virtual nodes for performing namespace mapping.

[0358] In one example, a hierarchical namespace service includes a plurality of directory blocks, wherein a globally unique identifier is assigned to a directory in response to directory creation, and wherein in response to renaming the directory, the globally unique identifier is not changed.

[0359] In one example, the hierarchical namespace service includes an entity block for each entity in the file system, the entity being one of a directory or a file, wherein the entity block for the entity can have multiple simultaneous versions in the cloud storage system.

[0360] In one example, the hierarchical namespace service includes a master directory table that identifies files and directories within each directory, wherein movement of directories or files is performed without moving or copying blobs.

[0361] In one example, a hierarchical namespace service is configured to map a file system path in a hierarchical namespace to a blob identifier in a flat namespace.

[0362] In one example, cloud storage also includes a location service for managing account namespace metadata across multiple storage clusters.

[0363] In one example, the hierarchical namespace service supports file system operations including: creating a file system, deleting a file system, listing the path of a file system, creating a directory, renaming a directory, deleting a directory, appending a file, flushing a file, and reading a file.

[0364] Fig.29 2900, on which or through which one or more example process embodiments described herein may be implemented or controlled. In alternative embodiments, machine 2900 may operate as a standalone device, or may be connected (e.g., networked) to other machines. In a network deployment, machine 2900 may operate in the capacity of a server machine, a client machine, or both in a server-client network environment. In one example, machine 2900 may act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. In addition, although only a single machine 2900 is illustrated, the term "machine" should also be understood to include any collection of machines that individually or collectively execute a set (or multiple sets) of instructions to perform any one or more methods discussed herein, such as via cloud computing, software as a service (SaaS), or other computer cluster configurations.

[0365] As described herein, examples may include logic, multiple components or mechanisms or be operated by them. A circuit device is a collection of circuits implemented in a tangible entity including hardware (e.g., simple circuits, gates, logic, etc.). Circuit device members may become flexible over time and with potential hardware variability. Circuit devices include members that can perform specified operations individually or in combination when operating. In one example, the hardware of the circuit device can be designed to implement specific operations (e.g., hardwired) invariably. In one example, the hardware of the circuit device may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.), which include physically modified (e.g., magnetically, electrically, by movable placement of constant mass particles, etc.) computer-readable media to encode instructions for specific operations. When connecting physical components, the basic electrical properties of the hardware components are changed (e.g., from insulators to conductors, and vice versa). Instructions enable embedded hardware (e.g., execution units or loading mechanisms) to create members of the circuit device in hardware via variable connections to implement parts of specific operations when operating. Therefore, when the device operates, the computer-readable medium is communicatively coupled to other components of the circuit device. In one example, any physical component can be used in more than one component of more than one circuit device. For example, in operation, an execution unit can be used in a first circuit of a first circuit device at one point in time, and can be reused by a second circuit in the first circuit device or a third circuit in the second circuit device at a different time.

[0366] The machine (e.g., computer system) 2900 may include a hardware processor 2902 (e.g., a central processing unit (CPU, FPGA), a hardware processor core, or any combination thereof), a graphics processing unit (GPU) 2903, a main memory 2904 (e.g., RAM, NVRAM), and a static memory 2906, some or all of which may communicate with each other via an interconnect (e.g., a bus) 2908. The machine 2900 may also include a display device 2910, an alphanumeric input device 2912 (e.g., a keyboard), and a user interface (UI) navigation device 2914 (e.g., a mouse). In one example, the display device 2910, the alphanumeric input device 2912, and the UI navigation device 2914 may be a touch screen display. The machine 2900 may additionally include a mass storage device (e.g., a drive unit, an SSD drive) 2916, a signal generating device 2918 (e.g., a speaker), a network interface device 2920, and one or more sensors 2921 (e.g., a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors). The machine 2900 may include an output controller 2928, such as a serial (e.g., universal serial bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate with or control one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0367] The mass storage device 2916 may include a machine-readable medium 2922 on which is stored one or more sets of data structures or instructions 2924 (e.g., software) implemented or utilized by any one or more of the techniques or functionality described herein. The instructions 2924 may also reside, in whole or at least in part, within the main memory 2904, within the static memory 2906, within the hardware processor 2902, or within the GPU 2903 during their execution by the machine 2900. Any combination of one of the hardware processor 2902, the GPU 2903, the main memory 2904, the static memory 2906, or the mass storage device 2916 may constitute a machine-readable medium.

[0368] Although the machine-readable medium 2922 is illustrated as a single medium, the term “machine-readable medium” may include a single medium or multiple media (eg, a centralized or distributed database and / or associated caches and servers) configured to store one or more instructions 2924.

[0369] The term "machine-readable medium" may include any medium that is capable of storing, encoding, or carrying instructions 2924 executed by the machine 2900 and causing the machine 2900 to perform any one or more of the techniques of the present disclosure, or that may store, encode, or carry data structures used by or associated with such instructions 2924. Non-limiting machine-readable medium examples may include solid-state memory and optical and magnetic media. In one example, a large-capacity machine-readable medium includes a machine-readable medium 2922 having a plurality of particles having a constant (e.g., stationary) mass. Therefore, to the extent that local law does not allow for claiming protection of a signal, the large-capacity machine-readable medium is not a transiently propagated signal. Specific examples of large-capacity machine-readable media may include non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices, magnetic disks (such as internal hard disks and removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0370] The instructions 2924 may also be transmitted or received over the communication network 2926 via the network interface device 2920 using a transmission medium.

[0371] Throughout this specification, multiple instances can implement components, operations or structures described as single instances. Although the individual operations of one or more methods are illustrated and described as separate operations, one or more operations in the individual operations can be performed concurrently, and the operations do not need to be performed in the order shown. The structure and function presented as a separate component in the example configuration can be implemented as a combined structure or component. Similarly, the structure and function presented as a single component can be implemented as a separate component. These and other variations, modifications, additions and improvements fall within the scope of this paper theme.

[0372] The embodiments described herein are described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used and derived therefrom, so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, the specific embodiments should not be understood in a limiting sense, and the scope of the various embodiments is limited only by the appended claims and the full scope of equivalents to which these claims are assigned.

[0373] As used herein, the term "or" may be interpreted in an inclusive or exclusive sense. In addition, multiple instances may be provided for resources, operations, or structures described herein as a single instance. In addition, the boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and specific operations are shown in the context of a specific illustrative configuration. Other allocations of functions may be expected and may fall within the scope of various embodiments of the present disclosure. Typically, structures and functions presented as separate resources in the example configuration may be implemented as combined structures or resources. Similarly, structures and functions presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of the present disclosure as represented by the appended claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive.< / account> < / account>

Claims

1. A cloud storage system, comprising: Multiple storage clusters, each storage cluster includes: Multiple storage nodes, each storage node includes: one or more computer processors; a partitioning layer that stores data in clumps within containers; and The stream layer, used to stream bulk data; a hierarchical namespace service, executed on the one or more computer processors, for implementing file system functions on the data stored in the blobs, the file system functions comprising directory functions and file functions, each directory being configured to include files and other directories requested via the file system functions, each file being stored in a blob, wherein the hierarchical namespace service comprises a master directory table, the master directory table identifying files and directories within each directory, wherein moving a directory or a file does not require moving or copying a blob; and A front-end layer, executed on the one or more computer processors, for processing user requests to access and manage data stored on the blob, the front-end layer accessing the partitioning layer to access the blob based on the address of the blob, the front-end layer accessing the hierarchical namespace service to identify the blob.

2. The cloud storage system of claim 1, wherein the partitioning layer is configured to identify a partition key for a flat namespace operation, the partition key comprising an account name, a container name, and a blob name.

3. The cloud storage system of claim 1, wherein the partition layer is configured to identify a partition key for hierarchical namespace operations, the partition key comprising an account name, a file system name, one or more directory names, and a file name. 4 . The cloud storage system according to claim 1 , wherein the hierarchical namespace service comprises one or more physical nodes, wherein the physical node has a plurality of virtual nodes for performing namespace mapping.

5. The cloud storage system of claim 1, wherein the hierarchical namespace service comprises a plurality of directory blocks, wherein a globally unique identifier is assigned to a directory in response to directory creation, and wherein the globally unique identifier does not change in response to renaming the directory.

6. The cloud storage system of claim 1 , wherein the hierarchical namespace service comprises a physical block for each entity in the file system, the entity being one of a directory or a file, wherein the physical block for an entity can have multiple simultaneous versions in the cloud storage system. 7 . The cloud storage system of claim 1 , wherein the hierarchical namespace service is configured to map a file system path in a hierarchical namespace to a blob identifier in a flat namespace.

8. The cloud storage system according to claim 1, further comprising: A location service is used to manage account namespace metadata across the multiple storage clusters.

9. The cloud storage system according to claim 1, wherein the hierarchical namespace service supports file system operations, the file system operations comprising: Create a file system, delete a file system, list the path of said file system, create a directory, rename a directory, delete a directory, append files, flush files, and read files.

10. A method for implementing a file system function, comprising: Multiple storage clusters are provided in the cloud storage system. Each storage cluster includes: Multiple storage nodes, each storage node includes: one or more computer processors; a partitioning layer that stores data in clumps within containers; and The stream layer, used to stream bulk data; a hierarchical namespace service, executed on the one or more computer processors, for implementing file system functions on the data stored in the blobs, the file system functions comprising directory functions and file functions, each directory being configured to include files and other directories requested via the file system functions, each file being stored on a blob, wherein the hierarchical namespace service comprises a master directory table, the master directory table identifying files and directories within each directory, wherein performing a move of a directory or a file does not require moving or copying blobs; and A front-end layer, executed on the one or more computer processors, for processing user requests to access and manage data stored on the blob, the front-end layer accessing the partitioning layer to access the blob based on the address of the blob, the front-end layer accessing the hierarchical namespace service to identify the blob.

11. The method of claim 10, wherein the partitioning layer is configured to identify a partition key for flat namespace operations, the partition key comprising an account name, a container name, and a blob name.

12. The method of claim 10, wherein the partitioning layer is configured to identify a partition key for hierarchical namespace operations, the partition key comprising an account name, a file system name, one or more directory names, and a file name.

13. The method of claim 10, wherein the hierarchical namespace service comprises one or more physical nodes having a plurality of virtual nodes for performing namespace mapping.

14. The method of claim 10, wherein the hierarchical namespace service comprises a plurality of directory blocks, wherein a globally unique identifier is assigned to a directory when the directory is created, and wherein the globally unique identifier does not change when the directory is renamed.

15. A non-transitory machine-readable storage medium comprising instructions that, when executed by a machine, cause the machine to perform operations comprising: Multiple storage clusters are provided in the cloud storage system. Each storage cluster includes: Multiple storage nodes, each storage node includes: a partitioning layer that stores data in clumps within containers; and The stream layer, used to stream bulk data; a hierarchical namespace service for implementing file system functions on the data stored in the blobs, the file system functions comprising directory functions and file functions, each directory being configured to include files and other directories requested via the file system functions, each file being stored in a blob, wherein the hierarchical namespace service comprises a master directory table, the master directory table identifying files and directories within each directory, wherein moving a directory or a file does not require moving or copying a blob; and A front-end layer is used to process user requests for accessing and managing data stored on the blob, the front-end layer accesses the partition layer to access the blob based on the address of the blob, and the front-end layer accesses the hierarchical namespace service to identify the blob.

16. The non-transitory machine-readable storage medium of claim 15, wherein the partitioning layer is configured to identify a partition key for flat namespace operations, the partition key comprising an account name, a container name, and a blob name.

17. The non-transitory machine-readable storage medium of claim 15, wherein the partition layer is configured to identify a partition key for hierarchical namespace operations, the partition key comprising an account name, a file system name, one or more directory names, and a file name.

18. The non-transitory machine-readable storage medium of claim 15, wherein the hierarchical namespace service comprises one or more physical nodes having a plurality of virtual nodes for performing namespace mapping.

19. The non-transitory machine-readable storage medium of claim 15, wherein the hierarchical namespace service comprises a plurality of directory blocks, wherein a globally unique identifier is assigned to a directory when the directory is created, and wherein the globally unique identifier does not change when the directory is renamed.

Images