Storage based on fault domains and storage classes

Abstract

Examples include a computing device configured by executable instructions to categorize a plurality of storage components into a plurality of storage fault domains, each storage fault domain corresponding to at least one failure scenario not shared by the other storage fault domains. The computing device may receive a data object for storage. The computing device may store at least a portion of the data object to a first storage fault domain, and may store at least another portion of the data object to a second storage fault domain that is different from the first storage fault domain.

Description

STORAGE BASED ON FAULT DOMAINS AND STORAGE CLASSESCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of, and priority to, U.S. Provisional Patent Application No. 63 / 735,930, filed December 19, 2024, which is incorporated by reference herein.TECHNICAL FIELD

[0002] This disclosure relates to the technical field of data storage and retrieval.BACKGROUND

[0003] Data storage systems, such as object storage systems or various other types of storage systems, may be configured to use a variety of storage media for physically storing their data. These storage systems may also store a variety of types of data having different requirements, such as regarding the level of protection, speed of access, encryption, compression, and / or various other parameters. Further, storage systems may utilize physical storage that is spread across multiple machines, racks, or other organizations in a single data center or in multiple data centers. However, in the case of a failure of a component of a storage system, it is desirable to avoid losing data.SUMMARY

[0004] Some implementations include a computing device configured by executable instructions to categorize a plurality of storage components into a plurality of storage fault domains, each storage fault domain corresponding to at least one failure scenario not shared by the other storage fault domains. The computing device may receive a data object for storage. The computing device may store at least a portion of the data object to a first storage fault domain, and may store at least another portion of the data object to a second storage fault domain that is different from the first storage fault domain.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The detailed description is set forth with reference to the accompanying figures. In the figures, the leftmost digit(s) of a reference number identifies the figure in which the referencenumber first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.

[0006] FIG. 1 illustrates an example architecture of a system configured to store data using storage classes and storage fault domains according to some implementations.

[0007] FIG. 2 illustrates the example of the system of FIG. 1 in which a data object is stored in multiple storage fault domains according to a second storage class.

[0008] FIG. 3 is a flow diagram illustrating an example process for storing data based on storage classes and storage fault domains according to some implementations.

[0009] FIG. 4 illustrates an example user interface that a user may use for defining storage components that may be used for storage fault domains according to some implementations.

[0010] FIG. 5 illustrates an example user interface that a user may use for creating storage classes that may be used for storing data according to a desired protection level in some implementations .

[0011] FIG. 6 illustrates an example of the storage class data structure that includes information related to a plurality of storage classes that may be used for storing data according to some implementations.

[0012] FIG. 7 illustrates an example user interface that a user may use for creating and / or defining storage fault domains according to some implementations.

[0013] FIG. 8 illustrates an example of the storage fault domain data structure that includes information related to a plurality of storage fault domains that may be used for storing data according to some implementations.

[0014] FIG. 9 illustrates select example components of the service computing device that may be used to implement at least some of the functionality of the systems described herein.

[0015] FIG. 10 illustrates select components of an example configuration of a storage computing device according to some implementations.DESCRIPTION OF THE EMBODIMENTS

[0016] Some implementations herein are directed to techniques and arrangements for an object storage system with varying types of physical storage media and varying requirements on the treatment of the stored data. For instance, the object storage system may include storage fault domains and storage classes that may be used to manage the treatment of data across different storage media and also guarantee a quality of service that may be specified for various different data objects. For examples, the storage class may define how the data is to be protected (e.g.,number of redundant copies, whether to use erasure coding, etc.), while the storage fault domains may device where data is protected (e.g., which storage components are sufficiently isolated from other storage components to avoid the likelihood of failure from a common point of failure. In some examples, a “storage component” may refer to a single storage medium that is configured in the object storage system herein for storing data. In some cases, when a plurality of storage components are configured in the object storage system, each storage component may be assigned to a single storage class and a single storage fault domain.

[0017] When a data object is written to the object storage system, the data object may be associated with a storage class. In some examples, a client associated with the data object may specify the storage class for the data object. In other examples, if the client has not specified a storage class for the data object, a default storage class may be assigned for the data object by the object storage system. When selecting which storage component to which to store a received data object, the object storage system may be configured to only select storage components corresponding to the same storage class as that associated with the data object. In the examples herein, the storage classes may have an assigned protection level, which defines, e.g., whether erasure coding is performed and / or whether multiple copies of the data object are stored.

[0018] When the specified storage class indicates that erasure coding is to be performed and / or multiple copies of data object are to be stored, the received data object is divided into multiple shards (referred to herein as “data object portions”). For instance, some of the data object portions may be data shards and some of the data object portions may be erasure coded parity shards, which are calculated to provide additional protection for the data object, such as in the case of hardware failure, data corruption, or the like. Each data object portion may be stored in a separate storage fault domain so that multiple data object portions of the same data object are not stored in the same storage fault domain. For instance, when one data object is divided into two data object portions, and a single parity data object portion is calculated for the two data object portions corresponding to the original data, each data object portion of the three data object portions may be stored in one of three separate storage fault domains.

[0019] Examples herein integrate storage classes with storage fault domains and tie the storage fault domains to protection levels that may correspond to protection levels specified for particular storage classes of a plurality of storage classes. The storage fault domains may be configured so that all storage components that are likely to fail together (e.g., due to common hardware, common power supply, or other common failure point) are grouped into a single storage fault domain. Accordingly, by ensuring that each shard of an object is stored in adifferent storage fault domain grouping, implementations herein ensure that no single hardware failure will render inaccessible more than one shard of a data object, thereby providing maximum protection of the data object. Additionally, or alternatively, some examples herein may allow multiple storage classes to be assigned to the same storage component in a storage fault domain.

[0020] Some implementations employ two logical concepts for organizing stored objects and storage media: storage classes and storage fault domains. Storage Classes can be chosen when objects are written. Storage hardware is assigned to specific storage classes, and when an object is written to the system, it will only be stored on hardware matching that storage class. Additionally, storage fault domains are logical groupings of storage media that are expected to likely fail at the same time. These groupings are used to prevent writing multiple pieces of one object “close to each other” in a way that makes it likely that multiple pieces are lost in a single failure.

[0021] Some examples herein may employ erasure coding as a data protection technique to, at least in part, provide redundancy protection to the stored data. Erasure coding may include receiving a data object, calculating parity data for the data of the data object, and storing the data and the parity data so that, if a portion of the stored data is lost, then the lost data can be recovered using the remaining object data and the parity data by recalculating the lost portion of the data from the parity data. Thus, erasure coding may be used by the object storage system herein as a technique for protecting data against loss. Further because the parity data can consume less storage space than the data of the object, erasure coding is able to protect against data loss while using storage capacity more efficiently as compared to keeping additional complete copies of the original data object.

[0022] The storage systems herein may use a variety of storage media for physically storing the data. The storage systems herein may also store a variety of different types of data with different requirements as to the level of protection, speed of access, encryption, and other parameters. As one example, an object storage system herein may include physical storage that is spread across multiple storage nodes, multiple racks, multiple arrays, multiple drives, and so forth, such as at a commercial or private data center, a commercial or private cluster of computing devices, or the like. In some examples, some of the storage may include network storage (e.g., cloud storage) provided by one or more commercial cloud-based storage providers, such as AMAZON®, MICROSOFT®, IBM®, GOOGLE®, HITACHI VANTARA®, or the like, such as may be provided at data centers operated by these entities.

[0023] For discussion purposes, some example implementations are described in the environment of one or more service computing devices in communication with one or more storage systems having storage components categorized into different storage fault domains for storing data according to specified storage classes. However, implementations herein are not limited to the particular examples provided, and may be extended to other types of computing system architectures, other types of storage components, other types of client configurations, other types of data, and so forth, as will be apparent to those of skill in the art in light of the disclosure herein.

[0024] FIG. 1 illustrates an example architecture of a system 100 configured to store data using storage classes and storage fault domains according to some implementations. The system 100 may serve as an object storage system including one or more service computing devices 102 that are able to communicate through one or more networks 106 with one or more storage computing devices 104, such as a first storage computing device 104(1) through an Nth storage computing device 104(N). Further, the service computing device(s) 102 may be able to communicate through the one or more networks 106 with one or more client computing devices 108. For example, the service computing device(s) 102 may include or may operate as a gateway node, an access node, a server node, a management node, and / or any of various other types of service computing devices that provide the client device(s) 108 with access to the storage provided by the storage computing devices 104 for enabling the client devices 108 to store and access data stored by the storage computing devices 104 on a plurality of storages 109, as well as performing other management and control functions, as discussed additionally below.

[0025] The storage computing devices 104( 1 )- 104(N) may each manage and / or access the plurality of storages 109 that may include storage devices, storage arrays, storage racks, storage nodes, or the like, and that may be separated or otherwise categorized into a plurality of storage fault domains 111. For example, as discussed additionally below, each storage fault domain 111 may be at least partially isolated from each other storage fault domain 111 such that a failure at one storage fault domain 111 may not affect the other storage fault domains 111 , at least to such an extent that data at the other storage fault domains is no longer accessible due to the failure at the storage fault domain 111 that suffered the failure.

[0026] In addition, one or more administrative computing devices 110 may be able to communicate with the storage computing devices 104 and / or the service computing device(s) 102, such as through the one or more networks 106. For example, an administrative user 114 may use the administrative computing device 110 to categorize the storages 109 into a pluralityof storage fault domains 111, such as a first storage fault domain 111(1), a second storage fault domain 111(2), . . ., through an Mth storage fault domain 111(M). As one example, the administrative user 1 14 may create and / or update a storage fault domain data structure (DS) 112 that categorizes the storages 109 into the respective separate storage fault domains 111(1) through 111(M). As one example, the administrative user 114 may use an administrative application 116 executed on the administrative computing device 110 to create and / or update the storage fault domain data structure 112. Each storage fault domain 111 may be identified in the storage fault domain data structure 112, such as by one or more of a storage fault domain identifier (SFD_ID), volume or folder path, storage bucket location or identifier, storage node identifier or address, rack identifier, array identifier, drive identifier or path, or the like.

[0027] Furthermore, in some examples, each storage fault domain 111 may be associated with a specified storage class 117. For instance, a first group of stores fault domains 1 11 may be associated with a first storage class for receiving data to be stored according to the first storage class, while a second group of stores fault domains 111 may be associated with a second storage class for receiving data to be stored according to the second storage class, and so forth for a plurality of storage classes that may be specified for the system 100.

[0028] In some examples, the service computing device(s) 102 may include one or more servers that may be embodied in any number of ways. For instance, the programs, other functional components, and at least a portion of data storage of the service computing device(s) 102 may be implemented on at least one server, such as in a cluster of servers, a server farm, a data center, a cloud-hosted computing service, and so forth, although other computer architectures may additionally or alternatively be used. As another example, the service computing device(s) 102 may be abstracted as, or otherwise treated as, a single node that is actually a cluster containing multiple service computing devices 102. Additional details of the service computing device(s) 102 are discussed below with respect to FIG. 9.

[0029] The one or more networks 106 may include any suitable network, including a wide area network, such as the Internet; a local area network (LAN), such as an intranet; a wireless network, such as a cellular network, a local wireless network, such as Wi-Fi, and / or short-range wireless communications, such as BLUETOOTH®; a wired network including Fibre Channel, fiber optics, Ethernet, or any other such network, a direct wired connection, or any combination thereof. Accordingly, the one or more networks 106 may include both wired and / or wireless communication technologies. Components used for such communications can depend at least in part upon the type of network, the environment selected, or both. Protocols for communicatingover such networks are well known and will not be discussed herein in detail. As one example, such as in the case that the service computing device(s) 102 is co-located with some or all of the storage computing devices 104(l)-104(N), such as in the same data center or the like, the network(s) 106 connecting the service computing device(s) 102 with the storage computing devices 104(l)-104(N) may include a private network, such as a LAN, storage area network (SAN), Fibre Channel network, and so forth. Additionally, the network(s) 106 connecting the service computing device(s) 102 with the client device(s) 108 and, in some cases, some of the storage computing devices (104) may be a public network that may include the Internet, or a combination of public and private networks. Implementations herein are not limited to any particular type of network as the one or more networks 106.

[0030] The service computing device(s) 102 may be configured to provide storage and data management services to the client device(s) 108. In some examples, the client devices 108 may be server-type computing devices or other high-performance computing devices that execute a client application 118 that stores and / or retrieves data, such as data objects 120, or the like on the storage computing devices 104. For example, the client application 118 may be an enterprise level application that has high input and output requirements for storing and / or retrieving data. Alternatively, in other examples, the client devices 108 may be any type of user computing device such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, and / or any other type of computing device able to send data over a network. In some cases, human users (not shown in FIG. 1) may be associated with client devices 108 such as through a respective user account, user login credentials, or the like. Furthermore, the client device(s) 108 may be configured to communicate with the service computing device(s) 102 through the one or more networks 106, through separate networks, or through any other suitable type of communication connection. Numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0031] The client application 118 executable on the client device 108 may communicate with a storage management program 122 executable on the service computing device(s) 102, such as for sending a data object 120, from the client device 108 to the service computing device(s) 102, for storage on the storages 109. Additionally, the client application 118 may request delivery of a stored data object from the storages 109 through a data instruction sent to the storage management program 122. Examples of data instructions that the client device 108 may send to the service computing device(s) 102 may include write operations, read operations, delete operations, and the like. The client application 118 may include any type of application havingcommunication functionality enabling communication over the one or more networks 106 with the storage management program 122 or other application(s) on the service computing device(s) 102. Accordingly, the service computing device(s) 102 may provide storage services for the client device(s) 108, such as by acting as a storage gateway, or the like, as mentioned above.

[0032] The administrative device 110 may be any suitable type of computing device such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, and / or any other type of computing device able to send data and communications over a network. The administrative user 114 may be associated with the administrative device 110, such as through a respective administrative account, administrative login credentials, or the like. Furthermore, the administrative device 110 may be able to communicate with the service computing device(s) 102 and / or the storage computing devices 104 through the one or more networks 106, through separate networks, through a direct connection, or through any other suitable type of communication connection.

[0033] Each administrative device 110 may include a respective instance of the administrative application 116 that may execute on the respective administrative device 110, such as for communicating with the storage management program 122 on the service computing device(s) 102 and / or for communicating with a storage program 126 executable on the storage computing devices 104. For example, the administrative user 114 may communicate with the storage program 126, such as for setting the storage fault domains 111 in the storage fault domain data structure 112. The administrative user 114 may also set the parameters for the storage classes 117, such as in a storage class data structure 1 19 (not shown in FIG. 1). In some cases, the administrative application 116 may include a browser or may operate through a browser, while in other cases, the administrative application 116 may include any other type of application having communication functionality enabling communication with the storage program 126 or other applications on the service computing device(s) 102 or the storage computing devices 104 over the one or more networks 106.

[0034] Each storage computing device 104(l)-104(N) may execute an instance of the storage program 126, respectively, for managing storage of respective data objects at the respective storage computing devices 104(1 )- 104(N). For instance, the storage program 126 may receive a data object 120 from the storage management program 122 at the service computing device(s) 102, may store the data object 120 on the storages in accordance with a storage class specified for the data object 120, and may retrieve and send requested data to the storage management program 122, such as in response to a client read request, or the like. The storage program 126may also provide information to the storage management program 122, such as performance information, capacity usage information, and so forth for the respective storage devices managed by the particular storage program 126 on a respective storage computing device 104.

[0035] The service computing device(s) 102 may execute the storage management program 122, which may provide access to the storage computing devices 104(l)-104(N) and the storages 109. For example, the service computing device 102 may receive a data object 120 from a client device 108 for storing on the storages 109 according to a specified storage class 117. In some examples, the storage class for a data object may be specified by the client user and / or by the client application 118. In other examples, the storage class 117 associated with the data object 120 may be a default storage class 117 selected by the storage management program 122, or previously specified for use for particular data types, or the like. As one example, a header included with a communication that transmits the data object 120 may specify the storage class for the associated data object 120.

[0036] Based on the storage class specified for the data object 120, the service computing device 102 may determine to which storage fault domains 111 and / or to which storage computing devices 104 to send the data object 120. Additionally, in some cases, the storage management program 122 may perform preprocessing of the data object 120 prior to sending the data object to the storage computing device(s) 104 for storage. In this example, suppose that the storage class 117 indicates that the data object 120 should be stored according to a first storage class that requires a protection level that includes erasure coded for data protection. Accordingly, the storage management program may divide the data object into a plurality of data shards otherwise referred to herein as a plurality of data object portions 123. Furthermore, the data management program 122 may calculate one or more additional data object portions 123 as erasure coded parity data corresponding to the other data object portions 123. Erasure coding techniques are well-known in the art and details of encoding and decoding particular erasure coded parity blocks are therefore not described herein.

[0037] As one example, suppose that the first storage class specifies two parity portions for every three data portions. Accordingly, the storage management program 122 may send a total of five data object portions 123 to the storage computing devices 104 for storage in a group 124 of storage fault domains 111 that correspond to the first storage class. For example, suppose at least storage fault domains 111(1) through 111(5) belong to a first group 124 of storage fault domains that are used for storing data according to the first storage class. In the illustrated example, a first data object portion 123(1) is stored in the second storage fault domain 111(2); asecond data object portion 123(2) is stored in the fifth storage fault domain 111(5); a third data object portion 123(3) is stored in the fourth storage fault domain 111(4); a fourth data object portion 123(4) is stored in the third storage fault domain 111(3); and a fifth data object portion 123(5) is stored in the first storage fault domain 111(1). In this data protection configuration, any two of the fault domains 111(1) through 111(5) could fail and the system 100 would be able to recover the complete data object 120 from the data portions 123 contained in the remaining three storage fault domains 111. The storage management program 122 may store metadata 130 for the stored data object 120, which may include the stored locations of the data object portions 123 and other information related to the data object 120, in a metadata data structure 132. Furthermore, while the storage management program 122 is described as dividing the data object 120 into data object portions 123 and determining the parity portions, in other examples, this function may be performed by one or more of the storage programs 126 on the storage computing devices 104.

[0038] Each separate storage fault domain 111 may correspond to a separate storage component. For example, a storage component may include at least one physical device able to store data. Furthermore each storage component may be physically dispersed from other storage components corresponding to the same group of storage fault domains. As mentioned above, the storage fault domains 111 may be grouped according to the storage class for which they are used to store data. Furthermore, in some examples, more than one storage class may be assigned to the same group of storage fault domains or portions of the same group. Accordingly, the storage component is the physical device that stores the data of the storage class. The storage class defines how the stored data is protected when stored, such as whether only a single copy is stored, multiple copies are stored, or the data is stored with erasure coded parity data, and so forth. The storage fault domains define where the stored data is stored and protected by storing the data to avoid the likelihood of a common or otherwise shared failure, and to thereby improve the resilience of the stored data and to ensure data availability if there is a failure of one of the storage components.

[0039] The storage fault domains 111 in the same storage fault domain group (e.g., group 124 in the illustrated example) may be partially isolated from the other storage fault domains 111 in the same group such that each storage fault domain in the group correspond to at least one failure scenario that is not shared by the other storage fault domains 111 in the same group. As one example, each storage fault domain may correspond to a separate storage component such as a separate storage drive such that if one storage drive fails the data on the other storage drives willstill be available. As another example, each storage component may be a group of storage drives, such as a storage array, that may be contained, for example, in a rack in a data center. Thus, each storage component in this scenario may be a separate storage array and in some cases, may be on a separate power circuit, separate power source, or the like, such that if one storage array corresponding to one storage fault domain fails, the other storage arrays corresponding to the other storage fault domains in the same group may continue to operate and can continue to provide their data to a computing device attempting to access the data. As yet another example, each storage component may include a plurality of storage drives and a storage computing device 104 configured to access the plurality of storage drives. In this scenario, each storage component includes a separate storage computing device corresponding to a separate storage fault domain, and similarly may be on a separate power supply, separate power circuit, or the like from other storage computing devices corresponding to other storage fault domains in the same storage fault domain group. Additionally, while several examples of storage fault domains and storage components that may be physically dispersed from each other have been described herein, numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0040] FIG. 2 illustrates the example of the system 100 of FIG. 1 in which a data object 202 is stored in multiple storage fault domains according to a second storage class. In the example of FIG. 2, as a partial continuation of the example discussed above with respect to FIG. 1 , suppose that the service computing device 102 receives a data object 202, and an indication that the data object should be stored according to a second data storage class that includes a data protection level of storing two complete copies of the data object 202. Further, suppose that a second group 204 of storage fault domains 111(10), 111(11), and 111(12) are assigned to the second storage class.

[0041] In some examples, the storage class for the data object 202 may be specified by a header included with the communication that transmits the data object 202 from the client device 108 to the service computing device 102. In other examples, the service computing device may already have been instructed to store all data received from the client device 108 according to the second storage class. As yet another example, if a storage class is not specified for the data object, the service computing device 102 may store the data object 202 according to a default storage class as discussed above.

[0042] Upon receipt of the data object 202, the storage management program 122 may determine, based on the specified storage class 117, which storage fault domains 111 to which tostore the data object 202. Since complete copies of the data object 202 are stored for the storage class hundred 117 in this example, the storage management program 122 may forward the data object to one or more storage computing devices 104 corresponding to the identified storage fault domains 111 without any additional preprocessing. Upon receipt of the data object 202, the one or more storage computing devices 104 may store a first copy of the data object 202 to one of the identified storage fault domains, e.g., tenth storage fault domain 111(10), and may store a second copy of the data object 202 to the twelfth storage fault domain 111(12). Furthermore, the storage management program 122 may store metadata 130 about the stored locations of the data object 202 to the metadata data structure 132.

[0043] FIG. 3 is a flow diagram illustrating an example process 300 for storing data based on storage classes and storage fault domains according to some implementations. The process is illustrated as a collection of blocks in a logical flow diagram, which represents a sequence of operations, some or all of which may be implemented in hardware, software or a combination thereof. In the context of software, the blocks may represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, program the processors to perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types. The order in which the blocks are described should not be construed as a limitation. Any number of the described blocks can be combined in any order and / or in parallel to implement the process, or alternative processes, and not all of the blocks need be executed. For discussion purposes, the process is described with reference to the environments, frameworks, and systems described in the examples herein, although the process may be implemented in a wide variety of other environments, frameworks, and systems.

[0044] In the example of FIG. 3, the process 300 may be performed at least in part by the storage management program 122 executing on the service computing device 102, and / or by the storage program 126 executing on a storage computing device 104, and / or by another computing device executing storage logic.

[0045] At 302, the computing device may receive a data object for storage. In some examples, the computing device may also receive an indication of a storage class for storing the object, such as in a communication header or the like. The storage class may include at least an indication of a protection level with which the data object is to be stored. In some examples, the computing device may receive the data object from a client device executing a client application.

[0046] At 304, the computing device may determine a storage class for storing the data object. As mentioned above, in some examples, the storage class for the data object may be included with the communication from the client device, or alternatively may have been specified in advance. As another example, if there is no storage class specified or otherwise indicated for storing the data object, the computing device may determine that the storage class for the data object is the default storage class.

[0047] At 306, the computing device may determine whether the identified storage class includes erasure coding. If so, the process goes to 308. If not, the process goes to 312.

[0048] At 308, based at least on determining that erasure coding is required, the computing device may divide the data object into a plurality of data portions based at least in part on the specified erasure coding scheme. Furthermore, when dividing the data object into the plurality of data portions, the size of the data object may also be taken into consideration such as for ensuring that a maximum data block size for the system is not exceeded by the size of the data portions.

[0049] At 310, the computing device may calculate erasure coded parity portions based on the erasure coding scheme for the storage class. For instance, in the case of a 3-2 erasure coding scheme, the computing device may calculate two erasure coded parity portions for every three data portions.

[0050] At 312, the computing device may determine available fault domains based on the storage class. For example, as discussed above, the storage fault domains may be grouped according to storage class. Thus, the computing device may identify a number of storage fault domains sufficient for storing separate data object portions on each separate storage fault domain.

[0051] At 314, the computing device may send the data object or data object portions for storage at the determined storage fault domains and according to the specified storage class. For instance, the computing device may send the data object, or, in the case that erasure coding is included in the specified protection level, the data object portions and parity portion(s) to one or more of the storage computing devices 104 discussed above with respect to FIGS. 1 and 2. The one or more storage computing devices 104 may store the data object and / or data object portions to the selected storage fault domains.

[0052] At 316, the computing device may store metadata for the stored data object. For example, the computing device may store at least location information for the stored data object so that the stored object can be retrieved if requested by a client computing device or the like.

[0053] FIG. 4 illustrates an example user interface 400 that a user may use for defining storage components that may be used for storage fault domains according to some implementations. As mentioned above, a storage component may include at least one physical device for storing data. In this example the user (e.g., the administrative user 114 discussed above with respect to FIG. 1) may select the components option 402 from the storage drop-down menu to cause the user interface 400 to be presented to the user. The user interface 400 presents information about the identified storage components such as a name 404, a type 406, a region 408, state 410, a fault domain 412, storage class 414, a percentage of capacity used 416, and a drive capacity 418. For instance, each storage component may have a unique name 404, and may have an indicated type of storage 406. Furthermore, the storage components are located in the Eastern US region as indicated at 408, are indicated to be active as indicated by the state 410, and are each associated with a separate fault domain as indicated at 412 such as being located at a separate rack such as within the same data center or the like. Furthermore, as indicated at 414, each storage component may be associated with a particular storage class such as storage class A in this example. Additionally, as indicated at 420, the user may add more storage components using a search field or the like.

[0054] FIG. 5 illustrates an example user interface 500 that a user may use for creating storage classes that may be used for storing data according to a desired protection level in some implementations. In this example the user (e.g., the administrative user 114 discussed above with respect to FIG. 1) may select the classes option 502 from the storage drop-down menu to cause the user interface 500 to be presented to the user. The user interface 500 presents information about the established storage classes such as a name 504 and a protection level 506. In addition, the user interface 500 may enable the user to select a default class as indicated at 508. In this example, the standard class is selected as the default class. Further, as indicated at 510, the user may edit any of the established storage classes. Moreover, as indicated at 512, the user may create more storage classes using the field at 512, or the like.

[0055] FIG. 6 illustrates an example of the storage class data structure 119 that includes information related to a plurality of storage classes that may be used for storing data according to some implementations. The storage class data structure includes columns for storage class name 602, tension level DCIV, data count 606, parity count 608, and an indication of the default storage class as indicated at 610. In this example, the stores class names include standard, which provides a protection level of one copy; Class A, which provides a protection level of two copies, i.e., one data count and one parity count; Class B, which provides a protection level ofthree plus two erasure coding and which includes a data count of three and a parity count of two; and Class C, which provides a protection level of two plus one erasure coding, and which includes a data count of two and a parity count of one.

[0056] FIG. 7 illustrates an example user interface 700 that a user may use for creating and / or defining storage fault domains according to some implementations. In this example the user (e.g., the administrative user 114 discussed above with respect to FIG. 1) may select the storage fault domain option 702 from the storage drop-down menu to cause the user interface 700 to be presented to the user. The user interface 700 presents information about the established storage fault domains such as a name 704 and corresponding storage class 706. In addition, the user interface 700 may show any tags 708 associated with the storage fault domain. For example, the tags 708 may indicate the location of a corresponding storage component within a data center and / or a name or type of the underlying storage component. Additionally, as indicated at 710 the path of the storage fault domain within the data center may be included in the user interface 700. Further, as indicated at 712, the user may edit any of the established storage fault domains.Moreover, as indicated at 714, the user may create more storage fault domains using the field at 714, or the like.

[0057] FIG. 8 illustrates an example of the storage fault domain data structure 112 that includes information related to a plurality of storage fault domains that may be used for storing data according to some implementations. In this example, the data structure 112 includes storage fault domain identifier 802, corresponding storage class 804, tags 806, and the path 808. For instance, as discussed above with respect to FIG. 4 each storage fault domain may have an identifier 802 that is unique within the system 100. Furthermore, each storage fault domain may be associated with at least one storage class and may have a unique path 808 within the data center or other computer system in which the storage fault domain is located. Additionally, the tags 806 may be selected by a user that creates the storage fault domain to provide an indication to other users as to information about the location of the storage fault domain within a data center or other system.

[0058] FIG. 9 illustrates select example components of the service computing device(s) 102 that may be used to implement at least some of the functionality of the systems described herein. The service computing device(s) 102 may include one or more servers or other types of computing devices that may be embodied in any number of ways. For instance, in the case of a server, the programs, other functional components, and data may be implemented on a single server, a cluster of servers, a server farm or data center, a cloud-hosted computing service, and so forth,although other computer architectures may additionally or alternatively be used. Multiple service computing device(s) 102 may be located together or separately, and organized, for example, as virtual servers, server banks, and / or server farms. The described functionality may be provided by the servers of a single entity or enterprise, or may be provided by the servers and / or services of multiple different entities or enterprises.

[0059] In the illustrated example, the service computing device(s) 102 includes, or may have associated therewith, one or more processors 902, one or more computer-readable media 904, and one or more communication interfaces 906. Each processor 902 may be a single processing unit or a number of processing units, and may include single or multiple computing units, or multiple processing cores. The processor(s) 902 can be implemented as one or more central processing units, microprocessors, microcomputers, microcontrollers, digital signal processors, system-on-chip processors, graphics processing units, artificial intelligence processing units, embedded processors, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. As one example, the processor(s) 902 may include one or more hardware processors and / or logic circuits of any suitable type specifically programmed or configured to execute the algorithms and processes described herein. The processor(s) 902 may be configured to fetch and execute computer-readable instructions stored in the computer- readable media 904, which may program the processor(s) 902 to perform the functions described herein.

[0060] The computer-readable media 904 may include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. For example, the computer-readable media 904 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape, magnetic disk storage, storage arrays, directly attached storage, network attached storage, storage area networks, cloud storage, combinations thereof, or any other medium that can be used to store the desired information and that can be accessed by a computing device. Depending on the configuration of the service computing device(s) 102, the computer-readable media 904 may be a tangible non-transitory medium to the extent that, when mentioned, non- transitory computer-readable media exclude media such as energy, carrier signals, electromagnetic waves, and / or signals per se. In some cases, the computer-readable media 904 may be at the same location as the service computing device 102, while in other examples, the computer-readable media 904 may be partially remote from the service computing device 102.For instance, in some cases, the computer-readable media 904 may include a portion of storage in the storage computing device(s) 104 discussed above with respect to FIG. 1.

[0061] The computer-readable media 904 may be used to store any number of functional components that are executable by the processor(s) 902. In many implementations, these functional components comprise instructions or programs that are executable by the processor(s) 902 and that, when executed, specifically program the processor(s) 902 to perform the actions attributed herein to the service computing device 102. Functional components stored in the computer-readable media 904 may include the storage management program 122, which may include one or more computer programs, applications, executable code, or portions thereof.

[0062] In addition, the computer-readable media 904 may store data, data structures, and other information used for performing the functions and services described herein. For example, the computer-readable media 904 may store the metadata data structure 132 containing the metadata 130 that may be maintained by the service computing device 102 such as for retrieving stored data objects from one or more of the storage computing devices 104, such as in response to a request from the client application or the like. In addition, the service computing device 102 may include the storage class data structure 919 that may contain information about the storage classes 1 17 currently active in the system 100, such as the protection requirements for each storage class 117. Additionally, the computer-readable media 904 may include the storage fault domain data structure 112 that includes information about the storage fault domains.

[0063] Further, while these data structures and programs are illustrated together in this example, during use, some or all of these data structures and / or programs may be stored on separate service computing device(s) 102. The service computing device 102 may also include or maintain other functional components and data, which may include programs, drivers, etc., and the data used or generated by the functional components. Further, the service computing device 102 may include many other logical, programmatic, and physical components, of which those described above are merely examples that are related to the discussion herein.

[0064] The one or more communication interfaces 906 may include one or more software and / or hardware components for enabling communication with various other devices, such as over the one or more network(s) 106. For example, the communication interface(s) 906 may enable communication through one or more of a LAN, the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi) and wired networks (e.g., Fibre Channel, fiber optic, Ethernet), direct connections, as well as close-range communications such as BLUETOOTH®, and the like, as additionally enumerated elsewhere herein.

[0065] FIG. 10 illustrates select components of an example configuration of a storage computing device 104 according to some implementations. The storage computing device 104 may include one or more storage computing devices 1002, which may include one or more servers or any other suitable computing device, such as any of the examples discussed above with respect to the service computing device 102. The storage computing device(s) 1002 may each include one or more processors 1004, one or more computer-readable media 1006, and one or more communication interfaces 1008. For example, the processor(s) 1004 may correspond to any of the examples discussed above with respect to the processors 902, the computer-readable media 1006 may correspond to any of the examples discussed above with respect to the computer-readable media 904, and the communication interface(s) 1008 may correspond to any of the examples discussed above with respect to the communication interfaces 906.

[0066] In addition, the computer-readable media 1006 may store the storage program 126, discussed above, as a functional component executed by the one or more processors 1004 for managing the storage of the data objects 1009 on the storage 109, or a portion thereof that may be associated with the storage computing device 104.

[0067] In addition, the computer-readable media 1006 may store data and data structures used by the storage program 126 such as the storage class data structure 119 and the storage fault domain data structure 112. The storage 109 may include one or more controllers 1012 associated with the storage 109 for storing data 1009, such as the data objects, or portions of the data objects, on one or more drives, arrays, trays, racks, extent groups, storage nodes, or other types of configurations of storage components 1016. For instance, the controller 152 may control the storage components 1016(1), 1016(2), ..., such as for configuring the storage component 1016(1) as a first storage fault domain 111(1), the second storage component 1016(2), as a second storage fault domain 111(2), and so forth. Further, the controller 1012 may be configured to store the data 1009 with various levels of protection based on the storage classes 1 17, such as in various erasure coded protection configurations, or any of various other configurations, such as a RAID configuration, JBOD configuration, or the like, and / or for presenting storage extents, logical units, logical volumes, or the like, based on the storage components 1016 to the storage program 126, and for managing the data 1009 stored on the underlying physical storage devices. The storage components 1016 may include any type of storage device, such as solid state drives, hard disk drives, optical drives, magnetic tape, combinations thereof, and so forth.

[0068] In some examples, the client device(s) 108 and / or the administrator device(s) 110 may have a hardware configuration and hardware elements similar to those discussed above for theservice computing device 102. However, the client device(s) 108 and / or the administrator device(s) 110 may have functional components and data that are different from those of the service computing device(s) 102.

[0069] The example processes described herein are only examples of processes provided for discussion purposes. Numerous other variations will be apparent to those of skill in the art in light of the disclosure herein. Further, while the disclosure herein sets forth several examples of suitable frameworks, architectures and environments for executing the processes, implementations herein are not limited to the particular examples shown and discussed. Furthermore, this disclosure provides various example implementations, as described and as illustrated in the drawings. However, this disclosure is not limited to the implementations described and illustrated herein, but can extend to other implementations, as would be known or as would become known to those skilled in the art.

[0070] Various instructions, processes, and techniques described herein may be considered in the general context of computer-executable instructions, such as programs, applications, executable code, program modules, and the like, stored on computer-readable media, and executed by the processor(s) herein. Generally, programs, applications, executable code, program modules, include routines, executable computer programs, programming objects, executable components, data structures, and so forth, for performing particular tasks or implementing particular abstract data types. These programs, applications, executable code, program modules, and the like, may be executed as native code or may be downloaded and executed, such as in a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of the programs, applications, executable code, program modules, and the like may be combined or distributed as desired in various implementations. An implementation of these programs, applications, executable code, program modules, and the like may be stored on computer storage media and / or transmitted across some form of communication media.

[0071] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

CLAIMS1. A method comprising: categorizing, by a computing device, a plurality of storage components into a plurality of storage fault domains, each storage fault domain corresponding to at least one failure scenario not shared by the other storage fault domains; receiving a data object for storage; and storing at least a portion of the data object to a first storage fault domain, and storing at least another portion of the data object to a second storage fault domain that is different from the first storage fault domain.

2. The method as recited in claim 1, further comprising: associating a first plurality of the storage fault domains with a first storage class, and associating a second plurality of the storage fault domains with a second storage class; receiving, in association with the data object an indication of the first storage class for storing the data object; and wherein storing at least the portion of the data object to the first storage fault domain, and storing at least the other portion of the data object to the second storage fault domain is based at least in part on determining that the first storage fault domain and the second storage fault domain are associated with the first storage class indicated for the data object3. The method as recited in claim 2, wherein based at least on the indication of the storage class indicating that the data object is to be stored according to the first storage class, the first storage class including erasure coding as a protection level, the method further comprises: dividing the data object into a plurality of data portions; calculating at least one parity portion for the plurality of data portions; storing at each data portion to a separate storage fault domain associated with the first storage class; and storing each at least one parity portion to a respective separate storage fault domain that is associated with the first storage class and that is also separate from the storage fault domains of the first storage class to which the data portions are stored.

4. The method as recited in claim 1, further comprising: receiving, with the data object, and indication of a storage class for storing the data object, wherein the storage class at least indicates a protection level for storing the data object; and wherein storing at least the portion of the data object to the first storage fault domain, and storing at least the other portion of the data object to the second storage fault domain that is different from the first storage fault domain further include selecting the first storage fault domain and the second storage fault domain based at least on the first and second fault domains being associated with the indicated storage class.

5. The method as recited in claim 4, further comprising receiving the indication of the storage class in a header of a communication associated with the data object.

6. The method as recited in claim 1, wherein the first storage fault domain and the second storage fault domain are separate from each other based at least in part on separate storage drives.

7. The method as recited in claim 1, wherein the first storage fault domain and the second storage fault domain are separate from each other based at least in part on separate arrays of storage devices.

8. The method as recited in claim 1, wherein the first storage fault domain and the second storage fault domain are separate from each other based at least in part on separate power supplies or separate power sources.

9. The method as recited in claim 1, wherein storing at least a portion of the data object to the first storage fault domain and storing at least the other portion of the data object to the second storage fault domain that is different from the first storage fault domain comprises storing a first copy of the data object to the first storage fault domain and storing a second copy of the data object to the second storage fault domain.

10. The method as recited in claim 1, further comprising identifying the plurality of storage components, wherein each storage component includes at least one storage medium that is subject to a different failure scenario from that of another storage component, and each storage component corresponds to a different storage failure domain.

11. The method as recited in claim 1, wherein the first and second storage failure domains are associated with a first storage class, the method further comprising: receiving another data object for storage; determining that a storage class has not been indicated for the other data object; and storing the other data object to a third storage failure domain corresponding to a default storage class.

12. A system comprising: a computing device configured by executable instructions to perform operations comprising: categorizing a plurality of storage components into a plurality of storage fault domains, each storage fault domain corresponding to at least one failure scenario not shared by the other storage fault domains; receiving a data object for storage; and storing at least a portion of the data object to a first storage fault domain, and storing at least another portion of the data object to a second storage fault domain that is different from the first storage fault domain.

13. The system as recited in claim 12, the operations further comprising: associating a first plurality of the storage fault domains with a first storage class, and associating a second plurality of the storage fault domains with a second storage class; receiving, in association with the data object an indication of the first storage class for storing the data object; and wherein storing at least the portion of the data object to the first storage fault domain, and storing at least the other portion of the data object to the second storage fault domain is based at least in part on determining that the first storage fault domain and the second storage fault domain are associated with the first storage class indicated for the data object.

14. A non- transitory computer-readable medium storing instructions executable by one or more processors to cause the one or more processors to perform operations comprising: categorizing a plurality of storage components into a plurality of storage fault domains, each storage fault domain corresponding to at least one failure scenario not shared by the other storage fault domains; receiving a data object for storage; and storing at least a portion of the data object to a first storage fault domain, and storing at least another portion of the data object to a second storage fault domain that is different from the first storage fault domain.

15. The non- transitory computer-readable medium as recited in claim 14, the operations further comprising: associating a first plurality of the storage fault domains with a first storage class, and associating a second plurality of the storage fault domains with a second storage class; receiving, in association with the data object an indication of the first storage class for storing the data object; and wherein storing at least the portion of the data object to the first storage fault domain, and storing at least the other portion of the data object to the second storage fault domain is based at least in part on determining that the first storage fault domain and the second storage fault domain are associated with the first storage class indicated for the data object.

Images