Ranged read of encrypted data

Abstract

Examples include a first computing device configured by executable instructions to receive, from a second computing device, a data object for storage. The first computing device divides the data object into a plurality of data segments and encrypts the data segments. The first computing device causes the plurality of encrypted data segments to be stored to a plurality of separate storage components. Additionally, the first computing device stores metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object to enable ranged read of a portion of the content data of the data object.

Description

RANGED READ OF ENCRYPTED DATACROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of, and priority to, U.S. Provisional Patent Application No. 63 / 735,927, filed December 19, 2024, which is incorporated by reference herein.TECHNICAL FIELD

[0002] This disclosure relates to the technical field of data storage and retrieval.BACKGROUND

[0003] Data storage systems, such as object storage systems or various other types of storage systems, may be configured to store data objects as encrypted data that is also protected by various data protection and redundancy techniques. These storage systems may store data having various different requirements, such as regarding the level of protection, speed of access, encryption, compression, and / or other storage parameters. However, because the data may be encrypted before the data is stored, the data typically cannot be read piecemeal. Instead, the entire string of data typically needs to be decrypted in order to use the data. This conflicts with the idea of a ranged read, which allows users access to data anywhere in the data object to save time. If an entire large object needed to be decrypted in order to read a piece near the end, that would make ranged read massively inefficient.SUMMARY

[0004] Some implementations include a first computing device configured by executable instructions to receive, from a second computing device, a data object for storage. The first computing device divides the data object into a plurality of data segments and encrypts the data segments. The first computing device causes the plurality of encrypted data segments to be stored to a plurality of separate storage components. Additionally, the first computing device stores metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object to enable a ranged read of a portion of the content data of the data object.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The detailed description is set forth with reference to the accompanying figures. In the figures, the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items or features.

[0006] FIG. 1 illustrates an example architecture of a system configured to store data while enabling a ranged read to be performed on the stored data according to some implementations.

[0007] FIG. 2 illustrates an example process performed for storing a data object using encryption and erasure coding, while also enabling a ranged read according to some implementations .

[0008] FIG. 3 illustrates an example of storage of encrypted and erasure-coded data portions in the system of FIG. 1 according to some implementations.

[0009] FIG. 4 is a flow diagram illustrating an example process for performing a ranged read on encrypted and erasure-coded data according to some implementations.

[0010] FIG. 5 illustrates select example components of the service computing device that may be used to implement at least some of the functionality of the systems described herein.

[0011] FIG. 6 illustrates select components of an example configuration of a storage computing device according to some implementations.DESCRIPTION OF THE EMBODIMENTS

[0012] Some implementations herein are directed to techniques and arrangements for a storage system, such as an object storage system or other data storage system, that enables ranged read of stored data, such as encrypted and erasure-coded data, including encrypted and erasure-coded data objects, and the like. For example, the storage system herein may encrypt stored data for providing increased protection of the stored data while enabling reading of only a desired portion of the data and without having to first decrypt an entire string of data or an entire data object. Thus, in examples herein, a user can read back only a desired smaller subset of the data of a data object (e.g., a row or set of rows from a database or table, a part of a data structure, a specified part of a media object, or the like) to avoid the added time and computational cost of reading back an entire data object, or the like, when only a portion of the stored data is desired for performing an operation.

[0013] Some implementations herein divide large data objects (e.g., larger than a threshold size) into a plurality of smaller parts (referred to herein as “data segments”) prior to performingencryption on the data segments. As one example, the size of the data segments may be fixed at a specific size (e.g., all data objects received for storage are broken into 4 MB segments prior to encryption) regardless of the size of the data object. As another example, the system may employ a fixed segment count in which all data objects received by the system are divided into the same number of segments regardless of the size of the data object (e.g., if a data object is larger than a threshold size, such as 1 MB, 10 MB, etc., the data object is divided into six equal segments). As another example, the segment count used by the system for received data objects may increase as the size of the data objects increases. For example, a data object that is larger than 1 MB and less than 50 MB may be divided into 5 segments, while a data object that is greater than 50 MB and less than 100 MB may be divided into 10 segments, a data object that is greater than 100 MB and less than 500 MB may be divided into 20 segments, and so forth. Numerous variations will be apparent to those of skill in the art having the benefit of the disclosure herein, with the foregoing being just several examples.

[0014] Of course, in some examples, the segmentation method might be not used at all, e.g., the entire received data object may be stored as one segment regardless of the size of the data object for some or all of the received data objects. For instance range read might not be implemented in the system for certain clients, certain types of data objects for which ranged read cannot be performed or is not useful, and / or for certain other situations in which ranged read is not desired. In addition, the segmentation method used by a particular system may be configurable by an administrative user or other authorized user. Thus, the segmentation method can be changed on the fly on a running system between any of the segmentation techniques discussed above, and / or the parameters of segmentation techniques may also be changed, such as the number of segments, etc.

[0015] Additionally, in some cases, very large data objects may be divided into multiple parts by a client prior to sending to the system gateway or other system computing device for storage. Thus, when a data object is received as a multi-part upload, the large parts of the data object may be treated similarly to large data objects, and each part may be segmented using the applicable data segmentation technique. The segments of the multi-part upload are tracked and managed using metadata similar to tracking of the segments of a large data object. Accordingly, after the parts are divided into segments, the data segments may be tracked only as segments regardless of the original part differentiation for the data object.

[0016] The segmentation techniques described herein include providing users with the security of encrypted and erasure-coded data as well as the added efficiency of enabling ranged read ofthe encrypted and erasure-coded data. Further, the flexibility of enabling a user to select a segmentation method allows for optimizing various different use cases that may arise. Additionally, if ranged read is not a concern, the system also provides the option of encrypting the entire object or object part without dividing into segments, which can improve system efficiency if range read will not be used.

[0017] Some examples herein may employ erasure coding as a data protection technique to, at least in part, provide redundancy protection for the stored data. Erasure coding may include receiving a data object, calculating parity data for the data of the data object, and storing the data and the parity data so that, if a portion of the stored data is lost, then the lost data can be recovered using the remaining object data and the parity data by recalculating the lost portion of the data from the parity data. Thus, erasure coding may be used by the object storage system herein as a technique for protecting data against loss. Further because the parity data can consume less storage space than the data of the data object, erasure coding is able to protect against data loss while using storage capacity more efficiently as compared to keeping additional complete copies of the original data object.

[0018] The storage systems herein may use any of a variety of storage media for physically storing the data. The storage systems herein may also store a variety of different types of data with different requirements as to the level of protection, speed of access, encryption techniques, and other parameters. As one example, an object storage system herein may include physical storage that may be spread across multiple storage nodes, multiple racks, multiple arrays, multiple drives, and so forth, such as at a commercial or private data center, a commercial or private cluster of computing devices, or the like. In some examples, some of the storage may include network storage (e.g., cloud storage) provided by one or more commercial cloud-based storage providers, such as AMAZON®, MICROSOFT®, IBM®, GOOGLE®, HITACHI V ANT ARA®, or the like, such as may be provided at data centers operated by these entities.

[0019] For discussion purposes, some example implementations are described in the environment of one or more service computing devices in communication with one or more storage systems having storage components for storing data, such as according to specified data protection levels or the like. However, implementations herein are not limited to the particular examples provided, and may be extended to other types of computing system architectures, other types of storage components, other types of client configurations, other types of data, and so forth, as will be apparent to those of skill in the art in light of the disclosure herein.

[0020] FIG. 1 illustrates an example architecture of a system 100 configured to store data while enabling a ranged read to be performed on the stored data according to some implementations. The system 100 may serve as an object storage system including one or more service computing devices 102 that are able to communicate through one or more networks 106 with one or more storage computing devices 104, such as a first storage computing device 104(1) through an Nth storage computing device 104(N). Further, the service computing device(s) 102 may be able to communicate through the one or more networks 106 with one or more client computing devices108. For example, the service computing device(s) 102 may include or may operate as a gateway node, an access node, a server node, a management node, and / or any of various other types of service computing devices that provide the client device(s) 108 with access to the storage provided by the storage computing devices 104 for enabling the client devices 108 to store and access data stored by the storage computing devices 104 on a plurality of storage components109, as well as performing other management and control functions, as discussed additionally below.

[0021] The storage computing devices 104(l)-104(N) may each manage and / or access the plurality of storage components 109(1) through 109(M), which may include storage devices, storage arrays, storage racks, storage nodes, or the like. As one example, each storage component 109 may include at least one separate storage drive, or, as another example, a separate group of storage drives, such as a separate storage array, that may be contained, for example, in a rack in a data center. As yet another example, each storage component 109 may include a plurality of storage drives and also a storage computing device 104 that is configured to access the plurality of storage drives. In this scenario, each storage component 109 includes a separate storage computing device 104 that is separate from the other storage computing devices 104 corresponding to other storage arrays. Additionally, while several examples of storage components 109 have been described herein, numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0022] In addition, one or more administrative computing devices 110 may be able to communicate with the storage computing devices 104 and / or the service computing device(s) 102, such as through the one or more networks 106. For example, an administrative user 114 may use the administrative computing device 110 to control the segmentation method that is employed by the service computing device(s) 102 when segmenting a data object that is received for storage on the storage components 109. As one example, the administrative user 114 may use an administrative application 116 executed on the administrative computingdevice 110 to manage the segmentation of data objects for enabling ranged read, as well as performing other management operations in the system 100.

[0023] In some examples, the service computing device(s) 102 may include one or more servers that may be embodied in any number of ways. For instance, the programs, other functional components, and at least a portion of data storage of the service computing device(s) 102 may be implemented on at least one server, such as in a cluster of servers, a server farm, a data center, a cloud-hosted computing service, and so forth, although other computer architectures may additionally or alternatively be used. As another example, the service computing device(s) 102 may be abstracted as, or otherwise treated as, a single node that is actually a cluster containing multiple service computing devices 102. Additional details of the service computing device(s) 102 are discussed below with respect to FIG. 5.

[0024] The one or more networks 106 may include any suitable network, including a wide area network, such as the Internet; a local area network (LAN), such as an intranet; a wireless network, such as a cellular network, a local wireless network, such as Wi-Fi, and / or short-range wireless communications, such as BLUETOOTH®; a wired network including Fibre Channel, fiber optics, Ethernet, or any other such network, a direct wired connection, or any combination thereof. Accordingly, the one or more networks 106 may include both wired and / or wireless communication technologies. Components used for such communications can depend at least in part upon the type of network, the environment selected, or both. Protocols for communicating over such networks are well known and will not be discussed herein in detail. As one example, such as in the case that the service computing device(s) 102 is co-located with some or all of the storage computing devices 104(1 )- 104(N), such as in the same data center or the like, the network(s) 106 connecting the service computing device(s) 102 with the storage computing devices 104(1)- 104(N) may include a private network, such as a LAN, storage area network (SAN), Fibre Channel network, and so forth. Additionally, the network(s) 106 connecting the service computing device(s) 102 with the client device(s) 108 and, in some cases, some of the storage computing devices 104 may be a public network that may include the Internet, or a combination of public and private networks. Implementations herein are not limited to any particular type of network as the one or more networks 106.

[0025] The service computing device(s) 102 may be configured to provide storage and data management services to the client device(s) 108. In some examples, the client devices 108 may be server-type computing devices or other high-performance computing devices that execute a client application 118 that stores and / or retrieves data, such as data objects 120, or the like onthe storage computing devices 104. For example, the client application 118 may be an enterprise level application that has high input and output requirements for storing and / or retrieving data. Alternatively, in other examples, the client devices 108 may be any type of user computing device such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, and / or any other type of computing device able to send data over a network. In some cases, human users (not shown in FIG. 1) may be associated with client devices 108 such as through a respective user account, user login credentials, or the like. Furthermore, the client device(s) 108 may be configured to communicate with the service computing device(s) 102 through the one or more networks 106, through separate networks, or through any other suitable type of communication connection. Numerous other variations will be apparent to those of skill in the art having the benefit of the disclosure herein.

[0026] The client application 118 executable on the client device 108 may communicate with a storage management program 122 executable on the service computing device(s) 102, such as for sending a data object 120, from the client device 108 to the service computing device(s) 102 for storage on the storage components 109. Additionally, the client application 118 may request delivery of a stored data object from the storage components 109 through a data instruction sent to the storage management program 122. Examples of data instructions that the client device 108 may send to the service computing device(s) 102 may include write operations, read operations, delete operations, and the like (not shown in FIG. 1). The client application 118 may include any type of application having communication functionality enabling communication over the one or more networks 106 with the storage management program 122 or other application(s) on the service computing device(s) 102. Accordingly, the service computing device(s) 102 may provide storage services for the client device(s) 108, such as by acting as a storage gateway, or the like, as mentioned above.

[0027] The administrative device 110 may be any suitable type of computing device such as a desktop, laptop, tablet computing device, mobile device, smart phone, wearable device, terminal, and / or any other type of computing device able to send data and communications over a network. The administrative user 114 may be associated with the administrative device 110, such as through a respective administrative account, administrative login credentials, or the like. Furthermore, the administrative device 110 may be able to communicate with the service computing device(s) 102 and / or the storage computing devices 104 through the one or more networks 106, through separate networks, through a direct connection, or through any other suitable type of communication connection.

[0028] Each administrative device 110 may include a respective instance of the administrative application 116 that may execute on the respective administrative device 110, such as for communicating with the storage management program 122 on the service computing device(s) 102 and / or for communicating with a storage program 126 executable on the storage computing devices 104. For example, the administrative user 114 may communicate with the storage program 126, such as for assigning storage components 109 to particular storage computing devices 104 or for performing any of various other management functions. In some cases, the administrative application 116 may include a browser or may operate through a browser, while in other cases, the administrative application 116 may include any other type of application having communication functionality enabling communication with the storage management program 122, the storage program 126, or other applications on the service computing device(s) 102 or the storage computing devices 104.

[0029] Each storage computing device 104(l)-104(N) may execute an instance of the storage program 126, respectively, for managing storage of respective data objects, or portions thereof, to the respective storage components 109(l)-109(M). For instance, the storage program 126 may receive a data object 120 from the storage management program 122 at the service computing device(s) 102, may store the data object 120 on one or more of the storage components 109 in accordance with a storage class specified for the data object 120, and may retrieve and send requested data to the storage management program 122, such as in response to a client read request (not shown in FIG. 1), or the like. The storage program 126 may also provide information to the storage management program 122, such as performance information, capacity usage information, and so forth for the respective storage components 109 managed by a particular instance of the storage program 126 executed on a respective storage computing device 104.

[0030] The service computing device(s) 102 may execute the storage management program 122, which may provide access to the storage computing devices 104(1 )- 104(N) and the storage components 109. The storage management program 122 may include segmentation and ranged read logic 127 that may be executed for segmenting received data objects according to a specified segmentation method. Further, the storage management program 122 may include encryption logic 128 that may be executed for encrypting the data object 120, such as segments of the data object 120 when segmentation is performed. Additionally, the storage management program 122 may include erasure coding logic 129 that may be executed for performing erasure coding, such as by further dividing object segments and calculating parity portions for dataportions. Erasure coding techniques are well-known in the art and details of encoding and decoding particular erasure coded parity blocks are therefore not described herein.

[0031] As one example, the service computing device 102 may receive a data object 120 from a client device 108 for storing on the storage components 109. In some examples, a storage class for a data object may be specified by the client user and / or by the client application 118. For instance, the storage class may specify at least a level of protection for the data object, e.g., an erasure coding scheme, a number of copies to be stored, no copies to be stored, an encryption scheme, or the like.

[0032] Upon receipt of the data object 120, the storage management program 122 may perform preprocessing of the data object 120 prior to sending the data object, or portions thereof, to the storage computing device(s) 104 for storage. In this example, suppose that the storage class indicates that just a single copy of the data object 120 should be stored in encrypted form. Accordingly, the storage management program 122 may segment the data object 120 into a plurality of data segments based on an applicable segmentation method. In this example, suppose that the data objects received from the client device 108 are specified to be segmented into four segments so long as the size of the data object is greater than 1 MB. Further, suppose that the data object in this example is greater than 1 MB. Accordingly, the segmentation and ranged read logic 127 may be executed to divide the data object into four segments.Additionally, the encryption logic 128 may then be executed to encrypt the four segments to provide four encrypted data segments 130.

[0033] Following encryption, the storage management program 122 may send the encrypted data segments 130 to one or more of the storage computing devices 104, which may store the encrypted data segments to the storage components 109. In this example, suppose a first encrypted data segment 130(1) is stored in the second storage component 109(2); a second encrypted data segment 130(2) is stored in the third storage component 109(3); a third encrypted data segment 130(3) is stored in the fourth storage component 109(4); and a fourth encrypted data segment 130(4) is stored in the first storage component 109(1).

[0034] Data objects typically include content data and metadata. For instance, the content data is the data that provides the substance of the data object, while the metadata describes various characteristics of the data object, such as data type, size of the data object, owner, author, date of creation, date last modified, storage location or path, and so forth. The storage management program 122 may store metadata 134 for the stored data object 120 in a metadata data structure 136, which may include the stored locations of the encrypted data segments 130( 1)- 130(4) andother information related to the data object 120. For example, the segmentation and ranged read logic 127 may be executed to determine metadata that includes the range of content data included within each data segment 130 of the plurality of data segments, and may record this information in the metadata data structure 136 with other metadata related to the received data object 120.

[0035] As one example, such as in the case that the data object 120 is a media item, such as a movie or audio recording, the metadata for each data segment may include a time range of the content. As one simplified example, in the case that the data object 120 is a two hour movie, suppose that the first half hour of the movie is included in the first encrypted data segment 130(1), the second half hour is included in the second encrypted data segment 130(2), the third half hour is included in the third encrypted data segment 130(3), and the fourth half hour is included in the fourth encrypted data segment 130(4). Accordingly, if a user requests to retrieve the first 15 minutes of the movie, the segmentation ranged read logic 127 may determine from the metadata database 136 that the first 15 minutes is contained in the first encrypted data segment 130(1), and may retrieve only the first encrypted data segment 130(1) for responding to the request, rather than retrieving the entire data object 120.

[0036] Furthermore, while the storage management program 122 is described as dividing the data object 120 into data segments and encrypting the data segments 130, in other examples, some or all of these functions may be performed by one or more of the storage programs 126 on one or more of the storage computing devices 104.

[0037] FIGS. 2 and 4 include flow diagrams illustrating example processes for storing data to enable ranged read according to some implementations. The processes are illustrated as collections of blocks in logical flow diagrams, which represents a sequence of operations, some or all of which may be implemented in hardware, software or a combination thereof. In the context of software, the blocks may represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, program the processors to perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types. The order in which the blocks are described should not be construed as a limitation. Any number of the described blocks can be combined in any order and / or in parallel to implement the process, or alternative processes, and not all of the blocks need be executed. For discussion purposes, the processes are described with reference to the environments, frameworks, and systems described in the examples herein,although the processes may be implemented in a wide variety of other environments, frameworks, and systems.

[0038] FIG. 2 illustrates an example process 200 performed for storing a data object using encryption and erasure coding, while also enabling a ranged read according to some implementations. In the example of FIG. 2, suppose that the service computing device 102 discussed above with respect to FIG. 1 receives a data object 201, and an indication that the data object 201 should be stored according to a data storage class that includes a data protection level with 2+1 erasure coding and encryption of the stored data object 201. Further, suppose that the data of the data object is represented by letters “ABCDEFGHIJKL” for convenience of discussion.

[0039] At 202, the service computing device 102 may execute the storage management program 122 to receive the data object 201 for storage. In this example, suppose that the segmentation method indicates that the data object 201 is to be divided into three data segments of approximately equal size.

[0040] At 204, upon receipt of the data object 201, the storage management program 122 may execute the segmentation and ranged read logic 127 to determine the applicable segmentation method and to perform segmentation of the received data object 201 according to the applicable segmentation method. As mentioned above, in this example, suppose that the segmentation method indicates that the data object 201 is to be divided into three data segments, i.e., segment 205(1), including the data ABCD, segment 205(2), including the data EFGH, and segment 205(3), including the data IJKL.

[0041] At 206, following segmentation of the data object, the storage management program 122 may execute the encryption logic 128 to encrypt the data segments 205(1) through 205(3) to generate encrypted data segments 207(1) through 207(3), respectively, including encrypted data.

[0042] At 208, following generation of the encrypted data segments 207(1) through 207(3), the storage management program 122 executes the erasure coding logic 129 to divide the encrypted data segments 207 into “shards”, referred to hereinafter as “erasure coded (EC) data portions” 209. In this example, suppose that the storage management program 122 generates six EC data portions 209(1) through 209(6) from the three encrypted data segments 207(1) through 207(3).

[0043] At 210, the storage management program 122 may calculate erasure coded (EC) parity portions 211 for the EC data portions 209. For instance, the storage management program 122 may calculate a first EC parity portion 211(1) based on the first two EC data portions 209(1) and 209(2), may calculate a second EC parity portion 211(2) from the next two EC data portions209(3) and 209(4), and may calculate a third EC parity portion 211(3) from the last two EC data portions 209(5) and 209(6).

[0044] At 212, the storage management program 122 may store the EC data portions and the EC parity portions according to the erasure coding scheme specified for the desired level of data protection. Details of the storage are discussed additionally below with respect to FIG. 3.

[0045] At 214, the storage management program may store metadata for the segments and other data object metadata to the metadata data structure 136 discussed above with respect to FIG. 1. As one example, suppose that the data object 201 is a database such as a large table or the like including rows and columns of data. Accordingly, the segment metadata for each segment 205(1) through 205(3) may indicate which columns and rows of the table are included in which of the individual segments 205(1) through 205(3).

[0046] FIG. 3 illustrates an example 300 of storage of encrypted and erasure-coded data portions in the system 100 according to some implementations. For instance, as discussed above with respect to FIG. 2, at 212, the storage management program 122 may cause the EC data portions 209 and the EC parity portions 211 to be stored to the storage components 109 according to the erasure coding scheme specified for the desired data protection level for the data object 201. For instance, in this example, the EC data portion 209(1), the second parity portion 211(2), and the EC data portion 209(5) are stored to the first storage component 109(1), the EC data portion 209(2), the EC data portion 209(3) and the third parity portion 211(3) are stored to the second storage component 109(2), and the first parity portion 211(1), the EC data portion 209(4), and the EC data portion 209(6) are stored to the third storage component 109(1). According to this erasure coding scheme, any single one of the storage components 109(1), 109(2), or 109(3) could fail, and the data for the data object 201 may be reconstructed from the data stored on the other two storage components 109(1), 109(2) or 109(3) that did not fail.

[0047] In addition, the metadata database 136 may be updated to include information about each of the EC data portions 209 and the EC parity portions 211, so the storage locations and corresponding data segments 205 of the EC data portions and EC parity portions 209 and 211 can be determined in case a range read is desired to be performed on a particular data segment.

[0048] FIG. 4 is a flow diagram illustrating an example process 400 for performing a ranged read on encrypted and erasure-coded data according to some implementations. In the example of FIG. 4, the process 400 may be performed at least in part by the storage management program 122 executing the segmentation and ranged read logic 127 on the service computing device 102. Alternatively, in other examples, the process 400 may be executed at least in part bythe storage program 126 executing on a storage computing device 104, and / or by another computing device executing the segmentation and ranged read logic 127.

[0049] At 402, the storage management program 122 may receive a read request for a specified data range of a data object stored by the system 100. As one example, the request may be received from the client computing device 108 or other user computing device.

[0050] At 404, the storage management program 122 may access the metadata data structure 136 to determine a segment of the data object containing the specified data range. For example, the metadata stored in the metadata data structure may indicate the range of content data of the data object stored in each data segment.

[0051] At 406, the storage management program 122 may determine one or more encrypted and erasure-coded data portions corresponding to the segment of the data object that includes the content data corresponding to the specified data range.

[0052] At 408, the storage management program 122 may determine the storage location(s) of the one or more encrypted and erasure-coded data portions. For example, the storage management program 122 may determine one or more storage computing devices 104 to which to send a request to retrieve the determined encrypted and erasure-coded data portions corresponding to the identified data segment.

[0053] At 410, the storage management program 122 may retrieve the one or more encrypted and erasure-coded data portions from the identified storage location(s).

[0054] At 412, the storage management program 122 may decrypt the one or more encrypted and erasure-coded data portions to obtain the decrypted data segment.

[0055] At 414, the storage management program 122 may extract the specified data range from the decrypted data segment.

[0056] At 416, the storage management program 122 may return, to the requester, the extracted data corresponding to the specified data range.

[0057] FIG. 5 illustrates select example components of the service computing device(s) 102 that may be used to implement at least some of the functionality of the systems described herein. The service computing device(s) 102 may include one or more servers or other types of computing devices that may be embodied in any number of ways. For instance, in the case of a server, the programs, other functional components, and data may be implemented on a single server, a cluster of servers, a server farm or data center, a cloud-hosted computing service, and so forth, although other computer architectures may additionally or alternatively be used. Multiple service computing device(s) 102 may be located together or separately, and organized, forexample, as virtual servers, server banks, and / or server farms. The described functionality may be provided by the servers of a single entity or enterprise, or may be provided by the servers and / or services of multiple different entities or enterprises.

[0058] In the illustrated example, the service computing device 102 includes, or may have associated therewith, one or more processors 502, one or more computer-readable media 504, and one or more communication interfaces 506. Each processor 502 may be a single processing unit or a number of processing units, and may include single or multiple computing units, or multiple processing cores. The processor(s) 502 can be implemented as one or more central processing units, microprocessors, microcomputers, microcontrollers, digital signal processors, system-on-chip processors, graphics processing units, artificial intelligence processing units, embedded processors, state machines, logic circuitries, and / or any devices that manipulate signals based on operational instructions. As one example, the processor(s) 502 may include one or more hardware processors and / or logic circuits of any suitable type specifically programmed or configured to execute the algorithms and processes described herein. The processor(s) 502 may be configured to fetch and execute computer-readable instructions stored in the computer- readable media 504, which may program the processor(s) 502 to perform the functions described herein.

[0059] The computer-readable media 504 may include volatile and nonvolatile memory and / or removable and non-removable media implemented in any type of technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. For example, the computer-readable media 504 may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape, magnetic disk storage, storage arrays, directly attached storage, network attached storage, storage area networks, cloud storage, combinations thereof, or any other medium that can be used to store the desired information and that can be accessed by a computing device. Depending on the configuration of the service computing device(s) 102, the computer-readable media 504 may be a tangible non-transitory medium to the extent that, when mentioned, non- transitory computer-readable media exclude media such as energy, carrier signals, electromagnetic waves, and / or signals per se. In some cases, the computer-readable media 504 may be at the same location as the service computing device 102, while in other examples, the computer-readable media 504 may be partially remote from the service computing device 102. For instance, in some cases, the computer-readable media 504 may include a portion of storage managed by the storage computing device(s) 104 discussed above with respect to FIG. 1.

[0060] The computer-readable media 504 may be used to store any number of functional components that are executable by the processor(s) 502. In many implementations, these functional components comprise instructions or programs that are executable by the processor(s) 502 and that, when executed, specifically program the processor(s) 502 to perform the actions attributed herein to the service computing device 102. Functional components stored in the computer-readable media 504 may include the storage management program 122, which may include one or more computer programs, applications, executable code, or portions thereof. For example, the storage management program 122 may include segmentation and ranged read logic 127, encryption logic 128, and erasure coding logic 129.

[0061] In addition, the computer-readable media 504 may store data, data structures, and other information used for performing the functions and services described herein. For example, the computer-readable media 504 may store the metadata data structure 136 containing the metadata 134 that may be maintained by the service computing device 102 such as for retrieving stored data segments of data objects from one or more of the storage computing devices 104, such as in response to a request from the client application or the like.

[0062] Further, while these data structures and programs are illustrated together in this example, during use, some or all of these data structures and / or programs may be stored on separate service computing device(s) 102. The service computing device 102 may also include or maintain other functional components and data, which may include programs, drivers, etc., and the data used or generated by the functional components. Further, the service computing device 102 may include many other logical, programmatic, and physical components, of which those described above are merely examples that are related to the discussion herein.

[0063] The one or more communication interfaces 506 may include one or more software and / or hardware components for enabling communication with various other devices, such as over the one or more network(s) 106. For example, the communication interface(s) 506 may enable communication through one or more of a LAN, the Internet, cable networks, cellular networks, wireless networks (e.g., Wi-Fi) and wired networks (e.g., Fibre Channel, fiber optic, Ethernet), direct connections, as well as close-range communications such as BLUETOOTH®, and the like, as additionally enumerated elsewhere herein.

[0064] FIG. 6 illustrates select components of an example configuration of a storage computing device 104 according to some implementations. The storage computing device 104 may include one or more servers or any other suitable computing device, such as any of the examples discussed above with respect to the service computing device 102. The storage computingdevice(s) 104 may each include one or more processors 604, one or more computer-readable media 606, and one or more communication interfaces 608. For example, the processor(s) 604 may correspond to any of the examples discussed above with respect to the processors 502, the computer-readable media 606 may correspond to any of the examples discussed above with respect to the computer-readable media 504, and the communication interface(s) 608 may correspond to any of the examples discussed above with respect to the communication interfaces 506.

[0065] In addition, the computer-readable media 606 may store the storage program 126, discussed above, as a functional component executed by the one or more processors 604 for managing the storage of segmented and encrypted data 609 (which also may be erasure coded in some examples) on a storage 610, or a portion thereof that may be associated with the storage computing device 104.

[0066] In addition, the computer-readable media 606 may store data and data structures used by the storage program 126 (not shown in FIG. 6). The storage 610 may include one or more controllers 612 associated with the storage 610 for storing the segmented and encrypted data 609, such as may correspond to data objects, or portions of the data objects, on one or more drives, arrays, trays, racks, extent groups, storage nodes, or other types of configurations of storage components 109. For instance, the controller 612 may control one or more storage components 109, such as the first storage component 109(1), the second storage component 109(2), and so forth. Further, the controller 612 may be configured to store the segmented and encrypted data 609 with various levels of protection, such as in various erasure-coded protection configurations, or any of various other configurations, such as a RAID configuration, JBOD configuration, or the like, and / or for presenting storage extents, logical units, logical volumes, or the like, based on the storage components 109, to the storage program 126, and for managing the segmented and encrypted data 609 stored on the underlying physical storage devices. The storage components 109 may include any type of storage device, such as solid state drives, hard disk drives, optical drives, magnetic tape, combinations thereof, and so forth, as additionally listed elsewhere herein.

[0067] In some examples, the client device(s) 108 and / or the administrator device(s) 110 may have a hardware configuration and hardware elements similar to those discussed above for the service computing device 102. However, the client device(s) 108 and / or the administrator device(s) 110 may have functional components and data that are different from those of the service computing device(s) 102.

[0068] The example processes described herein are only examples of processes provided for discussion purposes. Numerous other variations will be apparent to those of skill in the art in light of the disclosure herein. Further, while the disclosure herein sets forth several examples of suitable frameworks, architectures and environments for executing the processes, implementations herein are not limited to the particular examples shown and discussed. Furthermore, this disclosure provides various example implementations, as described and as illustrated in the drawings. However, this disclosure is not limited to the implementations described and illustrated herein, but can extend to other implementations, as would be known or as would become known to those skilled in the art.

[0069] Various instructions, processes, and techniques described herein may be considered in the general context of computer-executable instructions, such as programs, applications, executable code, program modules, and the like, stored on computer-readable media, and executed by the processor(s) herein. Generally, programs, applications, executable code, program modules, include routines, executable computer programs, programming objects, executable components, data structures, and so forth, for performing particular tasks or implementing particular abstract data types. These programs, applications, executable code, program modules, and the like, may be executed as native code or may be downloaded and executed, such as in a virtual machine or other just-in-time compilation execution environment. Typically, the functionality of the programs, applications, executable code, program modules, and the like may be combined or distributed as desired in various implementations. An implementation of these programs, applications, executable code, program modules, and the like may be stored on computer storage media and / or transmitted across some form of communication media.

[0070] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

CLAIMS1. A method comprising: receiving, by a first computing device, from a second computing device, a data object for storage; dividing the data object into a plurality of data segments; encrypting the data segments; storing the plurality of encrypted data segments to one or more storage components; and storing metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object to enable ranged read of a portion of the content data of the data object.

2. The method as recited in claim 1, further comprising: receiving a request to read the portion for performing a ranged read of the content data of the data object; determining, from the metadata, a data segment including the requested portion of the content data of the data object; decrypting the data segment determined from the metadata; and sending at least the requested portion of the content data obtained from the decrypted data segment in response to the request.

3. The method as recited in claim 2, further comprising: dividing each data segment of the plurality of data segments into a plurality of erasure- coded data portions following encryption of the plurality of data segments; generating at least one erasure-coded parity portion from the plurality of erasure-coded data portions; and storing the plurality of encrypted data segments to the one or more storage components by storing the erasure coded data portions according to a data protection scheme to protect at least in part against failure of a storage component.

4. The method as recited in claim 3, wherein: storing the metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object includes storing an indication of which erasure-coded data portions correspond to which data segments.

5. The method as recited in claim 3, further comprising retrieving multiple erasure coded data portions that make up the data segment from multiple different storage components.

6. The method as recited in claim 1, wherein dividing the data object into a plurality of data segments comprises: determining that a size of the data object exceeds a threshold size for performing segmenting of the data object.

7. The method as recited in claim 1, wherein dividing the data object into a plurality of data segments comprises: determining that an applicable segmentation method for the data object includes dividing the content data of the data object into a plurality of blocks of a predetermined fixed size.

8. The method as recited in claim 1, wherein dividing the data object into a plurality of data segments comprises: determining that an applicable segmentation method for the data object includes dividing the content data of the data object into a predetermined number of blocks.

9. The method as recited in claim 1, wherein a user is able to switch between a first segmentation method for dividing the data object into a plurality of data segments of a fixed sized, and a second segmentation method for dividing the data object into a plurality of data segments of a predetermined number.

10. A system comprising: a first computing device configured by executable instructions to perform operations comprising: receiving, from a second computing device, a data object for storage; dividing the data object into a plurality of data segments;encrypting the data segments; storing the plurality of encrypted data segments to one or more storage components; and storing metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object to enable ranged read of a portion of the content data of the data object.

11. The system as recited in claim 10, the operations further comprising: receiving a request to read the portion for performing a ranged read of the content data of the data object; determining, from the metadata, a data segment including the requested portion of the content data of the data object; decrypting the data segment determined from the metadata; and sending at least the requested portion of the content data obtained from the decrypted data segment in response to the request.

12. The system as recited in claim 11, the operations further comprising: dividing each data segment of the plurality of data segments into a plurality of erasure- coded data portions following encryption of the plurality of data segments; generating at least one erasure-coded parity portion from the plurality of erasure-coded data portions; and storing the plurality of encrypted data segments to the one or more storage components by storing the erasure coded data portions according to a data protection scheme to protect at least in part against failure of a storage component.

13. A non- transitory computer-readable medium storing instructions executable by one or more processors to cause the one or more processors to perform operations comprising: receiving, from a computing device, a data object for storage; dividing the data object into a plurality of data segments; encrypting the data segments; storing the plurality of encrypted data segments to one or more storage components; and storing metadata indicating which data segments of the plurality of data segments include which parts of content data of the data object to enable ranged read of a portion of the content data of the data object.

14. The non- transitory computer-readable medium as recited in claim 13, the operations further comprising: receiving a request to read the portion for performing a ranged read of the content data of the data object; determining, from the metadata, a data segment including the requested portion of the content data of the data object; decrypting the data segment determined from the metadata; and sending at least the requested portion of the content data obtained from the decrypted data segment in response to the request.

15. The non- transitory computer-readable medium as recited in claim 14, the operations further comprising: dividing each data segment of the plurality of data segments into a plurality of erasure- coded data portions following encryption of the plurality of data segments; generating at least one erasure-coded parity portion from the plurality of erasure-coded data portions; and storing the plurality of encrypted data segments to the one or more storage components by storing the erasure coded data portions according to a data protection scheme to protect at least in part against failure of a storage component.

Images