Content-secured object storage for backup and retention

Abstract

Systems, devices and methods for managing the lifecycle of objects in an object repository. A distributed ledger is used to record transactions between clients and object repositories. The distributed ledger records transactions and also proves object authenticity. Thus, transactions can be verified and can assist in resolving issues that arise with stored objects.

Description

Technical Field

[0001] Embodiments of the present invention relate to systems and methods for protecting data and performing data protection operations. More specifically, embodiments of the present invention relate to systems and methods for storing objects with confirmed content for storage and retention. Background Art

[0002] Ledger-based distributed technologies are widely used for various reasons. However, ledger-based technologies still have problems. For example, one of the problems with storing objects (e.g., data, files, content) in cloud systems is that despite the use of a traditional ledger, there is no guarantee that the objects have not been tampered with. Although signatures and error correction codes can be used to detect corrupted objects, the cloud provider may claim that the corruption of the object is not the cloud provider's fault but that the user stored the object in a corrupted form.

[0003] More specifically, there is no guarantee that an object stored or maintained at an untrusted cloud storage will be returned in exactly the same form. In other words, there is no guarantee that an object read from a cloud system will be exactly the same as an object written to the cloud system. For example, a user may store an object in a cloud system. When the same user later reads the object, a different object may be returned from the cloud system. The user cannot verify that the data returned from the cloud is different from the object originally stored in the cloud. The user cannot verify that the object was corrupted in the cloud, and the cloud cannot verify that the object was not corrupted in the cloud.

[0004] In other words, whenever there is a problem with an object that has been uploaded and stored in a cloud system, it is very difficult to determine how and when the problem occurred. As mentioned earlier, a user may claim that the object was corrupted by the cloud system, while the cloud system may claim that the user stored the corrupted object.

[0005] Similar issues may arise as to whether an object has been deleted or not. For example, a user may request an object, only to find that it no longer exists. In this case, the user cannot prove that a deletion order was not issued. Furthermore, the cloud is not immune to users who delete an object but later request it. In another example, the user cannot guarantee that the cloud system will comply with a deletion order. This is particularly relevant to some regulations (such as the GDPR's "Right to be Forgotten", DMCA (Digital Millennium Copyright Act) takedowns), which require certain objects or data to be destroyed. The user may have taken steps to destroy the object, but the cloud system did not actually destroy it - leaving the responsibility to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order to describe the manner in which at least some aspects of the present application can be obtained, a more specific description will be given with reference to the specific embodiments of the present application shown in the accompanying drawings. It should be understood that these drawings only illustrate exemplary embodiments of the present invention and are therefore not to be considered as limiting the scope of the present invention. The embodiments of the present invention will be described and explained with additional particularity and detail through the use of the accompanying drawings, in which:

[0007] Figure 1 An example of a system including a distributed ledger for storing objects and securing content for backup and retention is shown;

[0008] Figure 2 An example of a distributed ledger that records transactions and certifies objects related to the recorded transactions is shown;

[0009] Figure 3 An example of a method for writing an object into an object store using a distributed ledger is shown;

[0010] Figure 4 An example of a method for reading an object from an object repository using a distributed ledger is shown; and

[0011] Figure 5 An example of a method for deleting an object from an object repository using a distributed ledger is shown. DETAILED DESCRIPTION

[0012] Embodiments of the present invention relate to systems and methods for storing objects in a repository such as a cloud object repository (e.g., AWS S3). A cloud object repository may be referred to as a cloud or cloud system. A data center or distributed data center including hardware for storing objects may be an example of a cloud object repository. By way of example only, a cloud object repository may be a public cloud, a private cloud, or a hybrid cloud.

[0013] Embodiments of the present invention also relate to storing objects in a manner that ensures that objects written to a cloud object repository (referred to herein as a cloud or cloud storage) can be returned as originally written or can be deleted according to a delete command. If this cannot be ensured, embodiments of the present invention enable the source of the problem with the object to be determined and allow appropriate remedial measures to be provided. In one example, if the object is not returned as originally written (e.g., a damaged or partially damaged state) or if the object is not deleted according to a delete command, embodiments of the present invention enable the transaction to be verified and the error to be determined. As an example, this is achieved using smart contracts and a general ledger. Therefore, as an example, embodiments of the present invention allow the cloud object repository to be used as a backup or for backup purposes and allow objects to be retained and managed in the cloud object repository.

[0014] Embodiments of the present invention also relate to a distributed ledger that manages the lifecycle or state of objects stored in a cloud object repository. A ledger or distributed ledger may be implemented as a database that allows transactions to be recorded or stored. A distributed ledger may be or include data that is replicated, shared, and / or synchronized between multiple sites. In some examples, a distributed ledger may have no central manager.

[0015] According to an embodiment of the present invention, the distributed ledger proves the object itself in addition to recording transactions or proves the object itself together with the recorded transactions. Therefore, the general ledger may include an identifier (e.g., a hash) that allows the integrity of the object to be verified in addition to indicating that object Y is stored in the cloud object repository. Therefore, the general ledger proves the object itself and proves the content of the object. In addition, smart contracts can be used to form entries in the general ledger. Smart contracts are agreements that allow the negotiation or performance of a contract to be digitally promoted, verified, or implemented. Smart contracts may involve the use of private keys / public keys. For example, entries formed in the general ledger using a private key can be confirmed using a corresponding public key. This allows confirmation of the user's identification and allows the contract to be valid in a computing context. Using smart contracts, entries in the distributed ledger can become valid and enforceable.

[0016] Embodiments of the invention allow transactions to be recorded and also allow the integrity of the object to be ensured, at least initially. If the object is subsequently damaged or does not exist or if a command related to the object is not executed or is executed without authorization, the distributed ledger allows the client and the cloud to determine who is responsible.

[0017] When storing an object (e.g., writing an object to a cloud object repository), for example, a client (or user) may add information that the object has been written to the object repository to the distributed ledger. The client may also specify or provide a signature for the object (e.g., a hash signature or fingerprint that uniquely identifies the object). The client may also digitally sign the fingerprint and / or object to facilitate smart contracts. The client or user may also provide or define a retention policy for the object (how long the object must remain available) and availability requirements (e.g., 99.999%). These may form part of a contract or agreement reached regarding the object.

[0018] When the object is stored in the cloud object store, the cloud system (or cloud provider) is notified of the ledger transaction. In response, the cloud object store confirms that the object has been received and that the signature of the object recorded in the ledger is correct. For example, the cloud store may also perform a hash (the same hash performed by the client) to verify that the object received is the object that the client claims to have uploaded. The cloud may also agree to the retention requirements and availability requirements (and / or other requirements). Of course, the cloud store may also ignore or deny the request or notification of the ledger transaction. This allows the client to know that the cloud system received the correct object and not a corrupted object, and allows the cloud system to verify what the client actually uploaded.

[0019] If the cloud storage accepts the transaction, storing the object in the cloud object repository ensures that the conditions recorded in the transaction and the smart contract can be formed. The cloud provider may even agree to pay a penalty or provide other compensation or remedies if the object is unavailable due to availability requirements or if the object is subsequently found to be damaged. In one example, the object can be encrypted using a key known only to the client or user. When accepting the transaction, the cloud provider can check the hash of the encrypted object. The hash of the encrypted object can be used to verify whether the encrypted object has been damaged in the cloud. A similar process can also be performed for objects that are not encrypted. In other words, embodiments of the present invention can be performed regardless of whether the object is encrypted.

[0020] When reading an object from cloud storage, the client can read the object from the cloud object repository and the associated transaction from the ledger, which can also specify a hashing method. The client can calculate or determine the hash of the object being read to determine whether the object being read is corrupted or different from the object reflected in the ledger. If the object does not exist or if the client determines that the retrieved object is different from the object written to the cloud storage, the client or user can receive compensation or other remedies.

[0021] In one example, a trusted third party may serve as an arbitrator. The third party may be allowed access to the user's or client's cloud object repository and ledger. An application programming interface (API) may be used to immediately read objects from the cloud object repository and to check if the object is indeed lost or damaged. If the third-party service agrees that a service level agreement (SLA) for the object is not met, the cloud provider may be required to provide a predetermined remedy in accordance with the agreement.

[0022] When deleting an object, the client (or user) can request the cloud storage to delete the object by placing a delete request in the general ledger along with the object's identifier or hash. The client can simultaneously request the cloud storage to delete the object (e.g., by selecting the object and pressing delete or by dragging the object into a recycle bin or otherwise). After deleting the object, the cloud storage will confirm in the general ledger that the object was deleted. This ensures that the client does not attempt to read the deleted object and that the deleted object is deleted (at least from the client's perspective), which can exempt the user from liability. In one embodiment, the cloud can asynchronously check the general ledger for delete requests (e.g., when the delete command is not specifically received by the cloud storage) or commands, and delete the object even if the command or request does not come from the user.

[0023] Figure 1 is a block diagram illustrating an example of a computing environment in which embodiments of the invention may be implemented. Figure 1 Client 102 is shown. Client 102 may be a computing device such as a computer, tablet device, smartphone, etc. and may include a processor, memory, and other circuitry. Client 102 may communicate with cloud object repository 104 or servers associated therewith over a network such as the Internet. Cloud object repository 104 may store objects from multiple clients and may include various storage devices. Cloud object repository 104 may be a data center or other collection of processors, memory, and other hardware and circuitry such as switches, hardware interfaces, and the like.

[0024] In this example, client 102 stores object 108 in cloud object repository 104. Object 108 is represented as object 108a and object 108b. Object 108a represents the object when it is written or uploaded to cloud object repository 104 by client 102. Object 108b represents object 108 stored in cloud object repository 104. Ideally, object 108b is the same as object 108a.

[0025] Embodiments of the present invention ensure that object 108b is identical to object 108a and that the faulty entity can be identified if a difference is found. In one example, this is implemented using ledger 106, which may be a trusted or untrusted distributed ledger. Ledger 106 may be, for example, a distributed database.

[0026] In addition to noting that transactions occurred (e.g., client 102 wrote object 108 to cloud object store 104), ledger 106 also records information that allows the objects and their status (e.g., damaged, secure, lost) in cloud object store 104 to be determined. Ledger 106 advantageously prevents at least some potential fraudulent activity.

[0027] More specifically, for a given transaction, the ledger may include information from both the client 102 (and / or user) and the cloud object repository 104 (cloud provider). The ledger includes the transaction and allows the transaction to be verified, validated, and / or enforceable. In addition to witnessing the transaction as it occurred or recording the transaction (write, read, modify, delete, move, etc.), embodiments of the present invention allow both the client 102 and the cloud object repository 104 to confirm the transaction and confirm the state of the object and potentially agree on various responsibilities regarding the object 108. This can be accomplished using smart contracts.

[0028] In one example, the general ledger 106 effectively allows the cloud object repository 104 to verify that the object 108b is the same as the object 108a. Both the client 102 and the cloud object repository 104 may form an entry in the general ledger 106 related to the transaction or may enter information into the relevant general ledger entry. As discussed herein, the client 102 may indicate that a transaction with respect to the object is performed and may identify or provide an identifier of the object 108. The cloud object repository 104 may form an entry in the general ledger 106 that verifies that the object 108 is received and that the received object is the uploaded object. This is accomplished using a fingerprint or other identifier (such as a hash) of the object. If the fingerprint of the object generated by the cloud object repository 104 matches the fingerprint provided by the client 102, then the object 108a is the same as the object 108b and both the client 102 and the cloud object repository 104 know that the object was successfully received by the cloud object repository 104. This effectively prevents the cloud object repository 104 from claiming that a corrupted object 108 was uploaded and prevents the client 102 from claiming that the object is corrupted in the cloud object repository 104.

[0029] Figure 2 An example of a ledger 206 used by client 202 and cloud object repository 204 is shown. Ledger 206 stores transactions, which may include smart contracts. Each transaction may include a record of actions performed by client 202 and cloud object repository 204. Example transaction 210 includes information such as action 212 (e.g., read, write, delete, copy, etc.), identifier 214, and / or agreement 216.

[0030] Identifier 214 may include an identifier (e.g., a hash or other fingerprint) of object 218. For example, client 202 may generate a hash of object 218 and record the hash in transaction 210. Cloud object repository 204, upon receiving object 218, may generate an identifier from the object, for example, by performing the same hash. If the hash generated by cloud object repository 204 matches the hash provided by client 202, both client 202 and cloud object repository 204 know that object 218 stored in cloud object repository 204 is the same object uploaded by client 202. This is further confirmed in ledger 208. For example, transaction 210 may include space for a confirmation or signature or other indication of confirmation or agreement.

[0031] In one example, agreement 216 may include confirmation from cloud object repository 206 that cloud object repository 206 has object 218 in an acceptable form (e.g., the same form as uploaded). Agreement 216 may also include retention policies and / or availability requirements and / or remedial measures. The agreement 216 effectively constitutes a service level agreement (SLA) for object 218. The retention policy may specify how long object 218 will be stored, the availability may indicate how object 218 will be available, and the remedial measures may specify a predetermined penalty (e.g., fine, refund, etc.) if agreement 216 is violated. The penalty may depend on the extent of the violation. For example, an object that is damaged but still available is different from a lost object.

[0032] Figure 3 An example of a method for storing objects in a cloud object storage is shown. Figure 3 In the example, the object is written 302 to be stored in the cloud storage. Embodiments of the present invention contemplate other scenarios besides cloud storage. For example, embodiments of the present invention may be implemented in a local area network including a network storage.

[0033] After the object is written to the cloud object storage, the transaction is recorded 304 in the general ledger or distributed general ledger. The transaction may identify or include the action performed, the identifier of the object and / or the agreement. The agreement may already be part of the general ledger and be assumed to be used or attached to all transactions. The identifier may be generated in a repeatable manner and may be applied to both unencrypted objects and encrypted objects. The identifier is a way to uniquely identify the corresponding object.

[0034] The cloud object storage may then confirm 306 the object. Confirming the object may include validating the object, such as by generating an identifier from the written object and comparing the identifier to an identifier recorded by the client in the ledger. A match indicates that the object has been successfully received. If there is a mismatch, the ledger may also be delegated so that in a verifiable manner, the cloud object storage may assert that the object supposedly uploaded by the client was not received or that the object provided by the client is corrupted or does not correspond to the identifier. Confirming the object may optionally include agreeing to an agreement about the object (e.g., retention period, availability, etc.). If the identifiers match, the client is able to upload the object again. In one example, the client may be notified to re-upload the object.

[0035] When and if a problem then arises, the agreement can be enforced 308. Enforcement of the agreement can be performed by the cloud object storage and / or the client and / or by a third party service. For example, a client may claim that the cloud object storage did not store the object correctly and that the object was corrupted. The cloud object storage may claim that the original object or an uncorrupted object was never received. The general ledger allows this situation to be resolved. An inspection of the object in the cloud storage and the general ledger can determine whether the object was uploaded correctly or whether the object was corrupted at the cloud object storage.

[0036] Similarly, the ledger can be used to resolve client claims that an object is lost. The ledger can be used to determine that the object was uploaded and there is no subsequent delete command and in one example, the object storage guarantee period has not expired (for example, the contract can be 1 year of storage). Therefore, embodiments of the present invention allow for consideration of multiple aspects of the agreement, such as client commands, storage provider actions, agreed storage terms, etc.

[0037] Figure 4 An example of a method for reading an object from a cloud object repository is shown. Initially, an object is read 402 from a cloud object repository (or storage). Next, the object is verified 404. The verification may include determining that the object is missing. Verifying the object may also include comparing an identifier of the read object with an identifier associated with the object in the general ledger. If a match is made, the object is verified and a non-match indicates a problem. As discussed herein, a protocol is implemented 406 based on the details of the problem.

[0038] Figure 5An example of a method for deleting an object is shown. A request to delete an object may be received 502 by a cloud object store and / or by a general ledger. Next, the delete request is acknowledged 504. The delete request or other request in the general ledger may be serviced asynchronously by the cloud object store. As previously described, the delete request may include an identifier of the object. The cloud object store may then delete 506 the objects asynchronously or in batches by identifying the delete request in the general ledger in batches, and so on. The general ledger allows the cloud object store to use the identifier to verify that the correct object is being deleted.

[0039] In general, the distributed ledger allows transactions recorded therein to reflect actions and certifications against objects. For various actions, objects can be identified, if necessary, for example by using an identifier or fingerprint of the object and the identifier or fingerprint recorded in the ledger. The ledger helps ensure that delete requests are executed on the correct object, objects written to the cloud object store are successfully written, and read requests retrieve the correct object.

[0040] Whenever an error occurs (object is corrupted, cannot be found, not actually deleted, etc.), the ledger allows for inspection of transactions related to the object associated with the error, allowing the source of the error to be determined. Appropriate actions can then be taken based on the source of the error.

[0041] Thus, embodiments of the present invention allow data to be correctly stored, correctly read, correctly deleted, etc., such that both the client and the cloud object storage are responsible for their actions.

[0042] It should be understood that the present invention can be implemented in many ways, including processes, devices, systems, equipment, methods or computer readable media or computer networks such as computer readable storage media, in which computer program instructions are sent via optical communication links or electrical communication links. Application programs can take the form of software executed on a general-purpose computer or can be hard-wired or hard-coded in hardware. In the present application, these implementations or any other form that the present invention can take can be referred to as technology. Typically, the order of steps in the disclosed process can be changed and still within the scope of the present invention.

[0043] The embodiments disclosed herein may include the use of a special or general purpose computer including various computer hardware or software modules, as discussed in more detail below. The computer may include a processor and a computer storage medium carrying instructions that, when executed by the processor and / or cause the instructions to be executed by the processor, perform any one or more of the methods disclosed herein.

[0044] As mentioned above, embodiments within the scope of the present invention also include computer storage media, which are physical media for carrying or carrying computer executable instructions or data structures stored thereon. Such computer storage media can be any available physical media that can be accessed by a general or special purpose computer.

[0045] By way of example and not limitation, such computer storage media may include hardware such as a solid state drive (SSD), RAM, ROM, EEPROM, CD-ROM, flash memory, phase change memory ("PCM"), or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other hardware storage device that can be used to store program code in the form of computer-executable instructions or data structures that can be accessed and executed by a general or special purpose computer system to implement the functions disclosed in the present invention. Combinations of the above should also be included in the scope of computer storage media. These media are also examples of non-transitory storage media, which also include cloud-based storage systems and structures, however the scope of the present invention is not limited to these examples of non-transitory storage media.

[0046] Computer executable instructions include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a function or a group of functions. Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions disclosed herein are disclosed as example forms of implementing the claims.

[0047] As used herein, the term "module" or "component" may refer to a software object or routine executed on a computing system. The different components, modules, engines, and services described herein may be implemented as objects or processes executed on a computing system, for example, as separate threads. Although the systems and methods described herein may be implemented in software, it is also possible and expected to be implemented in hardware or a combination of software and hardware. In the present application, a "computing entity" may be any computing system as previously defined herein, or any module or combination of modules running on a computing system.

[0048] In at least some instances, a hardware processor is provided that is operable to execute executable instructions for performing a method or process, such as the methods and processes disclosed herein. The hardware processor may or may not include other hardware, such as elements of the computing devices and systems disclosed herein.

[0049] In terms of computing environment, embodiments of the present invention can be performed in a client-server environment, a network environment or a local environment, or any other suitable environment. Suitable operating environments for at least some embodiments of the present invention include cloud computing environments, in which one or more of the client, server, and target virtual machine can reside and operate in the cloud environment.

[0050] The present invention can be realized in other specific forms without departing from its spirit or essential characteristics. The described embodiments are considered only as illustrative and non-restrictive in all respects. Therefore, the scope of the present invention is indicated by the appended claims rather than by the previous description. All changes falling within the equivalent meanings and equivalent ranges of the claims are included in the scope of the claims.

Claims

1. A method for protecting data stored in an object storage, the method comprising: executing, by a client, a transaction of an object to the object store, wherein the client generates an identifier of the object; recording, by the client, a transaction in a ledger for the object such that the ledger attests to the transaction and the object, wherein the transaction includes an action performed on the object and an identifier of the object; receiving an acknowledgment from the object store, wherein the acknowledgment indicates that an identifier of the object generated by the object store matches an identifier of the object provided by the client; recording the confirmation from the object store in the general ledger, wherein the transaction also includes an agreement accepted by the object store or by a provider of the object store when the confirmation is recorded, and wherein the agreement specifies a penalty; receiving a delete request from the client into the ledger to delete the object, wherein the deletion is included in the agreement; servicing, by the object store, the delete request by deleting the object, wherein an identifier in the ledger and associated with the delete request ensures that the object store knows which object to delete; and The penalties are automatically enforced when the agreement is violated.

2. The method of claim 1, further comprising: A write to the object storage is performed.

3. The method of claim 1, further comprising: After the object is stored, the object is read from the object store, wherein the transaction is used to verify the object being read.

4. The method of claim 1, further comprising: Entering a delete request transaction into the ledger such that the ledger attests to the delete request.

5. The method of claim 1, wherein: The agreement is enforced by at least determining which of the client and the object store is at fault based on information stored in the ledger.

6. The method of claim 5, wherein: The transaction includes a smart contract between the client and the object storage or between a user and a provider of the object storage.

7. A non-transitory computer readable medium comprising instructions which, when executed, perform the method of claim 1.

8. A method for writing an object to a cloud object storage, the method comprising: Writing the object to the cloud object storage by a client, wherein the client provides an identifier of the object; recording in a ledger a transaction reflecting the writing of the object to the cloud object storage, wherein the transaction evidences the writing of the object to the cloud object storage and includes an identifier of the object and is associated with an agreement specifying the availability of the object; receiving an acknowledgement from the cloud object store, wherein the acknowledgement indicates that an identifier of the object generated by the cloud object store matches an identifier of the object provided by the client, and wherein the acknowledgement constitutes acceptance of the agreement; receiving a delete request from the client into the ledger to delete the object, wherein the deletion is included in the agreement; servicing, by the cloud object store, the delete request by deleting the object, wherein an identifier in the ledger and associated with the delete request ensures that the cloud object store knows which object to delete; and When a violation of the agreement regarding the object is determined, the agreement is enforced by imposing a penalty.

9. The method of claim 8, wherein: The object is encrypted.

10. The method of claim 8, further comprising: After the object is stored, the object is read from the cloud object store, wherein the transaction is used to verify the object being read by comparing an identifier of the object being read with an identifier included in the ledger.

11. The method of claim 8, wherein: The delete request is performed asynchronously.

12. The method of claim 8, wherein: The ledger may include a trusted or untrusted distributed ledger.

13. The method of claim 8, further comprising: The agreement is enforced, wherein the agreement is enforced at least by determining which of the client and the cloud object storage is at fault based on information stored in the ledger, wherein the agreement is enforced by an enforcement computer that accesses the ledger and the objects.

14. The method of claim 8, wherein: The transaction is associated with the client's private key to verify the client's identity.

15. A non-transitory computer readable medium comprising instructions which, when executed, perform the method of claim 8.

16. A computing device for storing an object, the computing device comprising a memory and a processor, the processor being configured to: Receive an object from the client; writing the object into the memory; accessing a general ledger to obtain an identifier of the object from a transaction stored in the general ledger; generating an identifier for the object and comparing the generated identifier to an identifier retrieved from the ledger; confirming the transaction in the ledger when the generated identifier matches an identifier retrieved from the ledger, wherein the confirmation completes the smart contract with the client; receiving a delete request from the client into the ledger to delete the object, wherein the deletion is included in the agreement; servicing the delete request by deleting the object, wherein an identifier in the ledger and associated with the delete request ensures that the store knows which object to delete; and The agreement is enforced, wherein the agreement includes a penalty that is automatically imposed upon a determination of a violation of the agreement.

Images