Implementing opportunistic authentication of encrypted data

Abstract

A computer-implemented method comprising: compressing an uncompressed data instance to create a compressed data instance; encrypting the compressed data instance in response to determining that the size of the compressed data instance is less than a predetermined threshold; creating a message authentication code (MAC) for the encrypted compressed data instance; and adding variable length zero padding and the MAC to the encrypted compressed data instance to create a formatted string.

Description

Background Art

[0001] The present invention relates to data security, and more particularly, to implementing end-to-end data integrity protection using a message authentication code (MAC).

[0002] A valuable way to implement data integrity protection is through the use of a message authentication code (MAC). For example, a MAC can be used to confirm the authenticity and integrity of received data. In addition, deterministic encryption is useful in data storage because it allows copies of encrypted data to be identified and removed within such data storage. Deterministic encryption can be combined with length-preserving compression (LPC) to enable a storage system to perform data deduplication on data that has been persistently encrypted at the host. However, current LPC implementations do not implement data integrity protection. Summary of the invention

[0003] A computer-implemented method according to one embodiment includes: compressing an uncompressed data instance to create a compressed data instance; encrypting the compressed data instance in response to determining that the size of the compressed data instance is less than a predetermined threshold; creating a message authentication code (MAC) for the encrypted compressed data instance; and adding zero padding and the MAC to the encrypted compressed data instance to create a formatted string.

[0004] According to another embodiment, a computer program product includes one or more computer-readable storage media, and program instructions stored together on the one or more computer-readable storage media, wherein the program instructions include instructions configured to cause one or more processors to perform a method, the method comprising compressing, by the processor, an uncompressed data instance to create a compressed data instance; encrypting, by the processor in response to determining that the size of the compressed data instance is less than a predetermined threshold, the compressed data instance; creating, by the processor, a message authentication code (MAC) for the encrypted compressed data instance; and adding, by the processor, zero padding and the MAC to the encrypted compressed data instance to create a formatted string.

[0005] According to another embodiment, a computer-implemented method includes compressing an uncompressed data instance to create a compressed data instance; in response to determining that the size of the compressed data instance is less than a predetermined threshold, encrypting the compressed data instance, creating a message authentication code (MAC) for the encrypted compressed data instance, and adding zero padding and the MAC to the encrypted compressed data instance to create a formatted string; and in response to determining that the size of the compressed data instance is greater than the predetermined threshold, discarding the compressed data instance and encrypting the uncompressed data instance to create the formatted string.

[0006] Other aspects and embodiments of the invention will become apparent from the following detailed description, which, when taken in conjunction with the accompanying drawings, illustrates, by way of example, the principles of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A cloud computing environment according to one embodiment of the present invention is described.

[0008] Figure 2 An abstract model layer according to one embodiment of the present invention is shown.

[0009] Figure 3 A cloud computing node according to one embodiment of the present invention is depicted.

[0010] Figure 4 A hierarchical data storage system according to an embodiment of the present invention is shown.

[0011] Figure 5 A flow chart of a method for implementing opportunistic authentication of encrypted data according to an embodiment of the present invention is shown.

[0012] Figure 6 A flow chart of a method for encrypting an uncompressed data instance according to an embodiment of the present invention is shown.

[0013] Figure 7 A flow chart of a method for encrypting and opportunistically verifying a data instance according to an embodiment of the present invention is shown.

[0014] Figure 8 A flow chart of a method for performing data deduplication according to an embodiment of the present invention is shown.

[0015] Fig. 9 An exemplary environment for performing wide block encryption of compressed plaintext instances according to one embodiment of the present invention is shown.

[0016] Fig.10 An exemplary environment for performing wide block encryption of uncompressed plaintext instances according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] The following description is intended to illustrate the general principles of the invention, but is not intended to limit the inventive concepts claimed herein. In addition, specific features described herein may be used in combination with other described features in all possible combinations and permutations.

[0018] Unless expressly defined otherwise herein, all terms are to be given their broadest possible interpretations, including the meanings implied in the specification and as understood by those skilled in the art and / or defined in dictionaries, treatises, and the like.

[0019] It must also be noted that, as used in the specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless otherwise indicated. It will also be understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0020] The following description discloses several embodiments that implement opportunistic authentication of encrypted data.

[0021] In a general embodiment, a computer-implemented method includes compressing an uncompressed data instance to create a compressed data instance; encrypting the compressed data instance in response to determining that the size of the compressed data instance is less than a predetermined threshold; creating a message authentication code (MAC) for the encrypted compressed data instance; and adding zero padding and the MAC to the encrypted compressed data instance to create a formatted string.

[0022] In another general embodiment, a computer program product includes one or more computer-readable storage media, and program instructions stored together on the one or more computer-readable storage media, wherein the program instructions include instructions configured to cause one or more processors to perform a method, the method comprising compressing, by the processor, an uncompressed data instance to create a compressed data instance; encrypting, by the processor in response to determining that a size of the compressed data instance is less than a predetermined threshold, the compressed data instance; creating, by the processor, a message authentication code (MAC) for the encrypted compressed data instance; and adding, by the processor, zero padding and the MAC to the encrypted compressed data instance to create a formatted string.

[0023] In another general embodiment, a computer-implemented method includes compressing an uncompressed data instance to create a compressed data instance; in response to determining that the size of the compressed data instance is less than a predetermined threshold, encrypting the compressed data instance, creating a message authentication code (MAC) for the encrypted compressed data instance, and adding zero padding and the MAC to the encrypted compressed data instance to create a formatted string; and in response to determining that the size of the compressed data instance is greater than the predetermined threshold, discarding the compressed data instance and encrypting the uncompressed data instance to create the formatted string.

[0024] It should be understood that although the present disclosure includes detailed descriptions about cloud computing, the implementation of the teachings set forth herein is not limited to cloud computing environments. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0025] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0026] Features are as follows:

[0027] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed without requiring manual interaction with the service provider.

[0028] Wide Area Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0029] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is location independence in the sense that consumers typically do not control or know the exact location of the resources provided, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0030] Rapid elasticity: In some cases, the ability to scale out quickly and in quickly can be provisioned quickly and elastically. To the consumer, the capacity available for provisioning often appears to be unlimited and can be purchased in any quantity at any time.

[0031] Metered Services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0032] The service model is as follows:

[0033] Software as a Service (SaaS): The capability provided to the consumer is to use the provider's applications running on the cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0034] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto cloud infrastructure, where the applications are created using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possible configuration of the application hosting environment.

[0035] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0036] The deployment model is as follows:

[0037] Private Cloud: The cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist inside or outside the building.

[0038] Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0039] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0040] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community or public) that remain a unique entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0041] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure consisting of a network of interconnected nodes.

[0042] Reference now Figure 1, depicts an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which a local computing device used by a cloud consumer can communicate, such as a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that Figure 1 The types of computing devices 54A-N shown in are intended for illustration only, and computing node 10 and cloud computing environment 50 may communicate with any type of computing device over any type of network and / or network addressable connection (eg, using a web browser).

[0043] Reference now Figure 2 , showing the cloud computing environment 50 ( Figure 1 ) provides a set of functional abstraction layers. It should be understood in advance that Figure 2 The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0044] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on RISC (Reduced Instruction Set Computer) architecture; server 63; blade server 64; storage device 65; and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0045] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.

[0046] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when resources are utilized in a cloud computing environment, as well as billing or invoicing for consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-scheduling and procurement of cloud computing resources, where future demand is anticipated based on the SLA.

[0047] The workload layer 90 provides examples of functions that can take advantage of the cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and message authentication 96.

[0048] Reference now Figure 3 , a schematic diagram of an example of a cloud computing node is shown. Cloud computing node 10 is only one example of a suitable cloud computing node and is not intended to impose any limitations on the scope of use or functionality of the embodiments of the present invention described herein. Regardless, cloud computing node 10 can be implemented and / or perform any of the functions set forth above.

[0049] In the cloud computing node 10, there is a computer system / server 12, which can operate with many other general or special computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with the computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, etc.

[0050] Computer system / server 12 may be described in the general context of computer system executable instructions, such as program modules, executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 12 may be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in local and remote computer system storage media including memory storage devices.

[0051] like Figure 3 As shown, the computer system / server 12 in the cloud computing node 10 is shown in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, a system memory 28, and a bus 18 that couples various system components including the system memory 28 to the processor 16.

[0052] Bus 18 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, these architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0053] Computer system / server 12 typically includes a variety of computer system readable media. Such media can be any available media that can be accessed by computer system / server 12, and it includes volatile and nonvolatile media, removable and non-removable media.

[0054] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer system / server 12 may also include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 may be provided for reading from and writing to a non-removable, non-volatile magnetic medium (not shown, and commonly referred to as a "hard drive"). Although not shown, a disk drive for reading from and writing to a removable, non-volatile disk (e.g., a "floppy disk") may be provided, as well as an optical drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media. In this case, each may be connected to the bus 18 via one or more data media interfaces. As will be further depicted and described below, the memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of an embodiment of the present invention.

[0055] A program / utility 40 having a set (at least one) of program modules 42, as well as an operating system, one or more application programs, other program modules, and program data, may be stored in memory 28 as an example and not by way of limitation. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, may include an implementation of a networking environment. Program modules 42 generally perform the functions and / or methods of embodiments of the present invention described herein.

[0056] The computer system / server 12 may also communicate with one or more external devices 14, such as a keyboard, pointing device, display 24, etc.; one or more devices that enable a user to interact with the computer system / server 12; and / or any device that enables the computer system / server 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may occur via an input / output (I / O) interface 22, however, the computer system / server 12 may communicate with one or more networks, such as a local area network (LAN), a general purpose wide area network (WAN), and / or a public network (e.g., the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other components of the computer system / server 12 via a bus 18, and it should be understood that, although not shown, other hardware and / or software components may be used in conjunction with the computer system / server 12, examples of which include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0057] Reference now Figure 4, shows a storage system 400 according to one embodiment. Note that according to various embodiments, Figure 4 Some elements shown in the figure may be implemented as hardware and / or software. The storage system 400 may include a storage system manager 412 for communicating with multiple media on at least one higher storage layer 402 and at least one lower storage layer 406. The (one or more) higher storage layers 402 may preferably include one or more random access and / or direct access media 404, such as hard disks in hard disk drives (HDDs), non-volatile memory (NVM), solid state memory in solid state drives (SSDs), flash memory, SSD arrays, flash memory arrays, etc., and / or other media mentioned herein or known in the art. The lower storage layer 406 may preferably include one or more lower performance storage media 408, including sequential access media, such as tapes and / or optical media in tape drives, slower access HDDs, slower access SSDs, etc., and / or other media mentioned herein or known in the art. One or more additional storage layers 416 may include any combination of storage memory media desired by the designer of the system 400. Moreover, any one of the higher storage layer 402 and / or the lower storage layer 406 may include some combination of storage devices and / or storage media.

[0058] Storage system manager 412 may communicate with storage system manager 412 via network 410, for example Figure 4 A storage area network (SAN), or some other suitable network type, is shown in communication with the storage media 404, 408 on the upper storage tier 402 and the lower storage tier 406. The storage system manager 412 may also communicate with one or more host systems (not shown) through a host interface 414, which may or may not be part of the storage system manager 412. The storage system manager 412 and / or any other components of the storage system 400 may be implemented in hardware and / or software, and may utilize a processor (not shown), such as a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc., for executing commands of a type known in the art. Of course, any arrangement of storage systems may be used, as will be apparent to those skilled in the art upon reading this specification.

[0059] In further embodiments, the storage system 400 may include any number of data storage layers, and may include the same or different storage memory media within each storage layer. For example, each data storage layer may include the same type of storage memory media, such as HDD, SSD, sequential access media (tapes in tape drives, optical disks in optical disk drives, etc.), direct access media (CD-ROM, DVD-ROM, etc.), or any combination of media storage types. In one such configuration, the higher storage layer 402 may include a large portion of the SSD storage media for storing data in a higher performance storage environment, and the remaining storage layers including the lower storage layer 406 and the additional storage layer 416 may include any combination of SSD, HDD, tape drives, etc. for storing data in a lower performance storage environment. In this way, more frequently accessed data, data with a higher priority, data that requires faster access, etc. can be stored in the higher storage layer 402, while data that does not have one of these attributes can be stored in the additional storage layer 416 including the lower storage layer 406. Of course, when reading this specification, those skilled in the art can design many other combinations of storage media types according to the embodiments presented herein to implement different storage schemes.

[0060] According to some embodiments, a storage system such as 400 may include logic configured to receive a request to open a data set, logic configured to determine whether the requested data set is stored in multiple associated parts to a lower storage tier 406 of the tiered data storage system 400, logic configured to move each associated part of the requested data set to a higher storage tier 402 of the tiered data storage system 400, and logic configured to combine the requested data set from the associated parts onto the higher storage tier 402 of the tiered data storage system 400.

[0061] Of course, according to various embodiments, the logic may be implemented as a method or computer program product on any device and / or system.

[0062] Reference now Figure 5 , a flow chart of a method 500 according to an embodiment is shown. In various embodiments, the method 500 may be performed according to the present invention. Figure 1-4 Of course, as will be understood by those skilled in the art upon reading this specification, method 500 may include more Figure 5 More or fewer operations may be performed as specifically described in .

[0063] Each step of method 500 can be performed by any suitable component of operating environment. For example, in various embodiments, method 500 can be performed partially or completely by one or more servers, computers or some other devices having one or more processors therein. One or more steps of method 500 can be performed by a processor (e.g., a processing circuit, chip and / or module implemented in hardware and / or software and preferably having at least one hardware component) in any device. Illustrative processors include but are not limited to central processing units (CPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc., combinations thereof, or any other suitable computing devices known in the art. Additional exemplary components include one or more coprocessors or accelerators, which may be on the same chip as the CPU or may not be on the same chip as the CPU or may be packaged on the same substrate or module.

[0064] like Figure 5 As shown, method 500 may begin with operation 502, where an uncompressed data instance is compressed to create a compressed data instance. In one embodiment, one or more compression algorithms may be used to compress the uncompressed data instance to create a compressed data instance. For example, a compression algorithm may be applied to the uncompressed data instance to create a compressed data instance.

[0065] Additionally, in one embodiment, the uncompressed data instance may include data identified as being to be encrypted. For example, a request to encrypt the uncompressed data instance may be received from an application, from a user, etc. In another embodiment, the uncompressed data instance may include a plaintext instance.

[0066] Additionally, method 500 may proceed to operation 504, where the compressed data instance is encrypted in response to determining that the size of the compressed data instance is less than a predetermined threshold. In one embodiment, the size of the compressed data instance may be compared to the predetermined threshold. In another embodiment, the predetermined threshold may include a threshold data size. In yet another embodiment, in response to determining that the size of the compressed data instance is greater than the predetermined threshold, the compressed data instance may be discarded, deleted, or removed.

[0067] For example, the compressed data instance can be discarded and overwritten by a future compression engine output.Then, the abandoned data can be deleted in response to determining that no additional data is incoming or a power outage has been detected or is about to occur.

[0068] In addition, in one embodiment, the predetermined threshold may include the size of the length field, the predetermined minimum zero padding length, and the sum of the message authentication code (MAC) field. For example, the length field, the predetermined zero padding, and the message authentication code (MAC) field may each have a predetermined size. In another embodiment, encrypting the compressed data instance may result in an encrypted compressed data instance.

[0069] In addition, in one embodiment, the compressed data instance may be encrypted using wide block encryption. For example, encrypting the compressed data instance using wide block encryption may include applying a wide block encryption algorithm to the compressed data instance. In another example, the wide block encryption algorithm may include an EME2-AES encryption algorithm, an XCB-AES encryption algorithm, etc.

[0070] Additionally, in one embodiment, the wide block encryption algorithm may behave as a single block cipher and may take as input a compressed data instance. In another embodiment, the wide block encryption algorithm may internally use a narrow block cipher (e.g., an Advanced Encryption Standard (AES) cipher) while encrypting the compressed data instance. In yet another embodiment, the wide block encryption algorithm may output an encrypted compressed data instance. In yet another embodiment, encrypting the compressed data instance using a wide block encryption may include applying the wide block encryption algorithm to the compressed data instance.

[0071] In addition, in one embodiment, the compressed data instance may be encrypted using an extended encryption mode (e.g., counter mode encryption, etc.) that includes an initialization vector. For example, the initialization vector may be created using the compressed data instance and a secret key. In another example, the secret key may include a data instance that is not publicly available for encryption (e.g., held by an entity that performs encryption of the data instance). In yet another example, the secret key may also be referred to as a private key.

[0072] Additionally, in one embodiment, the initialization vector may include a keyed hash. In another embodiment, the keyed hash may be generated by applying a hash function to the secret key and the compressed data instances. In yet another embodiment, the hash function may be applied to the secret key and the plaintext in any order (e.g., the secret key first, followed by the compressed instances of the data, or the compressed instances of the data first, followed by the secret key).

[0073] In addition, in one embodiment, the hash function may map the secret key and the compressed data instance to a fixed-size hash value that constitutes a keyed hash. In another embodiment, the hash function may include a secure hash algorithm-256 (SHA-256) cryptographic hash function. In yet another embodiment, the initialization vector may include a keyed hash message authentication code / hash-based message authentication code (HMAC) calculation involving the hash key and the plaintext. On the other hand, the hash key may be the same as the secret key used to encrypt the data, or may be related to the secret key through key derivation, and the keyed hash may be calculated using an encryption algorithm.

[0074] In addition, in one embodiment, a key for hashing can be derived from a secret key, and a keyed hash can be calculated using an encryption algorithm. For example, a keyed hash can be determined by encrypting a hash of a plaintext using a hash encryption key. A hash encryption key is a type of hash key, and therefore can be the same as a secret key used to encrypt data, or can be related to the secret key by key derivation. When a hash encryption key is used, the encryption algorithm used can be as simple as an electronic codebook (ECB) mode of an advanced encryption standard (AES) encryption algorithm (i.e., ECB-AES-128) using a 128-bit key. On the other hand, the hash key can also be generated independently of the secret key, and if so, it needs to be maintained so that it can be used with the secret key to perform further encryption under the same key to allow data deduplication. An independently generated hash key contributes to the uniqueness of an initialization vector (IV). If the IV is to be checked as a plaintext result of the decryption of a ciphertext created using the IV, the independently generated hash key will also have to be retained.

[0075] Additionally, in one embodiment, the compressed data instance may be encrypted using the initialization vector, the secret key, and the compressed data instance. For example, encrypting the compressed data instance may result in an encrypted compressed data instance. In another example, encrypting the compressed data instance may include applying an encryption mode (e.g., an encryption algorithm) to the compressed data instance using the initialization vector and the secret key.

[0076] In addition, in one embodiment, the encryption mode may include an encryption mode that relies on an IV or a random number such as CBC, PCBC, CFB, OFB, OCB, CTR, CCM, or GCM. For example, a counter-based encryption mode (e.g., CTR, GCM, etc.) may combine an initialization vector with an arbitrary counter value to produce a first value. The first value may then be incremented as needed to create a count stream. The count stream may then be block-cipher encrypted (e.g., Advanced Encryption Standard (AES) encryption, etc.) using a secret key to create a cipher stream. The data instance may then be combined with the cipher stream (e.g., using an exclusive-OR (XOR) operation) to create an encrypted data instance.

[0077] In addition, in one embodiment, the encryption mode may include a Galois Counter Mode (GCM) encryption mode. For example, the GCM encryption mode may determine the number of blocks within the compressed data instance. In another example, the number of blocks may be combined with an initialization vector to produce a first value. The first value may then be incremented whenever necessary to create a count stream. The count stream may then be block encrypted (e.g., Advanced Encryption Standard (AES) encryption, etc.) using a secret key to create a cipher stream. The data instance may then be combined with the cipher stream (e.g., using an XOR operation) to create an encrypted data instance.

[0078] Furthermore, in one embodiment, the encrypted compressed data instance may include an instance of ciphertext (eg, encrypted text produced by applying an encryption algorithm to the compressed data instance, etc.).

[0079] Additionally, method 500 may proceed to operation 506, where a message authentication code (MAC) is created for the encrypted compressed instance of the data. In one embodiment, the MAC may include data for authenticating the encrypted compressed instance of the data after it is sent to the recipient. In another embodiment, the MAC may be generated and verified using the same secret key. In yet another embodiment, the key generation algorithm may select the secret key from a key space (e.g., in a random manner, etc.).

[0080] Furthermore, in one embodiment, the key generation algorithm may select a predetermined secret key. In another embodiment, the signature algorithm may output a tag (e.g., a MAC) when given a selected secret key and an encrypted compressed data instance as input. In yet another embodiment, the verification algorithm may utilize the tag and the selected secret key to accept or reject a message containing an encrypted compressed instance of the data and the tag (e.g., a ciphertext string).

[0081] In this way, a MAC can be utilized to verify the authenticity of a cryptographically compressed instance of the data.

[0082] Additionally, in one embodiment, the MAC may include a Galois message authentication code (GMAC or GCM-MAC) created as part of a GCM encryption mode. For example, the GCM encryption mode may be used to encrypt the compressed data instance, and the MAC may be created as part of the same encryption calculation (e.g., using the same secret key, etc.).

[0083] In addition, in one embodiment, the MAC may include a key-controlled hash message authentication code or a hash-based message authentication code (e.g., HMAC). For example, the HMAC may be created based on an underlying cryptographic hash function (e.g., SHA2-256 or SHA3-256 hash function, etc.). In another example, a two-pass hash calculation may be used to create the HMAC.

[0084] For example, the secret key may calculate two keys (an inner key and an outer key) from the encrypted compressed data instance. In another embodiment, a first pass of the cryptographic hash function may use the encrypted compressed data instance and the inner key to produce an inner hash. In yet another embodiment, a second pass of the cryptographic hash function may use the inner hash and the outer key to produce a final HMAC.

[0085] Additionally, method 500 may proceed to operation 508, where zero padding and a MAC are added to the encrypted compressed data instance to create a formatted string. In one embodiment, a length field may also be added to the encrypted compressed data instance, where the length field may include a field having a predetermined length (e.g., two bytes, etc.) indicating the length of the encrypted compressed data instance. In another embodiment, the length of the encrypted compressed data instance may be determined and the length may be encoded into a length field that is concatenated to the encrypted compressed data instance.

[0086] In another embodiment, the size of the length field may be calculated based on the size of the encrypted compressed data instance.In one embodiment, the HMAC may be calculated after adding one or more of the length field and zero padding to the encrypted compressed data instance.

[0087] In addition, in one embodiment, the minimum zero padding may include a predetermined minimum number of bytes having some highly compressible and easily detectable value, such as all bytes set to zero. In another embodiment, the zero pad may have a length greater than a predetermined threshold. For example, the predetermined threshold may include a variable number of bytes having a minimum length. In yet another embodiment, the zero padding may be concatenated to an encrypted compressed instance of the data, a length field, etc.

[0088] In addition, in one embodiment, the MAC may include a field having a predetermined length. In another embodiment, the MAC may be concatenated to the encrypted compressed instance of the data, the length field, zero padding, etc. In yet another embodiment, the length of the ciphertext string (e.g., the length of the length field, the encrypted compressed data instance, the zero padding, and the MAC) may be equal to the length of the uncompressed data instance. In yet another embodiment, if an initialization vector (IV) is used to encrypt the compressed data instance, the IV may be concatenated to the encrypted compressed data instance, the length field, the zero padding, the MAC, etc.

[0089] In addition, in one embodiment, the length of the length field, the encrypted compressed data instance, the zero padding, the MAC, and the IV may be equal to the length of the uncompressed data instance. In this way, length-preserving compression (LPC) may be achieved while compressing, encrypting, and providing integrity protection for the data instance.

[0090] Furthermore, in one embodiment, the formatted string may be stored and / or transmitted. For example, the formatted string may be stored in a database, a distributed storage system, a cloud computing environment, etc. In another embodiment, the formatted string may be transmitted to one or more users, one or more devices, one or more remote storage systems, etc. via one or more networks.

[0091] In addition, in one embodiment, zero padding can be used to identify a cryptographically compressed instance of data (e.g., in response to receiving a cryptographically compressed instance of data or reading a cryptographically compressed instance of data from a storage device). For example, upon receiving a formatted string, the receiving entity can search the string for zero padding (e.g., at a predetermined position within the ciphertext string, etc.). In another example, if zero padding is identified, the formatted string can be identified as an encrypted, compressed instance of data. In yet another example, if zero padding is not identified, the formatted string can be identified as an encrypted, uncompressed instance of data.

[0092] Additionally, in one embodiment, the MAC can be used to verify the data integrity and authenticity of the ciphertext string. For example, upon receiving the ciphertext string, the MAC can be extracted from the ciphertext string. For example, the location of the MAC within the ciphertext string can be provided to the recipient, and the MAC can be extracted using the location. In another example, a verification algorithm (provided with a secret key) can determine the MAC using the ciphertext string and a secret key (e.g., a secret key used to create the MAC, etc.).

[0093] Furthermore, in one embodiment, the determined MAC may be compared to the extracted MAC. In another embodiment, if the determined MAC is the same as the extracted MAC, the ciphertext may be confirmed as valid. In yet another embodiment, if the determined MAC is different from the extracted MAC, the ciphertext may be rejected as invalid.

[0094] In this way, completely deterministic ciphertext strings can be created and stored for data instances. The ciphertext strings are completely deterministic because, using the wide block encryption technique described above, if a first data instance is identical to a second data instance and a first key is identical to a second key, then a first ciphertext string created for a first data instance using a first key can be identical to a second ciphertext string created for a second data instance using a second key. As a result, duplicate ciphertext strings can be identified within a hardware storage device and can be removed, thereby increasing available storage space within the hardware storage device and improving performance of the hardware storage device while maintaining the security of the stored data via robust encryption.

[0095] Additionally, by including a MAC in the ciphertext string, the ciphertext string can be checked upon receipt to see if it has been corrupted (e.g., via transmission errors, via malware, etc.) This can provide end-to-end integrity protection during storage and transmission of the ciphertext string.

[0096] Reference now Figure 6 , a flowchart of a method 600 for encrypting an uncompressed data instance according to one embodiment is shown. In various embodiments, the method 600 may be implemented in accordance with the present invention. Figure 1-4Of course, as will be understood by those skilled in the art upon reading this specification, method 600 may include more Figure 6 More or fewer operations may be performed as specifically described in .

[0097] Each step of method 600 can be performed by any suitable component of operating environment. For example, in various embodiments, method 600 can be performed partially or completely by one or more servers, computers or some other device having one or more processors therein. One or more steps of method 600 can be performed by a processor (e.g., a processing circuit, chip and / or module implemented in hardware and / or software and preferably having at least one hardware component) in any device. Illustrative processors include but are not limited to central processing units (CPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc., combinations thereof, or any other suitable computing devices known in the art. Additional exemplary components include one or more coprocessors or accelerators, which may be on the same chip as the CPU or may not be on the same chip as the CPU or may be packaged on the same substrate or module.

[0098] like Figure 6 As shown, method 600 may begin at operation 602, where it is determined that the size of the compressed data instance is greater than a predetermined threshold. In one embodiment, the uncompressed data instance may include data identified as being to be encrypted. For example, a request to encrypt the uncompressed data instance may be received from an application, from a user, etc.

[0099] In addition, in one embodiment, the uncompressed data instance may include a plaintext instance. For example, the plaintext may include unencrypted information that is desired to be encrypted via one or more encryption algorithms. In another embodiment, the uncompressed data instance may be compressed using one or more compression algorithms to create a compressed data instance.

[0100] In addition, in one embodiment, the size of the compressed data instance can be compared to a predetermined threshold. In another embodiment, the predetermined threshold can include a threshold data size. In yet another embodiment, the predetermined threshold can include space for the sum of a length field and a predetermined minimum zero padding length. In yet another embodiment, in response to determining that the size of the compressed data instance is greater than the predetermined threshold, the compressed data instance can be discarded (e.g., removed, deleted, etc.).

[0101] Additionally, method 600 may proceed to operation 604, where the uncompressed data instance is encrypted to create a ciphertext string without first compressing the uncompressed data instance. In one embodiment, encrypting the uncompressed data instance may produce an encrypted uncompressed data instance. In another embodiment, the encrypted uncompressed data instance may have the same size / length as the uncompressed data instance.

[0102] Moreover, in one embodiment, the uncompressed data instance may be encrypted using Advanced Encryption Standard (AES) encryption. In another embodiment, the uncompressed data instance may be encrypted using wide block encryption. For example, encrypting the uncompressed data instance using wide block encryption may include applying a wide block encryption algorithm to the uncompressed data instance.

[0103] Additionally, in one embodiment, the wide block encryption algorithm may include an EME2-AES encryption algorithm, an XCB-AES encryption algorithm, etc. In another embodiment, the wide block encryption algorithm may be presented as a single block cipher and may take an uncompressed data instance as input. In yet another embodiment, the wide block encryption algorithm may internally use a narrow block cipher (e.g., an Advanced Encryption Standard (AES) cipher) while encrypting the uncompressed data instance.

[0104] In addition, in one embodiment, the wide block encryption algorithm can output an encrypted uncompressed data instance. In another embodiment, the encrypted uncompressed data instance can include a ciphertext instance (e.g., encrypted text generated by applying an encryption algorithm to an uncompressed plaintext instance, etc.).

[0105] In addition, in one embodiment, the ciphertext string can be stored and / or transmitted. For example, the ciphertext string can be stored in a database, a distributed storage system, a cloud computing environment, etc. In another example, the ciphertext string can be sent to one or more users, one or more devices, one or more remote storage systems, etc. via one or more networks.

[0106] Reference now Figure 7 , according to one embodiment, shows a flow chart of a method 700 for encrypting and opportunistically verifying a data instance. In various embodiments, the method 700 can be implemented according to the present invention. Figure 1-4 Of course, as will be understood by those skilled in the art upon reading this specification, method 700 may include more Figure 7 More or fewer operations may be performed as specifically described in .

[0107] Each step of method 700 can be performed by any suitable component of operating environment. For example, in various embodiments, method 700 can be performed partially or completely by one or more servers, computers or some other devices having one or more processors therein. One or more steps of method 700 can be performed by a processor (e.g., a processing circuit, chip and / or module implemented in hardware and / or software and preferably having at least one hardware component) in any device. Illustrative processors include but are not limited to central processing units (CPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc., combinations thereof, or any other suitable computing devices known in the art. Additional exemplary components include one or more coprocessors or accelerators, which may be on the same chip as the CPU or may not be on the same chip as the CPU or may be packaged on the same substrate or module.

[0108] like Figure 7 As shown, method 700 may begin at operation 702, where an uncompressed data instance is compressed to create a compressed data instance. Additionally, method 700 may proceed to operation 704, where, in response to determining that the size of the compressed data instance is less than a predetermined threshold, the compressed data instance is encrypted, a message authentication code (MAC) is created for the encrypted compressed data instance, and zero padding and the MAC are added to the encrypted compressed data instance to create a formatted string.

[0109] Additionally, method 700 may continue with operation 706 where, in response to determining that the size of the compressed data instance is greater than a predetermined threshold, the compressed data instance is discarded and the uncompressed data instance is encrypted to create a formatted string.

[0110] Reference now Figure 8 , shows a flow chart of a method 800 for performing data deduplication according to one embodiment. In various embodiments, the method 800 may be performed according to the present invention. Figure 1-4 and 9-10. Of course, as will be appreciated by those skilled in the art upon reading this specification, method 800 may include more Figure 8 More or fewer operations may be performed as specifically described in .

[0111] Each step of method 800 can be performed by any suitable component of operating environment. For example, in various embodiments, method 800 can be performed partially or completely by one or more servers, computers or some other device having one or more processors therein. One or more steps of method 800 can be performed by a processor (e.g., a processing circuit, chip and / or module implemented in hardware and / or software and preferably having at least one hardware component) in any device. Illustrative processors include but are not limited to central processing units (CPUs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc., combinations thereof, or any other suitable computing devices known in the art. Additional exemplary components include one or more coprocessors or accelerators, which may be on the same chip with the CPU or may not be on the same chip with the CPU or may be packaged on the same substrate or module.

[0112] like Figure 8 As shown, method 800 may begin at operation 802, where a plurality of ciphertext strings in a data storage device are analyzed. In one embodiment, the data storage device may include a database, a distributed storage system, a cloud computing environment, etc. In another embodiment, the analysis may include comparing a single ciphertext string in the data storage device with other ciphertext strings in the data storage device to determine whether there are any matches (e.g., duplicates) in the data storage device.

[0113] In yet another embodiment, one or more of the plurality of ciphertext strings may be generated by performing wide block encryption on an uncompressed plaintext instance to produce an encrypted uncompressed plaintext instance having a length that matches the length of the uncompressed plaintext instance. In yet another embodiment, one or more of the plurality of ciphertext strings may be created by performing wide block encryption on a compressed plaintext instance to create an encrypted compressed plaintext instance. A length field and variable length zero padding may be added to the encrypted compressed plaintext instance to create a ciphertext string having a length that matches the length of the uncompressed plaintext instance.

[0114] Additionally, method 800 may proceed to operation 804, where one or more redundant ciphertext strings are removed from the data storage based on the analysis. In one embodiment, in response to determining that one or more matches / copies of a given ciphertext string are found within the data storage, one or more instances of the given ciphertext string may be removed (e.g., deleted, migrated, etc.) from the data storage. In another embodiment, duplicate ciphertext strings may be removed from the data storage such that a single instance of the ciphertext string remains in the data storage.

[0115] As a result, deduplication may be performed within the data store, which may maximize the amount of available storage space within the data store, thereby improving the performance of the hardware implementing the data store.

[0116] Fig. 9 An exemplary environment 900 is shown for performing encryption 906 of a compressed instance of plaintext 902 according to an exemplary embodiment. As shown, an uncompressed plaintext instance 904 is received with instructions to encrypt the uncompressed plaintext 904. In response, the uncompressed plaintext instance 904 is compressed 914 to create a compressed plaintext instance 902.

[0117] Additionally, the size of compressed plaintext instance 902 is compared to a predetermined size threshold (e.g., the size of uncompressed plaintext instance 904 is less than the size of zero padding 912 and the size of length field 910) and is found to be less than the predetermined size threshold. In response to this determination, encryption 906 is performed on compressed plaintext instance 902 to create an encrypted compressed plaintext instance 908 having the same size as compressed plaintext instance 902.

[0118] Additionally, the length of compressed plaintext instance 902 is determined and added to encrypted compressed plaintext instance 908 as length field 910. MAC 916 is calculated for the encrypted compressed instance of plaintext 908 and appended to the encrypted compressed instance of plaintext 908. Variable length zero padding 912 is created with a predetermined minimum length and appended to MAC 916.

[0119] In another embodiment, zero padding 912 may first be appended to the cryptographically compressed instance of plaintext 908 , and MAC 916 may be calculated for both zero padding 912 and the cryptographically compressed instance of plaintext 908 .

[0120] In this manner, a ciphertext string is created by combining the length field 910, the encrypted compressed instance of the plaintext 908, and the zero padding 912. The length of the ciphertext string is the same as the uncompressed instance of the plaintext 904 maintaining length preserving compression (LPC).

[0121] Fig.10 An exemplary environment 1000 is shown for performing encryption 1008 of an uncompressed plaintext instance 1004 according to an exemplary embodiment. As shown, an uncompressed plaintext instance 1004 is received, along with instructions to encrypt the uncompressed plaintext 1004. In response, the uncompressed plaintext instance 1004 is compressed 1002 to create a compressed plaintext instance 1006.

[0122] In addition, the size of compressed plaintext instance 1006 is compared to a predetermined size threshold and found to be greater than the predetermined size threshold. In response to this determination, compressed plaintext instance 1006 is discarded and encryption 1008 is performed on uncompressed plaintext instance 1004 to create an encrypted uncompressed plaintext instance 1010 having the same size as uncompressed plaintext instance 1004.

[0123] In this way, length preserving compression (LPC) is maintained.

[0124] Extended LPC encryption for opportunistic authentication

[0125] In one embodiment, a message authentication code (MAC) is inserted into the format of the data that is sufficiently compressible. The addition of the MAC makes the stored data more resilient to manipulation because the ciphertext is no longer malleable. More specifically, it becomes infeasible to replace the portion of the ciphertext of an encrypted sector protected by the MAC.

[0126] Insertion of the MAC is opportunistic in that the MAC is only inserted when the data can be compressed sufficiently to allow space for inserting the MAC.

[0127] In one embodiment, the MAC may be calculated separately from the plaintext (e.g., before encryption). This is referred to as Authentication Then Encryption (AtE). In another embodiment, the MAC may be calculated separately based on the ciphertext produced by encryption. This is referred to as Encryption Then Authentication (EtA).

[0128] In another embodiment, the MAC may be calculated as part of the encryption using an authenticated encryption algorithm such as GCM. In yet another embodiment, when the MAC is calculated separately, an HMAC calculation may be used to create a secure MAC.

[0129] In one embodiment, a method for verifying data units includes: compressing each data unit into compressed data; calculating the length of the compressed data and comparing it to a threshold, wherein if the compressed data length is less than the threshold, the data is sufficiently compressible; if the data of the unit can be sufficiently compressed, encrypting the data and inserting a message authentication code into the format of the data unit; and if the data unit cannot be sufficiently compressed, encrypting the data without inserting a message authentication code.

[0130] In another embodiment, encryption can take multiple forms. For example, a first way to encrypt sufficiently compressible data is to use an initialization vector (IV) field and use an IV-related encryption mode, and the formatting of the data unit includes both the IV field and the MAC; a second way to encrypt sufficiently compressible data is to encrypt the compressed data with a wide block encryption code, and the formatting of the data unit includes a MAC field instead of an IV field.

[0131] In this way, data integrity can be improved by adding authentication to the compressed data (which may or may not be encrypted).

[0132] The present invention may be a system, method and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform an embodiment of the present invention.

[0133] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0134] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0135] The computer-readable program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider via the Internet). In some embodiments, in order to perform embodiments of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions. Additional exemplary circuits include one or more coprocessors or accelerators, which may or may not be on the same chip as the CPU or packaged on the same substrate or module.

[0136] Embodiments of the present invention are described herein with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It will be understood that each frame of the flow chart and / or block diagram and the combination of frames in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0137] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing an embodiment of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0138] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0139] Flowcharts and block diagrams in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames shown continuously can actually be implemented as a step, and are performed simultaneously, substantially simultaneously, in a partially or entirely time-overlapping manner, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of the frames in each frame of the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be implemented by a dedicated hardware-based system that performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0140] In addition, a system according to various embodiments may include a processor and logic integrated with and / or integrable by the processor, the logic being configured to perform one or more of the process steps described herein. By integrated therewith, it is meant that the processor has logic embedded therewith as hardware logic, such as an application specific integrated circuit (ASIC), FPGA, and the like. By executable by the processor, it is meant that the logic is hardware logic; software logic, such as firmware, part of an operating system, part of an application; and the like, or some combination of hardware and software logic that is accessible by the processor and configured to cause the processor to perform a certain function when executed by the processor. The software logic may be stored on local and / or remote memory of any memory type, as is known in the art. Additional exemplary processors include one or more coprocessors or accelerators, which may or may not be on the same chip as the CPU, or packaged on the same substrate or module

[0141] It is clear that the various features of the foregoing systems and / or methods may be combined in any manner, resulting in multiple combinations from the description presented above.

[0142] It should also be appreciated that embodiments of the present invention may be provided as a service deployed on behalf of a client to provide services on demand.

[0143] The description of various embodiments of the present invention has been given for the purpose of illustration, but it is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements existing in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A computer-implemented method comprising: compressing an uncompressed data instance to create a compressed data instance; responsive to determining that the size of the compressed data instance is less than a predetermined threshold, encrypting the compressed data instance, wherein the compressed data instance is encrypted using an encryption technique selected from the group consisting of: wide block encryption, and an extended encryption mode including an initialization vector; creating a message authentication code (MAC) for the encrypted compressed data instance, wherein the MAC comprises a keyed hash message authentication code or a hash-based message authentication code; adding at least zero padding, a length field, and the MAC to the encrypted compressed data instance to create a formatted string having a length equal to the length of the uncompressed data instance, wherein the predetermined threshold comprises a size of a sum of the length field, a predetermined minimum amount of zero padding, and the MAC; comparing the formatted string with other formatted strings in a data storage device to determine if any duplicates exist in the data storage device; In response to determining that a duplicate exists in the data storage device, removing the duplicate; and In response to determining that the size of the compressed data instance is greater than the predetermined threshold: discarding the compressed data instance, and The uncompressed data instance is encrypted to create the formatted string, wherein the encrypted uncompressed data instance has the same length as the uncompressed data instance. 2 . The computer-implemented method of claim 1 , wherein the length field comprises a field indicating the length of the encrypted instance of the compressed data. 3 . The computer-implemented method of claim 1 , wherein the zero padding comprises a variable number of bytes of a highly compressible and easily detectable data pattern. 4 . The computer-implemented method of claim 1 , further comprising storing or transmitting the formatted string, wherein the encrypted compressed data instance comprises at least one ciphertext instance.

5. The computer-implemented method of claim 1, wherein the MAC comprises a Galois Message Authentication Code (GMAC) created as part of a Galois Counter Mode (GCM) encryption mode.

6. The computer-implemented method of claim 1, wherein the MAC is used to verify data integrity and authenticity of the formatted string.

7. A computer-implemented method comprising: compressing an uncompressed data instance to create a compressed data instance, wherein the compressed data instance is encrypted using an encryption technique selected from the group consisting of: wide block encryption, and an extended encryption mode including an initialization vector; In response to determining that the size of the compressed data instance is less than a predetermined threshold: encrypting the compressed data instance, creating a message authentication code MAC for the encrypted compressed data instance, wherein the MAC comprises a Galois Message Authentication Code GMAC created as part of a Galois Counter Mode GCM encryption mode, and adding at least zero padding, a length field, and the MAC to the encrypted compressed data instance to create a formatted string having a length equal to the length of the uncompressed data instance, wherein the predetermined threshold comprises a size of a sum of the length field, a predetermined minimum amount of zero padding, and the MAC; and In response to determining that the size of the compressed data instance is greater than the predetermined threshold: discarding the compressed data instance, and encrypting the uncompressed data instance to create the formatted string, wherein the encrypted uncompressed data instance has the same length as the uncompressed data instance; comparing the formatted string with other formatted strings in a data storage device to determine if any duplicates exist in the data storage device; and In response to determining that a duplicate exists in the data storage device, the duplicate is removed.

8. A computer program product, comprising one or more computer-readable storage media and program instructions stored together on the one or more computer-readable storage media, the program instructions comprising instructions configured to cause one or more processors to execute the method according to any one of claims 1 to 6 or claim 7.

9. A system comprising: Processing unit; as well as A memory, coupled to the processing unit and having instructions stored thereon, which, when executed by the processing unit, perform the method according to any one of claims 1 to 6 or claim 7.

Images