Data deduplication

Abstract

The embodiments of the present invention provide a concept for handling the handover of ownership of data from a source to a referrer in a data deduplication environment. By implementing the handover of ownership of data from a source to a referrer, the number of processes required to access the data can be reduced, and thus the performance of the system can be improved. The identity of the source for implementing the handover can be implemented via a volatile cache.

Description

Technical Field

[0001] The present invention generally relates to the field of data deduplication in computer systems, and more particularly to systems and methods for handling data deduplication in IT systems.

Background Art

[0002] There are multiple implementations for implementing storage deduplication in operation management and IT service management systems. Storage deduplication is designed to ensure that a set of data is stored in the system only once at the grain level. Many host volumes may then refer to this single instance of the stored data rather than requesting separately stored data sets for each host volume.

[0003] A widely adopted model of storage deduplication is called the source and referrer model. The source includes an address to the physical location of data associated with the virtual address of a host volume. When reading the source, the system should access the metadata from the source regarding the physical address of the stored data, extract the physical address of the data from the metadata, and issue a read request to the physical storage to retrieve the stored data.

[0004] The referrer may be written for deduplication that hits an existing source. The referrer indicates a virtual address in the source, and the virtual address indicates the physical location of the stored data.

Summary of the Invention

[0005] The present invention strives to provide a system for handling the handover of data ownership from a source to a referrer. The present invention also strives to provide a system for handling the handover of data ownership from a source to a referrer via a volatile cache. The present invention further strives to provide a method for handling the handover of data ownership from a source to a referrer. Such a method may be performed on a computer. That is, such a method may be performed on a computer infrastructure that explicitly embodies computer executable code on a computer-readable storage medium having programming instructions configured to perform the proposed method. The present invention further strives to provide a computer program product that includes computer program code for performing the proposed concept when executed on a processor.

[0006] According to an aspect of the present invention, a system is provided for handling the handover of data ownership from a source to a referrer, wherein the source includes a virtual address, the virtual address includes the physical address of a set of data in system memory, the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, and the system is adapted to identify the source accessed by the request directed to the source by the referrer, and to perform the handover of ownership of the virtual address of the identified source to the referrer.

[0007] According to another aspect of the present invention, a method is provided for handling the handover of ownership of data from a source to a referrer in a system, wherein the source includes a virtual address, the virtual address includes a physical address of a set of data in system memory, and the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, and the method includes the steps of identifying the source accessed by the request directed to the source by the referrer, and performing a handover of ownership of the virtual address of the identified source to the referrer.

[0008] The embodiments may be used in combination with conventional / existing data handling systems, data deduplication systems, or both. In this way, the embodiments may be integrated with legacy systems to improve or extend their functionality and capabilities, or both. An improved data deduplication system may therefore be provided by the proposed embodiments.

[0009] According to another embodiment of the present invention, a computer program product is provided for handling the handover of data ownership from a source to a referrer, wherein the source includes a virtual address, the virtual address includes a physical address of a set of data in system memory, the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, and the computer program product comprises a computer-readable storage medium embodying program instructions, and the program instructions are executable by a processing unit such that the processing unit implements the method according to one or more proposed embodiments when executed by at least one processor of a data processing system.

[0010] In yet another embodiment, a processing system is provided comprising at least one processor and a computer program product according to one or more embodiments, wherein the at least one processor is adapted to execute the computer program code of the computer program product.

[0011] Therefore, there may be proposed concepts for handling the handover of data ownership from source to referrer. For example, an embodiment may provide means for performing the handover of ownership of the source's virtual address to the referrer. By handing over ownership of the virtual address in this way, the number of processes required to access the data can be reduced, and thus the performance of the deduplication system can be improved.

[0012] The present invention will be described in the following detailed description with reference to the drawn drawings, as a non-limiting example of exemplary embodiments of the present invention. [Brief explanation of the drawing]

[0013] [Figure 1] This is a diagram of a cloud computing node according to an embodiment of the present invention. [Figure 2] This is a diagram of a cloud computing environment according to an embodiment of the present invention. [Figure 3] This is a diagram of an abstraction model layer according to an embodiment of the present invention. [Figure 4] This is a diagram of a cloud computing node according to another embodiment of the present invention. [Figure 5] This is a schematic diagram of a system comprising a source and a referrer according to an embodiment of the present invention. [Figure 6] This is a schematic diagram of a system equipped with a volatile cache according to a further embodiment of the present invention. [Figure 7] This is a schematic diagram of a system including multiple sources according to a further embodiment of the present invention. [Figure 8] This is a schematic diagram of a system including multiple referrers according to a further embodiment of the present invention. [Figure 9] This is a schematic diagram of a system comprising a handover component according to a further embodiment of the present invention. [Figure 10] This is a diagram illustrating a method for handling the handover of data ownership from a system source to a referrer, according to an embodiment of the present invention. [Modes for carrying out the invention]

[0014] Please understand that the diagrams are only schematic and are not drawn to be scaled. Also, please understand that the same reference numbers are used throughout the diagrams to indicate the same or similar parts.

[0015] In the context of this application, where embodiments of the present invention constitute a method, it should be understood that such a method is a process for execution by a computer, i.e., a computer-executable method. The various steps of the method thus reflect various components of a computer program, such as various components of one or more algorithms.

[0016] Furthermore, in the context of this application, the (processing) system may be a single device or a collection of distributed devices adapted to perform one or more embodiments of the method of the present invention. For example, the system may be a personal computer (PC), a server, or a collection of PCs or servers connected via a network such as a local area network or the Internet to cooperate in performing at least one embodiment of the method of the present invention.

[0017] Furthermore, in the context of this application, the system may be a single device or a collection of distributed devices adapted to perform one or more embodiments of the method of the present invention. For example, the system may be a personal computer (PC), a portable computing device (such as a tablet computer, laptop, or smartphone), a set-top box, a server, or a collection of PCs or servers connected via a network such as a local area network or the Internet to cooperate in performing at least one embodiment of the method of the present invention.

[0018] The technical features of the present invention generally relate to data handling, and more particularly, for example, to the concept of data deduplication that can implement the handover of data ownership in a system. More specifically, an embodiment of the present invention provides a system for handling the handover of data ownership from a source to a referrer, where the source includes a virtual address, the virtual address includes the physical address of a set of data on system memory, the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, the system identifies the source accessed by the request directed to the source by the referrer, and is adapted to perform the handover of the ownership of the identified source's virtual address to the referrer.

[0019] The present invention provides means for handling the handover of data ownership from a source to a referrer. The source and referrer models are a widely adopted implementation of storage deduplication intended to ensure that data is stored only once at the grain level. The source includes the physical location of data associated with the virtual address of a volume. When reading the source, the system may extract the physical location of the data from the source's metadata and then issue a read request to physical storage to retrieve the data itself. The referrer is written for deduplication that hits an existing source. The referrer indicates a virtual address in the source, and the virtual address indicates the physical location of the data. By performing the handover of the ownership of the source's virtual address to the referrer, the number of processes required to access the data can be reduced, and thus the performance of the deduplication system can be improved.

[0020] The drawback of the source and referrer models is due to the referrer requiring additional metadata access operations that can have an adverse impact on system performance when attempting to access the referenced data to perform a read to the address.

[0021] A further drawback of the source and referrer model is that it can occur when a rehome procedure needs to be performed. The rehome procedure is a procedure that occurs when the source can no longer hold data, such as when the user requests to delete the source volume, and is an action to change the ownership of the data from the original source to one of the referrers. When multiple referrers are linked to the same source data, it is ideal for these referrers to all point to the new source after the rehome procedure, but this may be prevented by other processes running on the system, and the deduplication ratio is reduced by multiple instances of the stored data.

[0022] Therefore, there is a need to improve the means of handling data deduplication in the system.

[0023] In an embodiment, the system comprises a volatile cache adapted to identify the source accessed by a request directed to the source by a referrer. By using the volatile cache to identify the source accessed by a request directed to the source by a referrer, the system resources required to identify the source can be reduced without the need to strengthen or mirror the data on the volatile cache. In an embodiment, the volatile cache is read-only.

[0024] In an embodiment, the system comprises a plurality of sources, and the volatile cache is adapted to generate a list of sources accessed by requests directed to each source by the associated referrers. In this way, the volatile cache may include a list of sources on the system that may be candidates for performing a handover process.

[0025] In this embodiment, the volatile cache is adapted to monitor the characteristics of the source over time, and the volatile cache is adapted to sort the list of sources according to the monitored characteristics. Thus, the most impactful sources may be brought to the top of the volatile cache based on the given monitored characteristics of the source.

[0026] In this embodiment, the characteristic being monitored is the frequency of source access. Since the more frequently a source is accessed, the greater its impact on system performance, the most frequently accessed source may be placed at the top of the source list.

[0027] In one embodiment, the system includes multiple referrers pointing to a source, and the volatile cache is further adapted to generate a list of referrers that direct requests to the source. In this way, it may be possible to monitor where requests to access the source originate.

[0028] In one embodiment, the volatile cache is adapted to sort the list of referrers according to the frequency of requests directed to the source by each referrer. Thus, the referrer that directs the largest number of access requests to the source may be identified as a more impactful referrer for a given source.

[0029] In the embodiment, the system further comprises a handover component adapted to perform a handover of ownership of the identified source's virtual address to the referrer.

[0030] In the embodiment, the system includes a volatile cache adapted to identify multiple sources accessed by requests directed to each of the sources by associated referrers, the volatile cache adapted to generate a list of the multiple sources accessed, and a handover component adapted to select one or more sources from the list of sources and perform a handover of virtual address ownership from the selected one or more sources to the referrers associated with each selected source. Thus, the handover component may select one or more sources from the list of sources as candidates for performing the handover process.

[0031] In the embodiment, the volatile cache is adapted to monitor the characteristics of the source over time, and the handover component is adapted to select one or more sources from a list of sources based on the monitored characteristics. Thus, the handover component may perform a handover process to one or more sources from a list of sources that are considered to have the greatest impact on system performance according to the monitored characteristics.

[0032] In the embodiment, the handover component is adapted to perform a handover of ownership of the virtual address of the identified source to the referrer when the source is accessed by a request. Thus, the handover component may opportunistically perform the handover process when the source is being accessed.

[0033] In the embodiment, the system is further adapted to update the list of sources to remove the source that has undergone the handover process after the handover of ownership of a virtual address from the source to the referrer has been performed. Thus, the volatile cache may be dynamically updated in response to sources that have undergone the handover process, and as a result, the list of sources will dynamically display the source that has the most impact on the system at that time.

[0034] In the embodiment, the system is further adapted to update the source and referrer to direct incoming requests to the virtual address in the referrer after the handover of ownership of the virtual address from the source to the referrer has been performed. Thus, references to the source that have not yet been updated will be directed to the referrer after the handover process has been performed.

[0035] Embodiments of the present invention further provide a method for handling the handover of data ownership from a source to a referrer in a system, wherein the source includes a virtual address, the virtual address includes the physical address of a set of data in system memory, and the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, and the method includes the steps of identifying the source accessed by the request directed to the source by the referrer, and performing a handover of ownership of the virtual address of the identified source to the referrer.

[0036] In embodiments, the system includes a plurality of sources, and the method further includes generating a list of sources accessed by requests directed to each source by associated referrers via a volatile cache, selecting one or more sources from the list of sources via a handover component, and performing a handover of virtual address ownership from the selected one or more sources to the referrers associated with each selected source via the handover component.

[0037] In the embodiment, the method further includes monitoring the characteristics of a source over time via a volatile cache, and selecting one or more sources from a list of sources via a handover component based on the monitored characteristics.

[0038] Embodiments of the present invention further provide a computer program product for handling the handover of ownership of data from a source to a referrer, wherein the source includes a virtual address, the virtual address includes a physical address of a set of data in system memory, the referrer is adapted to direct an incoming request for the set of data to the virtual address of the source, and the computer program product comprises a computer-readable storage medium embodying program instructions, the program instructions are executable by a processing unit to perform a method including identifying a source accessed by a request directed to the source by the referrer, and performing a handover of ownership of the virtual address of the identified source to the referrer.

[0039] In the embodiment, the computer program product comprises a computer-readable storage medium embodying program instructions, and the processing unit can perform a method that further includes the steps of generating a list of sources accessed by requests directed to each source by associated referrers via a volatile cache, selecting one or more sources from the list of sources via a handover component, and performing a handover of virtual address ownership from the selected one or more sources to the referrers associated with each selected source via the handover component.

[0040] Embodiments of the present invention further provide a processing system comprising the above-described at least one processor and computer program product, wherein the at least one processor is adapted to execute the computer program code of the computer program product.

[0041] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementations of the techniques enumerated herein are not limited to cloud computing environments. Rather, embodiments of the present invention are capable of being implemented in conjunction with any other type of computing environment, whether currently known or to be developed later.

[0042] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly delivered and released with minimal administrative effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0043] The characteristics are as follows:

[0044] On-demand self-service: Cloud users can unilaterally access computing power, such as server time and network storage, automatically and as needed, without requiring human interaction with service providers.

[0045] Broad network access: Capabilities are available over a network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0046] Resource pooling: A provider's computing resources are pooled to serve multiple users using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated as needed. Location independence generally means that users have no control or knowledge of the exact location of the resources provided, although they may be able to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0047] Rapid resilience: Capabilities can be provided quickly and flexibly, sometimes automatically, to scale out rapidly, and released quickly to scale in rapidly. To the user, the available capacity for provision often appears unlimited and can be purchased in any quantity at any time.

[0048] Measured Services: Cloud systems automatically control and optimize resource usage by leveraging metric capabilities at several levels of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage is monitorable, controllable, and reportable, providing transparency to both service providers and users.

[0049] The service model is as follows:

[0050] Software as a Service (SaaS): The ability provided to the user is to use the provider's applications running on a cloud infrastructure. These applications are accessible from various client devices through thin-client interfaces, such as web browsers (e.g., web-based email). Users have no management or control over the underlying cloud infrastructure, including the network, servers, operating system, storage, or possibly individual application capabilities, with the exception of limited user-specific application configuration settings.

[0051] Platform as a Service (PaaS): The ability provided to the user is to deploy user-created or acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The user does not manage or control the underlying cloud infrastructure, including the network, servers, operating system, or storage, but controls the deployed applications and, in some cases, the configuration of the application hosting environment.

[0052] Infrastructure as a Service (IaaS): The capability provided to the user is to offer processing, storage, networking, and other basic computing resources that the user can deploy and run any software, including operating systems and applications. The user does not manage or control the underlying cloud infrastructure, but has control over the operating system, storage, deployed applications, and, in some cases, limited control over selected networking components (e.g., host firewalls).

[0053] The deployment model is as follows:

[0054] Private Cloud: The cloud infrastructure is operated solely for the organization. The cloud infrastructure may be managed by the organization or a third party, and may be located on or off-site.

[0055] Community Cloud: Cloud infrastructure is shared by several organizations and supports a unique community that has shared concerns (e.g., mission, security requirements, policies, and compliance considerations). Cloud infrastructure may be managed by the organization or a third party, and may be located on-site or off-site.

[0056] Public cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0057] Hybrid Cloud: Cloud infrastructure remains a unique entity, but it is a composite of two or more clouds (private, community, or public) bound together by standard or proprietary technologies (e.g., cloud bursting for load balancing across clouds) that enable data and application portability.

[0058] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure consisting of a network of interconnected nodes.

[0059] Referring here to Figure 1, an schematic diagram of an example of a cloud computing node is shown. Cloud computing node 10 is merely one example of a suitable cloud computing node and is not intended to imply any limitation on the scope of use or functionality of the embodiments of the present invention described herein. Nevertheless, cloud computing node 10 is capable of performing or implementing any or both of the functions described above.

[0060] The cloud computing node 10 has a computer system / server 12 that can operate with a number of other general-purpose or dedicated computing system environments or configurations. Examples of well-known computing systems, environments, or configurations, or combinations thereof, that may be suitable for use with the computer system / server 12 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, as well as similar ones.

[0061] The computer system / server 12 may be described in the general context of computer system executable instructions, such as program modules, that are executed by the computer system. Generally, a program module may include routines, programs, objects, components, logic, data structures, etc., that perform a specific task or execute a specific abstract data type. The computer system / server 12 may be practiced in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may reside on both local and remote computer system storage media, including memory storage devices.

[0062] As shown in Figure 1, the computer system / server 12 in the cloud computing node 10 is represented in the form of a general-purpose computing device. The components of the computer system / server 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components, including the system memory 28, to the processor 16.

[0063] Bus 18 represents one or more of several types of bus structures, including memory buses or memory controllers, peripheral buses, accelerated graphics ports, and processor or local buses using any of the various bus architectures. Examples, and not limited to, such architectures include the Industry Standard Architecture (ISA) bus, Microchannel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0064] The computer system / server 12 typically includes various computer system-readable media. Such media may be any available media accessible by the computer system / server 12, and may include both volatile and non-volatile media, as well as both removable and non-removable media.

[0065] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30, or cache memory 32, or both. The computer system / server 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. In just one example, the storage system 34 can be provided for reading and writing non-removable non-volatile magnetic media (not shown, but typically called “hard drives”). Not shown, a magnetic disk drive for reading and writing removable non-volatile magnetic disks (e.g., “floppy disks”) and an optical disk drive for reading and writing removable non-volatile optical disks such as CD-ROMs, DVD-ROMs, or other optical media may be provided. In such examples, each can be connected to the bus 18 by one or more data media interfaces. As further described and mentioned below, the memory 28 may include at least one program product having a set of program modules (e.g., at least one) configured to perform the functions of embodiments of the present invention.

[0066] The program / utility 40 has a set (at least one) of program modules 42, which may be stored in memory 28, as well as, for example and not limited to, an operating system, one or more application programs, other program modules, and program data. Each or several combinations of the operating system, one or more application programs, other program modules, and program data may include an implementation of a networking environment.

[0067] The program module 42 generally performs functions or methods, or both, of the embodiments of the present invention as described herein. For example, some or all of the functions of the DHCP client 80 can be performed as one or more of the program modules 42. In addition, the DHCP client 80 may be run as a separate dedicated processor, or as one or more processors, to provide the functions described herein. In embodiments, the DHCP client 80 performs one or more of the processes described herein.

[0068] The computer system / server 12 may also communicate with one or more external devices 14 such as a keyboard or pointing device, one or more devices such as a display 24 that enable a user to interact with the computer system / server 12, or any device (e.g., a network card, modem, etc.) or a combination thereof that enables the computer system / server 12 to communicate with one or more other computing devices. Such communication can occur via the I / O interface 22. Furthermore, the computer system / server 12 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof, via the network adapter 20. As depicted, the network adapter 20 communicates with other components of the computer system / server 12 via the bus 18. It should be understood that other hardware components or software components, or both, that are not shown, may be used in conjunction with the computer system / server 12. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID (redundant array of inexpensive disks, or redundant array of independent disks) systems, tape drives, and data archive storage systems.

[0069] Referring here to Figure 2, an illustrative cloud computing environment 50 is depicted. As shown in the figure, the cloud computing environment 50 comprises one or more cloud computing nodes 10 with which local computing devices used by cloud users can communicate, such as a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, or an automotive computer system 54N, or a combination thereof. The nodes 10 may communicate with each other. The nodes 10 may be physically or virtually grouped in one or more networks, such as a private, community, public, or hybrid cloud, or a combination thereof, as described above (not shown). This allows the cloud computing environment 50 to provide infrastructure, a platform, or software as a service, or a combination thereof, without requiring cloud users to maintain resources on their local computing devices. The types of computing devices 54A-N shown in Figure 2 are intended to be illustrative only, and it is understood that the computing node 10 and the cloud computing environment 50 can communicate with any type of computerized device via any type of network, or a network addressable connection (e.g., using a web browser), or both.

[0070] Referring here to Figure 3, a set of functional abstraction layers provided by the cloud computing environment 50 (Figure 2) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 3 are for illustrative purposes only, and embodiments of the present invention are not limited thereto. The following layers and corresponding functions are provided as described:

[0071] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include a mainframe 61, a RISC (Reduced Instruction Set Computer) architecture-based server 62, a server 63, a blade server 64, a storage device 65, and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0072] The virtualization layer 70 provides an abstraction layer from which examples of virtual entities can be derived, such as virtual servers 71, virtual storage 72, virtual networks 73 including virtual private networks, virtual applications and operating systems 74, and virtual clients 75.

[0073] In one example, the management layer 80 may provide the functions described below. Resource provision 81 dynamically procures computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 82 tracks costs as resources are used within the cloud computing environment and bills or invoices for the usage of these resources. In one example, these resources may include application software licenses. Security verifies the identity of cloud users and tasks, and protects data and other resources. The user portal 83 provides users and system administrators with access to the cloud computing environment. Service level management 84 allocates and manages cloud computing resources to ensure that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 pre-positions and procures cloud computing resources for anticipated future requirements in accordance with SLAs.

[0074] The workload layer 90 provides examples of functions that can be utilized in a cloud computing environment. Examples of workloads and functions that can be provided from this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education delivery 93, data analysis processing 94, transaction processing 95, and the data deduplication process 96 described herein. According to aspects of the present invention, the workload / function of the data deduplication process 96 operates to perform one or more of the processes described herein.

[0075] Figure 4 depicts a cloud computing node according to another embodiment of the present invention. In particular, Figure 4 is another cloud computing node, which includes the same cloud computing node 10 as in Figure 1. In Figure 4, the computer system / server 12 also includes, or communicates with, a data deduplication client 170 and a data deduplication server 160.

[0076] According to aspects of the present invention, the data deduplication client 170 is executable as one or more program codes in a program module 42 stored in memory as separate or combined modules. Furthermore, the data deduplication client 170 may run as a separate dedicated processor or as one or more processors to provide the functionality of these tools. While the computer program code is running, the processing unit 16 can read from and write to memory, a storage system, or an I / O interface 22, or a combination thereof. The program code performs the process of the present invention.

[0077] For example, the data deduplication client 170 may be configured to communicate with the data deduplication server 160 via a cloud computing environment 50. As discussed with reference to Figure 2, the cloud computing environment 50 may be, for example, the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. In embodiments of the proposed data deduplication mechanism, the data deduplication server 160 may provide data to the client 170. Those skilled in the art will understand that the data deduplication client 170 and the data deduplication server 160 may communicate directly. Alternatively, a relay agent may be used as an intermediate for relaying messages between the data deduplication client 170 and the data deduplication server 160 via the cloud computing environment 50.

[0078] The present invention may be a system, method, or computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium (or a set of mediums) having computer-readable program instructions for causing a processor to execute an aspect of the present invention.

[0079] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction-executing device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. A less-than-complete list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved-reinforced structures on which instructions are recorded, and any suitable combination thereof. Computer-readable storage media as used herein should not be interpreted as inherently transient signals such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0080] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. The network adapter card or network interface of each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on the computer-readable storage media within each computing / processing device.

[0081] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk(R), C++, or similar, and conventional procedural programming languages ​​such as the C programming language or similar programming languages. The computer-readable program instructions may be executed as a standalone software package, either entirely or partially on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, electronic circuit devices, including programmable logic circuit devices, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions by using state information of computer-readable program instructions to personalize the electronic circuit device in order to carry out aspects of the present invention.

[0082] Aspects of the present invention will be described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in the flowchart or block diagram, or both, and combinations of blocks in the flowchart or block diagram, or both, are executable by computer-readable program instructions.

[0083] These computer-readable program instructions may be provided to a general-purpose computer, a dedicated computer, or a processor of another programmable data processing device to generate a machine that generates means for instructions to be executed via the processor of a computer or other programmable data processing device to perform functions / actions specified in one or more blocks of a flowchart or block diagram, or both. These computer-readable program instructions may also be stored on a computer-readable storage medium so that the computer-readable storage medium containing the instructions can provide a product containing instructions to perform a manner of function / action specified in one or more blocks of a flowchart or block diagram, or both, and can instruct a computer, a programmable data processing device, or other device, or a combination thereof, to function in a particular manner.

[0084] Computer-readable program instructions may also be loaded onto a computer, other programmable device, or other device to perform a series of operational steps on the computer, other programmable device, or other device in order to generate computer execution processing in order to produce computer execution processing in order to perform a function / action specified in one or more blocks of a flowchart or block diagram or both.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for performing a specified logical function. In some alternative implementations, the functions described in a block may be performed independently of the order shown in the figure. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or blocks may sometimes be executed in reverse order depending on the functions they contain.

[0086] Figure 5 provides a schematic diagram of a system flow for a system 200 for handling the handover of data ownership from a source 210 to a referrer 220, according to one aspect of the present invention. The source has a virtual address, which includes the physical address of a set of data in system memory, and the referrer is adapted to direct incoming requests 230 for the set of data to the virtual address of the source.

[0087] The example shown in Figure 5 illustrates the typical operation of system 200 before the handover of data ownership from source to referrer takes place. In particular, Figure 5 shows an incoming request 230 received at referrer 220, which is then directed to source 210. The requested data 240 may then be provided to the origin of the request, such as the user or process that generated the received request.

[0088] For example, a user may start a process on a system that communicates with system 200 to handle the handover of data ownership from source 210 to referrer 220. The started process may request access to data that has undergone data deduplication, meaning that a data request 230 is received at referrer 220. The request is then directed by referrer 220 to source 210 to retrieve the requested data 240 that may then be provided to the process.

[0089] Figure 6 provides a schematic diagram of a system flow for a system 200 for handling the handover of data ownership from source 210 to referrer 220, according to one aspect of the present invention. In the example shown in Figure 6, the system is adapted to identify source 210 accessed by a request directed to source 210 by referrer 220, and to perform the handover of ownership of the virtual address of the identified source 210 to referrer 220.

[0090] In the specific example shown in Figure 6, the system includes a volatile cache 250 adapted to identify the source 210 accessed by a request 230 directed to the source 210 by a referrer 220. The volatile cache may be any form of volatile memory capable of storing data about the system's source 210. For example, the volatile cache 250 may be formed on a dynamic volatile memory unit or a static volatile memory unit.

[0091] The volatile cache 250 may be adapted by any suitable means to identify the source 210 accessed by a request 230 directed to the source 210 by a referrer 220. For example, the volatile cache may communicate with the system's referrer 220 and be adapted to retrieve or receive data from the referrer 220 regarding the incoming request and to which source 210 the referrer directed the incoming request. Alternatively, the volatile cache may communicate with the source 210 and be adapted to retrieve or receive data from the source 210 in response to the request being received at the source 210. In a further alternative, the volatile cache may communicate with a system input unit that receives the request 230 to the system and be adapted to retrieve or receive data about the request 230 as the communication progresses through the system in order to identify the source 210 to which the request 230 is directed. The volatile cache may be adapted to store identification information of the source 210 thus accessed.

[0092] Since the volatile cache 250 is volatile memory, there is no need to enhance or mirror the data stored in the volatile cache 250. This has the benefit of speeding up metadata access to the source 210 or source chunks held in the volatile cache 250. Furthermore, the volatile cache 250 may provide a list of impactful candidate sources for performing handovers as discussed further below. In other words, it is proposed to include the volatile cache 250 in addition to the conventional caches that form part of the system, and the volatile cache 250 is dedicated to the deduplication sources present in the system. The volatile cache 250 may be read-only and can be used to optimize data access in source 210 and referrer 220 systems where source 210 is frequently accessed, as will be discussed further below.

[0093] The volatile cache 250 can be relatively small in terms of memory footprint because it does not need to retain data stored by the source itself. Rather, the volatile cache only needs to store source metadata for specific data addresses of interest. Thus, using a volatile cache eliminates the need to perform metadata access operations to access source metadata that is well managed at a larger source group, rather than at the level of individual addresses. Therefore, the volatile cache 250 can make data deduplication systems more efficient.

[0094] Figure 7 provides a schematic diagram of a system flow for a system 200 for handling the handover of data ownership from a source 210 to a referrer 220, according to one aspect of the present invention. In the example shown in Figure 7, the system comprises a plurality of sources 210 (source 1, source 2, ..., source N) and a corresponding plurality of referrers 220 (referrer 1, referrer 2, ..., referrer N), which are adapted to direct an incoming request 230 to a plurality of sources 210.

[0095] In the example shown in Figure 7, the volatile cache 250 may be further adapted to generate a list 252 of sources accessed by requests 210 directed to each source via an associated referrer 220. In other words, since each source 210 is accessed by requests 230 directed to source 210 via referrer 220, each source 210 may be added to the list of sources 210 maintained in the volatile cache. The volatile cache 250 may be adapted to monitor the characteristics of the sources 210 over time and sort the list of sources 210 according to the monitored characteristics. For example, the monitored characteristics may be the frequency of access to source 210. However, the monitored characteristics may be any characteristics of source 210, such as the frequency of access, the number of referrers associated with a given source 210, the time since source 210 was last accessed, the duration of existence of source 210, the characteristics of the data stored in source 210, and so on.

[0096] If the volatile cache 250 is adapted to sort List 252 according to the access frequency of sources 210, for example using an active replacement cache (ARC) or heap protocol, then the sources 210 most frequently accessed by their corresponding referrers will rise to the top of List 252. Sources that are rarely accessed, even if they have many references, will move to the bottom of List 252 maintained in the volatile cache 250. Frequently accessed sources 210 may be called hot sources, and infrequently accessed sources 210 may be called cold sources.

[0097] When system 200 has been running for a sufficient amount of time to populate the volatile cache 250, and the most frequently accessed sources have risen to the top of the list 252 maintained in the volatile cache, the system may select one or more sources from the top of the list and perform a handover of data ownership from the sources to their corresponding referrers. The handover of data ownership may be performed proactively by selecting one or more sources from the top of the list and performing the handover, or opportunistically by performing a handover to a source at or near the top of the list when a source is accessed by a referrer.

[0098] Figure 8 provides a schematic diagram of a system flow for a system 200 for handling the handover of data ownership from a source 210 to a referrer 220, according to one aspect of the present invention. In the example shown in Figure 8, the system comprises a plurality of referrers 220 (referrer 1, referrer 2, ..., referrer N) pointing to a source 210. In this case, the volatile cache 250 may be further adapted to generate a list 254 of referrers 220 that direct requests to the source 210, and the volatile cache 250 may be adapted to sort the list 254 of referrers 220 according to how often a request 230 is directed to the source 210 by each referrer 220.

[0099] In other words, if source 210 has two or more referrers 220 associated with source 210, the volatile cache 250 may record data about the referrers 220, such as the frequency of access by each referrer 220 and referrer metadata. In this case, since the referrer 220 that directs requests 230 to source 210 with the highest frequency may have the greatest impact on system performance compared to the other referrers 220 associated with source 210, a handover of data ownership from source 210 to referrer 220 may be performed so that it may receive ownership of the data.

[0100] Figure 9 provides a schematic diagram of a system flow for a system 200 for handling the handover of data ownership from a source 210 to a referrer 220, according to one aspect of the present invention. In the example shown in Figure 9, the system further comprises a handover component 260 adapted to perform a handover 270 of ownership of the virtual address of an identified source 210 to the referrer 220. As described above with reference to Figure 7, the system may also comprise a volatile cache 250 adapted to identify multiple sources 210 accessed by requests directed to each of the sources 210 by the associated referrer 220 and to generate a list of the multiple sources 210 accessed.

[0101] The handover component 260 may be adapted to select one or more sources 210 from a list of sources and to perform a handover 270 of virtual address ownership from the selected one or more sources 210 to the referrers 220 associated with each selected source 210. For example, the volatile cache 250 may be adapted to monitor the characteristics of the sources 210 over time, such as the access frequency of each source 210, and the handover component 260 may be adapted to select one or more sources 210 from a list of sources based on the monitored characteristics.

[0102] For example, the handover component 260 may be adapted to select the top candidate source, i.e., source 210, which is at the top of the list maintained in the volatile cache 250, from the volatile cache 250, and to perform a handover 270 of ownership of the data stored in source 210 to referrer 220. The handover component 260 may also be adapted to leave a hint of the data ownership change in source 210, so that other referrers 220 looking at source 210 are redirected to look at the new owner of the data, i.e., the referrer that now holds ownership of the data. In other words, the system may be further adapted to update source 210 and referrer 220 to direct incoming requests 230 to the virtual addresses in referrer 220 after the handover of ownership of the virtual addresses from source 210 to referrer 220 has been performed.

[0103] When a handover is being performed, both the old and new sources (i.e., the original source and the original referrer) may store the number of references associated with the old source that have not yet been updated. This is the number of references associated with a source that is not yet aware of the handover of data ownership and may therefore be handled in a special way to avoid data integrity issues. For example, if an unupdated referrer points to the original source, the original source may point the referrer to the new source. Furthermore, the referrer may then be updated to automatically point to the new source in the future, and the number of unupdated references maintained in both the old and new sources may be reduced.

[0104] The system may be further adapted to update the list of sources maintained in the volatile cache to remove the source that underwent the handover process after the handover of ownership of a virtual address from source to referrer has been performed. Thus, the list of sources maintained in the volatile cache may be constantly updated to reflect the most recent handover of ownership within the system.

[0105] The present invention may consist of two main components: a volatile cache and a handover component, which work together to support proactive or opportunistic ownership changes of the most impactful data sources within the system. By combining these components in the manner described above, a system for handling data ownership handovers in a deduplication system can be adapted over time to consume minimal resources and greatly improve the performance of the deduplication system.

[0106] Figure 10 illustrates a method 300 for handling the handover of data ownership from a source to a referrer in a system, according to one aspect of the present invention. The method begins in step 310 by identifying the source accessed by a request directed to the source by the referrer.

[0107] In step 320, a list of sources accessed by requests directed to each source by the associated referrer is generated. The list of sources may be generated via a volatile cache, as described above. In step 330, one or more sources are selected from the list of sources, for example, via a handover component. One or more sources may be selected based on monitored characteristics of the sources, such as the frequency of source access. For example, the most frequently accessed source may be selected.

[0108] In step 340, ownership of virtual addresses is handed over from one or more selected sources to the referrers associated with each selected source via the handover components.

[0109] In embodiments of the present invention, it will be understood by those skilled in the art that the proposed concept offers numerous advantages over conventional data deduplication and data ownership handover approaches. These advantages include, but are not limited to, efficient and accurate handover of data ownership from source to referrer. In embodiments of the present invention, this technical solution is achieved via a volatile cache.

[0110] A further advantage over technical issues is that the systems and processes described herein provide computer execution methods for efficient data deduplication and data ownership handover delivered over (or via) a distributed communication network. In this case, a computer infrastructure such as the computer systems shown in Figures 1 and 4, or the cloud environment shown in Figure 2, becomes available, and one or more systems for carrying out the processes of the present invention can be acquired (e.g., created, purchased, used, modified, etc.) and deployed to the computer infrastructure. To this extent, the deployment of the systems may include one or more of the following:

[0111] (i) Install program code from a computer-readable medium onto a computing device such as the computer system shown in Figure 1.

[0112] (ii) Add one or more computing devices to the computer infrastructure, and more specifically, to the cloud environment.

[0113] (iii) Incorporating, modifying, or both, one or more existing systems of a computer infrastructure to enable the computer infrastructure to carry out the processes of the present invention.

[0114] While various embodiments of the present invention have been presented for illustrative purposes, they are not intended to be exhaustive or to limit oneself to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the embodiments described. The terminology used herein has been chosen to best describe the principles, practical applications, or technical improvements to the technologies available on the market of the embodiments, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computer execution method for volume allocation in dynamic volume pooling, Identifying a source accessed by a request directed to the source by a referrer, wherein the source includes a virtual address, the virtual address includes the physical address of a set of data in system memory, and the referrer directs an incoming request for the set of data to the virtual address of the source. The handover of ownership of the virtual address of the identified source to the referrer is performed by one or more computer processors. A computer execution method that includes, and by performing the handover, can reduce the number of processes required to access the data.

2. Identifying, on one or more computer processors, the source accessed by the request directed to the source by the referrer, using a list of referrers that direct requests in the volatile cache back to the source. The computer execution method according to claim 1, further comprising:

3. The aforementioned source includes multiple sources, One or more computer processors generate a list of sources for the multiple sources in a volatile cache that is accessed by the request directed to each of the multiple sources by the associated referrer of the multiple sources. It further includes, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer execution method according to claim 1.

4. Monitoring one or more characteristics of the aforementioned source over time using one or more computer processors, Sort the list of sources for the plurality of sources according to one or more of the characteristics described above using one or more computer processors. It further includes, The one or more characteristics mentioned above are the access frequency of each of the multiple sources, the number of referrers associated with a given source, the time since the source was last accessed, the period during which the source existed, or the characteristics of the data stored in the source. The computer execution method according to claim 3.

5. The computer execution method according to claim 4, wherein the one or more characteristics include at least the access frequency of each of the plurality of sources.

6. The aforementioned referrer includes multiple referrers, A list of referrers for the multiple referrers in the volatile cache that direct requests to the source is generated by one or more computer processors. It further includes, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer execution method according to claim 2.

7. The computer execution method according to claim 6, wherein the list of referrers is sorted based on the frequency of requests directed to the source by each of the plurality of referrers.

8. The computer execution method according to claim 7, wherein access to the volatile cache from the referrer is restricted to read-only.

9. A computer program for volume allocation in dynamic volume pooling, The computer is made to identify the source accessed by a request directed to the source by a referrer, the source includes a virtual address, the virtual address includes the physical address of a set of data in system memory, and the referrer directs an incoming request for the set of data to the virtual address of the source. The computer is instructed to perform a handover of ownership of the virtual address of the identified source to the referrer. By performing this handover, the number of processes required to access the data can be reduced. Computer program.

10. The computer program according to claim 9, wherein the computer is caused to identify the source accessed by the request directed to the source by the referrer, using a list of referrers that direct requests in a volatile cache to the source.

11. The aforementioned source includes multiple sources, The computer generates a list of sources for the plurality of sources in the volatile cache that are accessed by the request directed to each of the plurality of sources by the associated referrer of the plurality of sources, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer program according to claim 9.

12. To the aforementioned computer, Monitoring one or more characteristics of the aforementioned source over time, Sort the list of sources for the plurality of sources according to one or more of the aforementioned characteristics. Have them do it, The one or more characteristics mentioned above are the access frequency of each of the multiple sources, the number of referrers associated with a given source, the time since the source was last accessed, the period during which the source existed, or the characteristics of the data stored in the source. The computer program according to claim 11.

13. The computer program according to claim 12, wherein one or more of the characteristics include at least the access frequency of each of the plurality of sources.

14. The aforementioned referrer includes multiple referrers, The computer is made to generate a list of referrers for the plurality of referrers in the volatile cache that direct requests to the source, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer program according to claim 10.

15. A computer system for volume allocation in dynamic volume pooling, One or more computer processors, One or more computer-readable storage media, A program instruction stored in one or more computer-readable storage media for execution by at least one of the one or more computer processors, A program instruction for identifying a source accessed by a request directed to the source by a referrer, wherein the source includes a virtual address, the virtual address includes a physical address of a set of data in system memory, and the referrer directs an incoming request for the set of data to the virtual address of the source, and Program instructions for performing the handover of ownership of the virtual address of the identified source to the referrer The program instructions include and A computer system that includes such a feature and, by performing the handover, can reduce the number of processes required to access the data.

16. Identifying the source accessed by the request directed to the source by the referrer using a list of referrers that direct requests in the volatile cache back to the source. The computer system according to claim 15, further comprising program instructions stored in one or more computer-readable storage media for execution by at least one of the one or more computer processors for performing the above.

17. The aforementioned source includes multiple sources, To generate a list of sources in the volatile cache accessed by the request directed to each of the multiple sources by the associated referrer of the multiple sources. The program instructions stored in the one or more computer-readable storage media are for execution by at least one of the one or more computer processors, for the purpose of performing the above, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer system according to claim 15.

18. Monitoring one or more characteristics of the aforementioned source over time, Sort the list of sources for the plurality of sources according to one or more of the aforementioned characteristics. The program instructions stored in the one or more computer-readable storage media are for execution by at least one of the one or more computer processors, for the purpose of performing the above, The one or more characteristics mentioned above are the access frequency of each of the multiple sources, the number of referrers associated with a given source, the time since the source was last accessed, the period during which the source existed, or the characteristics of the data stored in the source. The computer system according to claim 17.

19. The computer system according to claim 18, wherein one or more of the characteristics include at least the access frequency of each of the plurality of sources.

20. The aforementioned referrer includes multiple referrers, To generate a list of referrers for the multiple referrers in the volatile cache that direct the request to the said source. The program instructions stored in the one or more computer-readable storage media are for execution by at least one of the one or more computer processors, for the purpose of performing the above, The aforementioned list is used to identify the source accessed by the request directed to the source by the referrer, The computer system according to claim 16.

Images