Storage tiering in unified storage environments

Abstract

A method for allocating and migrating files between tiers in a unified storage environment. The method includes one or more computer processors determining a plurality of files stored within a tiered storage environment accessed by a set of executing applications. The method also includes determining that one or more files of the plurality of files are accessed with two or more protocols. The method also includes identifying a set of two or more tiering policies used within the tiered storage environment. The method also includes determining a protocol-based hotness value corresponding to the plurality of files. The method also includes storing the plurality of files between tiers of the tiered storage environment based on the identified set of two or more tiering policies and the protocol-based hotness value corresponding to the plurality of files.

Description

Background Technology

[0001] This invention generally relates to the field of data storage, and more particularly to the allocation or migration of files between storage layers in a unified storage environment.

[0002] A unified storage environment, also known as a network unified storage environment or a multiprotocol storage architecture, is a storage environment that enables the utilization and management of files and applications from a single device. A unified storage environment allows for both file data storage and some types of block-based input / output (I / O) processing for applications. Compared to traditional storage systems based on file or block access, a multiprotocol storage architecture (i.e., a unified storage environment) combines file-based and block-based access within a single storage platform.

[0003] A unified storage environment can also reduce hardware requirements. Instead of separate storage platforms such as Network Attached Storage (NAS) for file-based storage and RAID arrays for block-based storage, a unified storage environment combines both modes within a single architecture. Users of a unified storage environment also benefit from features such as storage snapshots, replication, tiering, encryption, compression, and data deduplication. Summary of the Invention

[0004] According to one aspect of the present invention, a method, computer program product, and / or system for distributing and migrating files between layers within a unified storage environment is provided. The method includes at least one computer processor identifying a plurality of files stored within a tiered storage environment and accessed by a set of executing applications. The method further includes at least one computer processor determining one or more files among the plurality of files that are accessed using two or more protocols. The method further includes at least one computer processor identifying a set of two or more tiering policies used within the tiered storage environment. The method further includes at least one computer processor determining protocol-based heat count values ​​corresponding to the plurality of files. The method further includes at least one computer processor storing the plurality of files between layers of the tiered storage environment based on the identified set of two or more tiering policies and the protocol-based heat count values ​​corresponding to the plurality of files. Attached Figure Description

[0005] Figure 1 A networked computing environment according to an embodiment of the present invention is shown.

[0006] Figure 2A An illustrative example of allocating / migrating files between storage layers based on a hierarchical strategy and a protocol-based hot count value, according to an embodiment of the present invention, is described.

[0007] Figure 2BAn illustrative example of allocating / migrating files between storage tiers based on a hierarchical strategy and hot count value associated with priority and / or protocol, according to an embodiment of the present invention, is described.

[0008] Figure 3 A flowchart describing the steps of an enhanced layering procedure according to an embodiment of the present invention is provided.

[0009] Figure 4 This is a block diagram of the components of a computer according to an embodiment of the present invention.

[0010] Figure 5 A cloud computing environment according to an embodiment of the present invention is shown.

[0011] Figure 6 An abstract model layer according to an embodiment of the present invention is shown. Detailed Implementation

[0012] Embodiments of the present invention recognize that some storage environments are designed to support specific file systems and / or data storage structures. Other storage environments are designed to enable users or software applications to access file pools or file clusters via multiple protocols and data storage structures. A unified storage environment is an example of a multi-protocol storage environment or architecture. Embodiments of the present invention recognize that a unified storage environment can support file-based access protocols such as Server Message Block (SMB) and Network File System (NFS), as well as block-based access protocols such as Fibre Channel (FC) and Internet Small Computer System Interface (iSCSI). A unified storage environment can also support protocols such as object, Portable Operating System Interface (POSIX), and Hadoop Distributed File System (HDFS).

[0013] Embodiments of the present invention recognize that an inode (i.e., an index node) associated with a file or data object can store the file or object's attributes and disk block location within the file system. File system attributes associated with a file may include metadata such as owner information; permission data; and time-based data such as the last modified time, access count, access protocol, process ID utilizing the file, etc.

[0014] Embodiments of the present invention recognize that storage systems and storage environments may also include an Information Lifecycle Management (ILM) system, which includes instructing periodic monitoring of stored files to determine the number of times a file is accessed, to determine the access temperature or “hot” value associated with the file, and subsequently to determine whether and when a file is migrated from one tier of the storage system to another within the tiered storage system. The hot value of a file can be an attribute or metadata element that provides a means of optimizing file allocation or migration within the tiered storage system based on a strategy of the ILM system associated with the tiered storage environment. A hot count value is a relative attribute indicating whether a file is “hotter” or “colder” than other files within a file pool, such as a namespace. In various scenarios, hot count-based ILM strategies are used to determine when to migrate hotter files to a higher (e.g., faster) storage tier and when to migrate other colder files to a lower (e.g., slower or archival) storage tier.

[0015] Embodiments of the present invention also recognize that, within a clustered file system environment or namespace comprising multiple files, file heat is maintained as a file attribute, and a file's heat increases or decreases based on the number of times it is evaluated compared to other files in the cluster or namespace within a configured monitoring interval. The file heat count value is not protocol-specific. In some instances, the heat count value corresponding to a file is affected by accessing the file via any protocol.

[0016] Embodiments of the present invention recognize that while one protocol may affect performance related to accessing files, another protocol for accessing files may affect the overall storage performance of the system storing the files (e.g., frequent block moves between storage tiers lead to unnecessary power consumption, network traffic, memory waste, etc.). Furthermore, embodiments of the present invention recognize that applications utilizing files may suffer from inconsistencies in performance related to migrating files between storage tiers. Moreover, embodiments of the present invention recognize that file-hot-based storage tiering is designed to optimize non-volatile storage, not memory usage. Therefore, when calculating file hot count values, cache hits or in-memory accesses associated with the file are not included.

[0017] Embodiments of the present invention also recognize that the computational resources used by different protocols to access files used by multiple applications are not factors used by the unified storage environment to determine whether a file is migrated between storage tiers. Furthermore, embodiments of the present invention also recognize that the tiering strategy within the unified storage environment does not include other factors, such as whether a file is designated as task-critical or whether the file is accessed by an application (app) designated as high priority when determining whether to migrate the file to another storage tier.

[0018] Embodiments of the present invention improve file-related storage tiering decisions within a unified storage environment by enhancing file-related in-memory inode table fields, on-disk information inode tables, and / or metadata. The inode table is used to capture processes accessing the corresponding erase code fragment inode, map file metadata to the process ID of the protocol server, and at least track file-related hot count values ​​via the protocol, which is further used to optimize file allocation or migration decisions between storage tiers. If the inode metadata corresponding to a file cannot be modified to track the file's hot count value via protocol and / or priority, other embodiments of the present invention utilize extended file attributes associated with the file to store metadata related to protocol-based and / or priority-based hot metadata information, such as the corresponding hot count value.

[0019] One embodiment of the present invention is used in a unified storage environment for storing data via erasure coding. Erasure coding is a data protection method in which data is fragmented, expanded and encoded with redundant data fragments, and stored across a set of different locations or storage media. However, embodiments of the present invention are not limited to file storage utilizing erasure coding. Other embodiments of the present invention are used in conjunction with other file storage systems, copying and / or data protection methods, or combinations thereof.

[0020] Various situations, instances, and examples relating to the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments.

[0021] The invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a functional block diagram illustrating a networked computing environment 100 according to an embodiment of the present invention. In this embodiment, the networked computing environment 100 includes clients 110-1 and clients 110-2 through 110-N, and a data processing environment 120, all interconnected via a network 160. In one embodiment, the networked computing environment 100 represents part of a data center within a single physical location. In some embodiments, one or more components of the networked computing environment 100 are distributed across different physical and / or geographical locations, such as a unified storage system distributed within a cloud computing environment.

[0022] Clients 110-1 and 110-2 through 110-N, and data processing environment 120, can be personal computers, desktop computers, mainframe computers, etc. In some embodiments, data processing environment 120 (e.g., management environment 130 and unified storage environment 140) represents a computer system utilizing clustered computers and components (e.g., database server computers, application server computers, etc.) that act as a single seamless resource pool when accessed via network 160, which is common in data centers and in conjunction with cloud computing architectures. Typically, clients 110-1 and 110-2 through 110-N, data processing environment 120, management environment 130, and unified storage environment 140 represent any programmable electronic device or combination of programmable electronic devices capable of executing machine-readable program instructions and communicating via network 160. According to embodiments of the invention, clients 110-1 and 110-2 through 110-N, and data processing environment 120 may include references Figure 4 Components that are described and explained in further detail.

[0023] In some embodiments, one or more of clients 110-1 and clients 110-2 to clients 110-N represent a computing system, a personal computing device, a mobile device such as a smartphone; an interface for hosting applications; etc., which enable a user to remotely execute one or more applications of client app 134 within the data processing environment 120. In embodiments, clients 110-1 and clients 110-2 to clients 110-N include programs and data (not shown), such as an operating system, web interface, communication software, client interface, etc., respectively associated with an application included within client app 134.

[0024] In other embodiments, one or more of clients 110-1 and clients 110-2 through 110-N represent managed computing entities within the network computing environment 100, such as virtual machines (VMs), containers, application programming interfaces (APIs), etc., which execute one or more applications of client app 134 within the data processing environment 120 without human intervention. In the example, client 110-2 could be a financial transaction processing application that interfaces with a fraud detection application (e.g., app 134-5 (not shown)), or an API that performs another function in response to an event, such as app 134-2 (not shown). For example, an IoT device activates and transmits an image in response to a proximity trigger.

[0025] In one embodiment, data processing environment 120 represents an enterprise data processing environment. In another embodiment, data processing environment 120 represents a cloud-based data processing environment, such as a Platform as a Service (PaaS) environment or a cloud computing system (regarding...).Figure 5 and Figure 6 (In further detail), according to various embodiments of the present invention, in which the data processing environment 120 includes one or more instances of a management environment 130 and / or one or more instances of a unified storage environment 140. In various embodiments, the data processing environment 120 includes computing power (not shown) separate from the management environment 130 and the unified storage environment 140. The management environment 130 and the unified storage environment 140 may communicate via various aspects of a network 160, utilizing a network (not shown) within the data processing environment 120, or a combination thereof. In further embodiments, according to various embodiments of the present invention, the management environment 130 and / or the unified storage environment 140 may serve as a cloud computing environment (regarding...). Figure 5 and Figure 6 One or more compute node operations (in further detail)

[0026] In one embodiment, management environment 130 may be a computer or a group of computers used to manage and control various aspects of data processing environment 120. Management environment 130 includes management controls 131, service level agreements (SLAs) 132, client applications (apps) 134, application information 135, and various other programs and data (not shown). Other programs and data may include system management facilities (SMFs), hypervisors, virtualization software, application orchestration systems, communication software, etc. In various embodiments, some programs and data within management environment 130 are used to interface with and / or control various aspects of unified storage environment 140; access, update, and / or control files stored within unified storage environment 140; transmit information related to clients (e.g., users) and applications associated with clients; receive data from clients for inclusion within unified storage environment 140; and so on.

[0027] In addition, management environment 130 handles remote procedure calls (RPCs) between entities associated with data processing environment 120, such as RPCs initiated by client devices or entities, or RPCs executed by applications within management environment 130, to access files and data within unified storage environment 140. Information included in the RPC handshake can be assigned and / or collected for the application based on IDs corresponding to the application and associated with one or more files invoked by that application. In one example, the RPC handshake associated with the application executing within management environment 130 and unified storage environment 140 may also include access protocols associated with the files, priority values, criticality information, weighting factors related to the protocol and / or priority, application ID, etc., respectively. In another example, the RPC handshake from a client or user may also include a "nice" value associated with the executing application and / or the preferred storage tier required by the application. In some instances, the priority associated with the application is determined by the user, but may be subject to a service level agreement (SLA) associated with the application and / or the user of the application.

[0028] Management control 131 may include metering and billing procedures, management console procedures, authentication and security procedures, schedulers, load balancers, etc. (not shown). In an embodiment, management control 131 controls the priority or priority range allocated to the app based on the service level agreement (SLA) associated with the execution of the app in app 134 and various other factors associated with the data processing environment 120, such as the availability of computing resources and / or storage capacity within the unified storage environment 100, a set of applications executing within the data processing environment 120, and the SLA 132 corresponding to the set of applications executing app 134 within the data processing environment 120.

[0029] In another embodiment, management control 131 uses information within application information 135 to determine the set or scope of resources to be supplied or allocated to the application. In some scenarios, management control 131 also controls the allocation of compute and storage resources within data processing environment 120 among multiple executing applications during a given time period. In other scenarios, management control 131 allocates compute and storage resources of data processing environment 120 to client environments (not shown), such as logical partitions (LPARs), VMs, and / or virtual servers, rather than allocating resources to individual applications. In other embodiments, management control 131 is used to control aspects of unified storage environment 140, such as creating or updating storage tiering policies (i.e., tiering policies) of tiering policy 145. Furthermore, management control 131 shares information with unified storage environment 140, such as information associated with application RPCs and other application-related information included in application information 135.

[0030] In some embodiments, management control 131 responds to dynamic changes, such as user-initiated changes to the "nice" value corresponding to an application in execution. In various scenarios, management control 131 assigns kernel-level or root-level priority values ​​to applications to be executed based on associated SLAs, while users or clients utilize "nice" values ​​to modify the priority assigned to the executing application. In embodiments, a "nice" value refers to user-level input that affects one or more aspects of the executing application. One type of "nice" value modifies the priority assigned to an application regarding processing power or processor core allocation. Another type of "nice" value affects I / O activity priority. The range of priority values ​​and user-modifiable "nice" values ​​can vary depending on the configuration of data processing environment 120 and / or the operating system. In some cases, "nice" values ​​may be limited to increasing or decreasing an application's priority value by 15% to 20%. Thus, a low-priority application with a large "nice" value cannot utilize more processing power or time slices of data processing environment 120 than another application executing at a high priority or associated with a higher-level SLA having a zero "nice" value.

[0031] Service Level Agreement (SLA) 132 includes a list of different service packages provided and associated Quality of Service (QoS) information, metrics (e.g., details) corresponding to each SLA, and / or End User Agreement (EUA) information. An SLA may include a list of services provided, performance levels (e.g., reliability and responsiveness), monitoring procedures, issue / problem reporting indications, response time indications, resolution timeframe metrics, a range of assignable execution priorities, the amount or availability of compute resources, constraints related to storage tiering, etc. In one example, a lower-level SLA may specify that the priority assigned to an application / client can be reduced to improve the performance of another application / client with a higher-level SLA. In another example, SLA 132 may include indications associated with "nice" values, such as constraining the range of "nice" values ​​that a user or client can apply to an application, or the impact of "nice" values ​​on priority values ​​assigned to an application. In yet another example, a lower-level SLA may restrict tier-based file allocation or file migration within a tier 5 storage system to tiers 2-5, while a higher-level SLA may bias the set of tiering policy values ​​toward one or more higher-performance tiers.

[0032] In one embodiment, SLA 132 may include a customer-specific SLA, such as customer 110-1, and / or a set of general SLAs associated with each customer, such as gold, silver, and bronze level SLAs. In some embodiments, each client or group of client applications may be associated with a different SLA, such as internal business applications being governed by a first SLA, consumer-facing web applications by a second SLA, and applications designated as mission-critical by a third SLA. Alternatively, a client may have various applications or services covered by different levels of multi-level SLAs. Furthermore, SLAs may specify a range of critical values ​​or values ​​that users can assign to a criticality level.

[0033] Client application 134 includes multiple software and applications for users, enterprises, etc., represented by applications 134-1 to 134-N. In various embodiments, one or more applications within client application 134 are initiated by client devices or entities (such as client 110-1 and client 110-N) outside the data processing environment 120.

[0034] Application information 135 may include a database or other data storage structure (not shown) containing data and information associated with applications hosted by data processing environment 120, and data and information associated with clients utilizing data processing environment 120 (such as client 110-2 and client 110-N). In embodiments, application information 135 includes a list of multiple files invoked or utilized by each application in client applications 134; configuration information corresponding to the client and / or application, such as computing resources allocated to the application or supplied to the environment associated with executing the application; SLAs associated with the application; and so on. Some application-related information within application information 135 is passed in an RCP handshake between components within networked computing environment 100, such as priority values, application IDs, file access protocols, etc. Application information 135 may also include historical priorities, "nice" values ​​from previous executions of the application, some RPC information, etc.

[0035] In another embodiment, application information 135 also includes application-related information such as the preferred initial location of files accessed by the application, criticality values ​​assigned to the application, criticality provisions other than SLA considerations, etc. For example, one client may designate an application as highly important (criticality value = 75), but a second client may designate the same application as task critical (criticality value = 90) and execute it with high priority. In some embodiments, application information 135 includes dynamic information or data assigned by management control 131, such as the execution priority or priority range to be assigned to the application, management information related to the dynamic information of the executed application input by the client / user, the process ID corresponding to the application, the "nice" value changed by the user or client to be assigned to the executed application, etc.

[0036] In one embodiment, the unified storage environment 140 includes a file system 141, an information lifecycle management (ILM) system 142, and a tiered storage system 150. The unified storage environment 140 may also include various other programs and data (not shown), such as one or more operating systems, virtualization software, communication programs, resource management systems, etc. In this embodiment, some aspects of the unified storage environment 140 are controlled by a management environment 130, such as computing and storage resources allocated to applications, tiered policy constraints associated with SLAs for clients or applications, etc.

[0037] In some embodiments, the data processing environment 120 defines rules; standards; and various hierarchies, such as the analysis hierarchy between access protocols, application priority values, SLAs, or application criticality values; specifies algorithms and / or values ​​associated with the ILM system 142 and associated hierarchical policies; and so on. In some cases, the administrator of the data processing environment 120 defines rules, standards, and various hierarchies, algorithms, and values ​​used within elements such as those associated with the ILM system 142. In other scenarios, SLAs corresponding to users or their applications allow users to modify certain rules, criteria, hierarchies, algorithms, and values ​​used within the unified storage environment 140. For example, a user with an SLA associated with an application designated as mission-critical can modify the analysis hierarchy between access protocols and / or modify values, such as weighting factors within algorithms associated with the priority-based hierarchical policies of the ILM system 142, by identifying one or more access protocols used by the mission-critical application within application information 135.

[0038] In one embodiment, file system 141 supports a set of protocols for accessing (e.g., creating, reading, writing, updating, deleting, etc.) multiple files stored within tiered storage system 150 based on information and / or commands received from management environment 130. For example, file system 141 supports SMB, NFS, object, POSIX, and HDFS protocols. In another embodiment, file system 141 also supports inode structures, extended data attributes, metadata tracking and modification, block and file structures, etc. File system 141 utilizes information obtained from management environment 130 to determine the initial or preferred tier allocation of files within tiered storage system 150. In various embodiments, file system 141 also includes software and algorithms for storing files in various formats or data structures, such as the Reed-Solomon algorithm for data erasure encoding and other fault-tolerant storage methods and schemes, such as RAID (Redundant Array of Independent Disks) storage.

[0039] In some embodiments, file system 141 further includes a file monitoring program and a daemon (not shown) that tracks file access counts between persistent and volatile storage elements, such as persistent memory included within tiered storage system 150 and memory and / or cache (not shown) included within memory within data processing environment 120. File system 141 also includes a hot-mapping program (not shown) for determining file access counts relative to a configurable monitoring period (e.g., a time window) and based on various file clusters and / or namespaces. In some scenarios, file system 141 updates metadata (i.e., information) within metadata 144 corresponding to files stored within tiered storage system 150, such as the access protocols used, hot count values ​​associated with the access protocols, information included in RPC handshakes and / or RPCs, etc. In another embodiment, the determined hot count value corresponding to a file may be modified by one or more factors to generate an effective hot count value. An example of an effective hot count value is a priority-based hot count, referencing... Figure 2B To describe in more detail.

[0040] In embodiments, the ILM system 142 includes a file list 143, file metadata 144, a tiering policy 145, an enhanced tiering procedure 300, and other procedures and data (not shown). In various embodiments, the ILM system 142 obtains information and data from various aspects of the unified storage environment 140 and the file system 141 to determine access counts and / or file heat counts associated with multiple files stored within the tiered storage system 150, identified within the file list 143, and also associated with the protocol used to access the files and / or the priority of the application accessing the files. The ILM system 142 uses the obtained file-related information and data to update instances of the file metadata 144 corresponding to the files.

[0041] In some embodiments, the ILM system 142 further includes a set of algorithms (not shown) that modify or determine (e.g., calculate) a set of heat count values ​​corresponding to a set of storage layers associated with a tiered policy of tiered policy 145, based on the number of storage layers within the tiered storage system 150, the number of protocols used to access the shared space, and one or more other factors. In one example, the algorithms of the ILM system 142 may determine a set of cumulative (e.g., summation, total) heat count values ​​and / or a set of valid heat count values ​​associated with the layers of the tiered policy, based on one or more factors such as the number of applications running simultaneously accessing a public file, protocol-related values, priority ranges of running applications, etc. In another example, other algorithms of the ILM system 142 utilize priority information and other factors to determine valid heat count values ​​corresponding to a file, such as priority-based heat counts (PBHC). In another embodiment, ILM system 142 specifies that, within a tiered group of tiered policies, analyzing the heat count value corresponding to a file with a subsequent tiered policy may upgrade (e.g., improve) the storage tier allocation of shared files, but based on the lower tiered policy in the tier, downgrading the allocation of shared (e.g., public) files is not permitted (e.g., disallowed, prevented).

[0042] In an embodiment, file list 143 represents a list of files used or accessed by multiple executing applications (such as applications 134-1 to 134-N) within the data processing environment 120. In some embodiments, file list 143 tracks the layer position corresponding to the file prior to access, the layer position determined based on one or more factors including the file heat count value, and the storage layer assigned to the file in response to the cessation of access (e.g., termination of the application using the file).

[0043] File metadata 144 includes multiple instances of metadata corresponding to files (not shown) within file list 143. In embodiments, instances of file metadata 144 include one or more process IDs for the file, file access counts, information identifying file access protocols (e.g., SMB, NFS objects, POSIX, HDFS, etc.). In some embodiments, instances of file metadata 144 are included within an inode mapping corresponding to the file. In other embodiments, instances of file metadata 144 are included within extended attributes corresponding to the file or within extended attributes of an inode mapping corresponding to the file. In various embodiments, file system 141 updates the metadata associated with each file, such as adding new access protocols within extended attributes and / or modifying hot count values ​​based on different configurations of a given protocol.

[0044] In one embodiment, the tiering strategy 145 includes multiple strategies that define (e.g., determine) the storage tiers in the tiered storage system 150 where files are allocated or migrated based on one or more standards, such as a set of protocol-based heat count values, a set of priority-based heat count values, and a set of tiering strategies (regarding...). Figure 2A and Figure 2B (More detailed description) The number of layers within the set of tiering strategies 145 utilized by the enhanced tiering procedure 300 is based on the configuration of the tiered storage system 150.

[0045] The tiered storage system 150 is an architecture based on a set of two or more types of storage media. The tiered storage system 150 is divided into a set of tiers based on thermal designations assigned to each tier. For example, an instance of the tiered storage system 150 may include a set of three tiers, each associated with a file access temperature rating, such as a “hot” tier including solid-state drives (SSDs); a “warm” tier including hard disk drives (HDDs); and a “cold” tier including archival storage media such as magnetic tape. In some embodiments, the tiered storage system 150 includes groups of more than three tiers and / or sub-tiers. In one example, the tiered storage system 150 may also include sub-tiers within an HDD tier based on HDD capacity and / or speed, such as 5.4K rpm HDDs, 7.2K rpm HDDs, and 15K rpm HDDs. Similarly, a “hot” SSD tier may include one sub-tier of SSDs designated for storing files associated with high write rates and another sub-tier of SSDs designated for storing files associated with a high percentage of read accesses. In another instance, the tiered storage system 150 may include one or more layers above an SSD layer based on other types of storage media, such as an inner memory layer utilizing volatile memory (e.g., dynamic random access memory (DRAM)) and / or a layer based on storage-class memory (SCM).

[0046] In some embodiments, each tier of the tiered storage system 150 is further configured or subdivided based on file clusters and / or namespaces, wherein groups of files are logically isolated but can be stored across the same storage medium (e.g., the storage device of the tier).

[0047] The enhanced tiering procedure 300 is a procedure for determining the storage tiers of files within a unified storage environment 140 based on heat count values ​​associated with various standards, such as protocol-based heat count values, priority-based heat count values, and analysis levels between tiering policies. In one embodiment, the enhanced storage procedure 300 utilizes information within a set of metadata associated with a file (e.g., an instance of file metadata 144) and one or more policies within tiering policy 145 to determine whether to migrate a file between tiers of the tiered storage system 150. Furthermore, the enhanced tiering procedure 300 can respond to changes between tiering policies, as the number of applications executing the file changes. In some embodiments, the enhanced tiering procedure 300 utilizes analysis levels between tiering policies to determine whether a file is migrated between tiers of the tiered storage system 150 in response to determining that two or more conditions are met, such as differences in the file's heat count value relative to different access protocols.

[0048] In various embodiments, the enhanced tiering procedure 300 determines whether file migration between layers of the tiered storage system 150 is protocol-based or priority-based by analyzing information related to two or more concurrently executing applications accessing common (e.g., identical) files. In some scenarios, in response to determining that file migration between layers is priority-based, the enhanced tiering procedure 300 determines a corresponding priority-based weighting factor for each executing application. The corresponding weighting factor is applied to the heat count value corresponding to the file accessed by the application. In other scenarios, the enhanced tiering procedure 300 determines file migration between layers of the tiered storage system 150 based on a combination of factors related to concurrently executing applications accessing the file set, said combination of factors being based on another weighting factor determined and applied to the heat count value corresponding to each utilized access protocol, and a tiering strategy based on protocols other than priority.

[0049] In one embodiment, clients 110-1 and 110-2 through 110-N communicate with data processing environment 120 via network 160. Network 160 can be, for example, a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunications network (e.g., part of a cellular network), a wireless local area network (WLAN) such as an intranet, or any combination thereof, and may include wired, wireless, or fiber optic connections. Typically, according to embodiments of the invention, network 160 can be any combination of connections and protocols supporting communication between clients 110-1 and 110-2 through 110-N and data processing environment 120. In some embodiments, network 160 also provides communication between different parts of data processing environment 120, such as management environment 130 and unified storage environment 140. In various embodiments, network 160 operates locally via wired, wireless, or optical connections and can be any combination of connections and protocols (e.g., personal area network, near field communication (NFC), laser, infrared, ultrasound, etc.).

[0050] Figure 2A This is an illustrative example of migrating and / or allocating files between storage layers based on protocols, file hot counting, and tiering strategies according to embodiments of the present invention. In the embodiments, Figure 2A The unified storage environment 140 is indicated to include three protocol-based hot-count tiering policies within tiering policies 145, 145A, 145B, and 145C, as well as three-tier (e.g., tier 1, tier 2, and tier 3) instances of the tiered storage system 150.

[0051] exist Figure 2AIn the illustrative example, the unified storage environment 140 depicts an analysis hierarchy among three tiering policies. In illustrative example policy 145A, the protocol #2-based tiering policy is first invoked to analyze the heat count value corresponding to the file. Policy 145B invokes the multi-protocol tiering policy, followed by analysis of the heat count value corresponding to the file. Policy 145C, the protocol #1-based tiering policy, is invoked, and finally, the heat count value corresponding to the file is analyzed. In some embodiments, the analysis hierarchy among the tiering policies is based on information included in application information 135, SLA 132, and / or ILM system 142. Furthermore, various aspects of ILM system 142 are used to calculate a set of heat count values ​​for one or more policies that specify the storage tier for an allocated file or a received migrated file.

[0052] Figure 2A The illustrative example also depicts the unified storage environment 140 as storing files 201 (no padding), 202 (crosshair padding), 203 (dotted padding), and 204 (diagonal hash padding) across the layers of the hierarchical storage system 150. Furthermore, files 201, 202, and 204 are accessed by one or more applications (such as application 134-1 utilizing protocol #1). Files 201, 202, and 203 are accessed by one or more different applications (such as application 134-4 (not shown) utilizing protocol #2). Continuing with this example, during a monitoring period, the file system 141 determines heat count values ​​corresponding to the depicted files and associated with the protocols used to access those files. For example, file 201 is depicted as having a heat count value of 50 with respect to protocol #1 and a heat count value of 35 with respect to protocol #2.

[0053] As previously discussed with respect to ILM system 142, one or more algorithms included in ILM system 142 and information associated with at least one of the two executing applications that access file 201, file 202, file 203 and / or file 204 are used to determine (e.g., calculate) a set of heats (i.e., heat count values) associated with at least strategy 145B.

[0054] In the illustrative example, policy 145A specifies that file 201 is allocated (e.g., stored or migrated) to layer 2 based on protocol #2 hot=35, file 202 is allocated to layer 2 based on protocol #2 hot=17, and file 203 is allocated to layer 3 based on protocol #2 hot=9. File 204 is not allocated to a layer based on policy 145A because file 204 is not accessed using protocol #2.

[0055] Next, policy 145B stipulates that based on the sum of the corresponding heats (cumulative heat = 85) for protocols #1 and #2, file 201 is promoted to layer 1; and file 202 (cumulative heat = 22) remains assigned to layer 2. Files 203 and 204 are unaffected by policy 145B and each file is accessed using a single protocol.

[0056] Subsequently, policy 145C instructs that file 201 should be assigned to layer 2, protocol #1 hot = 50, and policy 145C also instructs that file 202 should be assigned to layer 3, protocol #1 hot = 5. However, since in this embodiment, ILM system 142 prevents downgrading of file layer assignments during the same monitoring period, file 201 remains assigned to layer 1, and file 202 remains assigned to layer 2. Policy 145C instructs that file 204 be assigned to layer 2, protocol #1 hot = 20. File 203 is unaffected by policy 145C and remains assigned to layer 3; file 203 is not accessed using protocol #1.

[0057] Figure 2B This is an illustrative example of migrating and / or allocating files between storage layers based at least on application priority, file hot and tiering strategies, and exemplary priority-based hot algorithms, according to embodiments of the present invention.

[0058] In another example embodiment, Figure 2B The unified storage environment 140 is indicated to include two priority-based hot-count tiering policies within tiering policy 145, and a three-tier (e.g., tier 1, tier 2, and tier 3) instance of the tiered storage system 150. In an embodiment, policy 145D is based on a set of priority-modified hot-count values ​​associated with a single protocol (e.g., an unspecified protocol), while policy 145E is based on hot-count values ​​associated with two protocols. In the current example, policies 145D and 145E are not described based on an analytics hierarchy. Figure 2B The illustrative example also depicts the unified storage environment 140 as storing files 201 (no padding), 202 (crosshair padding), 203 (dotted padding), and 204 (diagonal hash padding) between the layers of the hierarchical storage system 150. Additionally, the example shown here includes algorithm 142A.

[0059] exist Figure 2BIn the example shown, Algorithm 142A represents a general algorithm for determining a priority-based effective heat count value, such as the PBHC corresponding to the described file. For example, file 201 has a file heat of 50 associated with APP 134-1 and the corresponding PBHC-1=20, respectively. In the illustrative embodiment, Algorithm 142A determines the PBHC by modifying the heat value corresponding to the file and associated with the application, by utilizing another value determined by a function using multiple inputs, such as the priority (Pr) corresponding to the application, the "nice" value (NV) associated with the application, and the weighting factor (WF). The WF associated with Algorithm 142A is based on another function using the application's criticality value and a value related to the protocol used by the application to access the file. A given instance of Algorithm 142A can be created by an administrator, client, and / or user of the data processing environment 120. Each instance of Algorithm 142A or another algorithm within the ILM system 142 can be tailored for a specific instance of the unified storage environment 140, a namespace within the instance of the unified storage environment 140, a set of users, etc.

[0060] exist Figure 2B In the illustrative example, APP 134-1 and APP 134-N access files 201 and 202. File 203 is accessed only by APP 134-N, and file 204 is accessed only by APP 134-1. Additionally, in the illustrative example, APP 134-1 and APP 134-N access files within the unified storage environment 140 based on different protocols (not shown). Each depicted file includes a heat value corresponding to at least one application. Furthermore, each depicted file is also associated with one or more corresponding priority-based heat counts (PBHCs), which are associated with applications, such as PBHC-1 and PBHC-N calculated using an instance of algorithm 142A and various factors / inputs (not shown). In the embodiment, a PBHC value of 0 indicates that the application has not accessed the file by a given protocol.

[0061] In the example shown, if the priority associated with the application is not a factor, policy 145E assigns file 201 to layer 2 (total file heat = 80). However, based on algorithm 142A, file 201 is assigned to layer 1 (cumulative PBHC = 83). Similarly, without considering priority factors, policy 145E assigns file 202 to layer 3 (total file heat = 15). However, based on algorithm 142A, file 202 is assigned to layer 2 (cumulative PBHC = 23).

[0062] Regarding files 203 and 204, policy 145D is invoked. Without considering priority, policy 145D would assign file 203 to layer 3 based on hot=8. However, using algorithm 142A, the corresponding PBHC-N=17 for file 203 is determined, indicating that file 203 is assigned to layer 2. Similarly, without considering priority, policy 145D would assign file 204 to layer 3 based on hot=20. However, using algorithm 142A, policy 145D determines that file 203 has PBHC-1=8, indicating that file 203 is assigned to layer 3.

[0063] Figure 3 This is a flowchart depicting the operational steps of an enhanced tiering procedure 300 according to an embodiment of the present invention, which determines the storage tier for allocated files or files receiving migration based at least in part on heat count values ​​associated with file access protocols and / or priorities corresponding to executing client applications (apps). In various embodiments, the enhanced tiering procedure 300 re-executes one or more steps in response to management environment 130 transmitting information related to changes associated with a set of concurrently executing applications (such as another application starting execution, the current application terminating, a user initiating a change to the "nice" value associated with the executing application, etc.). In some cases, the enhanced tiering procedure 300 executes continuously as a function of ILM system 142. In other cases, the enhanced tiering procedure 300 executes periodically in response to file system 141 determining that the access rate changes of one or more files exceed a threshold amount.

[0064] In step 302, the enhanced tiering procedure 300 identifies metrics associated with applications, respectively, within a set of executing applications. In embodiments, the enhanced tiering procedure 300 identifies metrics associated with applications, respectively, within a set of concurrently executing applications within the data processing environment 120, based on information included in application information 135 and the SLA of the application-associated SLA 132. Application-associated metrics may include priority information, criticality values, a specification of the analysis hierarchy in the access protocol associated with files accessed by two or more applications, weighting factors related to the protocol and / or criticality values, etc. The enhanced tiering procedure 300 may also determine changes associated with a set of metrics for the executing applications based on input received from users or clients utilizing the applications, such as modifying the “nice” value of the executing applications. In various embodiments, the enhanced tiering procedure 300 defines a set of concurrently executing applications as two or more applications active within the data processing environment 120 and intermittently accessing one or more files within the unified storage environment 140, or as applications executing during one or more monitoring periods used to determine the corresponding heat values ​​of files within the unified storage environment 140.

[0065] In step 304, the enhanced hierarchy procedure 300 determines a plurality of files accessed by the group of executing applications. In one embodiment, the enhanced hierarchy procedure 300 determines the plurality of files accessed by the group of executing applications based on information included in file list 143. In another embodiment, the enhanced hierarchy procedure 300 determines the plurality of files accessed by the group of executing applications based on RPCs associated with management environment 130 and applications, as well as information included in application information 135. In some embodiments, the enhanced hierarchy procedure 300 also determines the appropriate protocol used by each application in the group of executing applications to access the files in the determined plurality of files.

[0066] In step 306, the enhanced tiering procedure 300 identifies policies related to migrating files within the tiered storage system. In an embodiment, the enhanced tiering procedure 300 identifies information related to policies for allocating or migrating files within the tiered storage system 150 based on information obtained from one or more sources, such as RPC metadata, application information 135, and / or SLA 132. In an example, if the enhanced tiering procedure 300 determines that a single application accesses a file, the enhanced tiering procedure 300 may determine a policy from tiering policies 145 for allocating or migrating files within the tiered storage system 150 based on information associated with the application in the application information 135. In some embodiments, the enhanced tiering procedure 300 determines that two or more applications use a file. In response, the enhanced tiering procedure 300 accesses the application information 135, SLA 132, and / or ILM system 142 to determine whether an application or group of applications specifies a type and / or analytics hierarchy among tiering policies utilized by a set of executing applications.

[0067] In some scenarios, the enhanced tiering procedure 300 prioritizes the application with the highest-level SLA regarding selection within the tiering strategy of tiering policy 145, and regarding the allocation or migration of files within the tiered storage system 150. For example, the enhanced tiering procedure 300 determines that the highest-level SLA specifies multiple aspects between tiering policies, such as protocol-based hot-count tiering policies, an analysis hierarchy among a set of protocol-based hot-count tiering policies, and determines a set of hot-count values ​​for a multi-protocol tiering policy based at least on the number of concurrently executed applications accessing the same file. In other cases, the enhanced tiering procedure 300 prioritizes the application with the highest criticality value regarding selection within the priority-based tiering policies of tiering policy 145 and, in response to determining that two or more tiering policies have been selected, determines whether an analysis hierarchy has been defined. In a further embodiment, the enhanced tiering procedure 300 modifies (e.g., biases) a set of hot-count values ​​associated with one or more tiering policies based on various factors, such as the SLA associated with the application, the priority assigned to the application, the criticality value of the application, the number of concurrently executed applications, etc.

[0068] In step 308, the enhanced tiering procedure 300 determines the access count corresponding to a file. In one embodiment, the enhanced tiering procedure 300 determines the access count corresponding to a file based on information received from the file system 141 for use with the corresponding protocol. Subsequently, the enhanced tiering procedure 300 uses information related to the hot-mapping procedures of the tiered storage system 150 and the file system 141 to determine the hot count value corresponding to the corresponding protocol and stores the determined hot count value corresponding to the file in the file metadata 144. In some embodiments, the enhanced tiering procedure 300 stores the determined hot count value in another location (not shown) within the unified storage environment 140. In one embodiment, the enhanced tiering procedure 300 uses information associated with the ILM system 142 to determine the monitoring period (e.g., time window) configured for the file system 141 to determine the hot count value corresponding to the file. In other embodiments, the enhanced tiering procedure 300 determines additional access count information corresponding to the file based on metadata associated with accessing the file, which, in addition to accesses processed by the file system 141, also identifies cache hits and in-memory accesses of the file.

[0069] In decision step 309, the enhanced tiering procedure 300 determines whether storage tiering is based on application priority. In one embodiment, the enhanced tiering procedure 300 determines that the storage tiering of files associated with an application is based on application priority by analyzing application-related information, such as information within application information 135, application-related SLAs, and / or SLAs associated with the client (i.e., the user) executing the application. In some embodiments, the enhanced tiering procedure 300 determines that the storage tiering of files used by multiple applications is based on application priority by analyzing priority information associated with multiple applications (in step 306). In other embodiments, the enhanced tiering procedure 300 determines that the storage tiering of files associated with one or more applications is based on a file access protocol relative to application priority by analyzing application-related information, such as information within application information 135, application-related SLAs, and / or SLAs associated with the client (i.e., the user) invoking the application.

[0070] In response to determining that storage tiering is not based on application priority (No branch, decision step 309), the enhanced tiering procedure 300 determines the protocol-based heat count value of the file (step 310).

[0071] In step 310, the enhanced layering procedure 300 determines a protocol-based heat count value for a file. In one embodiment, the enhanced layering procedure 300 determines a protocol-based heat count value corresponding to a file based on retrieving the corresponding file's protocol-based heat count value within file metadata 144. In various embodiments, the enhanced layering procedure 300 determines a protocol-based heat count value corresponding to a file based on each protocol used to access the file. In some embodiments, the enhanced layering procedure 300 also determines a cumulative (e.g., summative or total) protocol-based heat count value corresponding to files accessed by two or more applications. In a further embodiment, the enhanced layering procedure 300 determines a cumulative protocol-based heat count value corresponding to files corresponding to combinations of access protocols.

[0072] Referring to decision step 309, in response to determining that storage tiering is based on application priority (is a branch, decision step 309), the enhanced tiering procedure 300 determines the application priority-based heat count value of a file (step 311).

[0073] In step 311, the enhanced tiering procedure 300 determines the application priority-based hot count value of a file. In various embodiments, the enhanced tiering procedure 300 determines the application priority-based hot count value and priority-based hot count of a file (such as a PBHC associated with at least the file and the application accessing the file, respectively) based on information previously determined by one or more algorithms included within the ILM system 142 (discussed previously with respect to FIG. 2b). In some embodiments, the enhanced tiering procedure 300 also determines the cumulative priority-based hot count value corresponding to a file accessed by two or more applications. In a further embodiment, if the enhanced tiering procedure 300 determines a set of priority-based tiering strategies to determine storage tiers based on factors other than hot count values, such as three protocols and various combinations of three protocols; then, the enhanced tiering procedure 300 correlates the determined hot count values ​​with factors of different groups or tiers (such as protocols 1 to 3, protocol combination A, protocol combination B, etc., and applications); in contrast to a protocol and its corresponding application.

[0074] In step 312, the enhanced tiering procedure 300 determines the storage tier of the file. In one embodiment, the enhanced tiering procedure 300 assigns the file to a storage tier based on information within application information 135. In another embodiment, the enhanced tiering procedure 300 determines the file's storage tier allocation by comparing the protocol-based heat value determined in step 310 with a set of heat count values ​​associated with one or more protocol-based tiering strategies identified in step 306. In some embodiments, the enhanced tiering procedure 300 determines the file's storage tier allocation based on analysis of the heat value or cumulative heat value corresponding to the file, a set of heat count values ​​associated with two or more protocol-based tiering strategies, and the analysis level among the two or more protocol-based tiering strategies identified in step 306; also previously discussed... Figure 2A The subject of discussion.

[0075] In other embodiments, the enhanced stratification process 300 is based on analysis of the effective calorific value or cumulative effective calorific value corresponding to the file determined in step 311, and one or more priority-based stratification strategies and a stratification strategy 145 included in relation to one or more applications in a set of executing applications (previously discussed). Figure 2B The corresponding set of heat count values ​​within the discussion are used to determine the storage layer allocation of the file. In a further embodiment, in response to determining the storage layer allocation based on criticality in addition to priority, the enhanced layering procedure 300 utilizes other priority-based layering strategies identified in step 306 to determine the storage layer allocation of the file.

[0076] In step 314, the enhanced tiering procedure 300 migrates files within the tiered storage system. In one embodiment, in step 312, the enhanced tiering procedure 300 migrates files within the tiered storage system 150 based on the tiers determined for the files. In another embodiment, the enhanced tiering procedure 300 determines in step 312 that the current storage tier allocation of a file within the tiered storage system 150 corresponds to the tiers determined for the files. In response, the enhanced tiering procedure 300 does not migrate the files. In some embodiments, the enhanced tiering procedure 300 updates instances of file metadata 144 corresponding to the files and / or application information 135 corresponding to the applications accessing the files to indicate the initial storage tier allocation associated with the files after a quiescent (i.e., inactive) period.

[0077] Figure 4A computer system 400 is depicted, representing clients 110-1, 110-2 to 110-N, a data processing environment 120, a management environment 130, and a unified storage environment 140. Computer system 400 is an example of a system including software and data 412. Computer system 400 includes a processor 401, a cache 403, memory 402, persistent memory 405, a communication unit 407, multiple input / output (I / O) interfaces 406, and a communication structure 404. Communication structure 404 provides communication between cache 403, memory 402, persistent memory 405, communication unit 407, and input / output (I / O) interfaces 406. Communication structure 404 can be implemented using any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within the system. For example, communication structure 404 can be implemented using one or more buses or crossover switches.

[0078] Memory 402 and persistent memory 405 are computer-readable storage media. In this embodiment, memory 402 includes random access memory (RAM). Typically, memory 402 may include any suitable volatile or non-volatile computer-readable storage medium. Cache 403 is a fast memory that enhances the performance of processor 401 by storing recently accessed data from memory 402 and data near the recently accessed data.

[0079] Program instructions and data for implementing embodiments of the present invention may be stored in persistent memory 405 and memory 402 for execution by one or more corresponding processors 401 via cache 403. In one embodiment, persistent memory 405 includes a magnetic hard disk drive (MDD). As an alternative to or supplement to an MDD, persistent memory 405 may include a solid-state drive (SSD), semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information. In some embodiments, various elements of persistent memory 405 and / or memory 402 may also be assigned to storage layers within a tiered storage system 150, such as non-volatile memory-class memory (SCM), memory-in-memory, and other types of storage media described above.

[0080] The medium used for persistent storage 405 can also be removable. For example, a removable hard disk drive can be used for persistent storage 405. Other examples include optical discs and disks, thumb drives, and smart cards, which are inserted into the drive for transfer to another computer-readable storage medium that is also part of persistent storage 405. Software and data 412 are stored in persistent storage 405 for access and / or execution by one or more corresponding processors 401 via cache 403 and memory 402. Regarding management environment 130, software and data 412 includes management controls 131, service level agreements 132, client applications 134, application information 135, and other programs and data (not shown). Regarding unified storage environment 140, software and data 412 includes file system 141, ILM system 142, file list 143, file metadata 144, tiering policy 145, enhanced tiering program 300, and other programs and data (not shown).

[0081] In these examples, communication unit 407 provides communication with other data processing systems or devices, including resources, management environment 130, and unified storage environment 140 for clients 110-1, 110-2, and to 110-N. In these examples, communication unit 407 includes one or more network interface cards. Communication unit 407 can provide communication by using one or both physical and wireless communication links. Program instructions and data for implementing embodiments of the invention can be downloaded to permanent memory 405 via communication unit 407.

[0082] I / O interface 406 allows data input and output to other devices that can be connected to each computer system. For example, I / O interface 406 can provide connectivity to external devices 408, such as keyboards, keypads, touchscreens, and / or other suitable input devices. External devices 408 may also include portable computer-readable storage media, such as, for example, thumb drives, portable optical discs or disks, and memory cards. Software and data used to practice embodiments of the invention can be stored on such portable computer-readable storage media and can be loaded onto persistent memory 405 via I / O interface 406. I / O interface 406 is also connected to display 409.

[0083] Display 409 provides a mechanism for displaying data to a user and can be, for example, a computer monitor. Display 409 can also be used as a touchscreen, such as the display of a tablet computer or smartphone. Alternatively, display 409 can display information to a user based on projection technology, such as a virtual retina display, a virtual monitor, or an image projector.

[0084] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0085] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0086] The features are as follows:

[0087] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider. WAN access: Capabilities are available on the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0088] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. Location independence has significance because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0089] Rapid Flexibility: In some cases, the ability to scale outwards and inwards quickly and flexibly can be provided. For consumers, the available capacity often appears unlimited and can be purchased in any quantity at any time.

[0090] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.

[0091] The service model is as follows:

[0092] Software as a Service (SaaS): The capability offered to consumers is the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0093] Platform as a Service (PaaS): This provides consumers with the ability to deploy consumer-created or acquired applications onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environments.

[0094] Infrastructure as a Service (IaaS): This provides consumers with the capability to deliver processing, storage, networking, and other basic computing resources that enable them to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0095] The deployment model is as follows:

[0096] Private cloud: The cloud infrastructure is operated solely by an organization. It can be managed by the organization or a third party and can reside either on-site or off-site. Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can reside either on-site or off-site.

[0097] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0098] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursting for load balancing between clouds).

[0099] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0100] Now for reference Figure 5The illustration depicts a cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 that can communicate with local computing devices used by cloud consumers, such as personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 5 The types of computing devices 54A-N shown are for illustrative purposes only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0101] Now for reference Figure 6 This demonstrates a cloud computing environment of 50 ( Figure 5 This provides a set of functional abstractions. It should be understood beforehand that... Figure 6 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0102] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a host 61; a server 62 based on a RISC (Reduced Instruction Set Computer) architecture; a server 63; a blade server 64; storage devices 65, such as a tiered storage system 150; and networks and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0103] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0104] In one example, management layer 80 may provide the functionality described below. In one embodiment, the various functions of management layer 80 are associated with aspects of management environment 130 and unified storage environment 140. Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking when utilizing resources in the cloud computing environment, as well as billing or invoicing for consuming these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, and protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-scheduling and procurement of cloud resources, where future needs are anticipated according to the SLA.

[0105] Workload layer 90 provides examples of functionalities that can leverage a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis and processing 94; transaction processing 95; and software 96. In various embodiments of the invention, software 96 represents the information lifecycle management system 142 and the enhanced layered program 300, or corresponding processing capabilities, respectively, regarding... Figure 1 and Figure 3 Further detailed description.

[0106] The programs described herein are identified based on applications that implement them in specific embodiments of the invention. However, it should be understood that any specific procedural terminology used herein is for convenience only, and therefore the invention should not be limited to use only in any specific application identified and / or implied by such terminology.

[0107] This invention can be a system, method, and / or computer program product. A computer program product may include one or more computer-readable storage media having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0108] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0109] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or via a network, such as the Internet, a local area network (LAN), a wide area network (WAN), and / or a wireless network, to an external computer or external storage device. The network may include copper cables, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the respective computing / processing device.

[0110] Computer-readable program instructions for performing the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., Smalltalk, C++, etc.) and conventional procedural programming languages ​​(e.g., the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform aspects of this invention, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions to personalize the electronic circuits by utilizing the information in the computer-readable program instructions.

[0111] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0113] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions indicated in the blocks may occur in a different order than indicated in the figures. For example, two blocks shown consecutively may actually be implemented as a single step, executed simultaneously, substantially simultaneously, with partial or complete time overlap, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0115] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the invention. The terminology used herein is chosen to best explain the principles of the embodiments, their practical application, or improvements to existing technologies on the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for distributing and migrating files between layers within a unified storage environment, comprising: One or more computer processors determine multiple files stored within a hierarchical storage environment and accessed by a set of executing applications; A set of two or more tiered strategies used within the tiered storage environment, identified by one or more computer processors; One or more computer processors determine the monitoring period configured by the file system to determine the heat count value corresponding to the plurality of files; Within a defined monitoring period, one or more computer processors determine access count information corresponding to files in the plurality of files based on information received from the file system. The access count information includes hot count values ​​corresponding to the plurality of files for the respective access protocols used to access each file. One or more computer processors store the determined heat count value corresponding to each file in the file metadata; One or more computer processors determine protocol-based hot count values ​​corresponding to the plurality of files, wherein the protocol-based hot count values ​​are generated at least in part based on retrieving protocol-based hot count values ​​corresponding to the files from the file metadata; One or more computer processors store the multiple files between layers of the hierarchical storage environment based on an identified set of two or more hierarchical strategies and protocol-based hot count values ​​corresponding to the multiple files; Sum the protocol-based heat count values ​​corresponding to the plurality of files; Upgrade the tier of the hierarchical storage environment storing the plurality of files, at least in part based on the sum of the protocol-based heat counts and the set of two or more tiering strategies; and The file is migrated within the upgraded tier of the tiered storage environment, and instances of file metadata corresponding to the file are updated to indicate the initial storage tier allocation of the file associated with the file after a period of inactivity.

2. The method according to claim 1, wherein, Identify a set of two or more tiered strategies used within the tiered storage environment, based at least in part on information associated with the set of executing applications, wherein a first application in the set of executing applications accesses a first file using a first protocol.

3. The method according to claim 2, wherein, Identifying a set of two or more tiered policies used within the tiered storage environment is also based on one or more indications related to a Service Level Agreement (SLA), which is associated with an application in the set of execution applications.

4. The method according to claim 1, wherein, Storing the multiple files across layers of the hierarchical storage environment based on a set of two or more identified hierarchical strategies also includes: The analysis hierarchy among the set of two or more hierarchical strategies is determined by one or more computer processors; One or more computer processors analyze protocol-based heat count values ​​corresponding to the plurality of files across the analysis hierarchy within a set of two or more hierarchical strategies; and The plurality of files are allocated among storage layers by one or more computer processors based on a determined set of heat count values ​​associated with the plurality of files respectively and also based on an analysis hierarchy among two or more identified hierarchical strategies.

5. The method according to claim 4, wherein, A service level agreement (SLA) associated with at least one of the set of execution applications specifies the analytics hierarchy among two or more identified hierarchical policies, wherein the SLA allows for upgrading the storage tier allocation of a file based on subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies, and prevents for downgrading the storage tier allocation of the file based on the subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies.

6. The method according to claim 1, wherein, The identified set of two or more hierarchical strategies also includes one or more multi-protocol hierarchical strategies associated with files accessed by two or more access protocols, and wherein a set of heat count values ​​associated with the multi-protocol hierarchical strategies is based on a combination of two or more access protocols used to access the files and the number of executed applications utilizing the combination of two or more access protocols used to access the files.

7. The method according to claim 4, wherein, The analysis of the protocol-based heat count values ​​corresponding to the multiple files also includes: In response to determining that a file in the plurality of files is accessed using two or more access protocols, one or more computer processors determine a multi-protocol hot count value corresponding to the file and also associated with a combination of access protocols, wherein the multi-protocol hot count value corresponding to the file is the sum of the hot count values ​​corresponding to each access protocol in the combination of access protocols; and One or more computer processors analyze protocol-based heat counts corresponding to the plurality of files and one or more multi-protocol heat counts corresponding to the files across the analysis hierarchy of the set of two or more hierarchical strategies.

8. A computer program product comprising: Program instructions, which can be executed by one or more computer processors to: Identify multiple files stored within a tiered storage environment that are accessed by a set of executing applications; Identify a set of two or more tiered strategies used within a tiered storage environment; A monitoring period configured by the file system is determined, wherein the monitoring period is used to determine the heat count value corresponding to the plurality of files; Within the defined monitoring period, based on information received from the file system, access count information corresponding to one of the plurality of files is determined, the access count information including hot count values ​​corresponding to the plurality of files for the corresponding access protocol used to access each file; The determined heat count value corresponding to each file is stored in the file metadata; Determine a protocol-based heat count value corresponding to the plurality of files, wherein the protocol-based heat count value is generated at least in part based on retrieving the protocol-based heat count value corresponding to the file from the file metadata; The multiple files are stored between layers of the hierarchical storage environment based on a set of two or more hierarchical strategies identified and a protocol-based hot count value corresponding to the multiple files. Sum the protocol-based heat count values ​​corresponding to the plurality of files; Upgrade the tier of the hierarchical storage environment storing the plurality of files, at least in part based on the sum of the protocol-based heat counts and the set of two or more tiering strategies; and The file is migrated within the upgraded tier of the tiered storage environment, and instances of file metadata corresponding to the file are updated to indicate the initial storage tier allocation of the file associated with the file after a period of inactivity.

9. The computer program product according to claim 8, wherein, Identify a set of two or more tiered strategies used within the tiered storage environment, based at least in part on information associated with the set of executing applications, wherein a first application in the set of executing applications accesses a first file using a first protocol.

10. The computer program product according to claim 9, wherein, Identifying a set of two or more tiered policies used within the tiered storage environment is also based on one or more instructions related to a Service Level Agreement (SLA), which is associated with an application in a set of applications.

11. The computer program product according to claim 8, wherein, Storing the multiple files across layers of the hierarchical storage environment based on a set of two or more identified hierarchical strategies also includes: Determine the analysis hierarchy among the two or more hierarchical strategies in the set; Analyze protocol-based heat count values ​​corresponding to the plurality of files across the set of two or more hierarchical strategies with respect to the analysis hierarchy; and The files are allocated among storage layers based on a determined set of heat count values ​​associated with each of the files, and also based on an analysis hierarchy among two or more identified hierarchical strategies.

12. The computer program product according to claim 11, wherein, A service level agreement (SLA) associated with at least one of the set of execution applications specifies the analytics hierarchy among two or more identified hierarchical policies, wherein the SLA allows for upgrading the storage tier allocation of a file based on subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies, and prevents for downgrading the storage tier allocation of the file based on the subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies.

13. The computer program product according to claim 8, wherein, The identified set of two or more hierarchical strategies also includes one or more multi-protocol hierarchical strategies associated with files accessed by two or more protocols, and wherein a set of heat count values ​​associated with the multi-protocol hierarchical strategies is based on a combination of two or more access protocols used to access the files and the number of executed applications utilizing the combination of two or more access protocols used to access the files.

14. The computer program product according to claim 11, wherein, The analysis of the protocol-based heat count values ​​corresponding to the multiple files also includes: In response to determining that a file in the plurality of files is accessed using two or more access protocols, a multi-protocol hot count value corresponding to the file and also associated with a combination of access protocols is determined, wherein the multi-protocol hot count value corresponding to the file is the sum of the hot count values ​​corresponding to each access protocol in the combination of access protocols; and Used to analyze protocol-based heat counts and one or more multi-protocol heat counts corresponding to the multiple files across the set of two or more hierarchical strategies with respect to the analysis hierarchy.

15. A computer system comprising: One or more computer processors; One or more computer-readable storage media; as well as Program instructions stored on the computer-readable storage medium for execution by at least one of the one or more computer processors, the program instructions comprising: Program instructions used to determine multiple files stored within a tiered storage environment and accessed by a set of executing applications; Program instructions used to identify a set of two or more tiered strategies used within a tiered storage environment; Program instructions for determining a monitoring period configured by the file system, wherein the monitoring period is used to determine a heat count value corresponding to the plurality of files; Used to determine, within a defined monitoring period, access count information corresponding to one of the plurality of files based on information received from the file system, the access count information containing program instructions corresponding to the plurality of files for the hot count value of the corresponding access protocol used to access each file; Program instructions for storing the determined heat count value corresponding to each file in the file metadata; Program instructions for determining protocol-based heat count values ​​corresponding to the plurality of files, wherein the protocol-based heat count values ​​are generated at least in part based on retrieving protocol-based heat count values ​​corresponding to the files from the file metadata; Program instructions for storing the plurality of files between layers of the hierarchical storage environment based on a set of two or more hierarchical strategies identified and a protocol-based hot count value corresponding to the plurality of files; Program instructions for summing protocol-based heat count values ​​corresponding to the plurality of files; Program instructions for upgrading a layer of the hierarchical storage environment storing the plurality of files, based at least in part on the sum of the protocol-based heat counts and the set of two or more hierarchical strategies; and Program instructions for migrating the file within an upgraded tier of the tiered storage environment and updating instances of file metadata corresponding to the file to indicate the initial storage tier allocation of the file associated with the file after a period of inactivity.

16. The computer system according to claim 15, wherein, Program instructions for identifying a set of two or more hierarchical strategies used within the hierarchical storage environment are based at least in part on information associated with the set of executing applications, wherein a first application in the set of executing applications accesses a first file using a first protocol.

17. The computer system according to claim 16, wherein, The program instructions used to identify a set of two or more tiered policies used within the tiered storage environment are also based on one or more instructions related to a Service Level Agreement (SLA), which is associated with the execution of an application within a set of applications.

18. The computer system according to claim 15, wherein, The method for storing the plurality of files across layers of the hierarchical storage environment based on a set of two or more identified hierarchical strategies also includes: Program instructions for determining the analysis hierarchy between the set of two or more hierarchical strategies; Program instructions for analyzing protocol-based heat count values ​​corresponding to the plurality of files across the analysis hierarchy within the set of two or more hierarchical strategies; and Program instructions for allocating the plurality of files among storage layers based on a determined set of heat count values ​​associated with the plurality of files respectively and also based on an analysis hierarchy among an identified set of two or more hierarchical strategies.

19. The computer system according to claim 18, wherein, A service level agreement (SLA) associated with at least one of the set of execution applications specifies the analytics hierarchy among two or more identified hierarchical policies, wherein the SLA allows for upgrading the storage tier allocation of a file based on subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies, and prevents for downgrading the storage tier allocation of the file based on the subsequent hierarchical policies of the analytics hierarchy among the hierarchical policies.

20. The computer system according to claim 18, wherein, The program instructions for analyzing protocol-based heat count values ​​corresponding to the multiple files also include: In response to determining that a file in the plurality of files is accessed using two or more access protocols, program instructions are provided to determine a multi-protocol heat count value corresponding to the file and also associated with a combination of access protocols, wherein the multi-protocol heat count value corresponding to the file is the sum of heat count values ​​corresponding to each access protocol in the combination of access protocols; and Program instructions for analyzing protocol-based heat counts corresponding to the plurality of files and one or more multi-protocol heat counts corresponding to the files across the set of two or more hierarchical strategies with respect to the analysis hierarchy.

Images