Deduplication of data integrity checks across systems

Abstract

A computer-implemented method includes receiving, in a clustered file system, a request from a formatted file system to perform a data integrity check on a portion of data. A determination is made as to whether the request includes a file system type of the portion of data, and in response to determining that the request includes a file system type of the portion of data, a further determination is made as to whether the clustered file system supports data integrity checking for the file system type. In response to determining that the clustered file system supports data integrity checking, a further determination is made as to whether the portion of data is currently available. The computer-implemented method also includes causing a data integrity check to be performed in response to determining that the portion of data is currently available. Results of performing the data integrity check are also sent to the formatted file system.

Description

[Technical Field]

[0001] The present invention relates to data storage, and more particularly, the present invention relates to deduplication of data integrity checking between different file systems. [Background technology]

[0002] Some file systems use inodes, which are data structures that each describe a file system object, such as a file or a directory. Each inode contains information about the corresponding file system object, such as the inode number (e.g., a unique identification number), the owner and group associated with the file, and the permission list associated with the file. Thus, an individual inode may contain actual file data, directory information (e.g., metadata), or it may be unused or "free."

[0003] Thus, the act of creating a new file involves changing the inode's state from free to allocated, actually writing the data to the new file, and updating the corresponding metadata to reflect these changes. However, a failure event may occur before this process of creating a new file can be completed, which may result in substantial file system corruption in certain circumstances. In an attempt to repair this corruption if it occurs, a data integrity check may be performed, such as a file system consistency check (FSCK) or other command that can address system corruption.

[0004] Such data integrity checks typically involve reading metadata, including inodes, and attempting to resolve inconsistency issues. While such data integrity checks can overcome corruption, they are computationally intensive and can be prone to unnecessary duplication in environments with different file system implementations. Summary of the Invention

[0005] A computer-implemented method according to one embodiment includes, in a clustered file system, receiving a request from a formatted file system to perform a data integrity check on a portion of data. A determination is made as to whether the request includes a file system type of the portion of data, and in response to determining that the request includes a file system type of the portion of data, a further determination is made as to whether the clustered file system supports data integrity checking for the file system type of the portion of data. In response to determining that the clustered file system supports data integrity checking for the file system type of the portion of data, a further determination is made as to whether the portion of data is currently available. The computer-implemented method also includes, in response to determining that the portion of data is currently available, causing a data integrity check to be performed on the portion of data. Results of the data integrity check are sent to the formatted file system.

[0006] Another embodiment of a computer program product includes a computer-readable storage medium having program instructions embodied therein, the program instructions being readable and / or executable by a processor to cause the processor to perform the method described above.

[0007] According to yet another embodiment, a system includes a processor and logic integrated with, executable by, or integrated with and executable by the processor, the logic configured to perform the method described above.

[0008] Other aspects and embodiments of the present invention will become apparent from the following detailed description, which, when taken in conjunction with the drawings, illustrate by way of example the principles of the invention. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates a network architecture according to one embodiment. [Figure 2] 2 illustrates respective hardware environments that may be associated with the server and / or client of FIG. 1, according to one embodiment. [Figure 3] FIG. 1 illustrates a tiered data storage system, according to one embodiment. [Figure 4] FIG. 1 is a partial representation of a storage system according to one embodiment. [Figure 5A] 1 is a flowchart illustrating a method according to one embodiment. [Figure 5B] 1 is a flowchart illustrating a method according to one embodiment. [Figure 6] FIG. 1 is a partial representation of a distributed file system according to an example use case. [Figure 7] FIG. 1 illustrates a cloud computing environment according to one embodiment of the present invention. [Figure 8] FIG. 2 illustrates abstraction model layers according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following description is made for the purpose of illustrating the general principles of this invention and is not intended to limit the inventive concepts claimed in this application. Furthermore, particular features described herein can be used in combination with other described features in each of the various possible combinations and permutations.

[0011] Unless otherwise defined herein, all terms are to be given their broadest possible interpretation, including the meaning implied by this specification and the meaning understood by those skilled in the art and / or defined in dictionaries, treatises, etc.

[0012] It should also be noted that, as used in this specification and the appended claims, the singular forms "a," "an," and "the" include the plural referent unless otherwise expressly stated. It should also be understood that the terms "comprises" and / or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, or components, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof.

[0013] The following description discloses several preferred embodiments of systems, methods, and computer program products for reducing consumed computational resources while ensuring overall system effective performance is maintained by eliminating duplicated data integrity checks between file systems. Moreover, as described in further detail below, for example, by selectively using resources with higher computational throughput to perform these deduplicated, computationally intensive data integrity checks, various embodiments herein can further improve computational performance by reducing latency and increasing throughput.

[0014] In one general embodiment, a computer-implemented method includes, in a clustered file system, receiving a request from a formatted file system to perform a data integrity check on a portion of data. A determination is made as to whether the request includes a file system type of the portion of data, and in response to determining that the request includes a file system type of the portion of data, a further determination is made as to whether the clustered file system supports data integrity checking for the file system type of the portion of data. In response to determining that the clustered file system supports data integrity checking for the file system type of the portion of data, a further determination is made as to whether the portion of data is currently available. The computer-implemented method also includes, in response to determining that the portion of data is currently available, causing a data integrity check to be performed on the portion of data. Results of the data integrity check are sent to the formatted file system.

[0015] In another general embodiment, a computer program product includes a computer-readable storage medium having program instructions embodied thereon, the program instructions being readable and / or executable by a processor to cause the processor to perform the above-described method.

[0016] In yet another general embodiment, a system includes a processor and logic integrated with, executable by, or integrated with and executable by the processor, and the logic is configured to perform the method described above.

[0017] Figure 1 illustrates an architecture 100 according to one embodiment. As shown in Figure 1, multiple remote networks 102 are provided, including a first remote network 104 and a second remote network 106. A gateway 101 may be coupled between the remote networks 102 and a neighboring network 108. In the context of this architecture 100, each of the networks 104, 106 may take any form, including, but not limited to, a local area network (LAN), a wide area network (WAN) such as the Internet, a public switched telephone network (PSTN), a private telephone network, etc.

[0018] In use, the gateway 101 serves as an entry point from the remote network 102 to the adjacent network 108. The gateway 101 can therefore function as a router that can direct a given packet of data arriving at the gateway 101, and as a switch that provides the actual path into and out of the gateway 101 for a given packet.

[0019] Additionally, at least one data server 114 is included, coupled to the neighboring network 108 and accessible from the remote network 102 via the gateway 101. Note that the data server 114 may include any type of computing device / groupware. Coupled to each data server 114 are multiple user devices 116. The user devices 116 may be directly connected by one of the networks 104, 106, 108. Such user devices 116 may include desktop computers, laptop computers, handheld computers, printers, or any other type of logic. Note that in one embodiment, the user devices 116 may be directly coupled to one of the networks.

[0020] A peripheral device 120 or a series of peripheral devices 120, such as a facsimile machine, a printer, a network storage unit or system, or a local storage unit or system, or both, may be coupled to one or more of the networks 104, 106, 108. It should be noted that databases and / or additional components may be used with or incorporated into any type of network element coupled to the networks 104, 106, 108. In the context of this description, a network element may refer to any component of a network.

[0021] According to some approaches, the methods and systems described herein may be implemented by and / or on virtual systems and / or systems that emulate one or more other systems, such as a UNIX system emulating an IBM® z / OS® environment (IBM and all IBM-based trademarks and logos are trademarks or registered trademarks of International Business Machines Corporation and / or its affiliates), a UNIX system virtually hosting a known operating system environment, or an operating system that emulates an IBM® z / OS® environment. This virtualization and / or emulation may be enhanced in some embodiments through the use of VMware® software.

[0022] In a further approach, one or more networks 104, 106, 108 may represent a cluster of systems commonly referred to as a "cloud." In cloud computing, shared resources, such as processing power, peripherals, software, data, and servers, are provided to any system in the cloud in an on-demand relationship, thereby enabling services to be accessed and distributed among many computing systems. Cloud computing typically requires internet connectivity between systems operating in the cloud, although other technologies for connecting systems can also be used.

[0023] Figure 2 illustrates a representative hardware environment associated with the user device 116 and / or server 114 of Figure 1, according to one embodiment. The diagram illustrates a typical hardware configuration of a workstation having a central processing unit 210, such as a microprocessor, and several other units interconnected via a system bus 212.

[0024] The workstation shown in FIG. 2 includes random access memory (RAM) 214, read-only memory (ROM) 216, input / output (I / O) adapter 218 for connecting peripheral devices such as disk storage unit 220 to bus 212, user interface adapter 222 for connecting keyboard 224, mouse 226, speakers 228, microphone 232 and / or other user interface devices such as a touch screen and digital camera (not shown) to bus 212, communications adapter 234 for connecting the workstation to communications network 235 (e.g., a data processing network), and display adapter 236 for connecting bus 212 to display device 238.

[0025] The workstation may have an operating system resident thereon, such as Microsoft Windows® Operating System (OS), macOS®, or UNIX® OS. It is understood that the preferred embodiment may be implemented on platforms and operating systems other than those mentioned. The preferred embodiment may be written using eXtensible Markup Language (XML), C, or C++, or a combination thereof, or other programming languages, along with object-oriented programming methods. Object-oriented programming (OOP), which is increasingly used to develop complex applications, may also be used.

[0026] Referring now to Figure 3, a storage system 300 according to one embodiment is shown. Note that some of the elements shown in Figure 3 may be implemented as hardware and / or software, according to various embodiments. The storage system 300 may include a storage system manager 312 for communicating with multiple media and / or drives on at least one upper storage tier 302 and at least one lower storage tier 306. The upper storage tier 302 may preferably include one or more random-access and / or direct-access media 304, such as hard disks in a hard disk drive (HDD), non-volatile memory (NVM), solid-state memory in a solid-state drive (SSD), flash memory, SSD arrays, flash memory arrays, or other media or combinations thereof described herein or known in the art. The lower storage tier 306 may preferably include one or more lower performance storage media 308, including sequential access media such as magnetic tape in a tape drive and / or optical media, slower access HDDs, slower access SSDs, etc., or other media described herein or known in the art, or any combination thereof. One or more additional storage tiers 316 may include any combination of storage and memory media desired by the designer of the system 300. Also, either the upper storage tier 302 or the lower storage tier 306 or both may include some combination of storage devices and / or storage media.

[0027] Storage system manager 312 may communicate with drives or storage media 304, 308 on upper storage tier 302 and lower storage tier 306, or both, via network 310, such as a storage area network (SAN) as shown in FIG. 3, or some other suitable type of network. Storage system manager 312 may also communicate with one or more host systems (not shown) via host interface 314, which may or may not be part of storage system manager 312. Storage system manager 312 and / or any other components of storage system 300 may be implemented in hardware and / or software and may utilize a processor (not shown) for executing commands of a type known in the art, such as a central processing unit (CPU), field programmable gate array (FPGA), application-specific integrated circuit (ASIC), or the like. Of course, any configuration of storage system may be used, as would be apparent to one of ordinary skill in the art upon reading this specification.

[0028] In further embodiments, storage system 300 may include any number of data storage tiers, each containing the same or different storage media. For example, each data storage tier may contain the same type of storage media, such as HDDs, SSDs, sequential access media (e.g., tapes in tape drives, optical disks in optical disk drives), direct access media (e.g., CD-ROMs, DVD-ROMs), or any combination of media storage types. In one such configuration, upper storage tier 302 may contain a majority of SSD storage media for storage of data in a higher-performance storage environment, while the remaining storage tiers, including lower storage tier 306 and additional storage tier 316, may contain any combination of SSDs, HDDs, tape drives, etc. for storage of data in a lower-performance storage environment. In this manner, more frequently accessed data, data of higher priority, data needing to be accessed more quickly, etc., may be stored in upper storage tier 302, while data without any of these attributes may be stored in lower storage tier 306 and additional storage tier 316. Of course, those skilled in the art, after reading this specification, will be able to devise many other combinations of storage media types to implement different storage schemes in accordance with the embodiments shown herein.

[0029] According to some embodiments, a storage system (e.g., 300) may include logic configured to receive a request to open a data set, logic configured to determine whether the requested data set is stored in multiple associated portions in a lower storage tier 306 of the tiered data storage system 300, logic configured to move each associated portion of the requested data set to an upper storage tier 302 of the tiered data storage system 300, and logic configured to aggregate the requested data set in the upper storage tier 302 of the tiered data storage system 300 from the associated portions.

[0030] Of course, this logic may be implemented as a method or computer program product on any device and / or system according to various embodiments.

[0031] As described above, applications configured in software-based entities such as containers or virtual machines each perform data operations (e.g., read operations and write operations) that are specific to their local file systems. As a result, data operations performed by such software-based entities are visible (e.g., known) only to the operating system of the software-based entities. Entities such as containers and virtual machines have their own local file systems, but also share an overall file system with other entities in the same location. As a result, these entities can achieve a reduction in overall size and computing resource consumption.

[0032] While the use of entities such as containers and virtual machines offers several advantages, traditional implementations that rely on these entities have significant practical inefficiencies. These inefficiencies stem from the fact that software-based entities have limited computing power, thereby making them a bottleneck for the data operations they perform. Each software-based entity's local file system also has difficulty coordinating with the overall file system, resulting in significant inefficiencies and performance losses during system operation.

[0033] In sharp contrast, various embodiments of the present disclosure may reduce the computational resources consumed during operation by eliminating redundant data integrity checks while ensuring that the effective performance of the overall system is maintained. This may desirably result in significant and substantial improvements in the efficiency with which a storage system may operate while ensuring that data operations are successful. Furthermore, by selectively using resources with higher computational throughput to perform these data integrity checks, various embodiments of the present disclosure may further improve computational performance by reducing latency and increasing throughput, for example, as described in further detail below.

[0034] Referring now to FIG. 4, a storage system 400 according to one embodiment is illustrated. As an option, the system 400 can be implemented in combination with features of any other embodiment enumerated herein, such as features described with reference to other figures. However, such system 400 and other systems illustrated herein can be used in various applications and / or permutations that may or may not be specifically described in the exemplary embodiments enumerated herein. Furthermore, the system 400 illustrated herein can be used in any desired environment. Therefore, FIG. 4 (and other figures) can be considered to include all possible permutations.

[0035] As shown, the overall storage system 400 includes a storage environment 402 and a container environment 404. It should be noted that the term "environment" is not intended to be limiting in any way, and either of these environments 402, 404 may include any desired components, software, etc. According to this approach, the storage environment 402 is illustrated as including multiple data storage devices 408 that combine to form a storage subsystem 410 of the overall storage system 400. The data storage devices 408 may include any type of memory, such as HDDs, SSDs, magnetic tape libraries, etc., depending on the approach. The data storage devices 408 are also coupled to a controller 412 that can be used to direct data flow, execute software programs, perform data manipulation, communicate with the container environment 404, etc. In some approaches, the controller 412 can be used, at least in part, to maintain a clustered file system in the storage environment 402.

[0036] Referring now to the container environment 404, a container orchestrator 414 is shown and can be used to manage the containers 416, 418, and 420 present (or at least functioning) in the container environment 404. The container orchestrator 414 can thereby function as a controller in some approaches. Each of the containers 416, 418, and 420 can also preferably map back to data physically stored in the data storage device 408 of the storage subsystem 410 and can be used to manage the data accordingly. For example, each of these containers 416, 418, and 420 can represent part of an application embodied as a single package that can be moved in its entirety from one computing environment to another. Thus, in some approaches, multiple application containers may be granted access to the same data, allowing these application containers to be created, moved, destroyed, etc., without actually affecting the underlying data to which they map. As a result, the data held in the container is preferably done so in a stateless manner so that, for example, the data remains the same regardless of how many times it is applied across different operating systems and / or applications.

[0037] 4 illustrates container environment 404 as including only containers, it should also be noted that virtual machines, applications, etc., or any other desired type of software-based entities may also exist (or at least function, be implemented, etc.) in container environment 404. Thus, for example, as will be understood by those skilled in the art upon reading this specification, container orchestrator 414 may be able to manage (e.g., monitor, control, interact with, etc.) any desired type of software-based entities present in container environment 404.

[0038] For example, virtual machine and container environments allow administrators to host and manage their application workloads, while providing users with advantages in development speed and operational efficiency. As a result, users and / or developers have the flexibility to move workloads around the system based on specific circumstances. In some approaches, users can move their application workloads between containers and virtual machines, or split application workloads within the same orchestrator. Therefore, the use of the term "container" herein is not intended to be limiting in any way and may implement any desired type of software-based entity.

[0039] Continuing with reference to FIG. 4, the storage environment 402 and the container environment 404 preferably communicate with each other by sending data, commands, requests, metadata, updates, etc. between them. In this approach, the storage environment 402 and the container environment 404 may communicate with each other via a network 406 spanning the two environments, although any form of communication may be used. For example, in other approaches, the storage environment 402 and the container environment 404 may communicate via a physical electrical connection between the two environments, although any type of communication channel may be used. It should also be noted that the network 406 shown in FIG. 4 may be of any type, depending, for example, on the desired approach. For example, in some approaches, the network 406 is a WAN such as the Internet. However, an example list of other network types on which the network 406 may be implemented includes, but is not limited to, a LAN, a PSTN, a SAN, a private network, etc. Thus, the storage environment 402 and the container environment 404 may communicate with each other regardless of the amount of separation that exists between them, for example, despite being located in different geographic locations.

[0040] As previously mentioned, applications configured in software-based entities such as containers (e.g., see 416, 418, and 420 in Figure 4) or virtual machines each perform data operations (e.g., read and write operations) that are specific to their local file system. The local file system provides a platform that can be used to perform local data operations without affecting the actual data in physical storage. However, the data operations performed by such software-based entities are thereby visible (e.g., known) only to the operating system of the software-based entities. Entities such as containers and virtual machines have their own local file systems, but also share an overall file system with other entities in the same location. As a result, these entities can achieve a reduction in overall size and computing resource consumption.

[0041] While the use of entities such as containers and virtual machines offers several advantages, traditional implementations that rely on these entities have significant practical inefficiencies. These inefficiencies stem from the fact that software-based entities have limited computing power, thereby making them a bottleneck for the data operations they perform. Each software-based entity's local file system also has difficulty coordinating with the overall file system, resulting in significant inefficiencies and performance losses during system operation, as the effects of any inefficiencies can be multiplied across multiple different locations.

[0042] In stark contrast to these conventional drawbacks, various of the processes included in method 500 can significantly improve performance by eliminating duplication of operations and reducing computational resource consumption. Method 500 can be performed by the present invention in various embodiments, such as, among others, in any of the embodiments illustrated in Figures 1-4. Of course, as will be understood by those skilled in the art upon reading this specification, method 500 may include more or fewer operations than those specifically depicted in Figures 5A and 5B.

[0043] Each of the steps of method 500 can be performed by any suitable component of an operating environment using known techniques and / or techniques that will be readily apparent to those skilled in the art upon reading this disclosure. In various other embodiments, method 500 can be performed in part or in whole by a controller, processor, or the like, or some other device incorporating one or more processors. A processor, e.g., a processing circuit, chip, or module, or combination thereof, implemented in hardware and / or software and preferably having at least one hardware component, can be used in any device to perform one or more steps of method 500. Examples of processors include, but are not limited to, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or the like, combinations thereof, or any other suitable computing device known in the art.

[0044] At least a portion of the processes included in method 500 may be performed by one of the processors (e.g., controller, orchestrator, CPU, etc.) included in system 400 of FIG. 4 . For example, the processes included in method 500 may be performed by controller 412 in storage environment 402 of FIG. 4 . As such, controller 412 would be able to communicate with container orchestrator 414 over network 406. In another approach, at least a portion of the processes included in method 500 may be performed by container orchestrator 414 in container environment 404 of FIG. 4 . As such, container orchestrator 414 would be able to communicate with container controller 412 over network 406. As such, performance of method 500 typically requires communication across at least two different environments, each having a different file system, for example, as described in further detail below. Furthermore, although the processes included in method 500 have been described in the context of being performed by a controller in a storage environment having a clustered file system (e.g., controller 412 in storage environment 402 of FIG. 4 ), this is not intended to be limiting in any way.

[0045] Referring now to FIG. 5A , operation 502 of method 500 includes establishing a communication channel between two different environments. As discussed above, each of these environments preferably has a different file system associated with it. For example, one of the environments may correspond to a clustered file system (e.g., the clustered file system in storage environment 402 of FIG. 4 ), while the other of the environments corresponds to a local file system (e.g., the local file system in each of containers 416, 418, and 420 of FIG. 4 ). It should also be noted that while various of the processes described herein involve two different environments, this is not intended to be limiting in any way. For example, as will be apparent to those skilled in the art upon reading this specification, one or more of these processes may be used to deduplicate data operations between three or more different file systems, depending on the desired approach.

[0046] According to some embodiments, the communication channel established in operation 502 can actually extend between processes running on different file systems (or at least between controllers running and / or managing the processes, respectively). For example, in some approaches, operation 502 includes establishing a communication channel extending between data integrity checking on a local formatted file system and data integrity checking on a clustered file system. Referring back to FIG. 4 , a communication channel can be formed between data integrity checking performed on a local formatted file system associated with any one of containers 416, 418, 420 in container environment 404 and data integrity checking performed by controller 412 on a clustered file system associated with storage environment 402. Thus, the communication channel can identify whether particular data stored in a back-end clustered file system is exported and used in a local formatted file system, for example, for application storage.

[0047] The specific type of communication channel may vary depending on the type of file system involved, the entities between which the communication channel is formed, etc., and the type of data integrity check may also vary depending on the approach. For example, in some approaches, the data integrity check is an FSCK process. Thus, in such approaches, a communication channel may be established extending between an FSCK process in a local formatted file system (e.g., having a container and / or virtual machine) and an FSCK process in a clustered file system (e.g., functioning as a storage subsystem). The communication channel itself may therefore implement any protocol associated with running an FSCK process and may be formed using any wired and / or wireless components contained in either file system location and any components that may already be used to facilitate communication between file systems (see, e.g., network 406 in FIG. 4).

[0048] Continuing with reference to FIG. 5A, operation 504 includes receiving a request to perform a data integrity check on a portion of the data. The request may be received from at least one of the local formatted file systems in the clustered file system of the storage environment via a communication channel existing therebetween. The data integrity check may also be requested in response to the occurrence of file system corruption in at least one of the local formatted file systems. Thus, in some approaches, the data integrity check is a FSCK process.

[0049] In a preferred approach, the request includes metadata that provides additional information associated with the desired data integrity check. For example, the request may include information (e.g., metadata) identifying the type of file system to which and / or in which the data corresponds (e.g., metadata). Some file systems use inode data structures, each of which corresponds to an object such as a file or a directory. An inode corresponding to a given file (e.g., a group of data) contains information about the given file, which may include an inode number (a unique identification number); the owner and / or group associated with the given file; the file and / or file system type; a permission list associated with the given file; file creation, access, and modification times; the size of the file; and the physical storage location of the given file. Thus, an individual inode may contain actual file data, directory information (e.g., metadata associated with the file), or may be unused (also referred to herein as "free"). Additionally, the act of creating a new file involves changing the status of the corresponding inode from free to allocated, writing data to the new file, and updating the metadata associated with the new file.

[0050] While this process of creating and updating files allows data in storage to be more effectively managed and accessed, computational failures (e.g., crashes) can occur before the process can be completed, potentially resulting in file system corruption. One way to overcome such file system corruption as it occurs over time is to perform data integrity checks, which typically involve examining metadata (e.g., inodes) and attempting to resolve identified inconsistencies. For example, FSCK operations can take several corrective actions to overcome computational failures that may occur during the operation. These corrective actions include updating allocation maps to mark incorrectly allocated blocks as free, creating directory entries for files and / or directories that have allocated inodes but no directory entries exist, removing directory entries that point to unallocated inodes or contain metadata that is inconsistent with the metadata stored in their respective inodes, updating link counts for files and / or directories to reflect accurate counts, etc.

[0051] While data integrity checks such as FSCK operations can improve overall system performance, it is important to monitor and prevent duplicate operations. By way of example, and not intended to be limiting, a disk subsystem issue can result in data corruption of a file at a specific disk address, and this inconsistency can be detected by both the container's local formatted file system and the backend clustered file system. Left unchecked, FSCK operations can proceed at both layers, resulting in performance losses due to duplicate execution of operations at the container's local formatted file system level and the backend storage file system level. As noted above, software-based entities such as containers have limited associated computing resources (e.g., CPU cores, RAM, cache, etc.), and therefore, performing unnecessary operations (e.g., FSCK operations) can cause significant downtime and performance degradation to running applications. However, by achieving coordination between the performance of the local formatted file system and the performance of the back-end clustered file system, various of the techniques included herein can significantly reduce computing resource consumption and reduce operational bottlenecks, for example by using the higher throughput of the back-end clustered file system when possible, as will become apparent below.

[0052] With further reference to FIG. 5A, method 500 further includes determining whether the request includes a file system type for that portion of data. See decision 506. The particular type of file system to which a given set of data corresponds impacts how or whether processes such as data integrity checks can be performed. Different file system types include different processes, protocols, permission levels, etc., and therefore significantly impact what types of data integrity checks can be performed and what components can perform those data integrity checks.

[0053] As described above, metadata can be used to identify the type of file system to which and / or in which the data corresponds. Thus, determination 506 can be made by examining metadata that may be received with the request and / or that may be stored in memory (e.g., in a lookup table). In some approaches, this determination is made by determining whether the actual piece of data has a metadata tag (also referred to herein as an “extended attribute”) corresponding to it. The metadata tag preferably contains information associated with the piece of data and can exist in any desired form. For example, in different approaches, the metadata tag may include one or more flags, be in the form of a file header, contain one or more keywords and / or terms associated with the data, correspond to an inode entry, etc. The metadata tag may also contain any desired type of information, such as the file system type, I / O activity associated with the received request, execution time information, priority associated with the request and / or the data itself, attached application container details, etc.

[0054] Thus, by examining any metadata associated with the data, information such as file system type and / or format can be desirably identified. After the identifying metadata and file system type and / or format are identified, the backend cluster file system can effectively perform a data integrity check for both file systems using only the backend clustered file system itself. For example, in an approach where the data integrity check includes an FSCK operation supported by the clustered file system, an FSCK daemon running on the clustered file system can be used to actually perform the data integrity check and implement corrective actions that improve performance of both the local formatted file system and the backend clustered file system. This desirably avoids unnecessary consumption of computing resources caused by duplicate operations and also improves the efficiency of performing the data integrity check, for example, as described above, the backend clustered file system can achieve significantly higher throughput than the local formatted file system.

[0055] In response to determining that the received request includes information such as the file system type of the portion of data, the information is preferably extracted for further use. Accordingly, in response to determining that the received request includes information such as the file system type of the portion of data, method 500 proceeds from decision 506 to operation 508, where operation 508 includes extracting metadata tags (or "extended attributes").

[0056] According to some techniques, information such as file system type can be extracted by exporting inode data and / or structural information from a file being formatted in a local formatted file system. The FSCK daemon of the backend clustered file system can be used to examine the identified file and extract inode metadata to identify the type of file system formatted by the container application. In some cases, the container orchestration layer also assists by passing formatted file system information from the container layer to the backend clustered file system using metadata, for example, as described above. Based on the inode structural inspection, the FSCK daemon can also determine information associated with the data, such as formatted block size, file system size, and other information relevant to performing FSCK operations on the file, for example, as would be understood by one of ordinary skill in the art after reading this specification.

[0057] With further reference to FIG. 5A , method 500 proceeds from operation 508 to decision 510, which includes determining whether the clustered file system supports data integrity checking for the file system type of that portion of data. As discussed above, the particular type of file system to which a given set of data corresponds impacts how or whether processes such as data integrity checking can be performed. Different file system types include different processes, protocols, permission levels, etc., which therefore significantly impact what types of data integrity checks can be performed and what components can perform those data integrity checks. Thus, decision 510 can be determined by comparing information in the extracted metadata tags to features and / or functionality supported by the clustered file system.

[0058] In response to determining that the clustered file system supports data integrity checking for the file system type of the portion of data, method 500 proceeds to decision 512, which includes determining whether the portion of data is currently available. In other words, decision 512 includes determining whether the portion of data (e.g., a file) associated with the received data integrity check is currently in use or available for performing an FSCK operation. In some approaches, this determination can be made by determining whether the portion of data has been exported from the clustered file system, whether it is currently being used as local storage by a local formatted file system, or both. The current usage information for the data can be stored in a logical-to-physical table, metadata tags associated with the given data, a predetermined location in memory, etc.

[0059] In response to determining in decision 512 that the portion of data is currently available, method 500 proceeds to operation 514, where operation 514 includes causing a data integrity check to be performed on the portion of data identified in the requested data integrity check. As described above, various of the processes included in method 500 may be performed by a controller in a storage environment having a clustered file system. Thus, at least a portion of the data integrity check may actually be performed by a daemon operating in the clustered file system of the storage environment. According to some approaches, causing a data integrity check to be performed on the portion of data may actually include sending one or more instructions to an FSCK daemon operating in the clustered file system and associated with an FSCK process in the clustered file system to perform the data integrity check.

[0060] Due to the efficiencies gained by using the higher computational throughput of the backend clustered file system, data integrity checks are preferably performed by the backend clustered file system rather than the local formatted file system. As discussed above, entities such as containers and virtual machines have limited computational capabilities and therefore represent a bottleneck for data operations performed in conventional implementations. The local file systems of each software-based entity also have difficulty coordinating with the overall file system, thereby causing significant inefficiencies and performance losses during system operation in conventional implementations. In sharp contrast to these conventional drawbacks, the techniques herein can reduce computational resources consumed during operation by eliminating duplicated data integrity checks while ensuring that the effective performance of the overall system is maintained. As a result, various embodiments of the present disclosure can desirably achieve significant and substantial improvements in the efficiency with which a storage system can operate while ensuring that data operations and data integrity checks are performed successfully. Furthermore, by selectively using resources with higher computational throughput to perform these deduplicated data integrity checks, various embodiments of the present disclosure can further improve computational performance by reducing latency and increasing overall throughput.

[0061] Method 500 proceeds from operation 514 to operation 516, which includes sending the results of the data integrity check to the local formatted file system. The information included in the results sent to the local formatted file system may vary depending on the particular approach, for example, based on the type of data integrity check performed, the file system type, user preferences, predetermined settings, etc. According to some approaches, the results of the data integrity check sent to the local formatted file system may include an FSCK timestamp, the status of the FSCK operation, details of corrupted files, etc. This information can then be used by the local formatted file system to implement the results of the data integrity check without actually performing the data integrity check in the local formatted file system. As noted above, this desirably reduces the amount of computing resources consumed while ensuring that the overall system remains efficient and performing well.

[0062] In response to performance of operation 516, method 500 may terminate. However, it should be noted that although method 500 may terminate in response to performance of operation 516, any one or more of the processes included in method 500 may be repeated to satisfy additional data integrity checks. In other words, any one or more of the processes included in method 500 may be repeated for subsequently received data integrity checks.

[0063] Returning now to decision 506, method 500 does proceed to operation 518 in response to determining that the request does not include a file system type for that portion of data. In other words, method 500 proceeds to operation 518 in response to determining that there is no way to identify a file system type associated with that portion of data. Here, operation 518 includes sending a request to the local formatted file system to update the request to include metadata that at least identifies a file system type associated with the given data. In some approaches, this request can be satisfied by the local formatted file system adding extended attributes to the data integrity checking request and then resubmitting the request. Thus, method 500 returns from operation 518 to operation 504 so that an updated request and / or another request can be received and processed in various ways described and / or suggested herein.

[0064] Referring next to decision 510, method 500 proceeds to operation 520 in response to determining that the clustered file system does not support data integrity checking for the file system type of that portion of data. As discussed above, the particular type of file system to which a given set of data corresponds impacts how or even whether processes such as data integrity checking can be performed. Different file system types include different processes, protocols, permission levels, etc., which therefore significantly impact what types of data integrity checks can be performed and what components can perform those data integrity checks. Accordingly, operation 520 includes failing the received data integrity check. In response to performing operation 520, method 500 can end. However, in some approaches, an indication of the failure of the request may be sent to the entity and / or program that originally issued the request so that, for example, the request can be resubmitted to a different file system type.

[0065] Also returning to decision 512, method 500 proceeds to operation 522 in response to a determination that the portion of data corresponding to the received request is currently unavailable. In other words, in response to a determination that the portion of data (e.g., a file) associated with the received data integrity check is currently in use and therefore cannot be used to perform the data integrity check, method 500 proceeds to operation 522. Here, operation 522 includes sending a request to re-run the data integrity check once the data becomes available again. Depending on the approach, this can be done in a number of different ways. For example, in some situations, operation 522 may simply add a time delay before re-running decision 512 itself. This allows time for the data to return to a “free” state before retrying to proceed with the requested data integrity check. In yet other approaches, the failed data integrity check may be stored in memory to maintain a record of data that has not yet been repaired, e.g., for future iterations. In still other approaches, the requested data integrity check may actually be returned to the entity and / or program that originally issued the request, e.g., so that it can be re-run at a later point in time.

[0066] Thus, by eliminating duplicate data integrity checks, various processes included in method 500 may desirably reduce computational resources consumed during operation while ensuring that overall system effective performance is maintained. In this way, various embodiments included herein may desirably achieve significant and substantial improvements in the efficiency with which a storage system may operate while ensuring that data operations and data integrity checks are performed successfully. Furthermore, by selectively using resources with higher computational throughput to perform these deduplicated data integrity checks, various embodiments included herein may further improve computational performance by reducing latency and increasing throughput. In certain embodiments, this is achieved, at least in part, by enhancing a daemon (e.g., the FSCK daemon) running in the backend clustered file system to offload certain container-specific, localized tasks to the backend clustered file system utilities, thereby avoiding duplicate data integrity checks and achieving improved performance and reduced application downtime.

[0067] Thus, by selectively using resources with higher computational throughput to perform these deduplicated data integrity checks, various embodiments herein may further improve computational performance by reducing latency and increasing overall throughput. Referring now to Figure 6, a distributed file system 600 is shown with a use example illustrating how FSCK operations may be offloaded from a container file system 602 to a backend clustered file system in a storage environment 604. Note that this use example is provided for illustrative purposes only and is not intended to limit the invention in any way.

[0068] As shown, container file system 602 includes containers 616 and 618 and container orchestrator 606. Additionally, container file system 602 may communicate with storage environment 604 using network 603 spanning the two environments, although any form of communication may be used. For example, in other approaches, container file system 602 and storage environment 604 may communicate via a physical electrical connection between the two environments, although any type of communication channel may be used. Note also that network 603 may be of any type, depending, for example, on the desired approach. Thus, container file system 602 and storage environment 604 may communicate with each other regardless of the amount of separation that exists between them, even if they are located in different geographic locations.

[0069] Thus, the container orchestrator 606 can send an indication to the controller 608 in the storage environment 604 of the files and / or devices currently being used by the containers 616, 618. This allows the storage environment 604 to preferably determine whether a file stored in the backend cluster file system storage subsystem 610 is exported or being used as local storage for a given container. As noted, some techniques use cloud orchestration hints sent from the container orchestrator to perform this identification of the file (e.g., group of data). It should also be noted that the storage subsystem 610 can include any desired type of storage device 612, such as an HDD, SSD, magnetic tape library, etc.

[0070] The orchestration layer may also actually include the relationship between the storage resources of the containers 616, 618 (and any other software-based entities in the container file system 602, e.g., virtual machines) with respect to the corresponding back-end storage. In addition to the orchestration layer, a storage driver such as the Container Storage Interface (CSI) may be used to facilitate the export of back-end storage files and / or devices, e.g., as the container local storage is enhanced to associate metadata characteristics to indicate the mapping of a given file, as will be appreciated by those skilled in the art upon reading this specification.

[0071] The controller 608 can use this information to create a communication channel between the data stored in the storage environment 604 and the data stored in the local file system storage of each of the containers 616, 618. In other words, the backend clustered file system preferably creates the communication channel based on a mapping between the data residing in the containers 616, 618 and the data residing in the storage environment 604. In some approaches, the framework establishes a dependency mapping between the container's formatted local file system and the data exported from the backend storage by appending additional metadata tags (or extended attributes) to the file containing information such as the attached application container details, priority, execution time, I / O activity, and local file system format and / or type.

[0072] The backend clustered file system may also export inode data and / or structure from a file formatted by the container's local file system. In other words, a daemon operating within the storage environment 604 may be augmented to examine the identified file, extract inode metadata, if any, and identify the corresponding file system type formatted by the given container application. In some cases, the container orchestration layer may assist this process by passing formatted file system information from the container layer to the backend clustered file system using the metadata. Based on the examination of this inode structure, the FSCK daemon of the backend clustered file system may also determine the formatted block size, file system size, and other information so that the FSCK process can be performed on that particular file using the local file system format. The FSCK daemon may also attach a metadata label to the given file based on the examined file system information.

[0073] Using the above mapping and an inspection of the local formatted file system in the container storage, the backend clustered file system can perform the FSCK process on the file identified in the first received request. Based on the identified mapping of the container's local file system to the actual file in the backend clustered file system, the FSCK framework of the backend file system will be enhanced to perform FSCK on the file based on the formatted file system. For example, if the local storage file system is formatted with EXT3 in a container application, whenever the backend clustered file system performs FSCK activity on that file, it will run the FSCK utility for EXT3 on that file.

[0074] The FSCK daemon on the backend clustered filesystem can either implement an external native filesystem check routine (e.g. ext3 FSCK) and perform the operation on a given file, or invoke a native OS FSCK process available on the clustered filesystem node. The appropriate container and the backend clustered filesystem will also coordinate when the FSCK operation actually occurs on that given file.

[0075] In response to a successful FSCK operation on a file in the backend storage environment 604, a consistency result, along with details such as the latest FSCK timestamp, the FSCK status (e.g., failed or passed), and details of the corrupted file, is passed to the container file system via RPC, a common logging infrastructure, etc. Additionally, if an inconsistency is identified in the block on the backend storage that actually stores the corresponding data, the backend storage preferably attempts to perform recovery actions. However, if these recovery attempts fail, the backend clustered file system translates the corruption details into a form that can be interpreted by the local container file system and sends the translated corruption details to the container so that the container application can determine what further recovery actions to pursue.

[0076] Again, various techniques described herein may desirably reduce computational resources consumed during operation by eliminating duplicated data integrity checks while ensuring that overall system effective performance is maintained. Additionally, various embodiments herein may further improve computational performance by selectively using resources with higher computational throughput to perform these deduplicated, computationally intensive data integrity checks, thereby reducing latency and increasing throughput. As noted above, in some situations, this is achieved, at least in part, by enhancing daemons (e.g., the FSCK daemon) running in the backend clustered file system to offload certain container-specific, localized tasks to backend clustered file system utilities rather than to thin-provisioned containers, thereby avoiding duplicated data integrity checks and achieving improved performance and reduced application downtime.

[0077] Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention may be implemented in conjunction with any other type of computing environment now known or later developed.

[0078] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or interaction with the service provider. This cloud model can include at least five characteristics, at least three service models, and at least four deployment models.

[0079] The features are as follows:

[0080] On-demand self-service: Cloud customers can unilaterally provision computing capacity, such as server time and network storage, automatically as needed, without the need for human interaction with the service provider.

[0081] Wide network access: Functionality is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., mobile phones, laptops, and PDAs).

[0082] Resource Pooling: A provider's computing resources are pooled to accommodate multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated depending on demand. Consumers generally have no control over or knowledge of the exact location of the resources provided, but there is a sense of location independence in that they may be able to specify a higher level of abstraction (e.g., country, state, or data center).

[0083] Rapid Elasticity: Rapid elasticity allows for rapid scaling out, sometimes automatically, by provisioning capabilities, and rapid release of capabilities to rapidly scale in. To the consumer, the capabilities available for provisioning often appear unlimited, and can be purchased in any quantity at any time.

[0084] Metered Services: Cloud systems automatically control and optimize resource utilization by utilizing metering capabilities at some level of abstraction appropriate for the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, giving both providers and consumers transparency into the services utilized.

[0085] The service model is as follows:

[0086] Software as a Service (SaaS): The functionality offered to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application functions, with the possible exception of limited user-specific application configuration settings.

[0087] Platform as a Service (PaaS): The capability offered to a consumer is to deploy applications they create or acquire, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, in some cases, the application hosting environment configuration.

[0088] Infrastructure as a Service (IaaS): The capability provided to a customer is to provision processing, storage, network, and other basic computing resources onto which the customer can deploy and run any software, which may include operating systems and applications. The customer does not manage or control the underlying cloud infrastructure, but does have control over the operating systems, storage, deployed applications, and possibly limited control over selected network components (e.g., host firewalls).

[0089] The deployment model is as follows:

[0090] Private Cloud: This cloud infrastructure is operated solely for the organization. It can be managed by the organization or a third party and can reside on-premise or off-premise.

[0091] Community Cloud: This cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance concerns). It can be managed by the organization or a third party and can reside on-premises or off-premises.

[0092] Public Cloud: This cloud infrastructure is available to the public or large industry organizations and is owned by an organization that sells cloud services.

[0093] Hybrid Cloud: This cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain their own entities but are joined by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0094] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0095] Referring now to FIG. 7, an exemplary cloud computing environment 750 is illustrated. As shown, the cloud computing environment 750 includes one or more cloud computing nodes 710 with which local computing devices used by cloud users, such as a personal digital assistant (PDA) or mobile phone 754A, a desktop computer 754B, a laptop computer 754C, or an automotive computer system 754N, or any combination thereof, can communicate. The nodes 710 can communicate with each other. The nodes 710 may be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or any combination thereof, as described above. This allows the cloud computing environment 750 to provide infrastructure-as-a-service, platform-as-a-service, or software-as-a-service, or any combination thereof, for which the cloud users are not required to maintain resources on their local computing devices. The types of computing devices 754A-754N shown in FIG. 7 are intended to be illustrative only, and it is understood that computing node 710 and cloud computing environment 750 can communicate with any type of computerized device via any type of network connection or network-addressable connection, or combination thereof (e.g., using a web browser).

[0096] Referring now to Figure 8, there is shown a set of functional abstraction layers provided by cloud computing environment 750 (Figure 7). It should be understood in advance that the components, layers, and functions shown in Figure 8 are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0097] Hardware and software layer 860 includes hardware and software components. Examples of hardware components include mainframes 861, reduced instruction set computer (RISC) architecture-based servers 862, servers 863, blade servers 864, storage devices 865, and networks and networking components 866. In some embodiments, software components include network application server software 867 and database software 868.

[0098] The virtualization layer 870 provides an abstraction layer that can instantiate virtual entities such as virtual servers 871, virtual storage 872, virtual networks including virtual private networks 873, virtual applications and operating systems 874, and virtual clients 875.

[0099] In one example, management layer 880 may provide the following functions: Resource provisioning 881 dynamically procures computing and other resources used to execute tasks within the cloud computing environment; Metering and pricing 882 tracks costs as resources are used within the cloud computing environment and bills or invoices for the consumption of those resources. In one example, these resources may include application software licenses; Security verifies the identity of cloud users and tasks and protects data and other resources; User portal 883 provides users and system administrators with access to the cloud computing environment; Service level management 884 allocates and manages cloud computing resources to ensure required service levels are met; and Service Level Agreement (SLA) planning and fulfillment 885 pre-arranges and procures cloud computing resources for anticipated future demand in accordance with SLAs.

[0100] Workload tier 890 provides examples of functions that can use a cloud computing environment. Examples of workloads and functions that can be provided from this tier include mapping and navigation 891, software development and lifecycle management 892, virtual classroom instruction delivery 893, data analytics processing 894, transaction processing 895, and deduplication of data integrity checking between different file systems 896 by any of the techniques described and / or suggested herein.

[0101] The present invention may be a system, method, or computer program product, or combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to implement aspects of the present invention.

[0102] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick®, floppy disk, punch cards or mechanically encoded devices such as ridge structures in grooves on which instructions are recorded, and any suitable combination thereof. As used herein, computer-readable storage media should not be construed as being ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses through fiber optic cable), or electrical signals transmitted over wires.

[0103] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device over a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, fiber optic transmission cables, wireless transmission cables, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to a computer-readable storage medium within the respective computing / processing device for storage.

[0104] Computer-readable program instructions for carrying out the operations of the present invention may be source or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or procedural programming languages, such as object-oriented programming languages ​​such as Smalltalk®, C++, and the "C" programming language, or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer as a standalone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can be personalized to carry out aspects of the present invention by executing computer readable program instructions using state information of the computer readable program instructions.

[0105] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0106] These computer-readable program instructions may be supplied to a processor of a computer or other programmable data processing apparatus such that the instructions, executed by the processor of the computer or other programmable data processing apparatus, form means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams to produce a machine. These computer-readable program instructions may be stored on a computer-readable storage medium, capable of instructing a computer, programmable data processing apparatus, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium on which the instructions are stored comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0107] The computer-readable program instructions may be loaded into a computer, other programmable data processing apparatus, or other device to provide a computer-implemented process that performs a series of operational steps on the computer, other programmable apparatus, or other device, such that the instructions, which execute on the computer, other programmable apparatus, or other device, implement the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0108] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be performed as a single step, in parallel, substantially in parallel, partially or fully overlapped, or the blocks may even be performed in reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or that implements a combination of dedicated hardware and computer instructions.

[0109] Furthermore, systems according to various embodiments may include a processor and logic integrated with and / or executable by the processor, where the logic is configured to perform one or more of the process steps described herein. The processor may be of any configuration described herein, such as a discrete processor or processing circuitry including numerous components, such as processing hardware, memory, and I / O interfaces. By integrated, it is meant that the processor has logic embedded as hardware logic, such as an application-specific integrated circuit (ASIC), FPGA, etc. By executable by the processor, it is meant that the logic is hardware logic, firmware, software logic such as part of an operating system or part of an application program, or some combination of hardware and software logic that is accessible by the processor and configured to cause the processor to perform some function when executed by the processor. The software logic may be stored in any type of memory, local and / or remote, known in the art. Any processor known in the art may be used, such as a software processor module and / or a hardware processor such as an ASIC, FPGA, central processing unit (CPU), integrated circuit (IC), graphics processing unit (GPU), etc.

[0110] It will be apparent from the above description that the various features of the above systems and / or methods may be combined in any manner, thereby creating multiple combinations.

[0111] It will also be appreciated that embodiments of the present invention may be provided in the form of a service that is deployed on behalf of a customer to provide the service on demand.

[0112] The descriptions of various embodiments of the present invention have been presented for illustrative purposes, but are not intended to be exhaustive or to be limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terms used herein are chosen to best explain the principles, practical applications, or technical improvements of the embodiments to the art found in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method comprising: In a clustered file system, receiving a request from a formatted file system to perform a data integrity check on a portion of data; determining whether the request includes a file system type of the portion of data; in response to determining that the request includes a file system type of the portion of data, determining whether the clustered file system supports the data integrity checking for the file system type of the portion of data; determining whether the portion of data is currently available in response to determining that the clustered file system supports the data integrity checking for the file system type of the portion of data; responsive to determining that the portion of data is currently available, causing the data integrity check to be performed on the portion of data; and sending the results of performing the data integrity check to the formatted file system.

2. The computer-implemented method of claim 1 , wherein the data integrity check is a file system consistency check (FSCK) operation.

3. establishing a communication channel between a FSCK process in the formatted file system and a FSCK process in the clustered file system; the request is received from the formatted file system via the communication channel; causing the data integrity check to be performed on the portion of data; 3. The computer-implemented method of claim 2, comprising sending one or more instructions to a FSCK daemon associated with the FSCK process in the clustered file system to perform the data integrity check.

4. 4. The computer-implemented method of claim 3, wherein causing the data integrity check to be performed on the portion of data comprises sending one or more instructions to an FSCK daemon to perform the FSCK operation.

5. 3. The computer-implemented method of claim 2, wherein the results of performing the data integrity check sent to the formatted file system include a result selected from the group consisting of a FSCK timestamp, a status of the FSCK operation, and corrupted file details.

6. Determining whether the portion of data is currently available 2. The computer-implemented method of claim 1, further comprising determining whether the portion of data has been exported from the clustered file system and / or is currently being used as local storage by the formatted file system.

7. determining whether the request includes a file system type of the portion of data; determining whether the portion of data has a metadata tag corresponding to the portion of data; the metadata tag comprises information associated with the portion of data; 2. The computer-implemented method of claim 1, wherein the information is selected from the group consisting of file system type, input / output (I / O) activity, execution time, priority, and application container details.

8. A computer program product which, when executed by a processor, causes the computer program product to carry out a method according to any one of claims 1 to 7.

9. 1. A system comprising: a processor; and logic integrated with, executable by, or integrated with and executable by said processor, said logic comprising: receiving, by the processor, a request from a formatted file system in a clustered file system to perform a data integrity check on a portion of data; determining, by the processor, whether the request includes a file system type of the portion of data; determining, by the processor, whether the clustered file system supports the data integrity checking for the file system type of the portion of data in response to determining that the request includes the file system type of the portion of data; determining, by the processor, whether the portion of data is currently available in response to determining that the clustered file system supports the data integrity checking for the file system type of the portion of data; causing the processor to perform the data integrity check on the portion of data in response to determining that the portion of data is currently available; and sending, by the processor, results of performing the data integrity check to the formatted file system.

10. 10. The system of claim 9, wherein the data integrity check is a file system consistency check (FSCK) operation.

11. The logic is: configured by the processor to establish a communication channel between a FSCK process in the formatted file system and a FSCK process in the clustered file system; the request is received from the formatted file system via the communication channel; causing the data integrity check to be performed on the portion of data; 11. The system of claim 10, further comprising sending one or more instructions to a FSCK daemon associated with the FSCK process in the clustered file system to perform the data integrity check.

12. 12. The system of claim 11, wherein causing the data integrity check to be performed on the portion of data comprises sending one or more instructions to an FSCK daemon to perform the FSCK operation.

13. 11. The system of claim 10, wherein the results of performing the data integrity check sent to the formatted file system include a result selected from the group consisting of a FSCK timestamp, a status of the FSCK operation, and corrupted file details.

14. Determining whether the portion of data is currently available determining whether the portion of data has been exported from the clustered file system and / or is currently being used as local storage by the formatted file system; determining whether the request includes a file system type of the portion of data; determining whether the portion of data has a metadata tag corresponding to the portion of data; the metadata tag comprises information associated with the portion of data; 10. The system of claim 9, wherein the information is selected from the group consisting of file system type, input / output (I / O) activity, execution time, priority, and application container details.

Images