42 results about "Global file system" patented technology

In a switched file system, a file switching device is logically positioned between clients and file servers and communicates with the clients and the file servers using standard network file protocols. The file switching device appears as a server to the client devices and as a client to the file servers. The file switching device aggregates storage from multiple file servers into a global filesystem and presents a global namespace to the client devices. The file switching device typically supports a “native” mode for integrating legacy files into the global namespace and an “extended” mode for actively managing files across one or more file servers. Typically, native-mode files may be accessed directly or indirectly via the file switching device, while extended-mode files may be accessed only through the file switching device. The file switching device may manage file storage using various types of rules, e.g., for managing multiple storage tiers or for applying different types of encoding schemes to files. Rules may be applied to pre-existing files.

A virtual file system and method. The system architecture enables a plurality of underlying file systems running on various file servers to be “virtualized” into one or more “virtual volumes” that appear as a local file system to clients that access the virtual volumes. The system also enables the storage spaces of the underlying file systems to be aggregated into a single virtual storage space, which can be dynamically scaled by adding or removing file servers without taking any of the file systems offline and in a manner transparent to the clients. This functionality is enabled through a software “virtualization” filter on the client that intercepts file system requests and a virtual file system driver on each file server. The system also provides for load balancing file accesses by distributing files across the various file servers in the system, through migration of data files between servers.

In a switched file system, a file switching device is logically positioned between clients and file servers and communicates with the clients and the file servers using standard network file protocols. The file switching device appears as a server to the client devices and as a client to the file servers. The file switching device aggregates storage from multiple file servers into a global filesystem and presents a global namespace to the client devices. The file switching device typically supports a “native” mode for integrating legacy files into the global namespace and an “extended” mode for actively managing files across one or more file servers. Typically, native-mode files may be accessed directly or indirectly via the file switching device, while extended-mode files may be accessed only through the file switching device. The file switching device may manage file storage using various types of rules, e.g., for managing multiple storage tiers or for applying different types of encoding schemes to files. Rules may be applied to pre-existing files.

The consistency callback mechanisms employed by local file systems such as NTFS and distributed file systems such as DDS, NFS and CIFS are extended to provide a shared memory foundation for efficiently broadcasting real-time high definition video from a source object to large numbers of viewers via the Internet. Distributed applications such as video viewing client applications establish connections to a common distributed file system object, and then each application registers with the underlying distributed file system to receive notifications whenever the video source modifies the source object. The data required to update images maintained by viewing clients is included in notification messages. The distributed file system employs a network of proxy cache nodes. Proxy cache nodes receive notification messages (complete with image update data) and update their cached images of the source object and then retransmit the notification messages towards the viewing clients using IP multicast techniques. In this manner, the distributed file system's consistency mechanism efficiently employs network resources to enable the real-time distribution of video content streams.

A content management system and method are disclosed having one or more shared file systems located on one or more networks having a firewall. The content management system includes an explorer client, a mounting client, and a gateway. The gateway is in communication with both the explorer client and the mounting client. The explorer client forms a profile of a user device by determining the characteristics of the user device. The mounting client determines the capabilities of the one or more file systems and where content resides on each of the one or more file systems. The gateway then creates a virtual abstraction layer based on the profile of the user device and the capabilities of the one or more file systems.

A shared storage distributed file system is presented that provides applications with transparent access to a storage area network (SAN) attached storage device. This is accomplished by providing clients read access to the devices over the SAN and by requiring most write activity to be serialized through a network attached storage (NAS) server. Both the clients and the NAS server are connected to the SAN-attached device over the SAN. Direct read access to the SAN attached device is provided through a local file system on the client. Write access is provided through a remote file system on the client that utilizes the NAS server. A supplemental read path is provided through the NAS server for those circumstances where the local file system is unable to provide valid data reads.Consistency is maintained by comparing modification times in the local and remote file systems. Since writes occur over the remote file systems, the consistency mechanism is capable of flushing data caches in the remote file system, and invalidating metadata and real-data caches in the local file system. It is possible to utilize unmodified local and remote file systems in the present invention, by layering over the local and remote file systems a new file system. This new file system need only be installed at each client, allowing the NAS server file systems to operate unmodified. Alternatively, the new file system can be combined with the local file system.

A mirror file systems (MFS) is a virtual file system that links two or more file systems together and mirrors between them in real time. The file systems linked and mirrored through the mirror file system can be a local file system connected to a physical device, or a network file system exported by a remote system on a network. The mirroring mechanism is established by linking a file system to another file system on a single directory through an MFS mounting protocol. User applications perform normal file system operation and file / directory operation system calls like open, read, write and close functions from the pathname of either file system. When updates occur, such as a write operation, the MFS mechanism ensures that all data updates go to both the file systems in real time.

Implicit journaling of a file operation relating to a file stored in a flash memory is performed by locking a semaphore corresponding to the file on which a file operation is to be performed, by initializing journaling of the file operation using the file map, by performing the file operation on the file, by completing journaling of the file operation using a file map corresponding to the file, and unlocking the semaphore. Additionally or alternatively, a file system is placed in a stable state following an interruption occurring during a file operation by scanning File Maps corresponding to the files, determining whether a file operation is incomplete based on validity flags contained in the file maps, and performing remediation so as to eliminate the incomplete file operation.

File system restoration is made more flexible through the use of indicators of individual file restoration status throughout the restoration process. Such indicators are particularly useful in the restoration of large file systems where, for example, the number of files may range upwards of several hundred million. Since file system restoration for large systems takes such a relatively long time, the present invention provides various degrees of individual file access even as the restoration process is being carried out. During file system restoration times, file access may be full or limited to a file's attributes. Most advantageously, the present invention is capable of providing full, dynamically driven (on-demand) file access even during file system restoration operations.

The invention prevents improper deletes of file system objects during rescan synchronizations of local and remote (cloud) file systems. File and folder deletes are validated (confirmed) during a rescan synchronization using events generated for a steady state synchronization process. Any improper deletes are fenced and not allowed to proceed during synchronization. A method for synchronizing local (LFS) and remote (RFS) file systems includes monitoring the LFS for changes, generating a first set of events as part of a first synchronization routine (e.g., a steady state synchronization routine), generating a second set of events as part of a second (rescan) synchronization routine, validating selected events of the second set with events of the first set to determine whether each of the selected events of the second set is valid or invalid, and synchronizing the file systems based on the events of the second set excluding those that have been invalidated. Storage systems are also disclosed.

Systems and methods are disclosed that track a document within a file system using document identifier (DOCID) that is preserved during safe save operations. The DOCID can be a universally unique identifier (UUID). The DOCID can be generated on a local file system or generated during a synchronization process with a remote synchronization service. Applications can opt in to document tracking. The kernel of the operating system can monitor file system operations on a per-application and per-thread basis and determine when a processing thread is about to perform a safe save. When a sequence of file system operations indicates that a safe save is to be performed, the kernel can generate and store a tombstone record that includes the DOCID and other metadata of the tracked document and pass the DOCID and metadata to the newly saved document.

In a switched file system, a file switching device is logically positioned between clients and file servers and communicates with the clients and the file servers using standard network file protocols. The file switching device appears as a server to the client devices and as a client to the file servers. The file switching device aggregates storage from multiple file servers into a global filesystem and presents a global namespace to the client devices. The file switching device typically supports a “native” mode for integrating legacy files into the global namespace and an “extended” mode for actively managing files across one or more file servers. Typically, native-mode files may be accessed directly or indirectly via the file switching device, while extended-mode files may be accessed only through the file switching device. The file switching device may manage file storage using various types of rules, e.g., for managing multiple storage tiers or for applying different types of encoding schemes to files. Rules may be applied to pre-existing files.

The invention provides a method for realizing a shared storage pool by utilizing a Linux global file system. A Linux global file system (GFS) is arranged in an SAN storage network, thereby realizing a small-scale cluster shared storage pool. According to the method, a shared storage pool for unified management, parallel reading and writing, data protection and elastic expansion can be provided for cluster nodes; SAN equipment-based cluster storage file system configures a physical storage raw device into a flexible logical storage volume group; storage space that is shared globally and is oriented to a node group is established on the logical volume group; a client mounts a GFS to a local end to realize parallel reading and writing operation on the file system; and a distributed log management system and a fault node isolation mechanism provide functions of data protection and disaster recovery for a shared storage pool.

A method for synchronizing a local file system (LFS) and a remote (cloud) file system (RFS) includes the step of storing events indicative of differences between the LFS and the RFS, where the events are associated with file system objects of the LFS and RFS. The method further includes the steps of prioritizing the events, generating file system operations for the events based at least in part on the prioritization of the events, and performing at least some of the file system operations to synchronize the LFS and RFS. Prioritizing the events includes defining a plurality of service classes (e.g., based on type of event, event attribute, etc.), assigning a priority to each of the service classes, and assigning each event to one of said service classes. Systems are also described. The invention allocates limited synchronization resources based on a predefined policy to synchronize important file system objects sooner.

The invention provides a method, a device and a system for sharing a storage pool by a cluster. The method comprises the following steps of: configuring a global file system (GFS) to be shared by the cluster based on a storage area network (SAN), and processing reading-writing operations of each client node in the cluster in parallel; and mounting a part of subareas in the global file system to a local server for the reading-writing operations according to needs by using each client node in the cluster. In the method, a network level high-efficiency data sharing way is provided, the parallel file reading-writing of a plurality of users is supported, and flexible online capacity expansion and data equilibrium distribution technologies are supported.

A method for synchronizing a local file system (LFS) and a remote (cloud) file system (RFS) includes the step of storing events indicative of differences between the LFS and the RFS, where the events are associated with file system objects of the LFS and RFS. The method further includes the steps of prioritizing the events, generating file system operations for the events based at least in part on the prioritization of the events, and performing at least some of the file system operations to synchronize the LFS and RFS. Prioritizing the events includes defining a plurality of service classes (e.g., based on type of event, event attribute, etc.), assigning a priority to each of the service classes, and assigning each event to one of said service classes. Systems are also described. The invention allocates limited synchronization resources based on a predefined policy to synchronize important file system objects sooner.

One aspect of the present invention provides an operation for recovering lost space and maintaining data consistency in a storage environment by providing a two-way orphan reconciliation method for extremely large file systems, such as a Hierarchical Storage Management (HSM) system which manages billions of files. This highly scalable and parallelizable orphan identification process may be used in a HSM controlled environment to enable the execution of a two-way file orphan check. In one embodiment, this orphan check identifies file system client orphans and server object orphans in a single pass by comparing a queue containing a list of migrated files on a storage repository server with a queue containing a list of stub files on a file system. If the queue elements do not match, a file system orphan or server object orphan can appropriately be identified.

Client node caching in a global file service. A client node receives a leaf image from a remote storage node storing a base image that is read-only accessible to the client node. The leaf image has a portion of the base image modified by the client node. The client node stores the leaf image in a cache on the client node. Unmodified portions of the base image are accessed from the remote storage node. Modified portions of the base image are accessed from the cache. The leaf images may comprise a branch modification log having portions of the base image modified by the client node and a metadata structure. The metadata structure has pointers to the modified data in the branch modification log and pointers to data unmodified by the client node in the base image. Alternatively, the leaf images may comprise a change volume for changes made to the base image by the client node.

The invention discloses an establishment method of a role-oriented virtual file system and belongs to the field of computers. The establishment method of the role-oriented virtual file system includes: establishing a corresponding role virtual file system configuration file for each role; giving a mapping relation among a virtual directory, a file and an overall situation file system in an operation system, a authority limit relation and an expanding access rule in the role virtual file system configuration file; dividing roles into basic roles and composite roles which comprising a plurality of the basic roles and the composite roles and removing a plurality of roles, and combining and removing role virtual file systems of the plurality of roles comprised in the composite roles to form role virtual file systems of the composite roles. The role-oriented establishment method of the virtual file system is capable of building any complex virtual file system in the overall situation file system of the operation system and provides a role-oriented individuation file system service for an application program in a mode of the role virtual file system.

In a file system recovery (FSRC) process, an FSRC module performs dynamic checks on file system objects to verify parent-child relationships among various objects in the system. The FSRC module stores a list of objects in the system and relevant object-related information for each object in a memory. A dynamic check on each listed object is then performed using the object-related information to verify parent-child relationships for that object. After the dynamic check is over, the object is removed from the list and its object-related information is deleted from the memory.

A cloud-native global file system, in which one or more filers are associated with a volume of a versioned files system in a private, public or hybrid cloud object store, is augmented to include a rapid ransomware recovery service. Upon detecting a ransomware attack associated with one or more files or directories of the volume, read and write access to the volume is restricted. A recovery filer is then activated or designated in the cloud. A restore operation is then initiated at the recovery filter. Following completion of the restore operation, a new clean (healthy) snapshot of the volume is then created using the recovery filer For any filer other than the recovery filer, a determination is made whether the filer has completed a merge operation with respect to the new clean snapshot. If so, read and write access to the volume is re-enabled from that filer.

Systems, methods and media for storing information in a common information store of a global file storage system are disclosed. Embodiments may include a common information store including a communications module to send and receive information to and from one or more servers in the global file storage system and a pattern database to store a plurality of subfiles for the global file storage system, where each subfile may include an indication of a recognizable pattern. The common information store may also include a common information store manager to analyze newly stored files and to reconstruct stored files of the global file storage system. In a further embodiment, the common information store manager may further comprise a pattern analyzer sub-module to analyze a file for recognizable patterns and to compare the recognizable patterns to the subfiles stored in the pattern database.

The invention discloses a building method of a virtual file system facing progresses, and belongs to the field of computers. The building method of the virtual file system facing the progresses includes that a progress virtual file system configuration files are built to store a mapping relation of a progress virtual file system and a global file system in an operation system; the progress virtual file system configuration files are appointed during a progress operation, a progress virtual file system configuration table of the progresses is built in internal storage by the operation system, and data of the progress virtual file system configuration files of the progresses are input; and the file operation of the progresses is converted into the operation of corresponding files of the global file system in the operation system through the mapping relation of the progress virtual file system configuration table. The building method effectively achieves logic or physics isolation of file systems among different progresses, therefore the damage degree of the global file system in the operation system by the progresses is reduced, and safety of the global file system is improved.

A method for synchronizing a file system (FS) and a remote file system (RFS) includes monitoring the FS for FS events, generating FS event records, receiving RFS event records of RFS events, generating file system operations (FSOs) based on the FS and RFS event records, and communicating the FSOs to the FS and RFS to synchronize them. A method for generating the FSOs includes accessing a plurality of FS and / or RFS event records, processing the accessed records to generate processed event records, generating the FSOs based on the processed event records, and outputting the FSOs to cause synchronization of the FS and RFS. Systems are also described. The invention facilitates event-based, steady-state synchronization of local and remote file systems.

One aspect of the present invention provides an operation for recovering lost space and maintaining data consistency in a storage environment by providing a two-way orphan reconciliation method for extremely large file systems, such as a Hierarchical Storage Management (HSM) system which manages billions of files. This highly scalable and parallelizable orphan identification process may be used in a HSM controlled environment to enable the execution of a two-way file orphan check. In one embodiment, this orphan check identifies file system client orphans and server object orphans in a single pass by comparing a queue containing a list of migrated files on a storage repository server with a queue containing a list of stub files on a file system. If the queue elements do not match, a file system orphan or server object orphan can appropriately be identified.

A cloud-native global file system in which a local filer creates objects and forward them to a cloud-based object store is augmented to include constant-time rekeying (CTR). At volume creation time on the filer, a random Intermediate Key (IK) is generated. The IK is encrypted using one or more public key(s) for the volume in question, and then stored in encrypted form in a volume metadata file (e.g., cloudvolume.xml) alongside the other volume information. Once created, the IK is treated like any other volume metadata. During startup of a volume manager on the filer, the one or more per-volume IK blobs (present) are decrypted using an appropriate secret key, and then cached in memory. All objects sent to the cloud are then symmetrically encrypted to the current IK for that volume. All objects read from the cloud are decrypted using the locally-cached IK.

A cloud-native global file system is augmented to include a file exporter (or, more generally, a file export tool) that facilitates an enterprise customer's use of a cloud-native tool that would otherwise be unable to operate against the global file system's underlying file system representation. In a typical use case, the file exporter is configured to extract in a native object format and to an unencrypted target (e.g., an S3 bucket, an Azure storage account, and the like) all or a portion of a volume's data from the underlying file system representation. In this manner, the exporter creates a copy of the data set that the enterprise user can then leverage against the desired cloud-native tool or other cloud services that are not under the management or control of the global file system service provider.

A cloud-native global file system used to provide primary file storage for enterprise data is augmented to provide “direct-to-cloud” (D2C) migration of an initial data set. In addition to providing a filer, the service provider configures a direct-to-cloud migration tool, typically as a downloadable virtual machine (VM), that is executed in association with the hardware of the enterprise filesystem whose data set is being migrated. The migration tool reads the existing filesystem, transforms the data into a format consistent with that done by a filer, and pushes the result (a single version) directly to an object store in the cloud. In a first pass (run), typically a large bulk of the data is moved and, depending on how the migration tool is configured, one or more additional passes may then be carried out. When migration is complete (or substantially complete), the filer then is activated to complete the migration (if need be) and take over the go-forward operations.