Storage allocation in a distributed segmented file system

Abstract

A file system (i) permits storage capacity to be added easily, (ii) can be expanded beyond a given unit, (iii) is easy to administer and manage, (iv) permits data sharing, and (v) is able to perform effectively with very large storage capacity and client loads. State information from a newly added unit is communicated (e.g., automatically and transparently) to central administration and management operations. Configuration and control information from such operations is communicated (e.g., automatically) back down to the newly added units, as well as existing units. In this way, a file system can span both local storage devices (like disk drives) and networked computational devices transparently to clients. Such state and configuration and control information can include globally managed segments as the building blocks of the file system, and a fixed mapping of globally unique file identifiers (e.g., Inode numbers) and / or ranges thereof, to such segments.

Description

I. CROSS-REFERENCES TO RELATED APPLICATIONS[0001]This is a continuation patent application which claims priority from U.S. patent application Ser. No. 10 / 425,550 (the '550 application) filed on Apr. 29, 2003 which is a continuation-in-part of and claims the benefit of U.S. application Ser. No. 09 / 950,555 filed Sep. 11, 2001, that claims the benefit of U.S. Provisional Application No. 60 / 232,102 filed Sep. 12, 2000. The '550 application also claims the benefit of U.S. Provisional Application No. 60 / 395,839 filed Jul. 15, 2002. The '550 application also claims the benefit of U.S. Provisional Application No. 60 / 447,720 filed Feb. 19, 2003. The '550 application also claims the benefit of U.S. Provisional Application No. 60 / 447,722 filed Feb. 19, 2003. The '550 application also claims the benefit of U.S. Provisional Application No. 60 / 447,723 filed Feb. 19, 2003. These applications are incorporated by reference in their entirety.II. BACKGROUND OF THE INVENTION[0002]A. Field of the Invent...

AI Technical Summary

Benefits of technology

The present invention provides methods, apparatus, and data structures for a distributed file system that can meet various needs. The invention allows for the efficient allocation of data across multiple file servers in a distributed-file system. It can determine which file server to forward a file system call based on the file identifier, physical storage media, and the location of the file server. The invention also allows for non-hierarchical storage policies to be followed, such as based on amounts of free storage space and relative amounts of free storage space and free data identifiers, as well as affinity groups. Overall, the invention improves the performance and reliability of distributed file systems.

Problems solved by technology

Unfortunately, however, SASs are only expandable internally—there is no transparent data sharing between file servers.

Further, with SASs, throughput is limited by the speed of the fixed number of busses internal to the file server.

The disadvantage of this technology is that a file system cannot expand beyond the limits of single NAS machine.

Consequently, administering and maintaining more than a few NAS units, and consequently more than a few file systems, becomes difficult.

Unfortunately, however, SANs are expensive.

Although SANs permit space to be shared among front-end server systems, they do not permit multiple SANs environments to use the same file system.

Also, various incompatible versions of SANs have emerged.

Maintaining and storing a map having every file could, however, limit scalability of the system as the number of files become large.

Images