Flat file system for organizing and interconnecting large datasets

Abstract

This disclosure relates to a flat file system for organizing and interrelated large datasets. In some embodiments, a computer-implemented method includes: receiving a flat file system configuration request from a user interface at a flat file system manager executing on one or more processors, the request including multiple planes and multiple plane relationships, wherein the multiple plane relationships include subplane relationships between planes and subplanes and orthogonal relationships between two independent planes among the multiple planes; configuring a storage environment to include the flat file system based on the flat file system configuration request by the flat file system manager; receiving a plane operation request from the user interface for performing an operation on a plane of the flat file system, wherein the plane operation request includes a plane identifier corresponding to the plane; and performing the operation on the plane based on the plane operation request.

Description

Technical Field

[0001] This disclosure relates in general to a flat file system for organizing and interconnecting large datasets. Background Technology

[0002] As the volume of data used for various scenarios of storing data for manipulation, analysis, or other purposes grows exponentially over time, organizing data can become increasingly difficult. For example, within the context of the high-performance computing (HPC) storage ecosystem—including, but not limited to, parallel file servers and file systems that generate large amounts of heterogeneous data—technologies for organizing large numbers of heterogeneous data files often require significant management overhead and may not provide robust techniques for creating logical relationships between data. Summary of the Invention

[0003] According to one aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to: receive, at a plane file system manager executing on the one or more processors, a plane file system configuration request from a user interface, including a plurality of planes, a plurality of subplanes of the plurality of planes, and a plurality of plane relationships, wherein the plurality of planes are a logical storage configuration for storing file groups, wherein: the plurality of plane relationships include a subplane relationship between a plane and a first subplane, a first orthogonal relationship between two independent planes among the plurality of planes, and a second orthogonal relationship between a second subplane and a third subplane among the plurality of subplanes, and the second subplane and the third subplane are different A subplane of a plane; the plane file system manager configures a storage environment to include a plane file system based on a plane file system configuration request, wherein configuring the storage environment to include the plane file system includes formatting a plurality of storage devices to include the plurality of planes and the plurality of subplanes; receiving from the user interface a plane operation request for performing an operation on a first plane among the plurality of planes of the plane file system, wherein the plane operation request includes a plane identifier corresponding to the plane; performing the operation on the first plane based on the plane operation request to obtain an initial analysis result; and performing an additional plane operation on a second plane among the plurality of planes of the plane file system based on the initial analysis result, the second plane being associated with the first plane through the first orthogonality relation, wherein the additional plane operation is performed on the second plane at least in part based on the first orthogonality relation.

[0004] According to another aspect of this disclosure, a computer-implemented method is provided, comprising: receiving, at a plane file system manager executing on one or more processors, a plane file system configuration request from a user interface, comprising a plurality of planes, a plurality of subplanes of the plurality of planes, and a plurality of plane relationships, wherein the plurality of planes are a logical storage configuration for storing file groups, wherein: the plurality of plane relationships include a subplane relationship between a plane and a first subplane, a first orthogonal relationship between two independent planes among the plurality of planes, and a second orthogonal relationship between a second subplane and a third subplane among the plurality of subplanes, and the second subplane and the third subplane are subplanes of different planes; configuring a storage ring by the plane file system manager based on the plane file system configuration request. The storage environment may include a planar file system, wherein configuring the storage environment to include the planar file system includes formatting a plurality of storage devices to include the plurality of planes and the plurality of sub-planes; receiving from the user interface a plane operation request for performing an operation on a first plane among the plurality of planes of the planar file system, wherein the plane operation request includes a plane identifier corresponding to the plane; performing the operation on the first plane based on the plane operation request to obtain an initial analysis result; and performing an additional plane operation on a second plane among the plurality of planes of the planar file system based on the initial analysis result, the second plane being associated with the first plane through a first orthogonality relation, wherein the additional plane operation is performed on the second plane at least in part based on the first orthogonality relation.

[0005] According to another aspect of this disclosure, a non-transitory computer-readable medium is provided, the non-transitory computer-readable medium storing a programming program for execution by one or more processors, the programming program including instructions for performing the following operations: receiving, at a plane file system manager executing on the one or more processors, a plane file system configuration request from a user interface, comprising a plurality of planes, a plurality of subplanes of the plurality of planes, and a plurality of plane relationships, wherein the plurality of planes are a logical storage configuration for storing file groups, wherein: the plurality of plane relationships include a subplane relationship between a plane and a first subplane, a first orthogonal relationship between two independent planes among the plurality of planes, and a second orthogonal relationship between a second subplane and a third subplane among the plurality of subplanes, and the second subplane and the third subplane are subplanes of different planes; by the plane A plane file system manager configures a storage environment to include a plane file system based on the plane file system configuration request, wherein configuring the storage environment to include the plane file system includes formatting multiple storage devices to include the multiple planes and the multiple sub-planes; receives from the user interface a plane operation request for performing an operation on a first plane among the multiple planes of the plane file system, wherein the plane operation request includes a plane identifier corresponding to the plane; performs the operation on the first plane based on the plane operation request to obtain an initial analysis result; and performs an additional plane operation on a second plane among the multiple planes of the plane file system based on the initial analysis result, the second plane being associated with the first plane through the first orthogonality relation, wherein the additional plane operation is performed on the second plane at least in part based on the first orthogonality relation. Attached Figure Description

[0006] Some embodiments discussed herein will be described with reference to the accompanying drawings listed below. However, the drawings illustrate only certain aspects or implementations of the embodiments described herein by way of example and are not intended to limit the scope of the claims.

[0007] Figure 1 The illustration shows a block diagram of an example system for implementing a flat file system for organizing and relating large datasets, according to one or more embodiments of the present disclosure.

[0008] Figure 2 The illustration provides an overview of example methods for implementing a flat file system according to one or more embodiments disclosed herein;

[0009] Figure 3 An example flat file system 300 according to one or more embodiments of the present disclosure is illustrated;

[0010] Figure 4An example flat file system 400 including example files is illustrated according to one or more embodiments of the present disclosure;

[0011] Figure 5A An example flat file system lookup table 500 according to one or more embodiments disclosed herein is shown;

[0012] Figure 5B An example of finding a directed acyclic graph in a planar file system according to one or more embodiments disclosed herein is shown;

[0013] Figure 6 An example planar file system 600, including orthogonal relationships between planes, is illustrated according to one or more embodiments of the present disclosure; and

[0014] Figure 7 A block diagram of a computing device according to one or more embodiments of the present disclosure is illustrated. Detailed Implementation

[0015] As storage, memory, computing, and network input / output (I / O) activities corresponding to high-performance computing applications become increasingly complex and large-scale, the amount of data generated, consumed, and / or otherwise used in such scenarios is growing exponentially. This data may grow in both the number of files containing the data and / or the size of such files. Such data can include vast amounts (e.g., billions, trillions, etc.) of heterogeneous files that can be interconnected in any diverse and complex way.

[0016] Examples of scenarios where such a large volume of files can be used include, but are not limited to: artificial intelligence, machine learning, autonomous vehicles, space exploration, the search for extraterrestrial life, genetic engineering, DNA modeling, computational biology, gene modeling, environmental modeling, and subatomic-level experiments. This vast amount of data can be accessed by any number and type of computing devices to perform various types of analysis and / or any other operations.

[0017] Data is typically stored in some form of hierarchical file system (HFS). Such file systems usually include constructs and concepts such as global namespaces, local namespaces, file hierarchies, and structures. At other times, object-based storage schemes can be used to store data, where data is managed as discrete objects, or block storage can be used, where data is managed as blocks within other data constructs (e.g., sectors, tracks, etc.). However, such concepts can make it difficult to operate on large amounts of related data simultaneously, or to organize data into relationships according to file system specifications.

[0018] To address at least the aforementioned problems concerning the organization of large amounts of data, the embodiments disclosed herein provide techniques for grouping data into plane groups stored in a plane and using relationships between planes to define various relationships between the data.

[0019] In one or more embodiments, as used herein, a plane refers to a logical storage configuration used to store related file groups. In one or more embodiments, a user can interact with a plane file system manager to define any number of planes for any number of plane file groups. In one or more embodiments, each plane file group is designed to store a related set of files.

[0020] In one or more embodiments, the user can also define any number of plane relationships between planes. In one or more embodiments, a plane can have any number of subplanes. In one or more embodiments, a subplane is intended to store a subset of related files or files with subplane relationships stored in a plane's file group, meaning that these files are related to files in the parent plane, but not necessarily to files in other planes or subplanes. In one or more embodiments, any number of subplanes can be configured in a plane. Additionally, a subplane can also have further subplanes that include a portion of the files of the subplane, wherein any number of nesting levels of subplanes are possible.

[0021] In one or more embodiments, subplanes within a plane can be discrete from one another. Alternatively, subplanes within a plane can be orthogonally correlated with each other, which may mean that there is a logical relationship between the data in the subplanes, but files in one subplane are not a subset of files in another subplane. Additionally, planes (e.g., not subplanes) can also be similarly orthogonally correlated.

[0022] In one or more embodiments, the user may also define the size of the plane, subplanes, etc., the maximum size of a single file that may exist within the plane, and / or define any other relevant parameters related to the plane's storage space, file size, etc. In one or more embodiments, such configuration details are provided to the plane file system manager.

[0023] In one or more embodiments, once the user has provided configuration details of the flat file system (including various planes, subplanes, and relationships), the flat file system manager can prepare any number of storage devices of any type to implement the user-defined flat file system. In one or more embodiments, preparing storage may include generating a table that includes each of the planes, subplanes, and plane relationships, as well as the location of the planes within one or more storage devices. In one or more embodiments, preparing storage may further include formatting any number of storage devices using various planes of various sizes specified by the user.

[0024] In one or more embodiments, the underlying storage of the flat file system is a parallel file system. In one or more embodiments, a parallel file system is a file system in which file data is striped across any number of homogeneous or heterogeneous storage devices and can be accessed concurrently by any number of entities (e.g., computing devices). In one or more embodiments, when the flat file system manager uses a parallel file system as the underlying storage mechanism for the flat file system, a user-defined flat can be striped across various storage devices in a similar manner, while also gaining the benefits provided by the parallel file system. Any other type of underlying storage device and solution can be used without departing from the scope of the embodiments disclosed herein.

[0025] In one or more embodiments, a plane, subplane, etc., as described above, may include data that is related to other data in some way. In one or more embodiments, a plane, subplane, etc., may operate as a whole or on a per-file basis. As an example, a user can configure any number of parameters (e.g., security, access control, encryption requirements, etc.) across the entire plane. Such permissions can flow down to subplanes. Additionally, subplanes may have defined additional parameters. As an example, a plane may be configured to have access permissions for a group of users but without requiring encryption, while subplanes within the plane may have the same access permissions (as part of the plane) but with encryption requirements.

[0026] Files in a plane and its subplanes can be identified by a file handle (e.g., a filename) and a plane identifier (ID). As an example, a plane ID can be any set of alphanumeric characters unique to that plane. Thus, a plane can have a plane ID of PL-A, another plane can have a plane ID of PL-B, and so on. In one or more embodiments, the plane ID represents the relationship between a plane and its subplanes. As an example, a plane PL-A can have a subplane PL-A1, which in turn can have a sub-subplane PL-A1A.

[0027] In one or more embodiments, a plane ID can be used to perform operations on the entire plane, and the relationships between planes can be used to perform additional operations. As an example, a user can provide a plane ID as an identifier for a set of documents that will be analyzed for any purpose (e.g., to determine whether a specific region of the universe has a planetary system including stars with specific characteristics), and if the analysis produces a specific result (e.g., the star has the desired characteristics), the analysis can continue to an orthogonal correlation plane that includes documents related to atmospheric conditions on the planets of that planetary system, which will be used to determine whether any planet in the planetary system has an atmosphere conducive to supporting life.

[0028] Some embodiments of this disclosure allow users to define any number of planes and plane relationships. Planes can be used to store large, related datasets, and the relationships between planes can be used to define relationships between data. Planes can be organized into a plane file system, which can be implemented on the underlying storage technology. The plane file system can be accessed by any number of related entities. The plane file system can simplify the storage and grouping of large datasets.

[0029] Figure 1 A block diagram of an example system for implementing a flat file system for organizing and interconnecting large datasets, according to one or more embodiments of the present disclosure, is illustrated. The system may include any number of computing devices, such as computing device A 100, computing device B 102, computing device C 104, computing device D 106, computing device E 108, and computing device N 110. The computing devices may be operatively connected to a flat file system manager 112. The flat file system manager 112 may be operatively connected to any number of storage devices (e.g., storage device A 114, storage device N 116). The storage devices (114, 116) may optionally be part of a parallel file system 118. Each of these components is described below.

[0030] In one or more embodiments, as used herein, a computing device (e.g., 100, 102, 104, 106, 108, 110) can be any single computing device, a group of computing devices, a portion of one or more computing devices, or any other physical, virtual, and / or logical grouping of computing resources. In one or more embodiments, a computing device is any device, a portion of a device, or any collection of devices capable of electronically processing instructions, and may include, but is not limited to, any of the following: one or more processors (e.g., components including circuitry) (not shown), memory (e.g., random access memory (RAM)) (not shown), one or more input and output devices (not shown), non-volatile storage hardware (e.g., solid-state drives (SSDs), hard disk drives (HDDs) (not shown)), one or more physical interfaces (e.g., network ports, storage ports) (not shown), any number of other hardware components (not shown), and / or any combination thereof.

[0031] Examples of computing devices include, but are not limited to, servers (e.g., blade servers in a blade server chassis, rack servers in a rack, etc.), desktop computers, mobile devices (e.g., laptops, smartphones, personal digital assistants, tablets, automotive computing systems, and / or any other mobile computing devices), storage devices (e.g., disk drive arrays, Fibre Channel storage devices, Internet Small Computer System Interface (iSCSI) storage devices, tape storage devices, flash storage arrays, network-attached storage devices, etc.), network devices (e.g., switches, routers, multilayer switches, etc.), virtual machines, virtualized computing environments, logical containers (e.g., for one or more applications), Internet of Things (IoT) devices, node arrays of computing resources, supercomputing devices, data centers or any part thereof, and / or any other type of computing device having the foregoing requirements. In one or more embodiments, any or all of the foregoing examples may be combined to create a system of such devices, or may be divided into separate logical devices, which may be collectively referred to as computing devices. Other types of computing devices may be used (e.g., Figure 7 The system may include any number and / or type of such computing devices in any arrangement and / or configuration without departing from the scope of the embodiments disclosed herein.

[0032] In one or more embodiments, the storage device (not shown) and / or memory (not shown) of the computing device or computing device system may be and / or include one or more data repositories for storing any number of data structures (which store any number of data, e.g., information)). In one or more embodiments, the data repository is any type of storage unit and / or device for storing data (e.g., file system, database, table collection, RAM, and / or any other storage mechanism or medium). Further, the data repository may include multiple different storage units and / or devices. These multiple different storage units and / or devices may or may not be of the same type or may or may not be located in the same physical location.

[0033] In one or more embodiments, any storage device (not shown) and / or memory (not shown) of a computing device or computing device system may be regarded, in whole or in part, as a non-transitory computer-readable medium storing software and / or firmware.

[0034] Such software and / or firmware may include instructions that, when executed by one or more processors (not shown) and / or other hardware (e.g., circuitry) of a computing device and / or a computing device system, cause one or more processors and / or other hardware components to perform operations according to one or more embodiments described herein.

[0035] Software instructions may be in the form of computer-readable program code for performing the methods, processes, etc. of the embodiments described herein, and by way of example, may be stored, in whole or in part, temporarily or permanently, on a non-transitory computer-readable medium such as an optical disc (CD), a digital multifunction disc (DVD), a storage device, a magnetic disk, a magnetic tape storage device, a flash storage device, a physical memory, or any other non-transitory computer-readable medium.

[0036] although Figure 1 Six computing devices are shown, but those skilled in the art who benefit from the specific implementation will understand that the system can include any number of computing devices, and the three dots between computing device E 108 and computing device N 110 are intended to convey this meaning. All or any part of the computing devices can be the same type or different types of computing devices.

[0037] In one or more embodiments, computing devices (e.g., 100, 102, 104, 106, 108, 110) can be configured to execute any number of processes (not shown). As used herein, a process is any set of operations performed by executing instructions through the processor of a computing device. A process can typically execute one or more threads, tasks, etc., to achieve a defined result. A process can be all or part of an application. An application can be software designed to perform any function. Examples of applications include, but are not limited to, high-performance computing applications, biological gene modeling applications, real-world simulations, astronomical data sampling, autonomous vehicle control, financial modeling, predictive analytics, machine learning / artificial intelligence applications, etc. A process can be part of a distributed application. In one or more embodiments, a distributed application is an application in which processes can execute on separate computing devices and / or separate processors. As an example, a distributed application may include processes that execute on separate computing devices in different physical locations (e.g., in separate data centers). As another example, a distributed application may execute processes on a separate processor, which may be part of the same computing device and / or a separate computing device. In one or more embodiments, processes execute to generate or otherwise obtain one or more results. In some cases, the results obtained by a process may be intermediate results. In one or more embodiments, an intermediate result is any result obtained by one process that will be used by another process. As an example, a process of a distributed application executing on one computing device may execute to obtain an intermediate result, and this intermediate result may be provided to another process of a distributed application executing on another computing device for further execution of the application.

[0038] In one or more embodiments, Figure 1Any one or more computing devices shown (e.g., 100, 102, 104, 106, 108, 110) can be configured to perform various operations, analyses, objectives, etc. In one or more embodiments, any one or more computing devices (e.g., 100, 102, 104, 106, 108, 110) can be configured to perform such analyses, operations, etc., using large amounts of data (e.g., terabytes, petabytes, exabytes, zettabytes, etc.) or otherwise based on such large amounts of data. Such large amounts of data can exist as corresponding large numbers of files (e.g., tens of thousands of gigabytes or more of audio / video files and / or several other heterogeneous files of different types with different sizes). Examples of scenarios in which computing devices (e.g., 100, 102, 104, 106, 108, 110) can be configured to perform operations using such large numbers of files include, but are not limited to: artificial intelligence, machine learning, autonomous vehicles, space exploration, search for extraterrestrial life, genetic engineering, DNA modeling, computational biology, gene modeling, environmental modeling, subatomic experiments, weather modeling, etc.

[0039] In one or more embodiments, computing devices (e.g., 100, 102, 104, 106, 108, 110) may be operatively connected to a flat file system manager 112 for storing and accessing large datasets. In one or more embodiments, the flat file system manager is any type of computing device (discussed above). In one or more embodiments, the flat file system manager is configured to implement and manage one or more flat file systems. In one or more embodiments, a flat file system is a technique for organizing files by grouping data into related file groups stored in a plane and defining various relationships between the data using relationships between planes.

[0040] In one or more embodiments, as used herein, a plane refers to a logical storage configuration for storing related groups of files. In one or more embodiments, a user can interact with the plane file system manager 112 to define any number of planes for any number of groups of files. In one or more embodiments, each plane is intended to store a related set of files. In one or more embodiments, a user can interact with the plane file system manager 112 using any suitable technology for interacting with computing devices, such as via one or more computing devices (e.g., 100, 102, 104, 106, 108, 110), by accessing a user interface via a network, etc.

[0041] In one or more embodiments, the user can also interact with the plane file system manager 112 to define any number of plane relationships between planes. In one or more embodiments, planes can have nested relationships, can be discrete subplanes, and / or orthogonally related to other planes.

[0042] In one or more embodiments, a nested plane relationship is a relationship when a plane has subplanes. In one or more embodiments, a plane can have any number of subplanes. In one or more embodiments, a subplane is intended to store a subset of related files of a plane file group stored in the plane, the related files having some additional level of interdependence relative to the parent plane of the subplane. In one or more embodiments, any number of subplanes can be configured in a plane. Additionally, a subplane can also have further subplanes that include a portion of the files of the subplane, thereby creating a nested relationship of any number of planes.

[0043] In one or more embodiments, subplanes can be discrete from one another, meaning that they are related, but not nested, as long as they are subplanes of the same parent plane.

[0044] In one or more embodiments, planes or subplanes may be orthogonally correlated with each other, which may mean that the files of a plane or subplane are logically correlated in some way, but the files of one orthogonally correlated plane or subplane are not a subset of the files of another orthogonally correlated plane or subplane.

[0045] In one or more embodiments, in addition to defining the number of planes in the flat file system and the relationships between them, users can also define the size of planes, subplanes, etc., the maximum size of a single file that may exist within a plane, and / or define any other relevant parameters related to the plane's storage space, file size, etc. Users can also define plane locks to facilitate concurrent access, various provisions for plane privileges in data structures, plane security attributes, etc.

[0046] In one or more embodiments, once the user has provided configuration details of the flat file system (including various planes, subplanes, and relationships), the flat file system manager 112 can prepare any number of storage devices of any type to implement the user-defined flat file system. In one or more embodiments, preparing the storage may include generating a data structure that includes each of the planes, subplanes, and plane relationships, as well as the location of the planes within one or more storage devices. In one or more embodiments, preparing the storage may further include formatting any number of storage devices using various planes of various sizes specified by the user.

[0047] In one or more embodiments, the flat file system manager can prepare any number of data structures for performing file or filegroup lookups on the flat file system. In one or more embodiments, file or filegroup lookups can be performed at least in part based on a plane identifier (ID) and / or file handle. Files in a plane and subplanes can be identified by a file handle (e.g., a filename) and a plane ID. As an example, a plane ID can be any set of alphanumeric characters unique to that plane. Thus, a plane can have a plane ID of PL-A, another plane can have a plane ID of PL-B, and so on. In one or more embodiments, a plane ID represents a relationship between a plane and a subplane. As an example, a plane PL-A can have a subplane PL-A1, which can have a subsubplane PL-A1A, and PL-A can be orthogonally correlated with a plane having a plane ID of PL-B. In one or more embodiments, in the context of discrete planes and / or orthogonally correlated planes, a plane namespace defined by a plane identifier can facilitate the use of the same file handles in different planes. In one or more embodiments, data structures that can be used to perform data lookups in the flat file system include, but are not limited to, tables (e.g., such as...). Figure 5A As shown), Directed Acyclic Graph (DAG) (e.g., as shown) Figure 5B As shown in the diagram), nested planar indices based on sparse matrices, etc.

[0048] In one or more embodiments, planes, subplanes, etc., may include associated data as described above. In one or more embodiments, planes, subplanes, etc., may operate as a whole (e.g., using a plane ID) or on a per-file basis (e.g., using a plane ID and a file handle). As an example, a user can configure any number of parameters (e.g., security, access control, encryption requirements, etc.) for the entire plane. Such permissions can flow down to subplanes. Additionally, subplanes may have defined additional parameters. As an example, a plane may be configured to have access permissions for a group of users but without requiring encryption, while subplanes within the plane may have the same access permissions (as part of the plane) but with additional encryption requirements.

[0049] In one or more embodiments, a plane ID can be used to perform operations on the entire plane, and the relationships between planes can be used to perform additional operations. As an example, a user can provide a plane ID as an identifier for a set of documents that will be analyzed for any purpose (e.g., determining whether a specific region of the universe has a planetary system including stars with specific characteristics), and if the analysis produces a specific result (e.g., the star has the desired characteristics), the analysis can continue to an orthogonal correlation plane that includes documents related to atmospheric conditions on the planets of that planetary system, which will be used to determine whether any planet in the system has an atmosphere conducive to supporting life.

[0050] In one or more embodiments, the flat file system manager 112 can be configured with any number of flat file systems, and each such flat file system can be configured for any number of computing devices (e.g., 100, 102, 104, 106, 108, 110) and / or otherwise accessible by any number of computing devices. As an example, a flat file system storing a specific type of data can be accessed by any number of independent computing devices that can use the data in different ways to perform different types of analysis, etc. In one or more embodiments, the plane and / or the files stored therein can be manipulated by a user of the computing device providing the flat file system manager 112 with commands including the plane ID and / or file handle of the files in the flat file system.

[0051] In one or more embodiments, the flat file system manager 112 configures the flat file system using at least part of configuration details received from one or more users, using any type of storage technology, paradigm, etc., and using any number of any type of storage devices (e.g., storage device A 114, storage device N 116). Although Figure 1 Two storage devices (114, 116) are shown, but those skilled in the art will understand from the specific implementation that the system can include any number of storage devices, and the three dots between storage device A 114 and storage device N 116 are intended to convey this meaning. All or any part of the storage devices can be storage devices of the same or different types.

[0052] In one or more embodiments, the underlying storage of the planar file system may optionally be a parallel file system 118, such as... Figure 1The parallel file system 118 is shown as a dashed line. In one or more embodiments, the parallel file system 118 is a file system in which file data is striped across any number of homogeneous or heterogeneous storage devices and can be accessed concurrently by any number of entities (e.g., computing devices). In one or more embodiments, when the flat file system manager 112 uses the parallel file system 118 as the underlying storage mechanism for the flat file system, a user-defined plane can be striped across various storage devices in a similar manner, while also gaining the benefits provided by the parallel file system. The flat file system manager 112 may include and / or be operatively connected to the parallel file system manager (not shown) and combine with the parallel system file manager operations to implement the flat file system on the parallel file system. Any other type of storage solution (e.g., network file system, distributed file system, etc.) may be used without departing from the scope of the embodiments disclosed herein. The flat file system can be implemented as a logical construct regardless of the underlying storage technology used to store the files of the flat file system. Alternatively or additionally, the flat file system can be used as part of a technique for formatting storage devices (e.g., flash storage devices), wherein the organization of the format is at least partially based on user-defined planes and plane relationships, which may be referred to as relationally configurable plane formatting. In one or more embodiments, the relationships in a planar format can be reconfigured at any time to modify the relationships of the datasets in the context of such a format, for example, when the amount of data stored is in the petabyte range or similar and / or when storing HPC applications and / or large datasets.

[0053] Although Figure 1 A specific configuration of the component is shown, but other configurations can be used without departing from the scope of the embodiments described herein. As an example, although... Figure 1 While some components may be shown as part of the same device, any component can be grouped into a collection of one or more components that can exist and perform as part of any number of independent and operatively connected devices. As another example, a single component can be configured to perform actions by… Figure 1 All or any part of the components shown perform all or any part of the functions. Therefore, the embodiments disclosed herein should not be limited to... Figure 1 The component configuration shown.

[0054] Figure 2 The illustration provides an overview of an example method for implementing a flat file system according to one or more embodiments disclosed herein. This method can be at least partially comprised of a flat file system manager (e.g., Figure 1 The planar file system manager 112 shown is used to execute this.

[0055] Although Figure 2 The steps in the flowchart shown are presented and described sequentially, but some or all steps may be performed in a different order, some or all steps may be combined or omitted, and some or all steps may be combined with... Figure 2 The other steps are executed in parallel.

[0056] In step 200, the method includes receiving a plane file system configuration request that includes multiple planes and multiple plane relationships. As an example, the plane file system configuration request can be received from a plane file system manager (e.g., Figure 1 The request is received at the flat file system manager 112. In one or more embodiments, the flat file system configuration request is received from a user attempting to configure a flat file system for storing a large number of files. As an example, an administrator working on a high-performance computing application might seek to configure a flat file system to store data to be analyzed by the high-performance computing application.

[0057] A plane file system configuration request may include requests for any number of planes (e.g., dozens, hundreds, etc.). In one or more embodiments, the plane file system configuration request includes a definition of relationships between the planes to be configured. As an example, a plane file system configuration request may specify that a portion of a plane is related by any number of nested plane relationships, that a plane has any number of subplanes, that some planes are orthogonally related to each other, etc. The plane file system configuration request may specify any other relevant information for configuring the storage file system without departing from the scope of the embodiments disclosed herein. Examples of other relevant information include, but are not limited to, the size of the plane and subplanes, the maximum and / or minimum file size allowed in the plane, the security parameters of the plane, the access permissions of the plane, the encryption requirements of the plane, the data retention policy of the plane, etc.

[0058] Flat file system configuration requests can be received via any technology through which the user interacts with the flat system file manager. As an example, the user can access the flat system file manager's user interface via a network connection.

[0059] In step 202, the method includes preparing a storage environment to include a flat file system based on a flat file system configuration request. In one or more embodiments, a flat file system manager (e.g., Figure 1A flat file system manager 112 can prepare a storage environment to include a flat file system. In one or more embodiments, preparing the storage environment includes formatting (this may be referred to as a plane format or format) any number of storage devices to be configured as a storage plane and any files to be ultimately stored in the plane. In one or more embodiments, preparing the storage environment includes instantiating any number of data structures for mapping the organization of the flat file system and / or for performing lookups of files, planes, subplanes, etc., included in the flat file system. In one or more embodiments, preparing the storage environment includes using or otherwise interacting with the underlying storage technology on which the flat file system is to be deployed.

[0060] As an example, a flat file system can be implemented using parallel file system technology, and a flat file system manager can interact with a parallel file system management entity to implement a flat file system on the underlying parallel file system by, for example, striping the plane across any number of homogeneous and / or heterogeneous storage devices. Similarly, plane deduplication, plane redundancy, plane index metadata management, etc., related to storage backup, recovery operations, logical volumes, etc., can be performed in the context of the plane and subplanes of the flat file system. Therefore, a plane implemented on a parallel file system can be accessed by any number of computing devices that may need to repeatedly access data to perform various operations and analyses, such as climate modeling (e.g., for weather science research), atomic / subatomic level models, AI-enabled models in the financial sector, cybersecurity, scientific computing, etc.

[0061] As another example, a flat file system can be implemented using physical volumes, logical volumes, and volume groups. In one or more embodiments, a volume group is a collection of one or more physical devices, each of which is referred to as a physical volume. In one or more embodiments, a logical volume is a virtual block device that can be used by the system or by an application. In one or more embodiments, relational flat / sub-flat logical volumes can be created by mounting a single-plane flat file system in each logical volume within a volume group. For example, logical volumes can be created from physical disk volumes (e.g., hdisk1, 2, 3). File systems (e.g., jfs, ext3, jfs2) can be mounted separately in such logical volumes. Alternatively, as an example, a flat file system can be mounted in a set of three logical volumes (e.g., 1, 2, and 3), and dataset-oriented relationships (e.g., orthogonal or logical superset: subset) can be established between the logical volumes. In one or more embodiments, relational flat logical volume groups can be created when using flat formatting based on a flat file system within a logical volume group. In one or more embodiments, a logical volume manager maintains a plane ID and correspondence between logical volumes. In one or more embodiments, when such a logical volume has other file systems (such as jfs, ext3, etc.), it may be impossible to establish relationships between different logical volumes (and the data included in different logical volumes) through file system technology.

[0062] In one or more embodiments, preparing the storage environment may include performing a plane mount. In one or more embodiments, a plane mount includes mounting the entire plane file system or any portion thereof (e.g., only a subset of the plane, subplanes, etc. of the plane file system). Typically, for non-plane file systems, when mounting a storage device that includes a file system (e.g., a USB flash drive with a New Technology File System (NTFS)), the entire file system is mounted at a mount point in the operating system. For example, in a Linux operating system, such a file system can be mounted using the mount command "mount / dev / sda1 / mnt / media1", which mounts the file system on the USB flash drive to the / mnt / media1 directory. Thus, by accessing / mnt / media1, a user can access all directories and files in the file system of the USB flash drive. In one or more embodiments, if the storage device (e.g., a USB flash drive) is alternatively formatted to include a plane file system that may have multiple plane formats, differential plane mounts can be performed according to the plane file system specification. Differential plane mounts can be used when only a portion of the file system is mounted. As an example, a command for mounting only a portion of a plane file system could be as follows: mount / sda2 / mnt / media1 {<optional planar IDs> In this example, if the storage device indicated by / dev / sda2 includes a flat file system, the user may have the option to mount the entire flat file system or only a subset of its planes. Therefore, in one or more embodiments, different planes of a flat system can be treated differently for mounting purposes. As an example, plane security can be implemented such that some planes can be mounted only using password or multi-factor authentication, while other planes can be designated as default planes that can be mounted without any security.

[0063] In step 204, the method includes receiving a plane operation request for performing an operation on a specific plane of the plane file system. In one or more embodiments, the plane operation request includes a plane ID. In one or more embodiments, the plane operation request is generated by a plane file system manager (e.g., Figure 1 The planar file system manager 112 receives the request. A planar operation request can be a request to perform any type of operation or operation on any one or more planes or subplanes of the planar file system. Examples include, but are not limited to: accessing one or more files stored in a plane or subplane (e.g., for performing data analysis on the files); adding, modifying, deleting, etc., plane attributes, characteristics, configuration parameters, etc. (e.g., access permissions, security parameters, retention policies, encryption policies, backup policies, etc.); adding, deleting, or modifying planes; adding, deleting, or modifying plane relationships, etc.

[0064] In step 206, the method includes performing a requested operation on one or more specific planes of the flat file system based on a plane operation request. As an example, a flat file system manager (e.g., Figure 1 The planar file system manager 112 can perform this operation. As an example, a planar operation request could be a request to access all files (e.g., millions of files) in a planar plane for use as input to a machine learning algorithm configured to produce one or more outputs. Through the embodiments disclosed herein, such a request can be simplified by requiring only the planar ID of the planar plane, rather than information about specific files within it. This allows the planar file system manager to use the planar ID to perform a lookup to obtain files within the planar plane. In one or more embodiments, the operation can be a storage operation. Storage operations can include, but are not limited to, backup operations, copy operations, deduplication operations, and / or disaster recovery operations (e.g., failover, failback, recovery, etc.), which can be performed on a planar plane or subplanar plane, or a subplanar granular dataset (e.g., for a planar plane and / or subplanar plane).

[0065] Figure 3 An example flat file system 300 according to one or more embodiments of the present disclosure is illustrated. Figure 3 The examples shown and discussed below are simplified examples intended for illustrative purposes only and are not intended to limit the scope of the embodiments described herein. Furthermore, while this example illustrates certain aspects of the embodiments described herein, not all possible aspects of such embodiments may be illustrated in this particular example.

[0066] Figure 3 An example flat file system 300 is shown. Figure 3 The view of the planar file system 300 shown can be a logical view representing the relationships between the planes of the planar file system 300, such as when the planes are striped across any number of storage devices in a parallel file system. Alternatively, Figure 3 The view of the flat file system 300 shown can represent a physical view of the layout of the planes in the flat file system formatted on one or more storage devices.

[0067] The example flat file system 300 includes two planes, PL-A and PL-B, at the highest level, which are the plane IDs for these two planes. Other planes and subplanes described as part of this example will have plane IDs that follow a similar naming convention defined by the user, and the user has also specified the relationships between the planes (described below). Figure 3 The structure of the flat file system shown can already be defined by a flat file system manager (e.g., Figure 1 The flat file system manager (112) is configured and implemented for users.

[0068] PL-A includes a nested subplane PL-A1, which comprises two nested subplanes PL-A11A and PL-A11B. Plane PL-A11A includes a further nested subplane PL-A11A1. Therefore, files in PL-A11A1 can have subplane relationships with files in PL-A11A, files in PL-A11A can have subplane relationships with files in PL-A11, files in PL-A11 can have subplane relationships with files in PL-A1, and files in PL-A1 can have subplane relationships with files in PL-A1. Files in PL-A11B can have subplane relationships relative to files in PL-A11, but otherwise are discrete with files in PL-A11A and its nested subplanes.

[0069] Plane PL-A may also include subplane PL-A1A. PL-A1A is orthogonal to PL-A1, which means that the files in PL-A1A are logically related to each other in some way, but the files stored in PL-A1A are not subplanes of the files in PL-A1.

[0070] Plane PL-A may also include subplane PL-A1A-1. PL-A1A-1 is orthogonal to PL-A1A, which means that the files in PL-A1A are logically related to each other in some way, but the files stored in PL-A1A-1 are not subplanes of the files in PL-A1A.

[0071] Plane PL-A may also include subplane PL-A1A-1A. PL-A1A-1A is orthogonal to PL-A1A-1, which means that the files in PL-A1A-1A are logically related to each other in some way, but the files stored in PL-A1A-1A are not subplanes of the files in PL-A1A-1A.

[0072] Plane PL-A may also include subplane PL-A1A-1A0. PL-A1A-1A0 is orthogonal to PL-A1A-1A, which means that the files in PL-A1A-1A are logically related to each other in some way, but the files stored in PL-A1A-1A0 are not subplanes of the files in PL-A1A-1A.

[0073] Plane PL-B can include subplane PL-B1. For example... Figure 3 As shown, the file subplane PL-B1 can be orthogonal to the file in the subplane PL-A1A-1.

[0074] Figure 4 An example flat file system 400 including example files is illustrated according to one or more embodiments of the present disclosure. Figure 4 The examples shown and discussed below are simplified examples intended for illustrative purposes only and are not intended to limit the scope of the embodiments described herein. Furthermore, while this example illustrates certain aspects of the embodiments described herein, not all possible aspects of such embodiments may be illustrated in this particular example.

[0075] Consider the following scenario: a user has interacted with a flat file system manager to configure a flat file system implemented using an underlying parallel file system, allowing data stored in the flat plane to be stored in a striped manner across any number of underlying heterogeneous storage devices. The user intends to use the flat file system to store any number of audio / video files and other related files that can later be accessed for analysis for some purpose.

[0076] Based on the user's flat file system configuration request, the flat file system manager can logically configure parallel file systems to include... Figure 4 The planar layout shown includes two planes, PL-A1 and PL-B1 (e.g., plane ID), where PL-A1 has a subplane PL-A10, which in turn has a subplane PL-A101. The user specifies the nested subplane relationships between PL-A1, PL-A10, and PL-A101, as well as the orthogonal relationship between PL-A1 and PL-B1, in the planar file system configuration request.

[0077] The user can then begin storing files in the flat file system 400, and each file will have a relationship based on the plane in which it is stored. The user can store the file Example.mp4 in plane PL-A1 by providing a plane operation request to the flat file system manager, which includes the plane ID: file handle combination PL-A1:Example.mp4. Similarly, the user can store the file PL-B1:ConsolidatedFeature.mov in PL-B1, the files PL-A10:Example.tiff, PL-A10:sample3.jpg, and PL-A10:collection.zip in PL-A10, and the files PL-A101:metadata1.docx, PL-A101:features.mov, and PL-A101:subset.pdf in PL-A101.

[0078] With the help of the planes that store them, the files PL-A101:metadata1.docx, PL-A101:features.mov, and PL-A101:subset.pdf in PL-A101 each have a subplane relationship with PL-A10:Example.tiff, and the files PL-A10:Example.tiff, PL-A10:sample3.jpg, and PL-A10:collection.zip have a subplane relationship with PL-A1:Example.mp4. Additionally, PL-A1:Example.mp4 is orthogonal to PL-B1:ConsolidatedFeature.mov because it is stored in PL-B1.

[0079] Figure 5A An example flat file system lookup table 500 according to one or more embodiments disclosed herein is shown. Table 500 represents tables shown as stored in... Figure 4 The files in the flat file system 400 include each file's plane ID, file handle, file size, and plane relation level. Therefore, when a plane operation request is received at the flat file system manager, the flat file system manager can access a single file (e.g., when the plane operation request includes a plane ID and a file handle), a group of files in the plane (e.g., when the plane operation request includes a plane ID), or files with some kind of relationship, as shown in Table 500 (e.g., the orthogonal relationship between PL-A1 and PL-B1).

[0080] Figure 5B An example of a DAG 550 lookup in a flat file system is shown. DAG 550 can be organized based on the flat ID as shown in the figure, and can be used to perform lookups in a flat file system, similar to the examples above. Figure 5A The search is discussed in the description.

[0081] Figure 6 An example plane file system 600, including orthogonal relationships between planes, is illustrated according to one or more embodiments of the present disclosure. Figure 6 The examples shown and discussed below are simplified examples intended for illustrative purposes only and are not intended to limit the scope of the embodiments described herein. Furthermore, while this example illustrates certain aspects of the embodiments described herein, not all possible aspects of such embodiments may be illustrated in this particular example.

[0082] Consider a scenario where a research organization investigates the possibility of life on other planets. Any number of probes have been launched in any number of directions, and these probes periodically send back documents with various sensor readings over time. As the probes travel for years, decades, and so on, a large number of documents may accumulate. To organize these documents for analysis, the research organization's users interact with a planar file system manager to implement a planar file system. The user specifies that the planar file system will consist of five planes, PL-A, PL-B, PL-C, PL-D, and PL-E, which are orthogonally correlated in this order, and any number of documents can be stored in each plane. The research organization has a high-performance computing (HPC) solution used to access the planar file system to perform a range of analyses as part of the search for life.

[0083] The first plane, PL-A, contains high-level data on the possibility of life forms in various star systems. The second plane, PL-B, contains documents related to atmospheric conditions on planets within the star system. The third plane, PL-C, contains documents related to temperature conditions on planets. The fourth plane, PL-D, contains documents related to water conditions on planets. The fifth plane, PL-E, contains documents related to information on the surface features of planets.

[0084] like Figure 6 As indicated by the middle arrow, the orthogonal relationships between the five planes are configured by the user to inform the analysis process performed by the research organization's HPC solution. Therefore, analysis is first performed using files in PL-A to determine if one or more probes sending data reside within a region of the universe where life may exist. If the analysis is "yes," the analysis can proceed to the orthogonally correlated PL-B to analyze the atmospheric conditions of planets in that region of the universe to determine if they are favorable for life (e.g., the planet's atmosphere includes ozone). If this analysis is "yes," the analysis can proceed to the orthogonally correlated PL-C to analyze whether the planet's temperature conditions are favorable for life. If this analysis is "yes," the analysis can proceed to the orthogonally correlated PL-D to analyze whether there is sufficient water on the planet to support life forms. If this analysis is "yes," the analysis can proceed to the orthogonally correlated PL-E to analyze whether the planet's surface features (e.g., number of volcanoes, seismic activity, etc.) are within a range favorable for supporting life forms. Thus, the search for life on distant planets, which involves analyzing a large number of relevant files, is improved by enabling analysis using a planar file system with orthogonally correlated planes (each plane containing relevant data). Despite Figure 6Not shown, but taking PL-B as an example, there may be multiple subplanes that are discrete subsets or orthogonally related, and may have inferences / analyses related to the aforementioned problem stored as files. Such an example (e.g., as an example of multiple additional subplanes that are discrete subsets or orthogonally related) can be applied within each plane (e.g., PL-A, PL-B, PL-C, PL-D, PL-E).

[0085] Figure 7 A block diagram of a computing device according to one or more embodiments of the present disclosure is illustrated. As discussed above, the embodiments described herein can be implemented using a computing device. For example, Figure 1 All or any part of the components shown may be implemented using one or more computing devices, at least in part. The computing device 700 may include one or more computer processors 702, non-persistent storage devices 704 (e.g., volatile memory such as random access memory (RAM), cache memory, etc.), persistent storage devices 706 (e.g., hard disks, optical disc drives such as CD drives or DVD drives, flash memory, etc.), communication interfaces 712 (e.g., Bluetooth interfaces, infrared interfaces, network interfaces, optical interfaces, etc.), input devices 710, output devices 708, and many other elements (not shown) and functions. Each of these components is described below.

[0086] In one or more embodiments, one or more computer processors 702 may be integrated circuits for processing instructions. For example, one or more computer processors may be one or more cores or microcores of a processor. Processor 702 may be a general-purpose processor configured to execute program code included in software executed on computing device 700. Processor 702 may be a special-purpose processor in which certain instructions are incorporated into the processor design. Although in Figure 7 Only one processor 702 is shown, but computing device 700 may include any number of processors without departing from the scope of the embodiments disclosed herein.

[0087] The computing device 700 may also include one or more input devices 710, such as a touchscreen, keyboard, mouse, microphone, touchpad, electronic pen, motion sensor, or any other type of input device. Input devices 710 allow users to interact with the computing device 700. In one or more embodiments, the computing device 700 may include one or more output devices 708, such as a screen (e.g., a liquid crystal display (LCD), plasma display, touchscreen, cathode ray tube (CRT) monitor, projector, or other display device), printer, external storage device, or any other output device. One or more output devices may be the same as or different from one or more input devices. One or more input and output devices may be locally or remotely connected to one or more computer processors 702, non-persistent storage devices 704, and persistent storage devices 706. Many different types of computing devices exist, and the aforementioned one or more input and output devices may take other forms. In some instances, a multi-mode system allows users to provide multiple types of input / output to communicate with the computing device 700.

[0088] Furthermore, the communication interface 712 can facilitate the connection of the computing device 700 to a network (not shown) (e.g., a local area network (LAN), a wide area network (WAN) such as the Internet, a mobile network, or any other type of network) and / or another device (such as another computing device). The communication interface 712 can perform or facilitate the reception and / or transmission of wired or wireless communications using wired and / or wireless transceivers, including utilizing audio jacks / plugs, microphone jacks / plugs, Universal Serial Bus (USB) ports / plugs, Apple® Lightning® ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, proprietary wired ports / plugs, Bluetooth® wireless signal transmission, BLE wireless signal transmission, IBEACON® wireless signal transmission, RFID wireless signal transmission, Near Field Communication (NFC) wireless signal transmission, Dedicated Short Range Communication (DSRC) wireless signal transmission, and 802.11. Wireless signal transmission via WiFi, WLAN, Visible Light Communication (VLC), Global Microwave Access Interoperability (WiMAX), IR communication, Public Switched Telephone Network (PSTN), Integrated Services Digital Network (ISDN), 3G / 4G / 5G / LTE cellular data network, ad hoc network, radio wave, microwave, infrared, visible light, ultraviolet light, and electromagnetic spectrum, or combinations thereof. The communication interface 712 may also include one or more Global Navigation Satellite System (GNSS) receivers or transceivers for determining the location of the computing device 700 based on one or more signals received from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the US GPS, the Russian GLONASS, the Chinese BeiDou Navigation Satellite System (BDS), and the European Galileo GNSS. There are no limitations on operation on any particular hardware arrangement, and therefore, as hardware or firmware arrangements evolve, the basic features described herein can be readily replaced with improved hardware or firmware arrangements.

[0089] The term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying one or more instructions and / or data. Computer-readable media may include non-transitory media, where data may be stored and does not include carrier waves and / or transient electronic signals propagated wirelessly or via a wired connection. Examples of non-transitory media include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as CDs or DVDs, flash memory, memory, or memory devices. Code and / or machine-executable instructions may be stored on a computer-readable medium, which may represent any combination of processes, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. A code segment can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted by any suitable means, including memory sharing, messaging, token passing, network transmission, etc.

[0090] All or any part of the components of computing device 700 may be implemented in a circuit. For example, the components may include and / or may be implemented using electronic circuitry or other electronic hardware, which may include one or more programmable electronic circuits (e.g., microprocessors, GPUs, DSPs, CPUs, and / or other suitable electronic circuits), and / or may include and / or be implemented using computer software, firmware, or any combination thereof to perform the various operations described herein. In some aspects, computer-readable storage devices, media, and memories may include wired or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

[0091] In the foregoing description, numerous details have been set forth as examples of the embodiments described herein. Those skilled in the art (who also benefit from this disclosure) will understand that one or more embodiments described herein can be practiced without these specific details, and that many variations or modifications can be made without departing from the scope of the embodiments described herein. Certain details known to those skilled in the art may have been omitted to avoid obscuring the description.

[0092] Specific details have been provided in the foregoing description to offer a thorough understanding of the aspects and examples presented herein. However, those skilled in the art will understand that these aspects can be practiced without these specific details. For clarity, in some instances, the technology may be presented as comprising functional blocks that may include devices, device components, steps or routines in a software-embodied method, or a combination of hardware and software. Additional components may be used in addition to those shown in the figures and / or described herein. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring these aspects with unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail to avoid obscuring these aspects.

[0093] The above can describe individual aspects as processes or methods depicted as flowcharts, flow diagrams, data flow diagrams, structure diagrams, or block diagrams. While flowcharts can describe operations as sequential processes, many operations can be performed in parallel or simultaneously. Furthermore, the order of operations can be rearranged. When an operation of a process completes, the process terminates, but may have additional steps not included in the diagram. A process can correspond to a method, function, flow, subroutine, subprogram, etc. When a process corresponds to a function, the termination of the process can correspond to the function returning to the calling function or the main function.

[0094] The processes and methods described in the examples above can be implemented using computer-executable instructions stored in or otherwise obtained from a computer-readable medium. Such instructions may include, for example, instructions and data to cause a general-purpose computer, special-purpose computer, or processing device to perform a particular function or group of functions, or to otherwise configure a general-purpose computer, special-purpose computer, or processing device to perform a particular function or group of functions. Some of the computer resources used may be accessible via a network. The computer-executable instructions may be, for example, binary files, intermediate format instructions (such as assembly language, firmware, source code, etc.). Examples of computer-readable media that may be used to store instructions, information used, and / or information created during the methods according to the described examples include disks or optical discs, flash memory, USB devices equipped with non-volatile memory, networked storage devices, etc.

[0095] In the foregoing description of the accompanying drawings, in the various embodiments described herein, any component described with respect to the drawings may be equivalent to one or more components with the same or similar names and / or numbers described with respect to any other drawing. For the sake of brevity, the description of these components may not be repeated for each drawing. Therefore, each embodiment of a component in each drawing is incorporated by reference and is assumed to be optionally present in every other drawing having one or more components with the same or similar names and / or numbers. Furthermore, any description of a component in the drawings according to the various embodiments described herein should be interpreted as an optional embodiment that may be implemented as a supplement, combination, or alternative to the embodiments described with respect to corresponding one or more components with the same or similar names and / or numbers in any other drawing.

[0096] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as adjectives for elements (i.e., any noun in the application). The use of ordinal numbers does not imply or create any particular order of elements, nor does it limit any element to being a single element unless explicitly stated otherwise, such as through the use of the terms “before,” “after,” “single,” and other such terms. Rather, ordinal numbers are used to distinguish elements. For example, the first element is different from the second element, and the first element may contain more than one element and be after (or before) the second element in the element order.

[0097] As used herein, the phrase “operably connected” or “operable connection” means that there exists a direct or indirect connection between elements / components / devices that allows the elements to interact with each other in a certain way. For example, the phrase “operably connected” can refer to any direct connection (e.g., a direct wired connection between two devices or components) or an indirect connection (e.g., a wired and / or wireless connection between any number of devices or components that are operably connected). Therefore, any path through which information can pass can be considered an operational connection.

[0098] While the embodiments discussed herein have been described with respect to a limited number of examples, those skilled in the art who benefit from this disclosure will understand that other embodiments can be devised without departing from the scope of the embodiments disclosed herein. Accordingly, the scope of the embodiments described herein should be limited only by the appended claims.

Claims

1. An electronic device, comprising: one or more processors; and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to: receive, at a flat file system manager executing on the one or more processors, from a user interface, a flat file system configuration request comprising a plurality of flats, a plurality of sub-flats of the plurality of flats, and a plurality of flat relationships, wherein the plurality of flats are logical storage constructs for storing groups of files, wherein: the plurality of flat relationships comprise a sub-flat relationship between a flat and a first sub-flat, a first orthogonal relationship between two independent flats of the plurality of flats, and a second orthogonal relationship between a second sub-flat of the plurality of sub-flats and a third sub-flat of the plurality of sub-flats, and the second sub-flat and the third sub-flat are sub-flats of different flats; configure, by the flat file system manager, a storage environment to include a flat file system based on the flat file system configuration request, wherein configuring the storage environment to include the flat file system comprises formatting a plurality of storage devices to include the plurality of flats and the plurality of sub-flats; receive, from the user interface, a flat operation request to perform an operation on a first flat of the plurality of flats of the flat file system, wherein the flat operation request comprises a flat identifier corresponding to the flat; perform the operation on the first flat based on the flat operation request to obtain an initial analysis result; and based on the initial analysis result, perform an additional flat operation on a second flat of the plurality of flats of the flat file system, the second flat related to the first flat by the first orthogonal relationship, wherein the additional flat operation is performed on the second flat based at least in part on the first orthogonal relationship. the storage environment comprises a parallel file system. 2.The electronic device of claim 1, wherein, the plurality of flats are striped across the plurality of storage devices of the parallel file system. 3.The electronic device of claim 2, wherein, the operation performed based on the flat operation request comprises invoking one or more flat locks to facilitate concurrent access to one or more flats of the plurality of flats by a plurality of computing devices. 4.The electronic device of claim 2, wherein, the operation performed based on the flat operation request is performed at least in part using a flat file system lookup directed acyclic graph (DAG). 5.The electronic device of claim 1, wherein, the operation performed based on the flat operation request comprises a modification to an attribute of the first flat, the modification applied to sub-flats of the first flat. 6.The electronic device of claim 1, wherein, the operation performed based on the flat operation request is a storage operation. 7.The electronic device of claim 1, wherein, 8. A computer-implemented method, comprising: receiving, at a flat file system manager executing on one or more processors, from a user interface, a flat file system configuration request comprising a plurality of flats, a plurality of sub-flats of the plurality of flats, and a plurality of flat relationships, wherein the plurality of flats are logical storage constructs for storing groups of files, wherein: ​ the plurality of plane relationships include a subplane relationship between a plane and a first subplane, a first orthogonal relationship between two independent planes of the plurality of planes, and a second orthogonal relationship between a second subplane of the plurality of subplanes and a third subplane of the plurality of subplanes, and the second subplane and the third subplane are subplanes of different planes; configuring, by the planar file system manager, a storage environment to include a planar file system based on the planar file system configuration request, wherein configuring the storage environment to include the planar file system includes formatting a plurality of storage devices to include the plurality of planes and the plurality of subplanes; receiving, from the user interface, a plane operation request to perform an operation on a first plane of the plurality of planes of the planar file system, wherein the plane operation request includes a plane identifier corresponding to the plane; performing the operation on the first plane based on the plane operation request to obtain an initial analysis result; and performing, based on the initial analysis result, an additional plane operation on a second plane of the plurality of planes of the planar file system, the second plane related to the first plane by the first orthogonal relationship, wherein the additional plane operation is performed on the second plane based at least in part on the first orthogonal relationship.

9. The computer-implemented method of claim 8, wherein, the storage environment includes a parallel file system.

10. The computer-implemented method of claim 9, wherein, the plurality of planes are striped across the plurality of storage devices of the parallel file system.

11. The computer-implemented method of claim 8, wherein, the operation performed based on the plane operation request is performed at least in part using a planar file system lookup data structure.

12. The computer-implemented method of claim 8, wherein, the operation performed based on the plane operation request includes a modification to an attribute of the first plane, the modification applied to subplanes of the first plane.

13. The computer-implemented method of claim 8, wherein, the operation performed based on the plane operation request includes a modification to an attribute of a subplane of the plane, and wherein the modification is not applied to the plane.

14. A non-transitory computer readable medium storing a program of programming for execution by one or more processors, the program of programming including instructions for: The planar file system manager executing on the one or more processors receives a planar file system configuration request comprising a plurality of planes, a plurality of sub-planes of the plurality of planes, and a plurality of plane relationships from a user interface, wherein, the plurality of planes are logical storage constructs for storing groups of files, wherein: the plurality of plane relationships include a subplane relationship between a plane and a first subplane, a first orthogonal relationship between two independent planes of the plurality of planes, and a second orthogonal relationship between a second subplane of the plurality of subplanes and a third subplane of the plurality of subplanes, and the second subplane and the third subplane are subplanes of different planes; configuring, by the planar file system manager, a storage environment to include a planar file system based on the planar file system configuration request, wherein configuring the storage environment to include the planar file system includes formatting a plurality of storage devices to include the plurality of planes and the plurality of subplanes; receiving, from the user interface, a plane operation request to perform an operation on a first plane of the plurality of planes of the planar file system, wherein the plane operation request includes a plane identifier corresponding to the plane; performing the operation on the first plane based on the plane operation request to obtain an initial analysis result; and performing, based on the initial analysis result, an additional plane operation on a second plane of the plurality of planes of the planar file system, the second plane related to the first plane by the first orthogonal relationship, wherein the additional plane operation is performed on the second plane based at least in part on the first orthogonal relationship. performing the operation on the first extent based on the extent operation request to obtain initial analysis results; and performing an additional extent operation on a second extent of the plurality of extents of the file system based on the initial analysis results, the second extent related to the first extent by the first orthogonal relationship, wherein the additional extent operation is performed on the second extent based at least in part on the first orthogonal relationship.

15. The non-transitory computer readable medium of claim 14, wherein, the storage environment comprises a parallel file system, and wherein the plurality of extents are striped across the plurality of storage devices of the parallel file system.

16. The non-transitory computer readable medium of claim 14, wherein, the operation performed based on the extent operation request is performed at least in part using an extent file system lookup data structure.

17. The non-transitory computer readable medium of claim 14, wherein, the operation performed based on the extent operation request comprises a modification to an attribute of the first extent, the modification applied to a sub-extent of the first extent.

18. The non-transitory computer readable medium of claim 14, wherein, the operation performed based on the extent operation request is a modification to an attribute of a sub-extent of the extent, and wherein the operation is not applied to the extent.

19. The non-transitory computer-readable medium of claim 14, wherein, to configure the storage environment, the non-transitory computer-readable medium further comprises instructions to format one or more storage devices to include the extent file system.

Images