Data access method, data storage method, device and file storage system
By introducing multiple file directory management shards and file storage clusters in the HDFS system, and using the raft protocol for data synchronization, the problem of limited file count and data volume caused by NameNode node memory limit is solved, and the availability and scalability of the system is improved .
Patent Information
- Application Number
- CN202010965297.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-09-14
AI Technical Summary
The memory limit of NameNode node in HDFS system leads to limited file number and data volume, low metadata reliability and service availability, and cannot effectively expand capacity horizontally.
By setting up multiple file directory management shards and file storage clusters in the file storage system, each shard and cluster includes both primary and secondary nodes, and data synchronization is adopted by raft protocol to realize file directory management shards and cluster management nodes. Horizontal capacity expansion and high availability.
The number of files stored in the system and the total data volume are improved, metadata loss and service availability problems caused by single point of failure are solved, and high availability and scalability of the file storage system are achieved.
Smart Images

Figure CN112100129B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of file storage technology, and in particular to a data access method, a data storage method, a device and a file storage system. Background Art
[0002] HDFS (Hadoop Distributed File System) is a typical and widely used open source distributed file storage system. In the HDFS system, the metadata of all files is stored in the NameNode node. The memory space of the NameNode node is limited, so the number of file metadata that can be stored in the NameNode node is limited, which limits the total number of files and the total amount of data that can be stored in the system as a whole. Summary of the invention
[0003] In view of this, an object of the present invention is to provide a data access method, a data storage method, an apparatus and a file storage system to increase the number of files stored in the system and the total amount of data.
[0004] In a first aspect, an embodiment of the present invention provides a data access method, the method comprising: determining, based on a data access request, a first target data volume to which a file to be accessed belongs; determining, based on a correspondence between a file directory management slice and a data volume in a file storage system, a file directory management slice to which the first target data volume belongs; wherein a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage metadata of files in a specified data volume; obtaining metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and accessing the file to be accessed based on the metadata of the file to be accessed.
[0005] Furthermore, each file directory management slice includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of the file to be accessed; the backup file directory management node synchronizes data with the primary file directory management node.
[0006] Furthermore, the primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; data of the backup file directory management node is synchronized with data of the primary file directory management node through a preset raft protocol.
[0007] Furthermore, multiple file storage clusters are set up in the file storage system; the metadata of the file to be accessed includes the file storage cluster where the file data of the file to be accessed is located; based on the metadata of the file to be accessed, the step of accessing the file to be accessed includes: obtaining the file to be accessed from the file storage cluster to which the file to be accessed belongs.
[0008] Furthermore, the file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; the step of obtaining the file to be accessed from the file storage cluster to which the file to be accessed belongs includes: determining the target data storage node where the file data of the file to be accessed is located through the cluster management node of the file storage cluster to which the file to be accessed belongs; and obtaining the file data of the file to be accessed from the target data storage node.
[0009] Furthermore, each file storage cluster includes multiple cluster management nodes; the multiple cluster management nodes include a primary cluster management node and at least one backup cluster management node; wherein the primary cluster management node is used to provide a target data storage node where the file data of the file to be accessed is located; the backup cluster management node synchronizes data with the primary cluster management node.
[0010] Furthermore, a data persistence service is provided in the file storage system; the data persistence service is used to provide a correspondence between file directory management slices and data volumes; before the step of determining the first target data volume to which the file to be accessed belongs based on the data access request, the method also includes: obtaining the correspondence between the file directory management slices and the data volumes through the data persistence service.
[0011] In a second aspect, an embodiment of the present invention provides a data storage method, the method comprising: determining, based on a data storage request, a second target data volume to which the file to be stored belongs; determining, based on a correspondence between a file directory management slice and a data volume in a file storage system, a file directory management slice to which the second target data volume belongs; wherein a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage metadata of files in a specified data volume; determining a target storage location for the file to be stored; based on the target storage location, generating metadata for the file to be stored in the file directory management slice to which the second target data volume belongs; and storing the file to be stored in the target storage location.
[0012] Furthermore, a plurality of file storage clusters are provided in the file storage system; the target storage location includes: a target file storage cluster storing file data of the file to be stored; the step of storing the file to be stored in the target storage location includes: storing the file to be stored in the target file storage cluster.
[0013] Furthermore, the file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; the step of storing the file to be stored in the target file storage cluster includes: determining the target data storage node that stores the file to be stored from the target file storage cluster; based on the target data storage node, generating, in the cluster management node of the target file storage cluster, the correspondence between the file data of the file to be stored and the target data storage node; and saving the file data of the file to be stored to the target data storage node.
[0014] In a third aspect, an embodiment of the present invention provides a file storage system, which includes a data request server, multiple file directory management shards and multiple file storage clusters; wherein each file storage cluster is used to store files in a specified data volume; each file directory management shard is used to manage metadata of files in the specified data volume; the metadata is used to indicate the file storage cluster where the files in the specified data volume are located; the data request server is used to process received data requests based on the correspondence between the file directory management shards and the data volumes.
[0015] Furthermore, a data persistence service is provided in the file storage system; the data persistence service is used to provide the corresponding relationship between the file directory management slices and the data volumes to the data request server.
[0016] Furthermore, each file directory management slice includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of files in a specified data volume; the backup file directory management node synchronizes data with the primary file directory management node.
[0017] Furthermore, the file storage cluster includes a cluster management node and a data storage node; the data storage node is used to store file data of each file in the file storage cluster; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node.
[0018] Furthermore, the file storage cluster includes multiple cluster management nodes; the multiple cluster management nodes include a primary cluster management node and at least one backup cluster management node; wherein the primary cluster management node is used to provide a data storage node where the file data of each file in the file storage cluster is located; the backup cluster management node synchronizes data with the primary cluster management node.
[0019] In a fourth aspect, an embodiment of the present invention provides a data access device, the device comprising: a first data volume determination module, used to determine, based on a data access request, a first target data volume to which a file to be accessed belongs; a first management slice determination module, used to determine, based on a correspondence between file directory management slices and data volumes in a file storage system, a file directory management slice to which the first target data volume belongs; wherein a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage metadata of files in a specified data volume; a file access module, used to obtain metadata of a file to be accessed from the file directory management slice to which the first target data volume belongs; and access the file to be accessed based on the metadata of the file to be accessed.
[0020] In a fifth aspect, an embodiment of the present invention provides a data storage device, the device comprising: a second data volume determination module, used to determine the second target data volume to which the file to be stored belongs based on a data storage request; a second management slice determination module, used to determine the file directory management slice to which the second target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage metadata of files in a specified data volume; a file storage module, used to determine a target storage location for the file to be stored; based on the target storage location, generate metadata for the file to be stored in the file directory management slice to which the second target data volume belongs; and store the file to be stored in the target storage location.
[0021] In a sixth aspect, an embodiment of the present invention provides an electronic device comprising a processor and a memory, the memory storing machine executable instructions that can be executed by the processor, and the processor executing the machine executable instructions to implement the data access method of any one of the first aspect, the data storage method of any one of the second aspect, or the file storage system of any one of the third aspect.
[0022] In the seventh aspect, an embodiment of the present invention provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement any data access method of the first aspect, any data storage method of the second aspect, or any file storage system of the third aspect.
[0023] The embodiments of the present invention bring the following beneficial effects:
[0024] The embodiment of the present invention provides a data access method, a data storage method, an apparatus and a file storage system, which determine the first target data volume to which the file to be accessed belongs based on a data access request; determine the file directory management slice to which the first target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with multiple file directory management slices; each file directory management slice is used to manage the metadata of the files in the specified data volume; obtain the metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and access the file to be accessed based on the metadata of the file to be accessed. In this method, the file storage system is provided with multiple file directory management slices, and when the file volume is large, the file target management slice can be expanded horizontally, avoiding the problem of the limited number of metadata of the files stored in a single file directory management node, and increasing the number of files stored in the system and the total data volume.
[0025] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0026] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0028] Figure 1 A flow chart of a data access method provided by an embodiment of the present invention;
[0029] Figure 2 A schematic diagram of the structure of a file storage system provided by an embodiment of the present invention;
[0030] Figure 3 A flowchart of another data access method provided by an embodiment of the present invention;
[0031] Figure 4 A flowchart of a data storage method provided by an embodiment of the present invention;
[0032] Figure 5 A schematic diagram of the structure of a file storage system provided by an embodiment of the present invention;
[0033] Figure 6 A schematic diagram of the structure of a data access device provided by an embodiment of the present invention;
[0034] Figure 7 A schematic diagram of the structure of a data storage device provided by an embodiment of the present invention;
[0035] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0037] At present, HDFS is a typical and widely used open source distributed file storage system. HDFS has the following main defects: 1. The size of stored files is limited: All file metadata are stored in the NameNode node, and the NameNode node has no horizontal expansion capability, resulting in the number of files and the total amount of data that the cluster can store being limited by the memory bottleneck of the NameNode node, and generally only hundreds of millions of files can be stored; 2. The metadata reliability is not high: The NameNode node of HDFS has two nodes, the master and the backup node. The metadata synchronization of the master and the backup node is asynchronous synchronization. When there is a metadata update, when the master node is synchronizing the update to the backup node, or has not yet synchronized the update to the slave node, the master node is down, and part of the latest metadata will be lost; 3. The service availability is not high: After the main NameNode node of HDFS hangs up, the backup NameNode node needs to load all metadata into the memory before it can provide services to the outside world. When the file size is at the hundreds of millions level, the switching time is at the level of several minutes. Based on this, the embodiment of the present invention provides a data access method, a data storage method, a device and a file storage system, and the technology can be applied to various hardware devices such as computers and notebooks, and can be applied to devices with data access functions in particular. This technology can be implemented through relevant hardware and software. It should be noted that the file storage system in this embodiment can be a distributed file storage system.
[0038] To facilitate understanding of this embodiment, a data access method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps:
[0039] Step S102, based on the data access request, determining the first target data volume to which the file to be accessed belongs;
[0040] The above-mentioned data volume can be a namespace used to store directories, files, etc. created by users; before using the file storage system, users first need to create a data volume for the files to be accessed; in actual implementation, users can send data access requests through devices such as clients, and the data access request includes the data volume name specified by the user. After the server of the file storage system receives the data access request, it can directly determine the first target data volume to which the files to be accessed belong through the data volume name included in the data access request. The server of the above-mentioned file storage system usually exposes operation interfaces such as volumes, directories, and files to the outside, and forwards the user's data access request to the corresponding module for processing.
[0041] Step S104, based on the correspondence between the file directory management slice and the data volume in the file storage system, determining the file directory management slice to which the first target data volume belongs; wherein the file storage system is provided with a plurality of file directory management slices; each file directory management slice is used to manage metadata of files in a specified data volume;
[0042] The file storage system mentioned in this implementation can be understood as a file system with a directory tree, where users store files uploaded by users and provide users with typical file system operations, such as: volume operations, directory operations, file operations, file links, changing file permissions and owners, etc. The above-mentioned file directory management slice can be called Slice. There are multiple file directory management slices set in the file storage system, for example, there can be three or five; each file directory management slice logically contains multiple data volumes, each data volume corresponds to a directory tree, and each data volume belongs to only one file directory management slice and cannot cross file directory management slices. Each file directory management slice is used to manage the metadata of files in a specified data volume; metadata can also be called inode id. Each file has unique metadata, which is also the unique identifier of the file. The metadata can be defined as data describing the file, which is descriptive information about files and information resources.
[0043] Since each data volume belongs to only one file directory management slice, the correspondence between the above file directory management slices and data volumes is a one-to-one correspondence, which can be stored through a specified service in the file storage system; specifically, after determining the first target data volume to which the file to be accessed belongs, the server of the file storage system can directly use the correspondence between the file directory management slices and the data volumes to first find the first target data volume from multiple data volumes, and then determine the file directory management slice to which the first target data volume belongs.
[0044] Step S106: acquiring metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and accessing the file to be accessed based on the metadata of the file to be accessed.
[0045] Since each file directory management slice is used to manage the metadata of files in a specified data volume, that is, each file directory management slice includes the metadata of the corresponding file. Therefore, after determining the file directory management slice to which the first target data volume belongs, the metadata of the file to be accessed can be obtained through the file directory management slice. Specifically, the metadata of the file to be accessed can be obtained through multiple file directory management nodes included in the file directory management slice. Among them, the file directory management node is mainly used to manage the user's directory tree and the mapping of files to metadata, and can provide the metadata of the file to be accessed.
[0046] The embodiment of the present invention provides a data access method, which determines the first target data volume to which the file to be accessed belongs based on a data access request; determines the file directory management slice to which the first target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with multiple file directory management slices; each file directory management slice is used to manage the metadata of the files in the specified data volume; obtains the metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and accesses the file to be accessed based on the metadata of the file to be accessed. In this method, the file storage system is provided with multiple file directory management slices, and when the file volume is large, the file target management slice can be expanded horizontally, avoiding the problem of the limited number of metadata of the files stored in a single file directory management node, and increasing the number of files stored in the system and the total data volume.
[0047] Each of the above-mentioned file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of the file to be accessed; the backup file directory management node synchronizes data with the primary file directory management node.
[0048] For details, see Figure 2 As shown in the structural diagram of the file storage system, the above-mentioned multiple file directory management slices correspond to Slice1, Slice2, etc. in the figure; the above-mentioned file directory management node corresponds to the NameNode in the figure; the above-mentioned main file directory management node corresponds to the Master NameNode in the figure, and the standby file directory management node corresponds to the SlaveNameNode in the figure; the data of the above-mentioned multiple file directory management nodes can be completely mirrored.
[0049] The above file directory management node mainly provides metadata of the file to be accessed through the mapping relationship between the file and the metadata. The data of the standby file directory management node and the main file directory management node can be asynchronously synchronized.
[0050] It should be noted that the file directory management shards included in the file storage system are not limited to the number shown in the figure, but may be more, usually including three or five, which is not limited here, and the specific number can be set according to actual needs. In addition, the at least one standby file directory management node included in the file directory management node is not limited to the number shown in the figure, but may be more, which is not limited here, and the specific number can be set according to actual needs.
[0051] In addition, the main file directory management node among the multiple file directory management nodes is determined based on the preset raft protocol; the data of the standby file directory management node is synchronized with the data of the main file directory management node through the preset raft protocol.
[0052] The above-mentioned raft protocol can make a cluster of servers form a replication state machine and maintain the data consistency of multiple file directory management nodes; usually in the initial state, there is no distinction between the primary and backup file directory management nodes in each file directory management shard. A primary file directory management node is selected from multiple file directory management nodes through the election mechanism of the raft protocol, and the remaining file directory management nodes directly become backup file directory management nodes. In this way, after the primary file directory management node hangs up, a new primary file directory management node can be selected from the backup file directory management nodes within 100 milliseconds and provide services; and due to the consistency algorithm of the raft protocol, the data of the new primary file directory management node is the same as the data of the primary file directory management node that has hung up, which solves the problem that the backup file directory management node takes several minutes to provide services to the outside after the primary file directory management node of the file storage system hangs up, and also solves the problem that some new metadata is lost after the primary file directory management node of the file storage system hangs up.
[0053] In the structure of the above-mentioned file storage system, the file directory management node that manages the directory tree is split into multiple file directory management shards, and the file directory management shards can be expanded horizontally and seamlessly; this solves the problem that the file storage system is limited by the memory bottleneck of a single file directory management node and can only store hundreds of millions of files, thereby increasing the number of files stored in the system and the total amount of data.
[0054] This embodiment provides another data access method. The file storage system is provided with multiple file storage clusters. The metadata of the file to be accessed includes the file storage cluster where the file data of the file to be accessed is located. Figure 2As shown in the structural diagram of the file storage system, a plurality of file storage clusters are arranged in the file storage system, corresponding to the Cluster in the figure; the metadata of the above-mentioned file to be accessed includes the address of the file storage cluster where the file data of the file to be accessed is located, namely, the cluster id.
[0055] like Figure 3 As shown, the method comprises the following steps:
[0056] Step S302, obtaining the correspondence between the file directory management slice and the data volume through the data persistence service;
[0057] For example, data persistence services can be services such as Zookeeper (distributed application coordination service), which can be used to persist IP (interconnection protocol between networks) information, data volumes and other information of file directory management nodes, cluster management nodes and data storage nodes in real time; data persistence services are usually set up in file storage systems; data persistence services are used to provide the correspondence between file directory management shards and data volumes; this correspondence is usually obtained by persisting the information of each file directory management node, cluster management node and data storage node in real time. For example, when the data of the file directory management node, cluster management node and data storage node is changed, the changed data will be persisted in real time to ensure that the data in the data persistence service is the latest data.
[0058] Specifically, when the system starts, the system server (corresponding to Figure 2 The Client in the system accesses the data persistence service once, obtains the correspondence between the file directory management slice and the data volume from the data persistence service, and saves the correspondence locally on the server. It should be noted that the server of the system only accesses the data persistence service once when the system is started, and saves the required correspondence between the file directory management slice and the data volume, as well as other related information locally. Avoid frequent access to the data persistence service during the data access process.
[0059] Step S304, based on the data access request, determining the first target data volume to which the file to be accessed belongs;
[0060] Step S306, based on the correspondence between the file directory management slices and the data volumes in the file storage system, determining the file directory management slice to which the first target data volume belongs;
[0061] Step S308, obtaining metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs;
[0062] Step S310: Obtain the file to be accessed from the file storage cluster to which the file to be accessed belongs.
[0063] Since the metadata of the file to be accessed includes the address of the file storage cluster where the file data of the file to be accessed is located, the corresponding file storage cluster can be directly accessed according to the address of the file storage cluster; and the file to be accessed is obtained through the cluster management node in the file storage cluster.
[0064] The above-mentioned file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node. The cluster management node mainly manages the mapping relationship between the file metadata and the file content (i.e., which pieces of content the file metadata consists of, and which data storage nodes each piece of content is distributed on); the data storage node is used to store the file data uploaded by the user, and each file is horizontally cut into multiple fixed-length blocks.
[0065] A possible implementation of the above step S310 is: determining the target data storage node where the file data of the file to be accessed is located through the cluster management node of the file storage cluster to which the file to be accessed belongs; and obtaining the file data of the file to be accessed from the target data storage node.
[0066] Specifically, the target data storage node corresponding to the file data of the file to be accessed can be determined through the correspondence between the file data of each file in the main cluster management node of the file storage cluster to which the file to be accessed belongs and the data storage node; after obtaining the address of the target data storage node, the file data of the file to be accessed can be directly obtained from the target data storage node.
[0067] Each file storage cluster includes multiple cluster management nodes; the multiple cluster management nodes include a primary cluster management node and at least one backup cluster management node; wherein the primary cluster management node is used to provide a target data storage node where the file data of the file to be accessed is located; the backup cluster management node synchronizes data with the primary cluster management node.
[0068] For details, see Figure 2 The structural diagram of the file storage system shown in FIG. 1 ; each file storage cluster usually includes three or five cluster management nodes, one of which is the main cluster management node and the rest are backup cluster management nodes; the cluster management node corresponds to Figure 2 MetaNode in the cluster; the main cluster management node corresponds to Figure 2 MasterMetaNode in the standby cluster management node Figure 2 The Slave MetaNode in the data storage node corresponds to Figure 2Each file storage cluster may include multiple data storage nodes; the data of the above multiple cluster management nodes may be completely mirrored. The above cluster management node mainly manages the distribution of all data in the file storage cluster and the mapping relationship between the metadata of the file and the file content.
[0069] It should be noted that the file storage clusters included in the file storage system are not limited to the number shown in the figure, but can be more, and there is no restriction here. The specific number can be set according to actual needs. In addition, the cluster management nodes are not limited to the number shown in the figure, but can be more, usually including three or five, and there is no restriction here. The specific number can be set according to actual needs; similarly, the data storage nodes are not limited to the number shown in the figure, but can be more or less, and there is no restriction here.
[0070] In addition, the active cluster management node among the multiple cluster management nodes is determined based on a preset raft protocol; the data of the standby cluster management node is synchronized with the data of the active cluster management node through the preset raft protocol.
[0071] Usually, in the initial state, there is no distinction between the primary and backup cluster management nodes in each file storage cluster. A primary cluster management node is selected from multiple cluster management nodes through the election mechanism of the raft protocol, and the remaining cluster management nodes directly become backup cluster management nodes. In this way, after the primary cluster management node hangs up, a new primary cluster management node can be selected from the backup cluster management nodes within 100 milliseconds and provide services; and due to the consistency algorithm of the raft protocol, the data of the new primary cluster management node is the same as the data of the hung primary cluster management node, ensuring that after the primary cluster management node of the file storage system hangs up, a new primary cluster management node is selected within 100 milliseconds and provides services to the outside world, and the data of the new primary cluster management node is completely consistent with the data of the old primary cluster management node, and no data will be lost. At the same time, it solves the storage capacity bottleneck problem caused by the file storage system being limited by the inability to expand horizontally in a single cluster.
[0072] In the structure of the above-mentioned file storage system, the cluster management nodes and data storage nodes are divided into multiple file storage clusters, and the file storage clusters can be expanded horizontally seamlessly; this solves the storage capacity bottleneck problem caused by the file storage system being limited by the inability of a single cluster to expand horizontally, and increases the number of files stored in the system and the total amount of data.
[0073] Since the typical and widely used open source file storage system HDFS has many defects, the most important of which is that the size of stored files is limited, all file metadata is stored in the NameNode node, and the NameNode node has no horizontal expansion capability, resulting in the number of files and the total amount of data that the cluster can store being limited by the memory bottleneck of the NameNode node. Generally, only hundreds of millions of files can be stored. Based on this, an embodiment of the present invention also provides a data storage method, such as Figure 4 As shown, the method comprises the following steps:
[0074] Step S402, based on the data storage request, determining a second target data volume to which the file to be stored belongs;
[0075] The second target data volume may have the same attributes as the aforementioned first target data volume, and may be a namespace for storing directories and files created by users.
[0076] Step S404, based on the correspondence between the file directory management slice and the data volume in the file storage system, determine the file directory management slice to which the second target data volume belongs; wherein the file storage system is provided with a plurality of file directory management slices; each file directory management slice is used to manage metadata of files in a specified data volume;
[0077] Step S406, determining a target storage location for the file to be stored; based on the target storage location, generating metadata for the file to be stored in the file directory management slice to which the second target data volume belongs; and storing the file to be stored in the target storage location.
[0078] The above metadata can also be called inode id, which is the unique identifier of the file to be stored. The metadata can be defined as data describing the file, descriptive information about the file and information resources. The target storage location of the file to be stored can be determined by the file storage system, and the target storage location of the file to be stored can be recorded at the same time to facilitate subsequent user access.
[0079] The above-mentioned file storage system is provided with multiple file storage clusters; the target storage location includes: a target file storage cluster storing file data of the file to be stored; the step of storing the file to be stored in the target storage location includes: storing the file to be stored in the target file storage cluster.
[0080] The above-mentioned file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; the step of storing the file to be stored in the target file storage cluster includes:
[0081] 1) Determine a target data storage node storing the file to be stored from the target file storage cluster;
[0082] Specifically, the target data storage node storing the file to be stored may be determined from the cluster management node of the target file storage cluster.
[0083] 2) Based on the target data storage node, in the cluster management node of the target file storage cluster, a correspondence between the file data to be stored and the target data storage node is generated; and the file data of the file to be stored is saved to the target data storage node.
[0084] The correspondence between the file data of the file to be stored and the target data storage node is generated to facilitate subsequent users to access the file data of the file to be stored.
[0085] In the above method, based on the data storage request, the second target data volume to which the file to be stored belongs is determined; based on the correspondence between the file directory management slice and the data volume in the file storage system, the file directory management slice to which the second target data volume belongs is determined; wherein, a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage the metadata of the files in the specified data volume; the target storage location of the file to be stored is determined; based on the target storage location, the metadata of the file to be stored is generated in the file directory management slice to which the second target data volume belongs; the file to be stored is stored in the target storage location. In this method, a plurality of file directory management slices are provided in the file storage system, and when the file volume is large, the file target management slice can be expanded horizontally, avoiding the problem of the limited number of metadata of the files stored in a single file directory management node, and increasing the number of files stored in the system and the total amount of data.
[0086] This embodiment also provides a file storage system, such as Figure 5 As shown, the system includes a data request server, multiple file directory management slices and multiple file storage clusters; wherein each file storage cluster is used to store files in a specified data volume; each file directory management slice is used to manage metadata of files in the specified data volume; metadata is used to indicate the file storage cluster where the files in the specified data volume are located; the data request server is used to process received data requests based on the correspondence between the file directory management slices and the data volumes.
[0087] The above-mentioned file storage system is provided with a data persistence service; the data persistence service is used to provide the corresponding relationship between the file directory management slice and the data volume to the data request server.
[0088] Among them, each file directory management slice includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of files in a specified data volume; the backup file directory management node synchronizes data with the primary file directory management node.
[0089] The above-mentioned file storage cluster includes a cluster management node and a data storage node; the data storage node is used to store the file data of each file in the file storage cluster; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node.
[0090] The above-mentioned file storage cluster includes multiple cluster management nodes; the multiple cluster management nodes include a primary cluster management node and at least one backup cluster management node; wherein the primary cluster management node is used to provide a data storage node where the file data of each file in the file storage cluster is located; the backup cluster management node synchronizes data with the primary cluster management node.
[0091] The file storage system provided in this embodiment includes a data request server, multiple file directory management slices and multiple file storage clusters; wherein each file storage cluster is used to store files in a specified data volume; each file directory management slice is used to manage metadata of files in a specified data volume; metadata is used to indicate the file storage cluster where the files in the specified data volume are located; the data request server is used to process received data requests based on the correspondence between the file directory management slice and the data volume. In this method, multiple file directory management slices are set in the file storage system. When the file volume is large, the file target management slice can be expanded horizontally, avoiding the problem of limited number of metadata of files stored in a single file directory management node, and increasing the number of files stored in the system and the total data volume.
[0092] Corresponding to the above method embodiment, this embodiment provides a data access device, such as Figure 6 As shown, the device comprises:
[0093] A first data volume determination module 61, configured to determine, based on a data access request, a first target data volume to which a file to be accessed belongs;
[0094] The first management slice determination module 62 is used to determine the file directory management slice to which the first target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with a plurality of file directory management slices; each file directory management slice is used to manage metadata of files in a specified data volume;
[0095] The file access module 63 is used to obtain metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and access the file to be accessed based on the metadata of the file to be accessed.
[0096] An embodiment of the present invention provides a data access device, which determines the first target data volume to which the file to be accessed belongs based on a data access request; determines the file directory management slice to which the first target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with multiple file directory management slices; each file directory management slice is used to manage the metadata of the files in the specified data volume; obtains the metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and accesses the file to be accessed based on the metadata of the file to be accessed. In this method, the file storage system is provided with multiple file directory management slices, and when the file volume is large, the file target management slice can be expanded horizontally, avoiding the problem of the limited number of metadata of the files stored in a single file directory management node, and increasing the number of files stored in the system and the total data volume.
[0097] Furthermore, each of the above-mentioned file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of the file to be accessed; the backup file directory management node synchronizes data with the primary file directory management node.
[0098] Furthermore, the primary file directory management node among the above-mentioned multiple file directory management nodes is determined based on a preset raft protocol; and the data of the above-mentioned standby file directory management node is synchronized with the data of the primary file directory management node through a preset raft protocol.
[0099] Furthermore, the file storage system is provided with a plurality of file storage clusters; the metadata of the file to be accessed includes the file storage cluster where the file data of the file to be accessed is located; and the file access module is further used to obtain the file to be accessed from the file storage cluster to which the file to be accessed belongs.
[0100] Furthermore, the above-mentioned file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; the above-mentioned file access module is also used to: determine the target data storage node where the file data of the file to be accessed is located through the cluster management node of the file storage cluster to which the file to be accessed belongs; and obtain the file data of the file to be accessed from the target data storage node.
[0101] Furthermore, each of the above-mentioned file storage clusters includes multiple cluster management nodes; the multiple cluster management nodes include a primary cluster management node and at least one backup cluster management node; wherein the primary cluster management node is used to provide a target data storage node where the file data of the file to be accessed is located; the backup cluster management node synchronizes data with the primary cluster management node.
[0102] Furthermore, a data persistence service is provided in the above-mentioned file storage system; the data persistence service is used to provide the correspondence between the file directory management slices and the data volumes; the above-mentioned device also includes an acquisition module, which is used to obtain the correspondence between the file directory management slices and the data volumes through the data persistence service.
[0103] The data access device provided in the embodiment of the present invention has the same technical features as the data access method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.
[0104] Corresponding to the above method embodiment, this embodiment also provides a data storage device, such as Figure 7 As shown, the device comprises:
[0105] A second data volume determination module 71, configured to determine, based on a data storage request, a second target data volume to which the file to be stored belongs;
[0106] The second management slice determination module 72 is used to determine the file directory management slice to which the second target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with a plurality of file directory management slices; each file directory management slice is used to manage metadata of files in a specified data volume;
[0107] The file storage module 73 is used to determine the target storage location of the file to be stored; based on the target storage location, generate metadata of the file to be stored in the file directory management slice to which the second target data volume belongs; and store the file to be stored in the target storage location.
[0108] In the above device, based on the data storage request, the second target data volume to which the file to be stored belongs is determined; based on the correspondence between the file directory management slice and the data volume in the file storage system, the file directory management slice to which the second target data volume belongs is determined; wherein, a plurality of file directory management slices are provided in the file storage system; each file directory management slice is used to manage the metadata of the files in the specified data volume; the target storage location of the file to be stored is determined; based on the target storage location, the metadata of the file to be stored is generated in the file directory management slice to which the second target data volume belongs; and the file to be stored is stored in the target storage location. In this method, a plurality of file directory management slices are provided in the file storage system. When the file volume is large, the file target management slice can be expanded horizontally, thereby avoiding the problem of the limited number of metadata of the files stored in a single file directory management node, and increasing the number of files stored in the system and the total amount of data.
[0109] Furthermore, the file storage system is provided with a plurality of file storage clusters; the target storage location includes: a target file storage cluster storing file data of the file to be stored; the file storage module is also used for: storing the file to be stored in the target file storage cluster.
[0110] Furthermore, the above-mentioned file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; the above-mentioned file storage module is also used to: determine the target data storage node for storing the file to be stored from the target file storage cluster; based on the target data storage node, generate the correspondence between the file data of the file to be stored and the target data storage node in the cluster management node of the target file storage cluster; and save the file data of the file to be stored to the target data storage node.
[0111] The data storage device provided in the embodiment of the present invention has the same technical features as the data storage method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.
[0112] This embodiment also provides an electronic device, including a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned data access party, data storage method or file storage system.
[0113] See also Figure 8 As shown, the electronic device includes a processor 100 and a memory 101, wherein the memory 101 stores machine executable instructions that can be executed by the processor 100, and the processor 100 executes the machine executable instructions to implement the above-mentioned data access method, data storage method or file storage system.
[0114] Further, Figure 8 The electronic device shown further includes a bus 102 and a communication interface 103 , and the processor 100 , the communication interface 103 and the memory 101 are connected via the bus 102 .
[0115] The memory 101 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used. The bus 102 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0116] The processor 100 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 100. The above processor 100 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 101, and the processor 100 reads the information in the memory 101 and completes the steps of the method of the above embodiment in combination with its hardware.
[0117] This embodiment also provides a machine-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the above-mentioned data access method, data storage method or file storage system.
[0118] The computer program products of the data access method, data storage method, device and file storage system provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be found in the method embodiments and will not be repeated here.
[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0120] In addition, in the description of the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0121] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0122] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0123] Finally, it should be noted that the above embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can still modify the technical solutions recorded in the above embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A data access method, It is characterized in that The method comprises: Based on the data access request, determining a first target data volume to which the file to be accessed belongs; Based on the correspondence between the file directory management slice and the data volume in the file storage system, determine the file directory management slice to which the first target data volume belongs; wherein the file storage system is provided with a plurality of file directory management slices; each of the file directory management slices is used to manage metadata of files in a specified data volume; and each data volume belongs to only one file directory management slice, and the correspondence between the file directory management slice and the data volume is a one-to-one correspondence; Acquire metadata of the file to be accessed from a file directory management slice to which the first target data volume belongs; and access the file to be accessed based on the metadata of the file to be accessed; Each of the file directory management slices includes a plurality of file directory management nodes; the plurality of file directory management nodes include a primary file directory management node and at least one backup file directory management node; The primary file directory management node is used to provide metadata of the file to be accessed; the backup file directory management node is synchronized with the data of the primary file directory management node; A primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; The data of the standby file directory management node is synchronized with the data of the main file directory management node through a preset raft protocol.
2. The method according to claim 1, It is characterized in that The file storage system is provided with a plurality of file storage clusters; the metadata of the file to be accessed includes the file storage cluster where the file data of the file to be accessed is located; The step of accessing the file to be accessed based on the metadata of the file to be accessed includes: acquiring the file to be accessed from a file storage cluster to which the file to be accessed belongs.
3. The method according to claim 2, It is characterized in that The file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; The step of obtaining the file to be accessed from the file storage cluster to which the file to be accessed belongs includes: Determine, by means of a cluster management node of a file storage cluster to which the file to be accessed belongs, a target data storage node where the file data of the file to be accessed is located; The file data of the to-be-accessed file is obtained from the target data storage node.
4. The method according to claim 3, It is characterized in that Each of the file storage clusters includes a plurality of cluster management nodes; the plurality of cluster management nodes include a primary cluster management node and at least one backup cluster management node; The active cluster management node is used to provide a target data storage node where the file data of the to-be-accessed file is located; and the standby cluster management node is synchronized with the data of the active cluster management node.
5. The method according to claim 1, It is characterized in that The file storage system is provided with a data persistence service; the data persistence service is used to provide a correspondence between the file directory management slice and the data volume; Before the step of determining the first target data volume to which the file to be accessed belongs based on the data access request, the method further includes: obtaining the corresponding relationship between the file directory management slice and the data volume through the data persistence service.
6. A data storage method, It is characterized in that The method comprises: Based on the data storage request, determining a second target data volume to which the file to be stored belongs; Based on the correspondence between the file directory management slice and the data volume in the file storage system, determine the file directory management slice to which the second target data volume belongs; wherein, the file storage system is provided with multiple file directory management slices; each of the file directory management slices is used to manage the metadata of the files in the specified data volume; and each data volume belongs to only one file directory management slice, and the correspondence between the file directory management slice and the data volume is a one-to-one correspondence; wherein, each of the file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide the metadata of the file to be accessed; the data of the backup file directory management node is synchronized with the data of the primary file directory management node; the primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; the data of the backup file directory management node is synchronized with the data of the primary file directory management node through a preset raft protocol; Determine a target storage location for the file to be stored; based on the target storage location, generate metadata for the file to be stored in a file directory management slice to which the second target data volume belongs; and store the file to be stored in the target storage location.
7. The method according to claim 6, It is characterized in that The file storage system is provided with a plurality of file storage clusters; the target storage location comprises: a target file storage cluster storing file data of the file to be stored; The step of storing the file to be stored in the target storage location includes: storing the file to be stored in the target file storage cluster.
8. The method according to claim 7, It is characterized in that The file storage cluster includes a cluster management node and a data storage node; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node; The step of storing the file to be stored in the target file storage cluster includes: Determine a target data storage node storing the file to be stored from the target file storage cluster; Based on the target data storage node, a correspondence between the file data of the file to be stored and the target data storage node is generated in the cluster management node of the target file storage cluster; and the file data of the file to be stored is saved to the target data storage node.
9. A file storage system, It is characterized in that The system includes a data request server, multiple file directory management slices and multiple file storage clusters; Wherein, each of the file storage clusters is used to store files in a specified data volume; each of the file directory management slices is used to manage metadata of files in a specified data volume; the metadata is used to indicate the file storage cluster where the files in the specified data volume are located; and each data volume belongs to only one file directory management slice, and the correspondence between the file directory management slice and the data volume is a one-to-one correspondence; wherein, each of the file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide metadata of files in the specified data volume; the backup file directory management node is synchronized with the data of the primary file directory management node; the primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; the data of the backup file directory management node is synchronized with the data of the primary file directory management node through a preset raft protocol; The data request server is used to process received data requests based on the correspondence between the file directory management slices and data volumes.
10. The system according to claim 9, It is characterized in that The file storage system is provided with a data persistence service; the data persistence service is used to provide the data request service end with the correspondence between the file directory management slice and the data volume.
11. The system according to claim 9, It is characterized in that The file storage cluster includes a cluster management node and a data storage node; The data storage node is used to store the file data of each file in the file storage cluster; the cluster management node stores the correspondence between the file data of each file stored in the file storage cluster and the data storage node.
12. The system according to claim 11, It is characterized in that The file storage cluster includes a plurality of cluster management nodes; the plurality of cluster management nodes include a primary cluster management node and at least one backup cluster management node; The active cluster management node is used to provide a data storage node where the file data of each file in the file storage cluster is located; and the standby cluster management node is synchronized with the data of the active cluster management node.
13. A data access device, It is characterized in that The device comprises: A first data volume determination module, used to determine a first target data volume to which the file to be accessed belongs based on the data access request; A first management slice determination module is used to determine the file directory management slice to which the first target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with multiple file directory management slices; each of the file directory management slices is used to manage the metadata of the files in the specified data volume; and each data volume belongs to only one file directory management slice, and the correspondence between the file directory management slice and the data volume is a one-to-one correspondence; wherein each of the file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide the metadata of the file to be accessed; the data of the backup file directory management node is synchronized with the data of the primary file directory management node; the primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; the data of the backup file directory management node is synchronized with the data of the primary file directory management node through a preset raft protocol; The file access module is used to obtain metadata of the file to be accessed from the file directory management slice to which the first target data volume belongs; and access the file to be accessed based on the metadata of the file to be accessed.
14. A data storage device, It is characterized in that The device comprises: A second data volume determination module, used to determine a second target data volume to which the file to be stored belongs based on the data storage request; A second management slice determination module is used to determine the file directory management slice to which the second target data volume belongs based on the correspondence between the file directory management slice and the data volume in the file storage system; wherein the file storage system is provided with multiple file directory management slices; each of the file directory management slices is used to manage the metadata of the files in the specified data volume; and each data volume belongs to only one file directory management slice, and the correspondence between the file directory management slice and the data volume is a one-to-one correspondence; wherein each of the file directory management slices includes multiple file directory management nodes; the multiple file directory management nodes include a primary file directory management node and at least one backup file directory management node; wherein the primary file directory management node is used to provide the metadata of the file to be accessed; the data of the backup file directory management node is synchronized with the data of the primary file directory management node; the primary file directory management node among the multiple file directory management nodes is determined based on a preset raft protocol; the data of the backup file directory management node is synchronized with the data of the primary file directory management node through a preset raft protocol; The file storage module is used to determine the target storage location of the file to be stored; based on the target storage location, generate metadata of the file to be stored in the file directory management slice to which the second target data volume belongs; and store the file to be stored in the target storage location.
15. An electronic device, It is characterized in that It includes a processor and a memory, the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the data access method described in any one of claims 1 to 5, the data storage method described in any one of claims 6 to 8, or the file storage system described in any one of claims 9 to 12.
16. A machine-readable storage medium, It is characterized in that The machine-readable storage medium stores machine-executable instructions, and when the machine-executable instructions are called and executed by the processor, the machine-executable instructions prompt the processor to implement the data access method described in any one of claims 1 to 5, the data storage method described in any one of claims 6 to 8, or the file storage system described in any one of claims 9 to 12.
Citation Information
Patent Citations
Cluster file system and file service method thereof
CN102158546A
Method for grouping space metadata based on cluster storage
CN102521386A
Distributed file system and data access method thereof
CN103647797A
Method and device for reading distributed metadata files
CN107315547A
Zookeeper backup management system and method based on salt
CN107357681A
