A storage method, device, system and computer storage medium

Abstract

The application discloses a storage method applied to an access device, and the method comprises the following steps: in the case that a storage request for a first file is received, an identification code is allocated to the first file; an index number of the first file is generated by using an existing parent directory of the first file and the identification code; an association between a name of the first file and the index number of the first file is stored; and the first file and the index number of the first file are sent to an index device. The application also discloses a device, a system and a computer storage medium, and the reliability of file storage is improved, and the storage efficiency of file storage is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to a file storage technology of a computer, and in particular to a storage method, device, system and computer storage medium. BACKGROUND

[0002] A distributed file system (DFS) can be used for data storage and management. At present, a common distributed file system is usually based on a single machine file system, and data is distributedly stored across servers by additionally adding a module, and is managed by sharing a same namespace.

[0003] Among various single machine file systems, a Zettabyte File System (ZFS) can cross a physical location of data due to the combination of file system and device management.

[0004] However, in a distributed file system, the ZFS cannot break through the limitation of the single machine file system, and still has many deficiencies in service availability. For example, it is usually difficult to provide services when a ZFS server is down or fails. SUMMARY

[0005] Therefore, the present application provides a storage method, device, system and computer storage medium to solve the technical problem that it is usually difficult to provide services when a ZFS server is down or fails in the prior art.

[0006] The technical scheme of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a storage method, which is applied to an access device and includes the following steps.

[0008] In a case where a storage request for a first file is received, an identification code is assigned to the first file;

[0009] An index number of the first file is generated by using an existing parent directory of the first file and the identification code;

[0010] An association between a name of the first file and the index number of the first file is stored;

[0011] The first file and the index number of the first file are sent to an index device;

[0012] The indexing device is used to, after determining the identifier of the storage node in the storage device used to store the first file, store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device.

[0013] Secondly, embodiments of the present invention provide a storage method, the method being applied in an indexing device, comprising:

[0014] Receive a first file and its index number from the access device; wherein the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device;

[0015] Determine the identifier of the storage node in the storage device used to store the first file;

[0016] The correspondence between the index number of the first file and the identifier of the storage node of the first file is stored in the first and second devices of the indexing device.

[0017] Thirdly, embodiments of the present invention provide an access device, including:

[0018] The allocation module is used to allocate an identification code to the first file upon receiving a storage request for the first file;

[0019] The generation module is used to generate an index number for the first file using the existing parent directory of the first file and the identifier code;

[0020] The first storage module is used to store the association between the name of the first file and the index number of the first file;

[0021] The sending module is used to send the first file and its index number to the indexing device;

[0022] The indexing device is used to, after determining the identifier of the storage node in the storage device used to store the first file, store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device.

[0023] Fourthly, embodiments of the present invention provide an indexing device, comprising:

[0024] The receiving module is used to receive a first file and the index number of the first file sent by the access device; wherein, the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device;

[0025] The determination module is used to determine the identifier of the storage node in the storage device used to store the first file;

[0026] The second storage module is used to store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device.

[0027] Fifthly, embodiments of the present invention also provide an access device, the access device comprising: a processor and a storage medium storing processor-executable instructions, the storage medium performing operations via a communication bus dependent on the processor, and when the instructions are executed by the processor, executing the storage method described in one or more of the above embodiments.

[0028] In a sixth aspect, embodiments of the present invention also provide an indexing device, the indexing device comprising: a processor and a storage medium storing processor-executable instructions, the storage medium performing operations dependent on the processor via a communication bus, wherein when the instructions are executed by the processor, the storage method described in one or more of the above embodiments is executed.

[0029] In a seventh aspect, embodiments of the present invention provide a computer storage medium storing executable instructions, wherein when the executable instructions are executed by one or more processors, the processors execute the storage method described in one or more of the above embodiments.

[0030] This invention provides a storage method, device, system, and computer storage medium. The method includes: upon receiving a storage request for a first file, an access device assigns an identifier to the first file; generates an index number for the first file using its existing parent directory and identifier; stores the association between the name of the first file and its index number; and sends the first file and its index number to an indexing device. The indexing device, after determining the identifier of the storage node in the storage device used to store the first file, stores the correspondence between the index number of the first file and the identifier of the storage node in the indexing device in both a first device and a second device. In other words, in this invention, upon receiving a storage request for a first file, the access device... When assigning an identifier to the first file, an index number for the first file is generated using its existing parent directory and the identifier. This allows the access device to store the association between the name and index number of each file, facilitating file reading and modification. Furthermore, the first file and its index number are sent to the indexing device. The indexing device can then determine the identifier of the storage node used to store the first file and associate the index number with the storage node identifier, storing this information in both the first and second indexing devices. In this way, if the first device fails, the second device can reconfigure the routing to ensure normal file storage and retrieval, thereby improving the reliability and efficiency of file storage. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of an optional file system structure in an embodiment of the present invention;

[0032] Figure 2 This is a flowchart illustrating an optional storage method in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the structure of an example of an optional file system in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of the architecture of an example of an optional file system in an embodiment of the present invention;

[0035] Figure 5 This is an optional catalog diagram in an embodiment of the present invention;

[0036] Figure 6 This is a flowchart illustrating an optional Raft algorithm in an embodiment of the present invention;

[0037] Figure 7 This is a flowchart illustrating an optional storage method in an embodiment of the present invention;

[0038] Figure 8 This is a flowchart illustrating another optional storage method in an embodiment of the present invention;

[0039] Figure 9 This is a schematic diagram of the structure of an optional access device in an embodiment of the present invention;

[0040] Figure 10 This is a schematic diagram of the structure of an optional indexing device in an embodiment of the present invention;

[0041] Figure 11 This is a schematic diagram of another optional access device in an embodiment of the present invention;

[0042] Figure 12 This is a schematic diagram of another optional indexing device in an embodiment of the present invention;

[0043] Figure 13 This is a schematic diagram of an optional file system structure in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] Example 1

[0046] This invention provides a storage method applied in a file system. Figure 1 This is a schematic diagram of an optional file system structure in an embodiment of the present invention, such as... Figure 1 As shown, the file system may include: an access device 11, an indexing device 12, and a storage device 13; wherein, the indexing device 12 includes two types of devices, namely a first device 121 and a second device 122; wherein,

[0047] Access device 11 and indexing device 12 have a communication connection. Access device 11 can be a user client or a gateway server, playing the role of a client in the file system. The first device 121 and the second device 122 of indexing device 12 are both computing servers. The first device 121 can be a ZFS server, and the second device 122 can be an Index server. The Index server is used to manage the routing of ZFS servers. Storage device 13 is usually a storage server used to store files.

[0048] Based on the above Figure 1 The present invention provides a storage method for a file system. Figure 2 This is a schematic diagram of an optional storage process interaction in an embodiment of the present invention, such as... Figure 2 As shown, the storage method may include:

[0049] S201: Upon receiving a storage request for the first file, the access device 11 assigns an identification code to the first file;

[0050] Currently, mainstream distributed file systems perform well in large file read / write operations. Performance bottlenecks are typically limited to massive small file read / write operations and index access, and the small file read / write problem can be attributed to index design issues. Mainstream distributed file system indexes generally fall into two categories: centralized indexes have scalability bottlenecks, while decentralized indexes suffer from basic performance limitations. Mainstream distributed file systems typically only address small file aggregation at the application layer or at the protocol access layer to alleviate bottlenecks in certain models.

[0051] Meanwhile, these distributed systems are hampered by traditional consensus algorithms (such as PAXOS or Multi-PAXOS), making it impossible to design efficient systems while implementing PAXOS semantics, and ultimately failing to prove the actual security of the designed systems. Split-brain scenarios and data loss are common in various mainstream distributed file systems in production environments, and fault self-healing to ensure eventual consistency is a regular occurrence. While the PAXOS algorithm itself possesses complete security provability and derivability, its biggest drawback is its difficulty in understanding and practical implementation. Its application in distributed file storage often requires significant additional overhead, ultimately failing to accurately guarantee security and necessitating further maintenance of eventual consistency.

[0052] To improve file storage efficiency, in this embodiment of the invention, after receiving a storage request for a first file, the access device 11 first assigns an identification code to the first file. Specifically, when the access device 11 receives the first file to be stored, it can generate an identification code for the first file according to a preset rule. For example, it can encode the file according to the file number, or it can number the file to be stored according to letters and numbers. Here, this embodiment of the invention does not make specific limitations on this.

[0053] S202: Access device 11 uses the existing parent directory and identifier code of the first file to generate an index number for the first file;

[0054] Specifically, after the access device 11 assigns an identifier to the first file, it can use the parent directory of the first file and the identifier of the first file to generate the index number of the first file. For example, if the identifier assigned to the first file is 345 and the parent directory of the first file is drive F, then the index number of the first file can be F / 345. In this way, the index number of the first file can be obtained.

[0055] S203: Access device 11 stores the association between the name of the first file and the index number of the first file;

[0056] After generating the index number of the first file, the name of the first file can be associated with the index number and stored. This ensures that when a file is read or lost, the file's index number can be found through the file's name, which is beneficial for reading or modifying files from the storage device.

[0057] S204: Access device 11 sends the name of the first file and the index number of the first file to the indexing device;

[0058] The indexing device typically includes multiple first devices and multiple second devices, and each second device manages multiple first devices. Therefore, in order to determine the device corresponding to the first file from among the first devices of the indexing device, in an optional embodiment, the above method may further include:

[0059] Access device 11 sends a request to the second device 122 to obtain routing information for the index device;

[0060] The second device 122 determines the routing information of the first device managed by the second device 122 as the routing information of the index device, and sends it to the access device 11;

[0061] Access device 11 determines the routing information of the index device received as the routing information of the determined index device.

[0062] Based on the routing information of the determined index device, the access device 11 determines the device corresponding to the first file from the first devices of the index device according to the existing parent directory of the first file;

[0063] Specifically, access device 11 sends a request to second device 122 to obtain routing information for the index device. Here, it should be noted that second device 122 refers to all second devices in the index device. Each second device is connected to at least two first devices for management. After receiving the request, second device 122 sends the routing information of the first devices under each second device as the routing information of the index device to access device 11. In this way, access device 11 will receive the routing information of the index device and determine it as the routing information of the index device.

[0064] In this way, the routing information of the index device is determined by sending an acquisition request, so that the determined routing information of the index device is valid, thereby ensuring the normal read and write of the file system.

[0065] Access device 11 stores routing information of the identified index device. Since the routing information of the index device stores the first device corresponding to the parent directory, access device 11 can determine the first device corresponding to the parent directory of the first file from the routing information of the identified index device, and determine the first device corresponding to the parent directory of the first file as the device corresponding to the first file.

[0066] It should be noted that the routing information of the identified index device stored in access device 11 can be valid routing information or invalid routing information. In order to ensure that the routing information of the identified index device is valid routing information and to guarantee the normal read, write and modification of the storage device, any of the following methods can be adopted. For example, it can be determined whether the routing information is valid by comparing the version number of the routing information. In an optional embodiment, the above method may further include:

[0067] When the version number of the routing information received from the indexing device is the same as the version number of the routing information of the stored indexing device, the access device 11 determines the routing information of the stored indexing device as the routing information of the determined indexing device.

[0068] When the version number of the routing information received from the indexing device is different from the version number of the routing information already stored in the indexing device, the access device 11 determines the routing information received from the indexing device as the routing information of the determined indexing device.

[0069] In other words, the access device 11 compares the version number of the routing information received from the index device with the version number of the routing information of the index device that has been stored. If they are the same, it means that the routing information of the index device that has been stored in the access device 11 is valid routing information. Therefore, the access device 11 determines the routing information of the index device that has been stored as the routing information of the determined index device.

[0070] If they are different, it means that the routing information of the index device stored in the access device 11 is invalid routing information. Therefore, the routing information of the index device received by the access device is determined as the routing information of the determined index device.

[0071] This ensures that the routing information of the identified indexing device is valid, preventing problems caused by access devices using invalid routing information.

[0072] After the access device 11 determines the index number of the first file, it sends the first file and the index number of the first file to the indexing device, so that the indexing device can store the correspondence between the index number of the first file and the identifier of the storage node in the storage device after determining the identifier of the storage node used to store the first file in the storage device.

[0073] It should be noted that S203 and S204 can be executed either S203 first, S204 first, or both S203 and S204 can be executed simultaneously. This application embodiment makes specific limitations on this.

[0074] S205: Index device 12 determines the identifier of the storage node in the storage device used to store the first file;

[0075] The aforementioned storage nodes may include primary nodes and backup nodes.

[0076] Specifically, after obtaining the first file and its index number, the first device of the indexing device 12 selects the identifier of the storage node used to store the first file from the available storage nodes of the storage device. The storage node may include a primary node and a backup node. It should be noted that, typically, there is one primary node and two or three backup nodes. However, this embodiment of the invention does not impose any specific limitations on this.

[0077] S206: The indexing device 12 stores the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first device and the second device of the indexing device 12;

[0078] Specifically, the first indexing device, which is the device that accompanies the first file determined from the first indexing device, needs to store the correspondence between the index number of the first file and the identifier of the primary node and the identifier of the backup node of the first file. In this way, when a user needs to read the stored file, he / she can read the file through the correspondence.

[0079] Furthermore, the first device also needs to send the aforementioned correspondence to the second device that manages the first device, so that the second device stores the correspondence. This prevents the inability to read or write files when a first device fails, thus ensuring the reliability of the file system.

[0080] To improve the consistency of reading and writing files in the file system, in one optional embodiment, the above method further includes:

[0081] When the device corresponding to the first file receives the reply message from the storage device 13 indicating that the first file has been successfully written, the device corresponding to the first file sends a reply message to the access device 11 indicating that the first file has been successfully written.

[0082] When the access device 11 receives a response message indicating that the first file has been successfully written, it generates a prompt message; the prompt message is used to notify the user that the first file has been successfully stored.

[0083] In other words, if the device corresponding to the first file successfully stores the first file to the storage device 13, the storage device 13 will send a reply message to the device corresponding to the first file indicating that the first file has been successfully written. After receiving the reply message, the device corresponding to the first file will send a response message indicating that the first file has been successfully stored to the access device 11, so that the access device 11 knows that the first file has been successfully stored. After knowing that the storage has been successful, the access device 11 will also generate a prompt message to remind the user that the first file has been successfully stored.

[0084] When a write operation fails, storage device 13 will not send a reply message indicating that the first file was successfully written to the device corresponding to the first file, and the device corresponding to the first file will not send a message to access device 11. In this way, when access device 11 does not receive a reply message indicating that the first file was successfully written within a certain period of time, it determines that the storage of the first file has failed. At this time, it will prompt the user that the storage of the first file has failed. Then, the file system does not store the first file. In this way, the consistency of data in the file system can be guaranteed through the response between access device 11 and index device 12.

[0085] In addition to storing files, the file system can also read stored files. In one optional embodiment, the above method may further include:

[0086] Access device 11 receives a request to read the stored second file;

[0087] Access device 11 determines the index number of the second file based on the association between the name of the stored file and the index number of the file;

[0088] Based on the routing information of the determined index device, the access device 11 determines the device corresponding to the second file from the first device of the index device according to the existing parent directory of the second file;

[0089] Access device 11 sends the index number of the second file to the device corresponding to the second file;

[0090] The device corresponding to the second file determines the storage node of the second file based on the correspondence between the index number of the stored file and the identifier of the storage node of the file;

[0091] The device corresponding to the second file reads the second file from the storage node of the second file.

[0092] In order to read the stored second file, the user sends a read request for the second file to the access device 11 through the interface of the access device 11. Since the access device 11 stores the association between the file name and the file index number, the access device 11 can determine the index number of the second file.

[0093] Then, from the routing information of the determined index device, the first device corresponding to the existing parent directory of the second file is determined, and it is identified as the device corresponding to the second file. The index number of the second file is then sent to the device corresponding to the second file.

[0094] Since the device corresponding to the second file stores the correspondence between the file's index number and the identifier of the file's storage node, the device corresponding to the second file determines the storage node of the second file based on this correspondence and reads the second file from the storage node.

[0095] In addition to storing files, the file system can also modify stored files. In one optional embodiment, the above method may further include:

[0096] When the data of the master node in the storage node of the stored third file changes, the access device 11 determines the index number of the third file based on the association between the name of the stored file and the index number of the file.

[0097] Based on the routing information of the determined index device, the access device 11 determines the device corresponding to the third file from the first device of the index device according to the existing parent directory of the third file;

[0098] Access device 11 sends the modified third file and the index number of the third file to the device corresponding to the third file.

[0099] The device corresponding to the third file determines the backup node in the storage node of the third file based on the correspondence between the index number of the stored file and the identifier of the storage node of the file.

[0100] The device corresponding to the third file is used to determine the backup node in the storage node of the third file based on the index number of the third file. The backup node of the third file is used to update the stored data to the modified third file.

[0101] In order to modify the stored third file, the user modifies the data of the master node of the third file through the access device 11. Since the access device 11 stores the association between the file name and the file index number, the access device 11 can determine the index number of the third file.

[0102] Then, the first device corresponding to the existing parent directory of the third file is determined from the routing information of the determined index device, and it is identified as the device corresponding to the third file. The index number of the third file is then sent to the device corresponding to the third file.

[0103] Since the device corresponding to the third file stores the correspondence between the file's index number and the file's storage node identifier, the device corresponding to the third file determines the backup node of the third file based on this correspondence. The demarcation point of the third file then uses the modified third file to update the data in the backup node of the third file.

[0104] In order to ensure the normal operation of the file system when a row file system fails, in an optional embodiment, the above method may further include:

[0105] When one of the first devices managed by the second device fails, the second device selects a first device from the other first devices managed by the second device.

[0106] The second device sends the mapping relationship between the index number of a file stored in one of the first devices and the identifier of the file's storage node to the selected first device for storage, and updates the routing information of the index device.

[0107] In other words, for a certain second device, when one of the first devices managed by the second device fails, in order to ensure the normal operation of the file system, the second device first selects a first device from the other first devices managed by the second device, and sends the pre-stored correspondence between the index number of the file stored in one of the first devices and the identifier of the file's storage node to the selected first device for storage.

[0108] This allows the file system to promptly utilize other working first devices when the first device fails, ensuring the file system can handle unexpected situations, increasing its reliability, and improving its read / write efficiency.

[0109] The following examples illustrate the storage methods described in one or more of the above embodiments.

[0110] Figure 3 This is a schematic diagram of the structure of an example of an optional file system in an embodiment of the present invention, such as... Figure 3 As shown, the file system may include: an access layer, an index layer, and a storage layer;

[0111] The access layer can be a user client or a gateway server (Posix File System Client), the index layer consists of an index cluster, including two types of computing servers, namely ZFSServer and IndexServer, and the storage layer consists of a storage cluster, including at least two storage servers (ZFS zpool).

[0112] It's important to note that the three-tier architecture described above is not an absolute 1:1:1 relationship. The storage cluster provides storage resources to the index cluster, and the interface depends entirely on the level of isolation for physical or logical fault domains. For example, multiple index clusters can share the resources of a single storage cluster, and these index clusters can also provide services to the same access gateway.

[0113] A file writing process:

[0114] The first step is for the devices in the access layer to assign an Inode number (equivalent to an identifier) ​​in the Distributed File System (DZFS) for the file to be written.

[0115] The second step involves the device in the access layer converting the parent directory of the file to be written (including the DZFS root directory and the directory used to divide the namespace) into prefix_id, and its own Inode number in DZFS into file_id; the two ids are concatenated to form the index number of the file to be written, and the file to be written is associated with the index number of the file to be written.

[0116] Third, because the access layer caches routes, if the route is not invalid, the device in the access layer can directly send the file to be written to the ZFSServer corresponding to the file; if the route is invalid, the device will retrieve the ZFSServer corresponding to the file from the IndexServer and send the file to that ZFSServer.

[0117] The fourth step is for ZFSServer to select a primary node and a redundant node (equivalent to a backup node) from the idle nodes in the storage layer, and write files to both the primary node and the redundant node simultaneously.

[0118] The fifth step involves the primary node and redundant nodes jointly writing the data, then replying to ZFSServer that the write was successful.

[0119] The sixth step is for ZFSServer to associate the index number of the file being written with the identifier of the primary node and the identifier of the redundant node, and then store the association.

[0120] Step 7: ZFSServer replies that the access layer has completed the write operation, and the access layer replies to the user that the write operation has completed.

[0121] A file reading process:

[0122] Step 1: The device in the access layer obtains the name of the file to be read, determines the index number of the file to be read, and sends the index number of the file to the corresponding ZFSServer;

[0123] The second step is for ZFSServer to determine the identifier of the primary node and / or the identifier of the redundant node for reading the file based on the index number of the file being read, and then read the file from the ZFS zpool.

[0124] Figure 4 This is a schematic diagram of the architecture of an optional file system instance in an embodiment of the present invention, such as... Figure 4 As shown, the access layer is responsible for parsing file semantics, transforming the semantics of application operations on files and directories into semantics recognizable by the DZFS backend index and storage layers. In public cloud environments, it is also responsible for multi-tenant isolation, etc. This layer is usually stateless, and the industry generally supports NFSv4 interfaces and SMB2.0, etc.

[0125] The index layer is responsible for mimicking the directory and file concepts of the local file system in a distributed architecture, typically shared by multiple tenants. The advantages of this architecture are:

[0126] Separating indexes and data allows for a more efficient use of the central processing unit (CPU-Intensive), while data is handled by enhanced input / output (IO-Intensive). Their different hardware requirements enable cost-effective deployments on targeted hardware, separate scaling, and independent architectural evolution. Mixing storage can lead to issues where data consumes little storage while indexes are heavily utilized, necessitating expansion and ultimately reducing overall resource utilization.

[0127] Reduce index and IO interference: Metadata operations actually account for the majority of the entire file system operations. For example, HEAD requests used by Content Delivery Network (CDN) to retrieve data from the origin do not need to access data in most cases. This is especially evident in multi-tenant scenarios. If indexes and data coexist, index-intensive applications can easily interfere with IO, causing both to degrade in performance.

[0128] It is more suitable for supporting elastic architecture: For example, in scenarios where indexes and data coexist, it is difficult to represent a very large file (e.g., PB level), while in a separate architecture, data scheduling and other operations can be performed more conveniently according to the data access model.

[0129] The storage layer is crucial for ensuring high data reliability and availability. It must also consider the IO access patterns required by business systems, particularly the need for efficient implementation of random IO for file storage. ZFS, already a single-machine file storage system, combines top-tier reliability and availability with comprehensive disk management. Therefore, the storage layer design prioritizes distributed availability and consistency.

[0130] The following describes the expansion of each layer:

[0131] For the access layer, there are two main points: the protocol layer performs semantic conversion of the protocol, such as the NFSv4 or SMB2.0 protocol mentioned above; communication between the access layer, index layer and storage layer is completed through the general remote procedure call (RPC) framework.

[0132] For the index layer, the directory and file indexes are implemented separately, referred to as DirIdx and FileIdx. These two parts are also stored separately for index data persistence.

[0133] DirIdx is managed by IndexServer and relies on an external database for persistent storage. Even in scenarios with a large number of small files, DirIdx does not appear to be large in size. It can be efficiently accessed on an external distributed database (Distribute DB, Distribute Database). Furthermore, the following text will explain that the access layer will not access IndexServer's indexes very frequently.

[0134] FileIdx is managed by ZFSServer and persisted by ZFS itself on each storage node. Meanwhile, IndexServer manages the first-level index of files up to the parent directory (including the root directory of the namespace).

[0135] In DZFS, each file is assigned a unique number in its namespace (equivalent to the innode number in traditional file storage). DZFS uses the file's parent directory (including the namespace root directory) number as prefix_id and the file's own number as file_id. Prefix_id / file_id serves as a unique key value for indexing throughout the entire namespace.

[0136] DirIdx only describes the basis for the existence of a directory entry and is used as the basis for real-time allocation of subfiles under the directory entry. After a directory is created on the client, DirIdx will be recorded in the IndexServer's database. The primary key is the unique ID of the directory entry, and the database entry will contain the real-time routing of its subfiles. In simpler terms, newly created directories will be assigned by the IndexServer to different ZFSServers. Once the ZFSServer is confirmed during normal cluster operation, client operations on the directory itself and its subfiles will be directly communicated with the ZFSServer.

[0137] Figure 5 This is an optional catalog diagram in an embodiment of the present invention, such as... Figure 5 As shown, the directory structure includes / dir0 / dirx / , dir0 / diry / , / dir0 / diry / dileα / , / dir1 / , and / dir2 / file0 / . Multiple subdirectories are distributed across multiple ZFS Servers for processing. Based on the routing design using the unique ID in DirIdx as the key, it's clear that clients reading and writing files can obtain the prefix_id. The route can then be retrieved using the prefix_id, ultimately achieving the goal of data distribution.

[0138] A more detailed description of the index layer:

[0139] Node discovery: The lease heartbeat between IndexServer and ZFSServer is initiated by IndexServer. This allows IndexServer to avoid exposing its services to ZFSServer, instead relying on external service discovery applications such as ZooKeeper to ensure node discovery. Therefore, IndexServer needs to keep a list of ZFSServers that are allowed to provide services within the current cluster, and only ZFSServers on this list are allowed to provide services.

[0140] Route maintenance: Under normal circumstances, the IndexServer needs to ensure that a leaf directory is served by only one ZFSServer. This requires maintaining the correctness of the routing table through mechanisms such as leases. The routes obtained by the client will only fall into two categories: One is correct routing: usually, the routes haven't changed, and the routing table maintained by the client is the same as that maintained by the IndexServer. The other is expired routing: the IndexServer has rescheduled the DirIdx routes, causing the routing table maintained by the client to expire.

[0141] After the client initially obtains the route to a subdirectory, it doesn't need to periodically communicate with the IndexServer to obtain the latest route. This is because route expiration is checked during routine communication with the ZFSServer, thus keeping the IndexServer under minimal load. Furthermore, since each subdirectory route entry only identifies a single directory's route to the ZFSServer, the pressure on client caching is extremely low. The ZFSServer and IndexServer implement a lease to prevent any ZFSServer from voluntarily retiring after a network partition.

[0142] ZFSServer is a core module in the critical I / O chain. Through interaction with the underlying storage, it writes data to the storage nodes. The key feature of ZFSServer's architecture is its statelessness. It achieves client file and file attribute persistence through the underlying storage layer module and ZFS, ensuring high data reliability.

[0143] Route scheduling: IndexServer only performs route scheduling in the following situations: One situation is directory splitting: If a single subdirectory has excessively high popularity and a large number of subfiles, the directory will split. This is done by splitting the directory using the same semantics as the client and changing the file key to prefix_id / num / file_id to achieve distribution. Popularity and usage are reported by ZFSServer via lease heartbeats. Another situation is directory migration: If the overall popularity and usage of ZFSServers are unbalanced, the directory route will be migrated to an idle ZFSServer (including newly added ZFSServers). This is also reported via lease heartbeats. A third situation is ZFSServer service failure: This is caused by the ZFSServer itself exiting via lease heartbeat reporting, or by client timeout triggering scheduling. Other situations include network failures, network partitions, etc.

[0144] IndexServer High Availability: Since the access pressure on the IndexServer is very low, the master-slave structure is also implemented through external services such as ZooKeeper. Only the master IndexServer can update the database, while the slave IndexServer only has the right to query the database to maintain its own cache space to facilitate rapid switching in case of anomalies.

[0145] Additional services: File system additional services such as QUOTA, QOS, snapshots and other tenant functions can all be categorized into setting the attributes of directory files within the distributed file system. The design of extracting the directory index separately in this embodiment of the invention can realize these functions.

[0146] In other words, the database index layer, using a unique ID as the primary key, also achieves high efficiency for lists (lists with a range of sizes). In summary, this implementation satisfies the goals of index separation and reduced index I / O interference.

[0147] For the storage layer, the ZFS single-machine file system already achieves the high efficiency of random I / O and high reliability required for file storage. The storage layer still needs additional modules to achieve high availability. Distributed file systems typically use redundancy to ensure this, hence the introduction of Raft as a consistency algorithm for redundancy scenarios.

[0148] The DirIdx indexes a single ZFSServer during indexing. When writing subfiles based on this DirIdx, the ZFSServer selects a single-machine ZFS process in the storage layer. This directory is treated as a primary replica on this ZFS, and the ZFS process controls the initiation of replica writes and synchronization to other nodes. After the replica write is complete, this primary ZFS machine will reply to the ZFSServer indicating which nodes the file was replicated to. The ZFSServer indexes and records the replication nodes for this file in memory. Under normal circumstances, file reads only go through the primary replica.

[0149] The most important implementation of the storage layer is maintaining consistency between replicas through Raft. Raft is an algorithm for maintaining the consistency of a replicated state machine. It achieves consistency between replicas by modifying the ZFS transaction layer. Figure 6 This is a flowchart illustrating an optional Raft algorithm in an embodiment of the present invention, as shown below. Figure 6 As shown, thanks to ZFS being a transactional file storage system with an entry point for Raft access, and the copy-on-write (COW) storage method ensuring the idempotency of transaction log operations, it is relatively simple to transform the ZFS transaction processing at the storage layer into a distributed consistency system.

[0150] Redundant synchronous write:

[0151] like Figure 6 As shown,

[0152] ①ZFS commits transactions to Raft nodes.

[0153] ② Raft records the wal log to persist operations on the transaction space (e.g., x=1). After most nodes are persisted, the state machine can be committed to apply these operations.

[0154] ③ The Raft StateMachine transforms the commit state application operation and operates on the ZFS transaction log storage space (including cache). (The log space content, such as x:9; y:22; z:33, becomes x:1; y:22; z:33 after this operation).

[0155] ④⑤ ZFS's own transaction process uses a transaction log to guarantee the atomicity of file I / O.

[0156] Asynchronous snapshots save the ZFS transaction log. These snapshots are generated and retrieved under the control of Raft and are used for Raft wal log compaction to prevent the Raft wal log from growing indefinitely.

[0157] As can be seen, the entire process achieves final distributed consistency by ensuring atomicity through ZFS's own transaction flow and leveraging the algorithmic security provided by Raft.

[0158] Data migration: As described above regarding routing scheduling, data migration will be triggered regardless of the circumstances that cause the index layer DirIdx scheduling to occur. The ZFS Server to which the directory is newly scheduled can initiate data migration by indicating the expiration of the lease and route. However, due to the COW characteristic of ZFS, the overhead of data migration has a very small impact on cluster performance.

[0159] File indexing: FileIdx, along with file information and attributes, is maintained by the ZFS zpool itself. The flattened metadata storage using prefix_id / file_id delivers extremely high performance. With prefix_id / file_id as the primary identifier, files in the ZFSPool no longer need to be distinguished by a directory tree; instead, they are listed based on prefix_id, and indexed directly by prefix_id / file_id, which is highly efficient.

[0160] Additional services: Utilize the inherent features of ZFS to provide additional services to the prefix_id list, which is extremely efficient.

[0161] As can be seen, based on this three-layer architecture design, the DZFS system can shield against single-module and single-node failures.

[0162] For example, if a ZFS zpool, a storage node in the storage layer, fails, it will first be detected by all its consumer ZFSServers. The ZFSServers can then modify all currently maintained file indexes, making other ZFS zpool nodes the primary nodes, to continue providing normal service. Next, the ZFSServers will send notification messages to these new primary nodes, instructing them to re-elect a secondary node to store the affected files (e.g., restoring from two remaining replicas to three replicas).

[0163] If an index layer ZFSServer fails, the IndexServer will detect it and redistribute the routes it maintains (DirIdx) to the unfailed ZFSServer nodes. The ZFSServer only needs to access all storage layer nodes to immediately gather the indexes of all files that should be maintained, using the prefix_id method as described above.

[0164] If the IndexServer fails, services such as ZooKeeper will detect it, notify the slave IndexServer to switch to master, and retrieve the latest route allocation from the database Query to continue maintaining the index layer route allocation.

[0165] Access layer client or gateway failures are actually managed by the user application itself. The proposed DZFS index layer obviously supports multiple client access, so the access layer can be built with multiple paths.

[0166] From a three-layer interface perspective, the access layer can allocate files to different index clusters based on namespaces, ensuring that a failure in one namespace will not affect another. Different index clusters can employ different redundancy settings, maintained by the IndexServer. For example, two index clusters using the same set of storage cluster physical resources can have different configurations: one can be configured with two-replica redundancy, while the other can have erasure coding redundancy, because the redundancy settings are included when sending writes to the storage layer.

[0167] The present invention provides a storage method comprising: upon receiving a storage request for a first file by an access device, assigning an identifier code to the first file; generating an index number for the first file using the existing parent directory and identifier code of the first file; storing the association between the name of the first file and the index number of the first file; and sending the first file and its index number to an indexing device. The indexing device, after determining the identifier of the storage node used to store the first file in the storage device, stores the correspondence between the index number of the first file and the identifier of the storage node in the indexing device in both a first device and a second device. In other words, in this invention, upon receiving a storage request, the access device assigns an identifier code to the first file. During the identification process, an index number for the first file is generated using its existing parent directory and identifier. This allows the access device to store the association between the file name and its index number, facilitating file reading and modification. Furthermore, the first file and its index number are sent to an indexing device. The indexing device then identifies the storage node used to store the first file and maps the first file's index number to this storage node identifier, storing this mapping in both the first and second indexing devices. This ensures that if the first device malfunctions, the second device can reconfigure the routing, guaranteeing normal file storage and retrieval, thus improving file storage reliability and efficiency.

[0168] Example 2

[0169] The above storage method is explained below using the various devices deployed in the file system as examples.

[0170] First, the storage method is described from the perspective of the access device.

[0171] This invention provides a storage method applied in an access device. Figure 7 This is a flowchart illustrating an optional storage method in an embodiment of the present invention, such as... Figure 7 As shown, the storage method may include:

[0172] S701: Upon receiving a storage request for the first file, assign an identifier code to the first file;

[0173] S702: Generate the index number of the first file using the existing parent directory and identifier code of the first file;

[0174] S703: Store the association between the name of the first file and the index number of the first file;

[0175] S704: Send the first file and its index number to the first device of the indexing device;

[0176] The indexing device is used to store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device after determining the identifier of the storage node used to store the first file in the storage device.

[0177] In an optional embodiment, the above method further includes:

[0178] Send a request to the second device to obtain routing information for the index device;

[0179] Received routing information for the indexing device from the second device;

[0180] When the version number of the routing information received from the indexing device is the same as the version number of the routing information of the stored indexing device, the routing information of the stored indexing device is determined as the routing information of the determined indexing device.

[0181] When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, the received routing information of the indexing device is determined to be the routing information of the determined indexing device.

[0182] Based on the routing information of the determined indexing device, the device corresponding to the first file is determined from the first devices of the indexing device according to the existing parent directory of the first file.

[0183] In an optional embodiment, the above method further includes:

[0184] Upon receiving a response message from the device corresponding to the first file indicating that the first file has been successfully written, a prompt message is generated; the prompt message is used to notify the user that the first file has been successfully stored.

[0185] In an optional embodiment, the above method further includes:

[0186] A request to read the stored second file has been received.

[0187] The index number of the second file is determined based on the relationship between the names of the stored files and their index numbers.

[0188] Based on the routing information of the determined indexing device, the device corresponding to the second file is determined from the first device of the indexing device according to the existing parent directory of the second file;

[0189] Send the index number of the second file to the device corresponding to the second file;

[0190] The index number of the second file is used to: determine the storage node of the second file and read the second file by the device corresponding to the second file.

[0191] In an optional embodiment, the above method further includes:

[0192] When the data of the primary node in the storage node of the stored third file changes, the index number of the third file is determined according to the association between the name of the stored file and the index number of the file.

[0193] Based on the routing information of the determined indexing device, the device corresponding to the third file is determined from the first device of the indexing device according to the existing parent directory of the third file;

[0194] Send the modified third file and its index number to the device corresponding to the third file;

[0195] The device corresponding to the third file is used to determine the backup node in the storage node of the third file based on the index number of the third file. The backup node of the third file is used to update the stored data to the modified third file.

[0196] Then, the storage method is described from the perspective of the indexing device.

[0197] This invention provides a storage method applied in an indexing device. Figure 8 This is a flowchart illustrating another optional storage method in an embodiment of the present invention, such as... Figure 8 As shown, the method may include:

[0198] S801: Receive the first file and the index number of the first file sent from the access device;

[0199] The index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device.

[0200] S802: Determine the identifier of the storage node in the storage device used to store the first file;

[0201] S803: Store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the index device.

[0202] In an optional embodiment, the above method further includes:

[0203] When a reply message is received from the storage device indicating that the first file has been successfully written, a response message indicating that the first file has been successfully written is sent to the access device; wherein, the response message is used by the access device to notify the user that the first file has been successfully stored.

[0204] In an optional embodiment, the above method further includes:

[0205] When the access device receives a read request for the stored second file, the first device receives the index number of the second file;

[0206] The first device determines the storage node of the second file based on the correspondence between the index number of the stored file and the identifier of the storage node of the file;

[0207] The first device reads the second file from the storage node of the second file.

[0208] In an optional embodiment, the above method further includes:

[0209] When the data of the master node in the storage node of the stored third file is modified, the first device receives the modified third file and the index number of the third file sent from the access device;

[0210] The first device determines the backup node in the storage node of the third file based on the correspondence between the index number of the stored file and the identifier of the file's storage node.

[0211] The backup node for the third file is used to update the stored data to the modified third file.

[0212] In an optional embodiment, the method further includes:

[0213] The second device receives a request from the access device to obtain routing information for the index device;

[0214] The second device determines the routing information of the first device managed by the second device as the routing information of the index device and sends it to the access device.

[0215] In an optional embodiment, the method further includes:

[0216] When one of the first devices managed by the second device fails, the second device selects a first device from the other first devices managed by the second device.

[0217] The second device sends the mapping relationship between the index number of a file stored in one of the first devices and the identifier of the file's storage node to the selected first device for storage, and updates the routing information of the index device.

[0218] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention provides an access device that is consistent with the access device provided in one or more of the above embodiments.

[0219] Figure 9 This is a schematic diagram of the structure of an optional access device in an embodiment of the present invention, such as... Figure 9As shown, the access device includes:

[0220] The allocation module 91 is used to allocate an identification code to the first file upon receiving a storage request for the first file;

[0221] The generation module 92 is used to generate the index number of the first file using the existing parent directory and identifier code of the first file;

[0222] The first storage module 93 is used to store the association between the name of the first file and the index number of the first file;

[0223] The sending module 94 is used to send the name of the first file and the index number of the first file to the indexing device;

[0224] The indexing device is used to store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device after determining the identifier of the storage node used to store the first file in the storage device.

[0225] In an optional embodiment, the access device is further used for:

[0226] Send a request to the second device to obtain routing information for the index device;

[0227] Received routing information for the indexing device from the second device;

[0228] When the version number of the routing information received from the indexing device is the same as the version number of the routing information of the stored indexing device, the routing information of the stored indexing device is determined as the routing information of the determined indexing device.

[0229] When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, the received routing information of the indexing device is determined to be the routing information of the determined indexing device.

[0230] Based on the routing information of the determined indexing device, the device corresponding to the first file is determined from the first devices of the indexing device according to the existing parent directory of the first file.

[0231] In an optional embodiment, the access device is further used for:

[0232] Upon receiving a response message from the device corresponding to the first file indicating that the first file has been successfully written, a prompt message is generated; the prompt message is used to notify the user that the first file has been successfully stored.

[0233] In an optional embodiment, the access device is further used for:

[0234] A request to read the stored second file has been received.

[0235] The index number of the second file is determined based on the relationship between the names of the stored files and their index numbers.

[0236] Based on the routing information of the determined indexing device, the device corresponding to the second file is determined from the first device of the indexing device according to the existing parent directory of the second file;

[0237] Send the index number of the second file to the device corresponding to the second file;

[0238] The index number of the second file is used to: determine the storage node of the second file and read the second file by the device corresponding to the second file.

[0239] In an optional embodiment, the access device is further used for:

[0240] When the data of the master node in the storage node of the stored third file changes, the index number of the third file is determined according to the association between the name of the stored file and the index number of the file.

[0241] Based on the routing information of the determined indexing device, the device corresponding to the third file is determined from the first device of the indexing device according to the existing parent directory of the third file;

[0242] Send the modified third file and its index number to the device corresponding to the third file;

[0243] The device corresponding to the third file is used to determine the backup node in the storage node of the third file based on the index number of the third file. The backup node of the third file is used to update the stored data to the modified third file.

[0244] In practical applications, the above-mentioned allocation module 91, generation module 92, first storage module 93 and transmission module 94 can be implemented by a processor located on the access device, specifically a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0245] This invention provides a first device, which is consistent with the first device described in one or more of the above embodiments.

[0246] Figure 10This is a schematic diagram of the structure of an optional indexing device in an embodiment of the present invention, such as... Figure 10 As shown, the indexing device includes:

[0247] The receiving module 101 is used to receive a first file and its index number from the access device; wherein the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device.

[0248] The determination module 102 is used to determine the identifier of the storage node in the storage device used to store the first file;

[0249] The second storage module 103 is used to store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device.

[0250] In an optional embodiment, the first device described above is further used for:

[0251] When a reply message is received from the storage device indicating that the first file has been successfully written, a response message indicating that the first file has been successfully written is sent to the access device; the response message is used by the access device to notify the user that the first file has been successfully stored.

[0252] In an optional embodiment, the first device described above is further used for:

[0253] When the access device receives a read request for the stored second file, it receives the index number of the second file;

[0254] The storage node of the second file is determined based on the correspondence between the index number of the already stored file and the identifier of the file's storage node.

[0255] Read the second file from the storage node of the second file.

[0256] In an optional embodiment, the first device described above is further used for:

[0257] When the data of the master node in the storage node of the stored third file changes, the modified third file and the index number of the third file are received from the access device.

[0258] Based on the correspondence between the index number of the stored file and the identifier of the file's storage node, determine the backup node in the storage node of the third file;

[0259] The backup node for the third file is used to update the stored data to the modified third file.

[0260] In an optional embodiment, the second device is further used for:

[0261] Received a request from the access device to obtain routing information for the index device;

[0262] The routing information of the first device managed by the second device is determined as the routing information of the index device and sent to the access device.

[0263] In an optional embodiment, the second device is further used for:

[0264] When one of the first devices managed by the second device fails, a first device is selected from the other first devices managed by the second device.

[0265] The mapping between the index number of a file stored in one of the first devices and the identifier of the file's storage node is sent to the selected first device for storage, and the routing information of the index device is updated.

[0266] In practical applications, the receiving module 101, determining module 102 and second storage module 103 can be implemented by a processor located on the indexing device, specifically a CPU, MPU, DSP or FPGA.

[0267] Figure 11 This is a schematic diagram of another optional access device in an embodiment of the present invention, such as... Figure 11 As shown, an embodiment of the present invention provides an access device 1100, comprising:

[0268] The processor 111 and the storage medium 112 storing instructions executable by the processor 111 are included. The storage medium 112 performs operations in dependence of the processor 111 via a communication bus 113. When the instructions are executed by the processor 111, the storage method described in Embodiment 1 is executed.

[0269] It should be noted that in practical applications, the various components in the terminal are coupled together via the communication bus 113. It can be understood that the communication bus 113 is used to achieve communication between these components. In addition to the data bus, the communication bus 113 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 11 The general labeled all buses as communication bus 113.

[0270] Figure 12 This is a schematic diagram of another optional indexing device in an embodiment of the present invention, such as... Figure 12 As shown, an embodiment of the present invention provides an indexing device 1200, comprising:

[0271] The processor 121 and the storage medium 122 storing instructions executable by the processor 121 are included. The storage medium 122 performs operations in dependence on the processor 121 via a communication bus 123. When the instructions are executed by the processor 121, the storage method described in Embodiment 1 is executed.

[0272] It should be noted that in practical applications, the various components in the terminal are coupled together via the communication bus 123. It can be understood that the communication bus 123 is used to achieve communication between these components. In addition to the data bus, the communication bus 123 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 12 The general labeled all buses as communication buses 123.

[0273] This invention provides a file system. Figure 13 This is a schematic diagram of an optional file system structure in an embodiment of the present invention, such as... Figure 13 As shown, the file system 1300 includes the access device described in one or more of the above embodiments, the indexing device described in one or more of the above embodiments, and the storage device.

[0274] This invention provides a computer storage medium storing executable instructions. When the executable instructions are executed by one or more processors, the processors execute the storage method described in Embodiment 1.

[0275] The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.

[0276] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0277] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0278] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0279] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 Figure 1 The steps of the function specified in one or more boxes.

[0280] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A storage method, characterized in that, The method is applied in an access device and includes: Upon receiving a storage request for the first file, an identifier code is assigned to the first file; Using the existing parent directory of the first file and the identifier code, generate the index number of the first file; Store the association between the name of the first file and the index number of the first file; Send the first file and its index number to the indexing device; Wherein, the indexing device is used to, after determining the identifier of the storage node in the storage device used to store the first file, store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device; The method further includes: Send a request to the second device to obtain routing information for the index device; Received routing information for the indexing device from the second device; When the version number of the received routing information of the indexing device is the same as the version number of the stored routing information of the indexing device, the stored routing information of the indexing device is determined as the determined routing information of the indexing device. When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, the received routing information of the indexing device is determined as the determined routing information of the indexing device. Based on the routing information of the determined indexing device, the device corresponding to the first file is determined from the first devices of the indexing device according to the existing parent directory of the first file.

2. The method according to claim 1, characterized in that, The method further includes: A request to read the stored second file has been received. The index number of the second file is determined based on the association between the names of the stored files and their index numbers; Based on the routing information of the determined indexing device, the device corresponding to the second file is determined from the first device of the indexing device according to the existing parent directory of the second file; Send the index number of the second file to the device corresponding to the second file; The index number of the second file is used to: determine the storage node of the second file and read the second file by the device corresponding to the second file.

3. The method according to claim 1, characterized in that, The method further includes: When the data of the master node in the storage node of the stored third file changes, the index number of the third file is determined according to the association between the name of the stored file and the index number of the file. Based on the routing information of the determined indexing device, the device corresponding to the third file is determined from the first device of the indexing device according to the existing parent directory of the third file; The modified third file and its index number are sent to the device corresponding to the third file. The device corresponding to the third file is used to determine the backup node in the storage node of the third file according to the index number of the third file, and the backup node of the third file is used to update the stored data to the modified third file.

4. A storage method, characterized in that, The method is applied to an indexing device and includes: Receive a first file and its index number from the access device; wherein the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device; Determine the identifier of the storage node in the storage device used to store the first file; The correspondence between the index number of the first file and the identifier of the storage node of the first file is stored in the first and second devices of the indexing device; The method further includes: When the data of the master node in the storage node of the stored third file changes, the first device receives the modified third file and the index number of the third file sent from the access device; The first device determines the backup node in the storage node of the third file based on the correspondence between the index number of the stored file and the identifier of the storage node of the file; The backup node of the third file is used to update the stored data to the modified third file.

5. The method according to claim 4, characterized in that, The method further includes: When the access device receives a read request for a stored second file, the first device receives the index number of the second file; The first device determines the storage node of the second file based on the correspondence between the index number of the stored file and the identifier of the file's storage node; The first device reads the second file from the storage node of the second file.

6. The method according to claim 4, characterized in that, The method further includes: The second device receives a request from the access device to obtain routing information for the index device; The second device determines the routing information of the first device managed by the second device as the routing information of the index device, and sends it to the access device.

7. The method according to claim 6, characterized in that, The method further includes: When one of the first devices managed by the second device fails, the second device selects a first device from the other first devices managed by the second device; The second device sends the correspondence between the index number of the file stored in one of the first devices and the identifier of the file's storage node to the selected first device for storage, and updates the routing information of the index device.

8. An access device, characterized in that, include: The allocation module is used to allocate an identification code to the first file upon receiving a storage request for the first file; The generation module is used to generate an index number for the first file using the existing parent directory of the first file and the identifier code; The first storage module is used to store the association between the name of the first file and the index number of the first file; The sending module is used to send the first file and its index number to the indexing device; Wherein, the indexing device is used to, after determining the identifier of the storage node in the storage device used to store the first file, store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device; The access device is also used for: Send a request to the second device to obtain routing information for the index device; Received routing information for the indexing device from the second device; When the version number of the received routing information of the indexing device is the same as the version number of the stored routing information of the indexing device, the stored routing information of the indexing device is determined as the determined routing information of the indexing device. When the version number of the received routing information of the indexing device is different from the version number of the stored routing information of the indexing device, the received routing information of the indexing device is determined as the determined routing information of the indexing device. Based on the routing information of the determined indexing device, the device corresponding to the first file is determined from the first devices of the indexing device according to the existing parent directory of the first file.

9. An indexing device, characterized in that, include: The receiving module is used to receive a first file and the index number of the first file sent by the access device; wherein, the index number of the first file is generated by the access device using the existing parent directory of the first file and the identification code assigned to the first file by the access device; The determination module is used to determine the identifier of the storage node in the storage device used to store the first file; The second storage module is used to store the correspondence between the index number of the first file and the identifier of the storage node of the first file in the first and second devices of the indexing device; The first device is further used for: When the data of the master node in the storage node of the stored third file changes, the first device receives the modified third file and the index number of the third file sent from the access device; The first device determines the backup node in the storage node of the third file based on the correspondence between the index number of the stored file and the identifier of the storage node of the file; The backup node of the third file is used to update the stored data to the modified third file.

10. An access device, characterized in that, The access device includes: A processor and a storage medium storing processor-executable instructions, the storage medium performing operations via a communication bus dependent on the processor, wherein when the instructions are executed by the processor, the storage method according to any one of claims 1 to 3 is performed.

11. An indexing device, characterized in that, include: A processor and a storage medium storing processor-executable instructions, the storage medium performing operations dependent on the processor via a communication bus, wherein when the instructions are executed by the processor, the storage method according to any one of claims 4 to 7 is performed.

12. A computer storage medium, characterized in that, The storage contains executable instructions, which, when executed by one or more processors, perform the storage method according to any one of claims 1 to 3, or the storage method according to any one of claims 4 to 7.

Images