Metadata processing method and device based on distributed storage and computer equipment

Abstract

The invention relates to a metadata processing method and device based on distributed storage, equipment and a storage medium. The method relates to cloud storage, and comprises the following steps: acquiring metadata service topology information corresponding to a metadata service request, and feeding back the metadata service topology information to a request client. The metadata service topology information is determined according to respective service attribute information of the active metadata service node and the target metadata service node bound with the active metadata service node, and is used for indicating the request client to perform request redirection processing based on the target metadata service request carrying the access request identifier; and redirecting the target metadata service request to the target metadata service node, so as to feed back target metadata information matched with the target metadata service request to the request client based on the target metadata service node. And the overall processing efficiency of the metadata service request is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, and in particular to a metadata processing method, apparatus, computer equipment, storage medium and computer program product based on distributed storage. Background Art

[0002] With the development of cloud technology and the gradual promotion and application of distributed storage systems, in actual application, distributed storage systems can provide multiple clients for different users and allow multiple clients to read, write and access operations at the same time, including operations such as creating directories, creating files, deleting directories, and reading directories, so as to meet the actual needs of different users.

[0003] Traditionally, when performing read, write, and access operations based on a distributed storage system, the processing requests corresponding to the read and write operations are usually sent to the active metadata service nodes in the system. The active metadata service nodes process the requests in sequence according to the processing request time to feedback the metadata information corresponding to the processing request.

[0004] However, since traditional active metadata service nodes use a serial request processing method, when it is necessary to read a directory containing a large number of files, the request processing takes a long time, and subsequent requests need to wait and cannot be processed in time, resulting in delayed client responses for other processing requests and the problem of low request processing efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide a metadata processing method, apparatus, computer device, computer-readable storage medium and computer program product based on distributed storage, which can improve the processing efficiency of each processing request based on the distributed storage system, in order to address the above technical problems.

[0006] In a first aspect, the present application provides a metadata processing method based on distributed storage. The method comprises:

[0007] receiving a metadata service request, and acquiring metadata service topology information corresponding to the metadata service request; the metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node;

[0008] Feedback the metadata service topology information to the requesting client that triggers the metadata service request; the metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node;

[0009] Based on the target metadata service node, target metadata information matching the target metadata service request is fed back to the requesting client.

[0010] In a second aspect, the present application provides a metadata processing method based on distributed storage. The method comprises:

[0011] Receive metadata service topology information;

[0012] If it is determined that the metadata service topology information carries the target state machine service information, a request redirection process is performed based on the target metadata service request carrying the access request identifier, and the target metadata service request is redirected to a target metadata service node that matches the target state machine service information;

[0013] Receive target metadata information matched with the target metadata service request fed back by the target metadata service node.

[0014] In a third aspect, the present application also provides a metadata processing device based on distributed storage. The device includes:

[0015] A metadata service topology information acquisition module, used to receive a metadata service request and acquire metadata service topology information corresponding to the metadata service request; the metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node;

[0016] A metadata service topology information feedback module, used to feed back the metadata service topology information to the requesting client that triggers the metadata service request; the metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node;

[0017] The target metadata information feedback module is used to feed back the target metadata information matching the target metadata service request to the requesting client based on the target metadata service node.

[0018] In a fourth aspect, the present application also provides a metadata processing device based on distributed storage. The device includes:

[0019] A metadata service topology information receiving module, used for receiving metadata service topology information;

[0020] A request redirection processing module, configured to, if it is determined that the metadata service topology information carries the target state machine service information, perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to a target metadata service node that matches the target state machine service information;

[0021] The target metadata information receiving module is used to receive the target metadata information matched with the target metadata service request fed back by the target metadata service node.

[0022] In a fifth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method in the first aspect or its implementation manners is implemented.

[0023] In a sixth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method in the second aspect or its implementation manners when executing the computer program.

[0024] In a seventh aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method in the first aspect or its respective implementations.

[0025] In an eighth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method in the second aspect or its respective implementations.

[0026] In a ninth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method in the first aspect or its respective implementation manners is implemented.

[0027] In a tenth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method in the second aspect or its respective implementation manner is implemented.

[0028] In the above-mentioned metadata processing method, device, computer equipment and storage medium based on distributed storage, by receiving a metadata service request, metadata service topology information corresponding to the metadata service request and determined according to the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node is obtained, and the metadata service topology information is fed back to the requesting client that triggers the metadata service request. According to the metadata service topology information, the requesting client can be instructed to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node, so that the target metadata information matching the target metadata service request can be fed back to the requesting client based on the target metadata service node, so that the access request that takes a long time is redirected to the target metadata service node for processing, reducing the long-term occupation of the active metadata service node, so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, thereby improving the overall processing efficiency of metadata service requests. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 The following is an application environment diagram of a metadata processing method based on distributed storage in one embodiment;

[0030] Figure 2 A schematic diagram of a process of a metadata processing method based on distributed storage in one embodiment;

[0031] Figure 3 A schematic diagram of a conventional process of a metadata service request for a requesting client in one embodiment;

[0032] Figure 4 A schematic diagram of request redirection processing based on a target metadata service request carrying an access request identifier in one embodiment;

[0033] Figure 5 is a flowchart of a metadata processing method based on distributed storage in another embodiment;

[0034] Figure 6 A schematic diagram of a traditional state transition of a metadata service in one embodiment;

[0035] Figure 7 A schematic diagram of state transition of a metadata service in a newly added standby readable state in an embodiment;

[0036] Figure 8 A schematic diagram of a binding process between an active metadata service node and a target metadata service node in one embodiment;

[0037] Fig. 9A schematic diagram of a process of performing data playback synchronization processing on a target metadata service node with the same sorting identifier according to a request to record data in an embodiment;

[0038] Fig.10 A schematic diagram of a data playback synchronization process in one embodiment;

[0039] Fig.11 A schematic diagram of a metadata processing method based on distributed storage in another embodiment;

[0040] Fig.12 is a flowchart of a metadata processing method based on distributed storage in yet another embodiment;

[0041] Fig.13 A schematic diagram of a process of requesting a client to receive metadata service topology information in one embodiment;

[0042] Fig.14 A schematic diagram of a request redirection process in one embodiment;

[0043] Fig.15 A schematic diagram of a timing diagram of a metadata processing method based on distributed storage in an embodiment;

[0044] Fig.16 is a structural block diagram of a metadata processing device based on distributed storage in one embodiment;

[0045] Fig.17 is a structural block diagram of a metadata processing device based on distributed storage in one embodiment;

[0046] Fig.18 is an internal structure diagram of a computer device in one embodiment;

[0047] Fig.19 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] The metadata processing method based on distributed storage provided in the embodiment of the present application specifically relates to cloud technology, which can be applied to the storage and reading application scenarios of files, data, caches, and objects. Among them, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. Specifically, cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the application of cloud computing business model, which can form a resource pool, which is used on demand and is flexible and convenient. Among them, the background service of the technical network system requires a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. Cloud computing technology will become an important support in the actual application process of the technical network system. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data need strong system backing support, which can only be achieved through cloud technology.

[0050] Among them, cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and business access functions.

[0051] At present, the storage method of the storage system is: create a logical volume, and when creating the logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a certain storage device or several storage devices. The client stores data on a certain logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object not only contains data but also contains additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object. Among them, the process of allocating physical storage space for the logical volume by the storage system is specifically: according to the capacity estimation of the object stored in the logical volume (the estimation often has a large margin relative to the capacity of the actual object to be stored) and the group of independent redundant disk array (RAID, Redundant Array of IndependentDisk), the physical storage space is pre-divided into stripes, and a logical volume can be understood as a stripe, so that the physical storage space is allocated to the logical volume.

[0052] With the development of the Internet, real-time data streams and diversified connected devices, as well as the promotion of demands for search services, social networks, mobile commerce and open collaboration, cloud technology has developed rapidly. At the same time, the emergence of cloud technology will drive tremendous changes in the entire Internet model and enterprise management model from a conceptual perspective.

[0053] The metadata processing method based on distributed storage provided in the embodiment of the present application specifically relates to cloud storage technology in cloud technology. The metadata processing method based on distributed storage provided in the embodiment of the present application can be applied to application scenarios of storage and reading of files, data, caches, and objects, and can be specifically applied to Figure 1In the application environment shown. Among them, the request client 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the request client 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices, and aircraft, etc. The Internet of Things devices can be smart speakers, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be an independent physical server, or it can be a server cluster composed of multiple physical servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The request client 102 and the server 104 can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application.

[0054] Among them, both the requesting client 102 and the server 104 can be used separately to execute the metadata processing method based on distributed storage provided in the embodiment of the present application, and the requesting client 102 and the server 104 can also cooperate to execute the metadata processing method based on distributed storage provided in the embodiment of the present application. For example, taking the requesting client 102 and the server 104 to cooperate to execute the metadata processing method based on distributed storage provided in the embodiment of the present application as an example, the server 104 receives the metadata service request triggered by the requesting client 102, and obtains the metadata service topology information corresponding to the metadata service request. Among them, the metadata service topology information is determined based on the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node. Further, the server 104 feeds back the metadata service topology information to the requesting client 102 that triggers the metadata service request. After the requesting client 102 receives the metadata service topology information, it further determines the target metadata service node based on the metadata service topology information, and performs request redirection processing based on the target metadata service request carrying the access request identifier, and redirects the target metadata service request to the target metadata service node. Then, the server 104 feeds back the target metadata information matching the target metadata service request to the requesting client 104 based on the target metadata service node.

[0055] Similarly, taking the request client 102 and the server 104 to collaboratively execute the metadata processing method based on distributed storage provided in the embodiment of the present application as an example, the request client 102 receives metadata service topology information, and when determining that the metadata service topology information carries the target state machine service information, performs request redirection processing based on the target metadata service request carrying the access request identifier, and redirects the target metadata service request to the target metadata service node that matches the target state machine service information. Further, the request client 102 receives the target metadata information that matches the target metadata service request fed back by the target metadata service node deployed by the server 104.

[0056] Among them, metadata service topology information, service attribute information of active metadata service nodes, service attribute information of target metadata service nodes, and target state machine service information, etc., can be stored in the cloud storage of server 104, or in the data storage system, or in the local storage of terminal 102. When metadata processing is required, it can be obtained from server 104, or the data storage system, or terminal 102.

[0057] In one embodiment, Figure 2 As shown, a metadata processing method based on distributed storage is provided, and the method is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0058] Step S202: receiving a metadata service request, and obtaining metadata service topology information corresponding to the metadata service request. The metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node.

[0059] Among them, metadata service requests specifically include read requests (such as file read requests, directory read requests, etc.), and create requests (such as file create requests, directory create requests, etc.). In a distributed storage system, multiple request clients can simultaneously initiate metadata service requests.

[0060] The metadata service topology information is determined based on the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node. The metadata service topology information may include metadata service nodes in different states, such as online metadata service nodes, offline metadata service nodes, and damaged metadata service nodes, as well as the name, creation time, and modification time of each metadata service node. Among them, the online metadata service nodes specifically include the active metadata service node and the target metadata service node bound to the active metadata service node.

[0061] Among them, the service attribute information of the metadata service node refers to the service status of the metadata service node itself. For example, the service status of the active metadata service node is "active", which is understood as an active and available service status, while the service status of the target metadata service node bound to the active metadata service node is "standby_read", which is understood as a standby and readable service status.

[0062] Specifically, by obtaining the active service attribute information of the active metadata service node and the target service attribute information of the target metadata service node bound to the active metadata service node, and based on the cluster management node, the active service attribute information and the target service attribute information are integrated and processed to obtain the metadata service topology information.

[0063] Among them, by obtaining the beacon messages of the active metadata service node and the target metadata service node bound to the active metadata service node (i.e., the heartbeat message of the metadata service node, if the metadata service node does not send the beacon message due to a fault, the cluster management node can determine that the metadata service node that did not send the beacon message has a fault, and then can handle the fault in time), and send the beacon message to the cluster management node. Among them, the beacon message specifically includes the service status, version, name and number of the metadata service node, and the service status of the metadata service node specifically includes: the active service attribute information of the active metadata service node (i.e., the service status information of "active"), and the target service attribute information of the target metadata service node bound to the active metadata service node (i.e., the service status information of "standby_read").

[0064] Furthermore, the active service attribute information of the active metadata service node (i.e., the service status information of "active"), the target service attribute information of the target metadata service node bound to the active metadata service node (i.e., the service status information of "standby_read"), and the version, name, number and other information of each metadata service node are sent to the cluster management node (i.e., the Mon cluster node, used to generate and store various topology information in the distributed storage system), and the cluster management node is called to integrate the active service attribute information, the target service attribute information, the version, name, number and other information of each metadata service node to generate metadata service topology information containing the service status of different metadata service nodes.

[0065] Step S204, the metadata service topology information is fed back to the requesting client that triggers the metadata service request. The metadata service topology information is used to instruct the requesting client to redirect the request based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node.

[0066] Specifically, by feeding back the metadata service topology information to the requesting client that triggers the metadata service request, after each requesting client receives the metadata service topology information, it can identify the service status of different metadata service nodes based on the metadata service topology information, including the "active" service status information of the active metadata service node and the "standby_read" service status information of the target metadata service node bound to the active metadata service node.

[0067] Furthermore, when each requesting client identifies the service status information of "standby_read", it further performs request redirection processing based on the target metadata service request carrying the access request identifier. That is, when it is determined that the triggered target metadata service request carrying the access request identifier is, the target metadata service request carrying the access request identifier is redirected to the target metadata service node, so that the corresponding target metadata service request carrying the access request identifier can be processed through the target metadata service node.

[0068] In one embodiment, for metadata service requests from different request clients, conventionally, all metadata service requests are sent to an active metadata service node, and the active metadata service node receives and processes each metadata service request in sequence.

[0069] Specifically, Figure 3 As shown, a traditional processing method for metadata service requests from requesting clients is provided, referring to Figure 3 It can be seen that multiple request clients (i.e. Ceph-fuse) accept different requests triggered by the use of objects, and send each request (including file creation request, directory creation request, and directory reading request, etc.) to the active MDS (i.e. the metadata service node in the "active" state, understood as the active metadata service node), and each request is received and processed in sequence by the active MDS.

[0070] Among them, when processing requests, the active MDS needs to apply for MDS resources (i.e., metadata service resources), and when the service node cannot obtain resources in its own memory, it initiates metadata operations to obtain resources from the metadata pool of Rados (i.e., the cornerstone for storing various data in the distributed storage system, in which various data are stored in the form of objects). Among them, a standby MDS (i.e., a metadata service node in the "standby" state, understood as a metadata service node in the standby state) is also set up. The standby MDS is used as a backup for the active MDS. When the active MDS goes down or fails, it switches to the standby MDS for service, that is, the standby MDS is used as a new active MDS to process various metadata service requests.

[0071] However, in the traditional processing method for metadata service requests, active MDS uses a serial processing method for each metadata service request. When it is necessary to read a directory containing a large number of files, the request processing takes a long time, and subsequent requests need to wait and cannot be processed in time, resulting in delayed client responses for other processing requests and low request processing efficiency.

[0072] Furthermore, if Figure 4 As shown, a request redirection processing process based on a target metadata service request carrying an access request identifier is provided, referring to Figure 4 It can be seen that a new standby_read state (i.e., standby readable service state) is added for the traditional standby MDS (i.e., metadata service node in standby state).

[0073] Among them, multiple request clients (i.e. Ceph-fuse) accept different requests triggered by the use object, and divert each request (including file creation request, directory creation request, and directory reading request, etc.) according to the processing time. That is, the file creation request and directory creation request with shorter processing time are sent to the active MDS (i.e. the active metadata service node) for processing, and the file reading request and directory reading request with longer processing time are redirected to the standby_read MDS bound to the active MDS (understood as the metadata service node in the standby readable service state, that is, the target metadata service node bound to the active metadata service node) for processing, so that each metadata service node can be reasonably utilized and the long-term occupation of the active metadata service node can be reduced, so that the active metadata service node can process other processing requests in time, avoid the situation where other processing requests wait for a long time, and improve the overall processing efficiency of metadata service requests.

[0074] Step S206: Based on the target metadata service node, target metadata information matching the target metadata service request is fed back to the requesting client.

[0075] Specifically, through the target metadata service node, the target metadata service request carrying the access request identifier is received, and the target metadata service request carrying the access request identifier is processed to obtain the file or directory that needs to be read by the metadata service request, so as to feed back the read file or directory to the request client that triggers the target metadata service request carrying the access request identifier.

[0076] In the above-mentioned metadata processing method based on distributed storage, by receiving a metadata service request, metadata service topology information corresponding to the metadata service request and determined according to the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node is obtained, and the metadata service topology information is fed back to the requesting client that triggers the metadata service request. According to the metadata service topology information, the requesting client can be instructed to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node, so that the target metadata information matching the target metadata service request can be fed back to the requesting client based on the target metadata service node, so that the access request that takes a long time is redirected to the target metadata service node for processing, reducing the long-term occupation of the active metadata service node, so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, thereby improving the overall processing efficiency of metadata service requests.

[0077] In one embodiment, Figure 5 As shown, a metadata processing method based on distributed storage is provided, which specifically includes the following steps:

[0078] Step S502: Acquire a backup metadata service node associated with the active metadata service node.

[0079] Specifically, the backup metadata service node (i.e., standby MDS) set for the active metadata service node (i.e., active MDS) is obtained. The standby MDS (i.e., the metadata service node in the "standby" state, understood as the metadata service node in the standby state) is used as a backup for the active MDS. When the active MDS is down or fails, it switches to the standby MDS for service, and the standby MDS can be used as a new active MDS to process various metadata service requests.

[0080] In one embodiment, Figure 6 As shown, a traditional state transition diagram of a metadata service is provided. Figure 6 It can be seen that traditionally, the states of each metadata service node include null state, active state and standby state. The state of the metadata service node is controlled by the state machine, and the state change is achieved through the conversion between state machines. Specifically, the null state can be converted to the standby state, the standby state can be converted to the null state, or to the active state, and the active state can be converted to the null state.

[0081] Among them, null indicates the state when the metadata service node is just started or the state after the metadata service is removed, standby indicates that the metadata service node is converted to the standby state after startup, and the standby state can be further converted to the active state. The active state means that when a file system is created, the standby MDS will be selected as the active MDS to provide metadata services for the file system.

[0082] Step S504: Add a standby readable state to the backup metadata service node to obtain a target metadata service node.

[0083] Specifically, for the backup metadata service node (ie, standby MDS), a standby readable state (ie, standby_read state) is added to obtain the target metadata service node.

[0084] In one embodiment, Figure 7 As shown, a state transition diagram of a metadata service with a newly added standby readable state is provided. Figure 7 It can be seen that in order to reasonably utilize the standby MDS (i.e., the metadata service node in the standby state), a new standby_read state (i.e., the standby readable state) is added. When a metadata service node in the standby state is detected, the metadata service node is switched to the standby_read state. Among them, the null state can be converted to the standby state, the standby state can be converted to the null state, or to the active state, or to the standby_read state, the active state can be converted to the null state, and the standby_read state can be converted to the null state, or to the active state.

[0085] Among them, when a metadata service node in the standby state is detected, the metadata service node is switched to the standby_read state, and the metadata service node in the standby_read state is further bound to the metadata service node in the active state, so that the target metadata service request carrying the access request identifier can be redirected from the metadata service node in the active state to the metadata service node in the standby_read state bound to the metadata service node in the active state. Similarly, when the metadata service node in the active state goes down, the metadata service node in the standby_read state can be converted to the metadata service node in the active state to continue to provide services to the requesting client.

[0086] Step S506: Bind the active metadata service node and the target metadata service node.

[0087] Specifically, by obtaining the sorting identifier of each log shard file in the system log file, and setting the same sorting identifier based on each active metadata service node and the corresponding target metadata service node, the active metadata service node and the target metadata service node are bound.

[0088] Among them, in the system log file, i.e., the journal log file, there are multiple log shard files, and each log shard file usually has the same object size. Specifically, the journal log file is sharded according to the object size of 4M (i.e., MB, megabytes) in Rados (i.e., the cornerstone for storing various data in the distributed storage system, in which each data is stored in the form of an object), i.e., multiple log shard files of the same object size are obtained. Among them, the prefix and sorting identifier of each log shard file correspond, and then the same sorting identifier can be set based on each active metadata service node (i.e., active MDS) and the corresponding target metadata service node (i.e., standby_read MDS), so that the active metadata service node (i.e., active MDS) and its corresponding target metadata service node (i.e., standby_read MDS) point to the same log shard file, thereby achieving the purpose of binding the active metadata service node and the target metadata service node.

[0089] The sorting identifier (i.e., rank) is used to indicate the active metadata service node and the target metadata service node, and performs data synchronization based on the same log shard file.

[0090] Furthermore, if Figure 8 , provides a binding process for active metadata service nodes and target metadata service nodes, refer to Figure 8 It can be seen that the journal log file is divided into multiple journal shards (i.e., log file shards), and the object name and sorting identifier (i.e., rank) of each journal shard correspond to each other. For example, multiple journal shards such as rank.00000001, rank.00000002, rank.00000003, ..., rank.n are set. Then, an active metadata service node (i.e., active MDS) and its corresponding target metadata service node (i.e., standby_read MDS) can be set to the sorting identifier of rank.00000001. By analogy, each active metadata service node (i.e., active MDS) and its corresponding target metadata service node (i.e., standby_read MDS) can be set to the sorting identifier of rank.00000002, rank.00000003, ..., rank.n, etc.

[0091] Step S508: Perform data synchronization based on the active metadata service node and the target metadata service node.

[0092] Specifically, based on the log shard file corresponding to the sorting identifier, the request record data written by the active metadata service node is obtained, and according to the request record data, data playback synchronization processing is performed on the target metadata service node with the same sorting identifier.

[0093] When using the standby_read MDS to provide metadata services for reading directories or files, it is necessary to maintain the same data as the active MDS in the memory of the standby_read MDS, and data playback synchronization can be performed through the journal log file (i.e., the system log file). Specifically, the standby_read MDS and active MDS with a binding relationship are determined according to the sorting identifier, and the data playback synchronization of the standby_read MDS is implemented according to the data stored in the journal shard (i.e., the log file shard) corresponding to the sorting identifier.

[0094] Furthermore, by obtaining the request record data written by the activeMDS (i.e., active metadata service node) in the journal shard (i.e., log file shard) corresponding to the sorting identifier, such as the file creation request record data and the directory creation request record data, and based on the file creation request record data and the directory creation request record data, etc., the standby_read MDS (i.e., target metadata service node) with the same sorting identifier is subjected to data playback synchronization processing.

[0095] In the above-mentioned metadata processing method based on distributed storage, the target metadata service node is obtained by acquiring the backup metadata service node associated with the active metadata service node and adding a standby readable state to the backup metadata service node, and then the active metadata service node and the target metadata service node can be bound, so that data synchronization processing can be achieved based on the active metadata service node and the target metadata service node, so that when the target metadata service request carrying the access request identifier is subsequently redirected, the file or directory and other data established in the active metadata service node can be read through the target metadata service node, thereby reducing the long-term occupation of the active metadata service node, reducing the request waiting time for response, and improving the processing efficiency of metadata service requests.

[0096] In one embodiment, Fig. 9 As shown, according to the request to record data, the step of performing data playback synchronization processing on the target metadata service node with the same sorting identifier specifically includes the following steps:

[0097] Step S902: calling the playback thread to parse the requested recording data to obtain each metadata operation recording event in the requested recording data.

[0098] Specifically, by calling the replay thread (ie, Replay_thread) set in the distributed storage system, the request record data read from the journal shard (ie, log file shard) is parsed to obtain each metadata operation record event (ie, event event) in the request record data.

[0099] The metadata operation record event includes information about the metadata operation, such as a metadata operation code (used to indicate whether it is a metadata operation of creating a directory or a metadata operation of creating a file), and operation objects, etc. Each metadata service request (such as a request to create a file, a request to create a directory, etc.) sent by a requesting client corresponds to a metadata operation record event.

[0100] In one embodiment, after calling the playback thread to parse the request record data and obtaining each metadata operation record event in the request record data, the method further includes:

[0101] Obtain the write position corresponding to each metadata operation record event, and store the write position in the top storage shard in the system log file, read the write position corresponding to each metadata operation record event from the top storage shard, and compare the read position in the target service storage of the target metadata service node with each write position to generate a comparison result, and determine the metadata operation record event to be played back based on the comparison result.

[0102] Among them, since in the actual application process, not all metadata operation record events need to be synchronized every time (that is, there are cases where one or some metadata operation record events have been synchronized), it is necessary to determine the metadata operation record events to be replayed, that is, specifically for each determined metadata operation record event to be replayed, metadata operation playback processing is performed. Among them, the active MDS (i.e., the active metadata service node) records the metadata operation record events corresponding to the received requests such as create file requests and create directory requests in the journal log shard file corresponding to the sorting identifier, and determines which data the standby_read MDS needs to replay and synchronize by synchronizing write_pos (i.e., the access address of the write position) and read_pos (i.e., the access address of the read position).

[0103] Specifically, the active metadata service node (i.e., active MDS) receives the metadata service request initiated by the requesting client and updates the metadata service storage of the active metadata service node itself. If it is determined that the received metadata service request is a modification operation (for example, a create file request requires a new file, while a create directory request requires a new directory), a metadata operation record event corresponding to the metadata service request is generated, and the metadata operation record event is written to the corresponding write position in the metadata pool, and then the write position corresponding to each metadata operation record event is obtained and stored in the top storage slice in the system log file.

[0104] Among them, the top storage shard, namely the header, is the 0th shard in the journal log file (i.e., the log file shard of rank.00000000), which is used to record the pos position of the logical space, such as write_pos (i.e., the access address of the write position), etc.

[0105] Furthermore, a comparison result is generated by comparing the read position and each write position in the target service storage of the target metadata service node. If, according to the comparison result, it is determined that the access address of the write position is smaller than the access address of the read position, each metadata operation record event located between the access address of the read position and the access address of the write position is determined as the metadata operation record event to be played back.

[0106] Among them, when a new metadata operation record event (i.e., event event) is generated, it will be appended to the write_pos position (i.e., the access address of the write position), and write_pos will move forward. And read_pos (i.e., the access address of the read position) indicates the address that has been played back. If the metadata operation record event at the read_pos position has been played back, read_pos will move forward, and then determine that read_pos (i.e., the access address of the write position) is less than write_pos (the access address of the read position), which indicates that there is a metadata operation record event that needs to be played back. Specifically, each metadata operation record event located between read_pos (i.e., the access address of the read position) and write_pos (i.e., the access address of the write position) is determined as the metadata operation record event to be played back.

[0107] Step S904: extract the metadata operation code carried by each metadata operation record event.

[0108] Specifically, the metadata operation code to be replayed carried by each metadata operation record event to be replayed is extracted, such as whether it is a metadata operation to create a directory or a metadata operation to create a file, and then according to the metadata operation code to be replayed, it can be determined which metadata operations need to be synchronized and replayed.

[0109] Step S906: Perform metadata operation playback processing in the target service storage of the target metadata service node with the same sorting identifier according to each metadata operation code.

[0110] Specifically, according to the metadata operation code to be replayed, the metadata operation loop replay process is performed in the target service storage of the target metadata service node with the same sorting identifier until the synchronous playback of each metadata operation record event to be replayed is completed.

[0111] Among them, according to the metadata operation code to be replayed, it is determined which metadata operations need to be synchronized and replayed specifically, and then in the target service storage of the target metadata service node with the same sorting identifier, the metadata operations that need to be synchronously replayed are cyclically replayed until the synchronous playback of each metadata operation record event to be replayed is completed, so that the target service storage of the target metadata service node can also store the metadata operation record events corresponding to the metadata operations that need to be synchronously replayed, thereby achieving the purpose of data synchronization, so that when the target metadata service request carrying the access request identifier is subsequently redirected, the target metadata service node can read the data such as files or directories established in the active metadata service node, thereby realizing the diversion of metadata service requests and reducing the long-term occupation of the active metadata service node.

[0112] In one embodiment, Fig.10As shown, a data playback synchronization process is provided, referring to Fig.10 It can be seen that the metadata service processing module used to implement data playback synchronization processing specifically includes:

[0113] (1) MDCache module: the metadata service storage module, which is used to cache system metadata. For example, if the metadata service request initiated by the request client needs to read the directory content, it can first determine whether there is the directory content to be read in MDCache. If there is, it is directly returned from MDCache. Among them, data playback synchronization can be understood as the need to ensure that the data stored in the MDCache of the standby_read MDS is consistent with the data stored in the MDCache of the active MDS.

[0114] (2) Journaler module: the log file management module, which is used to write event events (i.e., metadata operation record events) into the journal log in the rados metadata pool, and write the access address of the write position of the event event (i.e., write_pos) into the top storage shard (i.e., header) of the journal log file in the metadata pool. It is also used to read event events from the journal log.

[0115] (3) Replay_thread module: the replay thread module is used to parse the event read from the journal log and replay the event. The replay operation refers to replaying the metadata operation in the MDCache of the standby_read MDS synchronously according to the metadata operation code recorded in the event.

[0116] (4) Rados metadata pool: The journal files in the metadata pool are divided into storage functions, including event events, logical journals, and headers.

[0117] The event is a metadata operation record event, which includes information about the metadata operation, such as a metadata operation code (used to indicate whether it is a metadata operation to create a directory or a metadata operation to create a file), and operation objects, etc. Each metadata service request (such as a request to create a file, a request to create a directory, etc.) sent by a requesting client corresponds to a metadata operation record event.

[0118] The logical journal can be understood as all the journal log shards are logically regarded as a continuous address access space, that is, composed of events one by one, and the logical journal space is specified by write_pos to be written. Among them, for each event in the logical journal, specifically, each metadata operation record event located between read_pos (i.e., the access address of the read position) and write_pos (i.e., the access address of the write position) is determined as the metadata operation record event to be played back.

[0119] Among them, header is the top storage shard, which is the 0th shard in the journal log file (that is, the log file shard of rank.00000000), and is used to record the pos position of the logical space, such as write_pos (that is, the access address of the write position).

[0120] Specifically, combined Fig.10 It can be seen that the specific process of realizing data playback synchronization processing based on the metadata service processing module includes:

[0121] (1) Receive the metadata service request sent by the requesting client (i.e., Ceph-fuse) through the active MDS, and update the MDCache. If it is determined that the received metadata service request is a modification operation (for example, a create file request requires a new file, while a create directory request requires a new directory), a metadata operation record event (i.e., event event) corresponding to the metadata service request is generated.

[0122] (2) Through the Journaler module, the metadata operation record event (i.e., event event) is written to the write_pos position, and write_pos is updated to the header.

[0123] (3) Read the write_pos position from the header through the Standby_read MDS and compare it with the read_pos in the Standby_readMDS's own MDCache to determine whether there are still events that need to be played back.

[0124] (4) If it is determined that read_pos is less than write_pos, the journaler module of the Standby_read MDS reads the event at the read_pos position (i.e., reads each metadata operation record event located between the access address of read_pos and the access address of write_pos, and determines them as the metadata operation record event to be replayed), and passes each metadata operation record event to be replayed to the replay_thread module.

[0125] (5) Replay_thread parses each metadata operation record event to be replayed, and performs data synchronization replay processing on each metadata service operation in each metadata operation record event to be replayed in the MDCache of the Standby_read MDS.

[0126] (6) Steps (3) to (5) are cyclically repeated in the Standby_read MDS until the synchronous playback of each metadata operation record event to be played back is completed, that is, until it is determined that there is no event that needs to be played back.

[0127] In this embodiment, the request record data is parsed by calling the playback thread to obtain each metadata operation record event in the request record data, and the metadata operation code carried by each metadata operation record event is extracted, so that according to each metadata operation code, the metadata operation playback processing is performed in the target service storage of the target metadata service node with the same sorting identifier, so that the target service storage of the target metadata service node can also store the metadata operation record event corresponding to the metadata operation that needs to be synchronously played back, so as to achieve the purpose of data synchronization, so that when the target metadata service request carrying the access request identifier is subsequently redirected, the target metadata service node can read the data such as files or directories established in the active metadata service node, so as to realize the diversion of metadata service requests, thereby reducing the long-term occupation of the active metadata service node and improving the processing efficiency of each metadata service request.

[0128] In one embodiment, Fig.11 As shown, a metadata processing method based on distributed storage is provided, which specifically includes the following steps:

[0129] Step S1101: Acquire a backup metadata service node associated with the active metadata service node, and add a standby readable state to the backup metadata service node to obtain a target metadata service node.

[0130] Specifically, the backup metadata service node (i.e., standby MDS) set for the active metadata service node (i.e., active MDS) is obtained. The standby MDS (i.e., the metadata service node in the "standby" state, understood as the metadata service node in the standby state) is used as a backup for the active MDS. When the active MDS is down or fails, it switches to the standby MDS for service, and the standby MDS can be used as a new active MDS to process various metadata service requests.

[0131] Furthermore, for the backup metadata service node (ie, standby MDS), a standby readable state (ie, standby_read state) is added to obtain the target metadata service node.

[0132] In order to reasonably utilize the standby MDS (i.e. the metadata service node in the standby state), a new standby_read state (i.e. the standby readable state) is added. When a metadata service node in the standby state is detected, the metadata service node is switched to the standby_read state. The null state can be converted to the standby state, the standby state can be converted to the null state, or to the active state, or to the standby_read state, the active state can be converted to the null state, and the standby_read state can be converted to the null state, or to the active state.

[0133] Among them, when a metadata service node in the standby state is detected, the metadata service node is switched to the standby_read state, and the metadata service node in the standby_read state is further bound to the metadata service node in the active state, so that the target metadata service request carrying the access request identifier can be redirected from the metadata service node in the active state to the metadata service node in the standby_read state bound to the metadata service node in the active state. Similarly, when the metadata service node in the active state goes down, the metadata service node in the standby_read state can be converted to the metadata service node in the active state to continue to provide services to the requesting client.

[0134] Step S1102, obtaining the sorting identifier of each log shard file in the system log file, and setting the same sorting identifier based on each active metadata service node and the corresponding target metadata service node to bind the active metadata service node and the target metadata service node.

[0135] Specifically, in the system log file, i.e., the journal log file, there are multiple log shard files, and each log shard file usually has the same object size. Specifically, the journal log file is sharded according to the object size of 4M (i.e., MB, megabytes) in Rados (i.e., the cornerstone for storing various data in the distributed storage system, in which each data is stored in the form of an object), i.e., multiple log shard files of the same object size are obtained. Among them, the prefix and sorting identifier of each log shard file correspond, and then the same sorting identifier can be set based on each active metadata service node (i.e., active MDS) and the corresponding target metadata service node (i.e., standby_read MDS), so that the active metadata service node (i.e., active MDS) and its corresponding target metadata service node (i.e., standby_read MDS) point to the same log shard file, thereby achieving the purpose of binding the active metadata service node and the target metadata service node.

[0136] The sorting identifier (i.e., rank) is used to indicate the active metadata service node and the target metadata service node, and performs data synchronization based on the same log shard file.

[0137] Step S1103: based on the log shard file corresponding to the sorting identifier, obtain the request record data written by the active metadata service node.

[0138] When using the standby_read MDS to provide metadata services for reading directories or files, it is necessary to maintain the same data as the active MDS in the memory of the standby_read MDS, and data playback synchronization can be performed through the journal log file (i.e., the system log file). Specifically, the standby_read MDS and active MDS with a binding relationship are determined according to the sorting identifier, and the data playback synchronization of the standby_read MDS is implemented according to the data stored in the journal shard (i.e., the log file shard) corresponding to the sorting identifier.

[0139] Specifically, by obtaining the request record data written by the active MDS (i.e., active metadata service node) in the journal shard (i.e., log file shard) corresponding to the sorting identifier, such as the file creation request record data and the directory creation request record data, and based on the file creation request record data and the directory creation request record data, etc., the standby_read MDS (i.e., target metadata service node) with the same sorting identifier is subjected to data playback synchronization processing.

[0140] Step S1104: calling the playback thread to parse the requested record data to obtain each metadata operation record event in the requested record data.

[0141] Specifically, by calling the playback thread (i.e., Replay_thread) set in the distributed storage system, the request record data read from the journal shard (i.e., the log file shard) is parsed to obtain each metadata operation record event (i.e., event event) in the request record data. The metadata operation record event includes information about the metadata operation, such as a metadata operation code (used to indicate whether it is a metadata operation for creating a directory or a metadata operation for creating a file), and information such as the operation object. Each metadata service request sent by each request client (such as a request to create a file, a request to create a directory, etc.) corresponds to a metadata operation record event.

[0142] Step S1105, obtaining the write position corresponding to each metadata operation record event, and storing the write position in the top storage slice in the system log file.

[0143] Specifically, the active metadata service node (i.e., active MDS) receives the metadata service request initiated by the requesting client and updates the metadata service storage of the active metadata service node itself. If it is determined that the received metadata service request is a modification operation (for example, a create file request requires a new file, while a create directory request requires a new directory), a metadata operation record event corresponding to the metadata service request is generated, and the metadata operation record event is written to the corresponding write position in the metadata pool, and then the write position corresponding to each metadata operation record event is obtained and stored in the top storage slice in the system log file.

[0144] Among them, the top storage shard, namely the header, is the 0th shard in the journal log file (i.e., the log file shard of rank.00000000), which is used to record the pos position of the logical space, such as write_pos (i.e., the access address of the write position), etc.

[0145] Step S1106, read the write position corresponding to each metadata operation record event from the top storage slice, and compare the read position in the target service storage of the target metadata service node with each write position to generate a comparison result.

[0146] Among them, since in the actual application process, not all metadata operation record events need to be synchronized every time (that is, there are cases where one or some metadata operation record events have been synchronized), it is necessary to determine the metadata operation record events to be replayed, that is, specifically for each determined metadata operation record event to be replayed, metadata operation playback processing is performed. Among them, the active MDS (i.e., the active metadata service node) records the metadata operation record events corresponding to the received requests such as create file requests and create directory requests in the journal log shard file corresponding to the sorting identifier, and determines which data the standby_read MDS needs to replay and synchronize by synchronizing write_pos (i.e., the access address of the write position) and read_pos (i.e., the access address of the read position).

[0147] Specifically, the read position in the target service storage of the target metadata service node is compared with each write position to generate a comparison result. According to the comparison result, each metadata operation record event to be played back that needs to be processed for data playback can be determined from each metadata operation record event.

[0148] Step S1107, if it is determined according to the comparison result that the access address of the write position is smaller than the access address of the read position, each metadata operation record event between the access address of the read position and the access address of the write position is determined as the metadata operation record event to be played back.

[0149] Specifically, when a new metadata operation record event (i.e., event event) is generated, it will be appended to the write_pos position (i.e., the access address of the write position), and write_pos will move forward. Read_pos (i.e., the access address of the read position) indicates the address that has been played back. If the metadata operation record event at the read_pos position has been played back, read_pos will move forward, and then it is determined that read_pos (i.e., the access address of the write position) is less than write_pos (the access address of the read position), which indicates that there is a metadata operation record event that needs to be played back. Specifically, each metadata operation record event located between read_pos (i.e., the access address of the read position) and write_pos (i.e., the access address of the write position) is determined as the metadata operation record event to be played back.

[0150] Step S1108, extract the metadata operation code to be played back carried by each metadata operation record event to be played back, and according to the metadata operation code to be played back, perform metadata operation loop playback processing in the target service storage of the target metadata service node with the same sorting identifier until the synchronous playback of each metadata operation record event to be played back is completed.

[0151] Specifically, the metadata operation code to be replayed carried by each metadata operation record event to be replayed is extracted, such as whether it is a metadata operation to create a directory or a metadata operation to create a file, and then according to the metadata operation code to be replayed, it can be determined which metadata operations need to be synchronized and replayed.

[0152] Furthermore, according to the metadata operation code to be replayed, the metadata operation is replayed in a loop in the target service storage of the target metadata service node with the same sorting identifier until the synchronous playback of each metadata operation record event to be replayed is completed. According to the metadata operation code to be replayed, it is determined which metadata operations need to be synchronized and replayed, and then the metadata operations that need to be replayed in a loop are replayed in the target service storage of the target metadata service node with the same sorting identifier until the synchronous playback of each metadata operation record event to be replayed is completed, so that the target service storage of the target metadata service node also stores the metadata operation record events corresponding to the metadata operations that need to be replayed in a synchronous manner, so as to achieve the purpose of data synchronization, so that when the target metadata service request carrying the access request identifier is subsequently redirected, the target metadata service node can read the data such as files or directories established in the active metadata service node, so as to realize the diversion of metadata service requests, so as to reduce the long-term occupation of the active metadata service node.

[0153] Step S1109: receiving a metadata service request, and acquiring metadata service topology information corresponding to the metadata service request. The metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node.

[0154] The metadata service request specifically includes a read request (such as a file read request, a directory read request, etc.) and a create request (such as a file create request, a directory create request, etc.). In a distributed storage system, multiple request clients can simultaneously initiate metadata service requests. The metadata service topology information specifically includes metadata service nodes in different states, such as online metadata service nodes, offline metadata service nodes, and damaged metadata service nodes, etc., as well as the name, creation time, and modification time of each metadata service node.

[0155] Furthermore, online metadata service nodes specifically include active metadata service nodes and target metadata service nodes bound to active metadata service nodes. The service attribute information of metadata service nodes refers to the service status of the metadata service nodes themselves. For example, the service status of the active metadata service node is "active", which is understood as an active and available service status, while the service status of the target metadata service node bound to the active metadata service node is "standby_read", which is understood as a standby and readable service status.

[0156] Specifically, by obtaining the beacon messages of the active metadata service node and the target metadata service node bound to the active metadata service node (i.e., the heartbeat message of the metadata service node, if the metadata service node does not send the beacon message due to a fault, the cluster management node can determine that the metadata service node that did not send the beacon message has a fault, and can then handle the fault in a timely manner), and sending the beacon message to the cluster management node. Among them, the beacon message specifically includes the service status, version, name, and number of the metadata service node, and the service status of the metadata service node specifically includes: the active service attribute information of the active metadata service node (i.e., the service status information of "active"), and the target service attribute information of the target metadata service node bound to the active metadata service node (i.e., the service status information of "standby_read").

[0157] Furthermore, the active service attribute information of the active metadata service node (i.e., the service status information of "active"), the target service attribute information of the target metadata service node bound to the active metadata service node (i.e., the service status information of "standby_read"), and the version, name, number and other information of each metadata service node are sent to the cluster management node (i.e., the Mon cluster node, used to generate and store various topology information in the distributed storage system), and the cluster management node is called to integrate the active service attribute information, the target service attribute information, the version, name, number and other information of each metadata service node to generate metadata service topology information containing the service status of different metadata service nodes.

[0158] Step S1110, the metadata service topology information is fed back to the requesting client that triggers the metadata service request. The metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node.

[0159] Specifically, by feeding back the metadata service topology information to the requesting client that triggers the metadata service request, after each requesting client receives the metadata service topology information, it can identify the service status of different metadata service nodes based on the metadata service topology information, including the "active" service status information of the active metadata service node and the "standby_read" service status information of the target metadata service node bound to the active metadata service node.

[0160] Furthermore, when each requesting client identifies the service status information of "standby_read", it further performs request redirection processing based on the target metadata service request carrying the access request identifier. That is, when it is determined that the triggered target metadata service request carrying the access request identifier is, the target metadata service request carrying the access request identifier is redirected to the target metadata service node, so that the corresponding target metadata service request carrying the access request identifier can be processed through the target metadata service node.

[0161] Step S1111: Based on the target metadata service node, target metadata information matching the target metadata service request is fed back to the requesting client.

[0162] Specifically, through the target metadata service node, the target metadata service request carrying the access request identifier is received, and the target metadata service request carrying the access request identifier is processed to obtain the file or directory that needs to be read by the metadata service request, so as to feed back the read file or directory to the request client that triggers the target metadata service request carrying the access request identifier.

[0163] In the above-mentioned metadata processing method based on distributed storage, by receiving a metadata service request, metadata service topology information corresponding to the metadata service request and determined according to the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node is obtained, and the metadata service topology information is fed back to the requesting client that triggers the metadata service request. According to the metadata service topology information, the requesting client can be instructed to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node, so that the target metadata information matching the target metadata service request can be fed back to the requesting client based on the target metadata service node, so that the access request that takes a long time is redirected to the target metadata service node for processing, reducing the long-term occupation of the active metadata service node, so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, thereby improving the overall processing efficiency of metadata service requests.

[0164] In one embodiment, Fig.12 As shown, a metadata processing method based on distributed storage is provided, and the method is applied to Figure 1 Taking the request client 104 in the example as an example, the following steps are specifically included:

[0165] Step S1202: receiving metadata service topology information.

[0166] Specifically, the metadata service topology information is determined based on the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node. The metadata service topology information may specifically include metadata service nodes in different states, such as online metadata service nodes, offline metadata service nodes, and damaged metadata service nodes, etc., and also includes the name, creation time, and modification time of each metadata service node.

[0167] Among them, online metadata service nodes specifically include active metadata service nodes and target metadata service nodes bound to active metadata service nodes. The service attribute information of metadata service nodes refers to the service status of metadata service nodes themselves. For example, the service status of active metadata service nodes is "active", which means active and available service status, while the service status of target metadata service nodes bound to active metadata service nodes is "standby_read", which means standby and readable service status.

[0168] In one embodiment, Fig.13 As shown, the process of requesting the client to receive metadata service topology information is shown in Fig.13 It can be seen that the beacon messages (i.e., the heartbeat messages of the metadata service nodes) of the active metadata service node (i.e., active MDS) and the target metadata service node (i.e., standby_read MDS) bound to the active metadata service node are obtained, and the beacon messages are sent to the cluster management node (i.e., Mon cluster node). Among them, the beacon message specifically includes the service status, version, name, and number of the metadata service node, and the service status of the metadata service node specifically includes: the active service attribute information of the active metadata service node (i.e., the service status information of "active"), and the target service attribute information of the target metadata service node bound to the active metadata service node (i.e., the service status information of "standby_read").

[0169] Furthermore, specifically, the active service attribute information of the active MDS (i.e., the service status information of "active"), the target service attribute information of the standby_read MDS bound to the active MDS (i.e., the service status information of "standby_read"), and the version, name, number and other information of each metadata service node are sent to the cluster management node (i.e., the Mon cluster node), and the cluster management node is called to integrate the active service attribute information, the target service attribute information, the version, name and number and other information of each metadata service node to generate metadata service topology information (i.e., the service status of different metadata service nodes) Fig.13 The MDS Map in the metadata service topology information is sent to each requesting client (i.e. Fig.13 ceph-fuse in .

[0170] Step S1204: If it is determined that the metadata service topology information carries the target state machine service information, request redirection processing is performed based on the target metadata service request carrying the access request identifier, and the target metadata service request is redirected to the target metadata service node matching the target state machine service information.

[0171] Among them, if the requesting client detects that the metadata service topology information carries the target state machine service information, that is, it is determined that there is a standby_read MDS (that is, the metadata service node in the standby_read state), then the request redirection processing is performed based on the target metadata service request carrying the access request identifier, that is, the time-consuming access request (such as a directory read request, or a file read request, etc.) is redirected to the target metadata service node (that is, the standby_read MDS) that matches the target state machine service information for processing.

[0172] Specifically, by determining the active metadata service node corresponding to the target metadata service request and the target metadata service node bound to the active metadata service node, and performing request redirection processing based on the target metadata service request, the target metadata service request is redirected from the active metadata service node (i.e., active MDS) to the target metadata service node (i.e., standby_read MDS), thereby reducing the long-term occupation of the active metadata service node (i.e., active MDS), so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, and improving the overall processing efficiency of metadata service requests.

[0173] In one embodiment, Fig.14As shown, a schematic diagram of the request redirection process is provided. Fig.14 It can be seen that when the requesting client (i.e. Ceph-fuse) detects the existence of standby_read MDS, when sending a metadata service request, the service client (i.e. Fig.14 The MDS client in the activeMDS redirects the metadata service request (such as a directory read request or a file read request) carrying the access request identifier to the standby_read MDS bound to the activeMDS, thereby implementing the diversion of metadata service requests and avoiding long-term occupation of the active metadata service node (i.e., activeMDS).

[0174] Step S1206: receiving target metadata information matched with the target metadata service request fed back by the target metadata service node.

[0175] Specifically, after receiving the target metadata service request carrying the access request identifier, the target metadata service node processes the target metadata service request carrying the access request identifier, obtains the file or directory that needs to be read by the metadata service request, and feeds back the read file or directory to the requesting client that triggers the target metadata service request carrying the access request identifier, thereby triggering the requesting client of the target metadata service request carrying the access request identifier, and can receive target metadata information matching the target metadata service request, such as data information such as the file or directory that needs to be read.

[0176] In the above-mentioned metadata processing method based on distributed storage, by receiving metadata service topology information, and when determining that the metadata service topology information carries the target state machine service information, request redirection processing is performed based on the target metadata service request carrying the access request identifier, and the target metadata service request is redirected to the target metadata service node that matches the target state machine service information, so as to receive the target metadata information matching the target metadata service request fed back by the target metadata service node, thereby realizing the reasonable use of each metadata service node, reducing the long-term occupation of active metadata service nodes, so that the active metadata service nodes can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, and improving the overall processing efficiency of metadata service requests.

[0177] In one embodiment, Fig.15 As shown, a metadata processing method based on distributed storage is provided, which specifically includes:

[0178] Step S1501: The server obtains a backup metadata service node associated with the active metadata service node, and adds a standby readable state to the backup metadata service node to obtain a target metadata service node.

[0179] In step S1502, the server obtains the sorting identifier of each log shard file in the system log file, and sets the same sorting identifier based on each active metadata service node and the corresponding target metadata service node to bind the active metadata service node and the target metadata service node.

[0180] Step S1503: The server obtains the request record data written by the active metadata service node based on the log shard file corresponding to the sorting identifier.

[0181] Step S1504: the server calls the playback thread to parse the requested record data and obtains each metadata operation record event in the requested record data.

[0182] Step S1505: The server obtains the write position corresponding to each metadata operation record event, and stores the write position in the top storage slice in the system log file.

[0183] Step S1506, the server reads the write position corresponding to each metadata operation record event from the top storage slice, and compares the read position in the target service storage of the target metadata service node with each write position to generate a comparison result.

[0184] Step S1507, if the server determines that the access address of the write position is smaller than the access address of the read position based on the comparison result, each metadata operation record event between the access address of the read position and the access address of the write position is determined as the metadata operation record event to be played back.

[0185] In step S1508, the server extracts the metadata operation code to be played back carried by each metadata operation record event to be played back, and performs metadata operation loop playback processing in the target service storage of the target metadata service node with the same sorting identifier according to the metadata operation code to be played back, until the synchronous playback of each metadata operation record event to be played back is completed.

[0186] Step S1509: The server receives the metadata service request sent by the requesting client, and obtains metadata service topology information corresponding to the metadata service request. The metadata service topology information is determined based on the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node.

[0187] Step S1510: requesting the client to receive metadata service topology information sent by the server.

[0188] Step S1511: If it is determined that the metadata service topology information carries the target state machine service information, the requesting client determines the active metadata service node corresponding to the target metadata service request and the target metadata service node bound to the active metadata service node, and performs request redirection processing based on the target metadata service request, redirecting the target metadata service request from the active metadata service node to the target metadata service node.

[0189] Step S1512: The requesting client receives target metadata information matched with the target metadata service request fed back by the target metadata service node.

[0190] In the above-mentioned metadata processing method based on distributed storage, by receiving a metadata service request, metadata service topology information corresponding to the metadata service request and determined according to the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node is obtained, and the metadata service topology information is fed back to the requesting client that triggers the metadata service request. According to the metadata service topology information, the requesting client can be instructed to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node, so that the target metadata information matching the target metadata service request can be fed back to the requesting client based on the target metadata service node, so that the access request that takes a long time is redirected to the target metadata service node for processing, reducing the long-term occupation of the active metadata service node, so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, thereby improving the overall processing efficiency of metadata service requests.

[0191] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0192] Based on the same inventive concept, the embodiment of the present application also provides a metadata processing device based on distributed storage for implementing the metadata processing method based on distributed storage involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more metadata processing device embodiments based on distributed storage provided below can refer to the limitations of the metadata processing method based on distributed storage above, and will not be repeated here.

[0193] In one embodiment, Fig.16 As shown, a metadata processing device based on distributed storage is provided, including: a metadata service topology information acquisition module 1602, a metadata service topology information feedback module 1604 and a target metadata information feedback module 1606, wherein:

[0194] The metadata service topology information acquisition module 1602 is used to receive a metadata service request and acquire metadata service topology information corresponding to the metadata service request; the metadata service topology information is determined based on the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node;

[0195] The metadata service topology information feedback module 1604 is used to feed back the metadata service topology information to the requesting client that triggers the metadata service request; the metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node;

[0196] The target metadata information feedback module 1606 is used to feed back the target metadata information matching the target metadata service request to the requesting client based on the target metadata service node.

[0197] In the above-mentioned metadata processing device based on distributed storage, by receiving a metadata service request, metadata service topology information corresponding to the metadata service request and determined according to the service attribute information of the active metadata service node and the target metadata service node bound to the active metadata service node is obtained, and the metadata service topology information is fed back to the request client that triggers the metadata service request. According to the metadata service topology information, the request client can be instructed to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node, so that the target metadata information matching the target metadata service request can be fed back to the request client based on the target metadata service node, so that the access request that takes a long time is redirected to the target metadata service node for processing, reducing the long-term occupation of the active metadata service node, so that the active metadata service node can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, thereby improving the overall processing efficiency of metadata service requests.

[0198] In one embodiment, the metadata service topology information acquisition module is also used to: obtain active service attribute information of active metadata service nodes and target service attribute information of target metadata service nodes bound to the active metadata service nodes; based on the cluster management node, perform information integration processing on the active service attribute information and the target service attribute information to obtain metadata service topology information.

[0199] In one embodiment, a metadata processing device based on distributed storage is provided, which also includes a data synchronization processing module, which is used to: obtain a backup metadata service node associated with an active metadata service node; add a standby readable state to the backup metadata service node to obtain a target metadata service node; bind the active metadata service node and the target metadata service node; and perform data synchronization processing based on the active metadata service node and the target metadata service node.

[0200] In one embodiment, the data synchronization processing module is also used to: obtain the sorting identifier of each log shard file in the system log file; set the same sorting identifier based on each active metadata service node and the corresponding target metadata service node to bind the active metadata service node and the target metadata service node; the sorting identifier is used to indicate the active metadata service node and the target metadata service node to perform data synchronization processing based on the same log shard file.

[0201] In one embodiment, the data synchronization processing module is also used to: obtain the request record data written by the active metadata service node based on the log shard file corresponding to the sorting identifier; and perform data playback synchronization processing on the target metadata service node with the same sorting identifier according to the request record data.

[0202] In one embodiment, the data synchronization processing module is also used to: call the playback thread to parse the request record data to obtain each metadata operation record event in the request record data; extract the metadata operation code carried by each metadata operation record event; and perform metadata operation playback processing in the target service storage of the target metadata service node with the same sorting identifier according to each metadata operation code.

[0203] In one embodiment, a metadata processing device based on distributed storage is provided, which also includes a metadata operation record event determination module to be replayed, which is used to: obtain the write position corresponding to each metadata operation record event, and store the write position in the top storage slice in the system log file; read the write position corresponding to each metadata operation record event from the top storage slice, and compare the read position in the target service storage of the target metadata service node with each write position to generate a comparison result; and determine the metadata operation record event to be replayed based on the comparison result.

[0204] In one embodiment, the module for determining metadata operation record events to be replayed is also used to: if it is determined based on the comparison result that the access address of the write position is smaller than the access address of the read position, each metadata operation record event located between the access address of the read position and the access address of the write position is determined as the metadata operation record event to be replayed.

[0205] In one embodiment, the data synchronization processing module is also used to: extract the metadata operation code to be played back carried by each metadata operation record event to be played back; and perform metadata operation loop playback processing in the target service storage of the target metadata service node with the same sorting identifier according to the metadata operation code to be played back until the synchronous playback of each metadata operation record event to be played back is completed.

[0206] In one embodiment, Fig.17 As shown, a metadata processing device based on distributed storage is provided, including a metadata service topology information receiving module 1702, a request redirection processing module 1704 and:

[0207] The metadata service topology information receiving module 1702 is used to receive the metadata service topology information target metadata information receiving module 1706, wherein:

[0208] The request redirection processing module 1704 is used to redirect the target metadata service request to a target metadata service node matching the target state machine service information if it is determined that the metadata service topology information carries the target state machine service information, based on the target metadata service request carrying the access request identifier;

[0209] The target metadata information receiving module 1706 is used to receive the target metadata information matched with the target metadata service request fed back by the target metadata service node.

[0210] In the above-mentioned metadata processing device based on distributed storage, by receiving metadata service topology information and, when determining that the metadata service topology information carries target state machine service information, request redirection processing is performed based on the target metadata service request carrying the access request identifier, and the target metadata service request is redirected to the target metadata service node that matches the target state machine service information, so as to receive the target metadata information that matches the target metadata service request fed back by the target metadata service node, thereby realizing the reasonable use of each metadata service node, reducing the long-term occupation of active metadata service nodes, so that the active metadata service nodes can process other processing requests in a timely manner, avoiding the situation where other processing requests wait for a long time, and improving the overall processing efficiency of metadata service requests.

[0211] In one embodiment, the request redirection processing module is also used to: determine the active metadata service node corresponding to the target metadata service request, and the target metadata service node bound to the active metadata service node; perform request redirection processing based on the target metadata service request, and redirect the target metadata service request from the active metadata service node to the target metadata service node.

[0212] Each module in the above-mentioned metadata processing device based on distributed storage can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0213] In one embodiment, a computer device is provided. The computer device may be a server or a request client. The computer device is described as a server. The internal structure diagram of the computer device may be as follows: Fig.18As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store metadata service topology information, service attribute information of active metadata service nodes, service attribute information of target metadata service nodes, target metadata service requests carrying access request identifiers, and target metadata information data matching the target metadata service requests. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a metadata processing method based on distributed storage is implemented.

[0214] In another embodiment, a computer device is provided. The computer device may be a server or a requesting client. The internal structure diagram of the computer device may be as follows: Fig.19 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a metadata processing method based on distributed storage is implemented.

[0215] Those skilled in the art will understand that Fig.18 and Fig.19The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0216] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0217] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0218] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0219] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0220] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0221] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0222] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A metadata processing method based on distributed storage, characterized in that: The method comprises: receiving a metadata service request, and acquiring metadata service topology information corresponding to the metadata service request; the metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node; Feedback the metadata service topology information to the requesting client that triggers the metadata service request; the metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node; Based on the target metadata service node, target metadata information matching the target metadata service request is fed back to the requesting client.

2. The method according to claim 1, characterized in that The method of obtaining the metadata service topology information includes: Acquire active service attribute information of the active metadata service node and target service attribute information of a target metadata service node bound to the active metadata service node; Based on the cluster management node, information integration processing is performed on the active service attribute information and the target service attribute information to obtain the metadata service topology information.

3. The method according to claim 1 or 2, characterized in that: Before receiving the metadata service request and acquiring metadata service topology information corresponding to the metadata service request, the method further includes: Acquire a backup metadata service node associated with the active metadata service node; Add a standby readable state to the backup metadata service node to obtain a target metadata service node; Binding the active metadata service node and the target metadata service node; Data synchronization processing is performed based on the active metadata service node and the target metadata service node.

4. The method according to claim 3, characterized in that: The binding the active metadata service node and the target metadata service node includes: Get the sorting identifier of each log fragment file in the system log file; Based on each of the active metadata service nodes and their corresponding target metadata service nodes, the same sorting identifier is set to bind the active metadata service node and the target metadata service node; the sorting identifier is used to indicate that the active metadata service node and the target metadata service node perform data synchronization processing based on the same log shard file.

5. The method according to claim 4, characterized in that The performing data synchronization processing based on the active metadata service node and the target metadata service node includes: Based on the log shard file corresponding to the sorting identifier, obtaining the request record data written by the active metadata service node; According to the requested data recording, data playback synchronization processing is performed on the target metadata service nodes with the same sorting identifier.

6. The method according to claim 5, characterized in that The step of recording data according to the request and performing data playback synchronization processing on the target metadata service nodes with the same sorting identifier includes: Calling a playback thread to parse the request record data to obtain each metadata operation record event in the request record data; Extracting the metadata operation code carried by each metadata operation record event; According to each metadata operation code, metadata operation playback processing is performed in the target service storage of the target metadata service node with the same sorting identifier.

7. The method according to claim 6, characterized in that After the calling playback thread parses the request record data and obtains each metadata operation record event in the request record data, the method further includes: Obtaining the write position corresponding to each metadata operation record event, and storing the write position in the top storage slice in the system log file; Reading the write position corresponding to each metadata operation record event from the top storage slice, and comparing the read position in the target service storage of the target metadata service node with each write position to generate a comparison result; According to the comparison result, the metadata operation record event to be played back is determined.

8. The method according to claim 7, characterized in that Determining the metadata operation record event to be played back according to the comparison result includes: If it is determined according to the comparison result that the access address of the write position is smaller than the access address of the read position, each metadata operation record event located between the access address of the read position and the access address of the write position is determined as the metadata operation record event to be played back.

9. The method according to claim 7, characterized in that: The extracting the metadata operation code carried by each metadata operation record event includes: extracting the metadata operation code to be replayed carried by each metadata operation record event to be replayed; The metadata operation playback process is performed in the target service storage of the target metadata service node with the same sorting identifier according to each metadata operation code, including: According to the metadata operation code to be replayed, a metadata operation loop replay process is performed in the target service storage of the target metadata service node with the same sorting identifier until the synchronous playback of each metadata operation record event to be replayed is completed.

10. A metadata processing method based on distributed storage, characterized in that: The method comprises: Receive metadata service topology information; If it is determined that the metadata service topology information carries the target state machine service information, a request redirection process is performed based on the target metadata service request carrying the access request identifier, and the target metadata service request is redirected to a target metadata service node that matches the target state machine service information; Receive target metadata information matched with the target metadata service request fed back by the target metadata service node.

11. The method according to claim 1, characterized in that: Performing request redirection processing based on the target metadata service request carrying the access request identifier, redirecting the target metadata service request to a target metadata service node matching the target state machine service information, including: Determine an active metadata service node corresponding to the target metadata service request and a target metadata service node bound to the active metadata service node; Perform request redirection processing based on the target metadata service request to redirect the target metadata service request from the active metadata service node to the target metadata service node.

12. A metadata processing device based on distributed storage, characterized in that: The device comprises: A metadata service topology information acquisition module, used to receive a metadata service request and acquire metadata service topology information corresponding to the metadata service request; the metadata service topology information is determined based on service attribute information of an active metadata service node and a target metadata service node bound to the active metadata service node; A metadata service topology information feedback module, used to feed back the metadata service topology information to the requesting client that triggers the metadata service request; the metadata service topology information is used to instruct the requesting client to perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to the target metadata service node; The target metadata information feedback module is used to feed back the target metadata information matching the target metadata service request to the requesting client based on the target metadata service node.

13. A metadata processing device based on distributed storage, characterized in that: The device comprises: A metadata service topology information receiving module, used for receiving metadata service topology information; A request redirection processing module, configured to, if it is determined that the metadata service topology information carries the target state machine service information, perform request redirection processing based on the target metadata service request carrying the access request identifier, and redirect the target metadata service request to a target metadata service node that matches the target state machine service information; The target metadata information receiving module is used to receive the target metadata information matched with the target metadata service request fed back by the target metadata service node.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images