Blacklist processing method and device of distributed file system and storage medium

Abstract

The invention discloses a blacklist processing method and device of a distributed file system and a storage medium, when blacklist information is received, a first management node adds the blacklist information into a temporary service node topology table, generates corresponding topology table updating operation information, then generates corresponding topology table updating operation key value pairs, and sends the topology table updating operation key value pairs to the temporary service node topology table; a topology table updating operation key value pair is sent to all the second management nodes, then the service node topology table is updated into the temporary service node topology table according to the topology table updating operation key value pair, and corresponding blacklist updating instructions are sent to all the second management nodes; and enabling each second management node to update the service node topology table into the temporary service node topology table. The blacklist processing method provided by the embodiment of the invention can be suitable for distributed file systems of different operating systems, and has good general applicability. The embodiment of the invention can be applied to various cloud security application scenes such as blacklist registration and access security protection.

Description

Technical Field

[0001] The present invention relates to the field of cloud security technology, and in particular to a blacklist processing method, device and storage medium for a distributed file system. Background Art

[0002] For access control of distributed file systems, the method used in related technologies is to implement access control of users through Iptable. In this related technology, by setting Iptable in the kernel of the metadata service node of the distributed file system, the metadata service node can use the Iptable in the kernel to filter and judge the user's address when receiving an access request sent by the user. If the user's address meets the filtering rules in Iptable, the access request is forwarded to the metadata management service in the user state in the metadata service node for shared access. If the user's address does not meet the filtering rules in Iptable, the access request will be rejected.

[0003] However, this related technology requires the metadata service node to support the Iptable function. However, since the operating systems of different distributed file systems may be different and some operating systems may not support the Iptable function, the method of implementing user access control through Iptable has the problem of low universal applicability. Summary of the invention

[0004] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0005] The embodiments of the present invention provide a blacklist processing method, device and storage medium for a distributed file system, which can be applied to distributed file systems of different operating systems and thus have good universal applicability.

[0006] On the one hand, an embodiment of the present invention provides a blacklist processing method for a distributed file system, the method is applied to a first management node in the distributed file system, and the method includes the following steps:

[0007] When receiving the blacklist information, adding the blacklist information into the temporary service node topology table recording the address information of the metadata service node;

[0008] Using the temporary service node topology table with the blacklist information added as an operation value, generating topology table update operation information;

[0009] Generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information;

[0010] According to the value data in the topology table update operation key-value pair, the service node topology table in the first management node is updated to the temporary service node topology table with the blacklist information added, and a corresponding blacklist update instruction is generated;

[0011] The blacklist update instruction is sent to all the second management nodes, so that each of the second management nodes uses the value data in the topology table update operation key-value pair according to the blacklist update instruction, and updates the service node topology table in each of the second management nodes to the temporary service node topology table with the blacklist information added.

[0012] On the other hand, an embodiment of the present invention further provides a blacklist processing method for a distributed file system, the method being applied to a second management node in the distributed file system, the method comprising the following steps:

[0013] Receive a topology table update operation key-value pair sent by a first management node in the distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as an operation value, and the temporary service node topology table records address information of a metadata service node;

[0014] Receive a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation;

[0015] According to the blacklist update instruction, the service node topology table in the second management node is updated to the temporary service node topology table with the blacklist information added thereto by using the value data in the topology table update operation key-value pair.

[0016] On the other hand, an embodiment of the present invention further provides a blacklist processing device for a distributed file system, wherein the blacklist processing device is arranged in a first management node in the distributed file system, and the blacklist processing device comprises:

[0017] A blacklist adding unit, configured to, upon receiving blacklist information, add the blacklist information into a temporary service node topology table recording address information of a metadata service node;

[0018] An operation information generating unit, configured to generate topology table update operation information by using the temporary service node topology table with the blacklist information added thereto as an operation value;

[0019] A first information sending unit is used to generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information;

[0020] A first information updating unit, configured to update the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation, and generate a corresponding blacklist update instruction;

[0021] The second information sending unit is used to send the blacklist update instruction to all the second management nodes, so that each of the second management nodes uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each of the second management nodes to the temporary service node topology table with the blacklist information added.

[0022] Optionally, the first information sending unit is further used for:

[0023] Serializing the topology table update operation information to obtain serialized topology table update operation information;

[0024] The serialized topology table update operation information is used as value data to generate a topology table update operation key-value pair.

[0025] Optionally, the first information sending unit is further used for:

[0026] Determine the key value number corresponding to the serialized topology table update operation information;

[0027] A topology table update operation key-value pair is generated by using the key value number as key data and the serialized topology table update operation information as value data.

[0028] Optionally, the first information updating unit is further used for:

[0029] Generate a topology table update key-value pair according to the value data in the topology table update operation key-value pair, wherein the value data of the topology table update key-value pair is the temporary service node topology table with the blacklist information added thereto;

[0030] The key-value pairs are updated according to the topology table, and the service node topology table in the first management node is updated to the temporary service node topology table with the blacklist information added thereto.

[0031] Optionally, the value data in the topology table update operation key-value pair is serialized topology table update operation information; and the first information update unit is further used to:

[0032] Deserialize the serialized topology table update operation information to obtain the topology table update operation information;

[0033] Acquire the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added;

[0034] Determine the version number corresponding to the temporary service node topology table to which the blacklist information is added;

[0035] A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with the blacklist information added as value data.

[0036] Optionally, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair;

[0037] The blacklist processing device also includes:

[0038] A response information receiving unit, used to receive response information sent by each of the second management nodes after persistently storing the key-value pairs of the topology table update operation;

[0039] A response information setting unit is used to set a response mark corresponding to each second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the key-value pair of the topology table update operation is completed.

[0040] On the other hand, an embodiment of the present invention further provides a blacklist processing device for a distributed file system, wherein the blacklist processing device is arranged in a second management node in the distributed file system, and the blacklist processing device comprises:

[0041] A first information receiving unit is configured to receive a topology table update operation key-value pair sent by a first management node in the distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as an operation value, and the temporary service node topology table records address information of a metadata service node;

[0042] A second information receiving unit is used to receive a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation;

[0043] The second information updating unit is used to update the service node topology table in the second management node to the temporary service node topology table with the blacklist information added thereto by using the value data in the topology table update operation key-value pair according to the blacklist update instruction.

[0044] Optionally, the second information updating unit is further used for:

[0045] Generate a topology table update key-value pair using the value data in the topology table update operation key-value pair according to the blacklist update instruction, wherein the value data of the topology table update key-value pair is the temporary service node topology table with the blacklist information added thereto;

[0046] The key-value pairs are updated according to the topology table, and the service node topology table in the second management node is updated to the temporary service node topology table with the blacklist information added thereto.

[0047] Optionally, the value data in the topology table update operation key-value pair is serialized topology table update operation information; and the second information update unit is further used to:

[0048] Deserialize the serialized topology table update operation information according to the blacklist update instruction to obtain the topology table update operation information;

[0049] Acquire the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added;

[0050] Determine the version number corresponding to the temporary service node topology table to which the blacklist information is added;

[0051] A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with the blacklist information added as value data.

[0052] Optionally, the blacklist processing device further includes:

[0053] An information processing unit, used for persistently storing the key-value pairs of the topology table update operation and generating corresponding response information;

[0054] The third information sending unit is used to send the response information to the first management node, so that the first management node sets the response mark corresponding to the second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the key-value pair of the topology table update operation is completed.

[0055] Optionally, the second management node is a slave node, and the first management node is a master node; the blacklist processing device further includes:

[0056] The fourth information sending unit is used to forward the blacklist information to the first management node when receiving the blacklist information, so that the first management node adds the blacklist information into the temporary service node topology table.

[0057] On the other hand, an embodiment of the present invention further provides a blacklist processing device for a distributed file system, comprising:

[0058] at least one processor;

[0059] at least one memory for storing at least one program;

[0060] When at least one of the programs is executed by at least one of the processors, the blacklist processing method of the distributed file system as described above is implemented.

[0061] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program executable by a processor. When the computer program executable by the processor is executed by the processor, it is used to implement the blacklist processing method of the distributed file system as described above.

[0062] On the other hand, an embodiment of the present invention further provides a computer program product, including a computer program or computer instructions, wherein the computer program or the computer instructions are stored in a computer-readable storage medium, and a processor of a blacklist processing device of a distributed file system reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions, so that the blacklist processing device performs the blacklist processing method of the distributed file system as described above.

[0063] The embodiment of the present invention includes at least the following beneficial effects: when receiving blacklist information, the first management node first adds the blacklist information into a temporary service node topology table that records the address information of the metadata service node, and then uses the temporary service node topology table with the blacklist information added as the operation value to generate topology table update operation information, and generates a topology table update operation key-value pair according to the topology table update operation information, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information, and then sends the topology table update operation key-value pair to all second management nodes in the distributed file system; since the value data of the topology table update operation key-value pair is obtained by the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, the synchronization of the topology table update operation information between the first management node and each second management node can be achieved, thereby ensuring the consistency of all management nodes in the distributed file system in executing the topology table update operation information. After sending the topology table update operation key-value pair to all second management nodes in the distributed file system, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added; since the first management node will send the corresponding blacklist update instruction to all second management nodes to trigger each second management node to perform the same topology table update after updating its service node topology table to the temporary service node topology table with blacklist information added, it is possible to achieve synchronization between the first management node and each second management node for the service node topology table with blacklist information added, thereby ensuring the consistency of the service node topology table with blacklist information added by all management nodes in the distributed file system.In addition, since all management nodes in the distributed file system can achieve consistency in the execution of topology table update operation information and consistency in the service node topology table with blacklist information added, all management nodes in the distributed file system can have the same service node topology table with blacklist information added, and since the service node topology table with blacklist information added records the address information of the metadata service node, the distributed file system can be accessed through the service node topology table, so when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process of the client is implemented according to the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for distributed file systems of different operating systems, the management node will store the service node topology table, so the embodiment of the present invention can enable distributed file systems of different operating systems to implement client filtering based on blacklist information, that is, the embodiment of the present invention can be applicable to distributed file systems of different operating systems, so it has good universal applicability.

[0064] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0066] Figure 1 It is a schematic diagram of the related technology provided by the embodiment of the present invention using Iptable to implement access control for users;

[0067] Figure 2 is a schematic diagram of an implementation environment provided by an embodiment of the present invention;

[0068] Figure 3 is a schematic diagram of another implementation environment provided by an embodiment of the present invention;

[0069] Figure 4 It is a flow chart of a blacklist processing method of a distributed file system provided by an embodiment of the present invention;

[0070] Figure 5 is a schematic diagram of a communication connection between a command line client and a first management node provided by an embodiment of the present invention;

[0071] Figure 6 It is a schematic diagram of the communication connection between the command line client and the MON cluster provided by an embodiment of the present invention;

[0072] Figure 7 It is a schematic diagram of storing data through a KV database in a first management node provided by an embodiment of the present invention;

[0073] Figure 8 It is a schematic diagram of storing data through a KV database in a second management node provided by an embodiment of the present invention;

[0074] Fig. 9 It is a schematic diagram of sending key-value pairs for a topology table update operation performed by a first management node in a first stage provided by an embodiment of the present invention;

[0075] Fig.10 It is a schematic diagram of the first management node updating the service node topology table in the second stage provided by an embodiment of the present invention;

[0076] Fig.11 It is a flow chart of another blacklist processing method of a distributed file system provided by an embodiment of the present invention;

[0077] Fig.12 It is a schematic diagram of a blacklist verification process performed by a client according to an embodiment of the present invention;

[0078] Fig.13 It is a schematic diagram of a blacklist verification process performed by multiple clients provided by an embodiment of the present invention;

[0079] Fig.14 It is a flowchart of a blacklist processing method of a distributed file system according to a specific example provided by an embodiment of the present invention;

[0080] Fig.15 It is a schematic diagram of a blacklist processing device for a distributed file system provided by an embodiment of the present invention;

[0081] Fig.16 is a schematic diagram of another blacklist processing device for a distributed file system provided by an embodiment of the present invention;

[0082] Fig.17 It is a schematic diagram of another blacklist processing device for a distributed file system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0083] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present invention, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0084] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0085] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0086] Before further describing the embodiments of the present invention in detail, the nouns and terms involved in the embodiments of the present invention are described. The nouns and terms involved in the embodiments of the present invention are subject to the following interpretations.

[0087] 1) Cloud Technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool and be used on demand, which is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, image websites, e-commerce platform websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0088] 2) Cloud Security refers to the general term for security software, hardware, users, institutions, and security cloud platforms based on cloud computing business model applications. Cloud security integrates emerging technologies and concepts such as parallel processing, grid computing, and unknown virus behavior judgment. Through a large number of clients in a network, it monitors the abnormal behavior of software in the network, obtains the latest information of Trojans and malicious programs on the Internet, and sends it to the server for automatic analysis and processing, and then distributes virus and Trojan solutions to each client. The main research directions of cloud security include: 1. Cloud computing security, which mainly studies how to ensure the security of the cloud itself and various applications on the cloud, including cloud computer system security, secure storage and isolation of target object data, user access authentication, information transmission security, network attack protection, compliance audit, etc.; 2. Cloudification of security infrastructure, which mainly studies how to use cloud computing to build and integrate security infrastructure resources and optimize security protection mechanisms, including building a large-scale security event, information collection and processing platform through cloud computing technology, realizing the collection and correlation analysis of massive information, and improving the control and risk control capabilities of the entire network security events; 3. Cloud security services, which mainly studies various security services provided to users based on cloud computing platforms, such as antivirus services.

[0089] 3) Ceph (The Versatile Distributed Storage System) is an open source distributed storage system that can provide object, file, and block storage services at the same time.

[0090] 4) CephFS (Ceph File System), one of the main components of Ceph, is used to provide file storage services. It is a distributed file system that can provide interfaces and functions similar to traditional file systems. CephFS supports simultaneous access by multiple clients and can provide high-performance and high-reliability file storage.

[0091] 5) MDS (Metadata Server), also known as the metadata service node, is one of the main components of Ceph and is used to manage and maintain CephFS metadata, including directory structure, file attributes, permissions and other information. In CephFS, MDS is a centralized metadata service responsible for processing client metadata operation requests for CephFS. In addition, MDS is also responsible for handling CephFS locking and concurrent access control to ensure data consistency and integrity.

[0092] 6) MON (Monitor) is one of the main components of Ceph, which is used to manage and maintain the metadata information of the Ceph cluster (including status and topology information, etc.). In the Ceph cluster, MON is a centralized management node responsible for managing the status and health of each storage node (Object Storage Daemon, OSD) in the Ceph cluster, as well as the topology of the cluster. In addition, MON is also responsible for managing the configuration information, authentication and authorization of the Ceph cluster.

[0093] 7) Mdsmap (Metadata Server Map), which represents the topology table of MDS, is used to describe and manage the status and topology information of CephFS metadata service node (MDS). Mdsmap contains information such as MDS status, MDS IP address, CephFS topology, load balancing and fault recovery.

[0094] Distributed file systems (such as CephFS, etc.) can provide shared access to multiple users at the same time. For example, for CephFS, as long as the user can communicate with the components of CephFS over the network and knows the address of the MON node of the Ceph cluster, he can access CephFS. However, when there are many users accessing the distributed file system, it is difficult for the distributed file system to control a large number of accesses. If the user access is not controlled, it is easy to cause a large number of non-compliant users to access the distributed file system and affect the normal operation of the distributed file system. For example, suppose a CephFS is specially provided to project group A (this project group is in the same network segment). If other project group B also obtains the cluster information of the CephFS, then other project group B can also access and use the same CephFS, which will affect the normal use of the CephFS by project group A. In order to solve this problem, the relevant technology proposes a solution to use Iptable to implement user access control. Reference Figure 1As shown, the scheme sets an Iptable rule table 121 in the kernel 120 of the metadata service node 110 of the distributed file system, and can use the Iptable rule table 121 in the kernel 120 of the metadata service node 110 to filter the user's address, thereby achieving control over the access initiated by the user. Specifically, when multiple user clients 140 send access requests to the metadata service node 110, the metadata service node 110 will use the Iptable rule table 121 in the kernel 120 to filter and judge the address of each user client 140. If the address of the user client 140 meets the filtering rules in the Iptable rule table 121, the corresponding access request will be forwarded to the metadata management service 131 of the user state 130 in the metadata service node 110 for shared access. If the address of the user client 140 does not meet the filtering rules in the Iptable rule table 121, the corresponding access request will be rejected. However, the related technology requires the metadata service node to support the Iptable function. However, since the operating systems of different distributed file systems may be different, and some operating systems may not support the Iptable function, the method of implementing user access control through Iptable has the problem of low general applicability. In addition, although the related technology can implement access control by verifying the user's address, this verification process is performed in the metadata service node (MDS). When there are a large number of useless access requests from malicious clients, the metadata service node needs to verify these useless access requests, which will affect the metadata service node's processing of normal client access requests. In addition, there will be multiple metadata service nodes in the distributed file system. When Iptable is used to implement user access control, the Iptable in each metadata service node needs to be initialized or updated. If some metadata service nodes fail to update Iptable in time due to network delays or node failures, there will be a risk of missing modifications or inconsistent modifications, which will affect the stability of the distributed file system.

[0095] In order to make the processing of blacklists applicable to distributed file systems of different operating systems and have good universal applicability and improve the stability of distributed file systems, an embodiment of the present invention provides a blacklist processing method for a distributed file system, a blacklist processing device for a distributed file system, a computer-readable storage medium and a computer program product. When receiving blacklist information, a first management node first adds the blacklist information to a temporary service node topology table that records the address information of a metadata service node, and then uses the temporary service node topology table with the blacklist information added as an operation value to generate topology table update operation information, and generates a topology table update operation key-value pair according to the topology table update operation information. The value data of the topology table update operation key-value pair is obtained according to the topology table update operation information, and then the topology table update operation key-value pair is sent to all second management nodes in the distributed file system; since the value data of the topology table update operation key-value pair is obtained by the topology table update operation information, and the operation value of the topology table update operation information is a temporary service node topology table with blacklist information added, after the topology table update operation key-value pair is sent to all second management nodes in the distributed file system, the synchronization of the topology table update operation information can be achieved between the first management node and each second management node, thereby ensuring the consistency of all management nodes in the distributed file system in executing the topology table update operation information. After sending the topology table update operation key-value pair to all second management nodes in the distributed file system, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added; since the first management node will send the corresponding blacklist update instruction to all second management nodes to trigger each second management node to perform the same topology table update after updating its service node topology table to the temporary service node topology table with blacklist information added, it is possible to achieve synchronization between the first management node and each second management node for the service node topology table with blacklist information added, thereby ensuring the consistency of the service node topology table with blacklist information added by all management nodes in the distributed file system.In addition, since all management nodes in the distributed file system can achieve consistency in the execution of topology table update operation information and consistency in the service node topology table with blacklist information added, all management nodes in the distributed file system can have the same service node topology table with blacklist information added. Since the service node topology table with blacklist information added records the address information of the metadata service node, access to the distributed file system can be achieved through the service node topology table. Therefore, when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added. This can not only avoid attacks on the distributed file system by a large number of useless access requests, but also reduce network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process for the client is implemented based on the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, so it has good universal applicability and can improve the stability of the distributed file system.

[0096] Reference Figure 2 , Figure 2 2 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. The implementation environment may include a user terminal 210, a first management node 220, multiple second management nodes 230 and multiple metadata service nodes 240, wherein the first management node 220, multiple second management nodes 230 and multiple metadata service nodes 240 constitute a distributed file system. The user terminal 210 and the distributed file system may be directly or indirectly connected via wired or wireless communication. The user terminal 210 and each node in the distributed file system may be a node in a blockchain, which is not specifically limited in this embodiment.

[0097] The user terminal 210 may include but is not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, an intelligent voice interaction device, a smart home appliance, a vehicle terminal, an aircraft, etc. Optionally, the user terminal 210 may be installed with a distributed file system access client, and the distributed file system access client may be used to implement interaction between the user terminal 210 and the distributed file system. The distributed file system access client may include a management service client and a metadata service client. Through the management service client, the user terminal 210 may interact with the first management node 220 or each second management node 230; through the metadata service client, the user terminal 210 may interact with each metadata service node 240.

[0098] The first management node 220, each second management node 230 and each metadata service node 240 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Among them, the first management node 220 and each second management node 230 will store a service node topology table for recording the node information of each metadata service node. The node information of the metadata service node may include the status information and address information of the metadata service node, etc. In addition, the service node topology table also records blacklist information for restricting access. The first management node 220 and each second management node 230 can send a service node topology table to the user terminal 210 in response to an access request sent by the user terminal 210 through the management service client, so that the metadata service client in the user terminal 210 can implement access control to the user terminal 210 according to the service node topology table.

[0099] Reference Figure 2As shown, in an application scenario, it is assumed that the user terminal 210 is a computer, and the user terminal 210 is provided with a distributed file system access client including a management service client 211 and a metadata service client 212. In response to the user registering a blacklist with the distributed file system through the user terminal 210, the first management node 220 in the distributed file system receives the blacklist information sent by the user terminal 210, and adds the blacklist information to the temporary service node topology table that records the address information of the metadata service node 240, and then uses the temporary service node topology table with the blacklist information added as the operation value to generate topology table update operation information, and then generates a topology table update operation key-value pair according to the topology table update operation information, and sends the topology table update operation key-value pair to all second management nodes 230, wherein the value data of the topology table update operation key-value pair is based on the topology table update operation. After sending the topology table update operation key-value pair to all second management nodes 230, the first management node 220 updates the service node topology table in the first management node 220 to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, and generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes 230, so that each second management node 230 uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node 230 to a temporary service node topology table with blacklist information added. So far, the distributed file system has completed the processing of the blacklist registration initiated by the user. When the user starts the distributed file system access client in the user terminal 210, the management service client 211 in the distributed file system access client will randomly subscribe to the service node topology table from any management node in the distributed file system. When the first management node 220 receives the subscription information sent by the user terminal 210 through the management service client 211, the first management node 220 will send the service node topology table with blacklist information added to the user terminal 210; in response to receiving the service node topology table, the metadata service client 212 in the user terminal 210 will filter and determine the address information of the user terminal 210 according to the blacklist information in the service node topology table. When the blacklist information includes the address information of the user terminal 210, the distributed file system access client will exit the startup; when the blacklist information does not include the address information of the user terminal 210, the distributed file system access client will remain started. At this time, the user can access the distributed file system through the user terminal 210.

[0100] Reference Figure 3As shown, in another application scenario, it is assumed that the user terminal 210 is a computer, and the user terminal 210 is provided with a distributed file system access client including a management service client 211 and a metadata service client 212. In the process of the user registering the blacklist to the distributed file system through the user terminal 210, when a second management node 230 in the distributed file system receives the blacklist information sent by the user terminal 210, the second management node 230 will forward the blacklist information to the first management node 220 in the distributed file system. After the first management node 220 receives the blacklist information forwarded by the second management node 230, the first management node 220 adds the blacklist information to the temporary service node topology table that records the address information of the metadata service node 240, and then uses the temporary service node topology table with the blacklist information added as the operation value to generate topology table update operation information, and then generates a topology table update operation key-value pair according to the topology table update operation information, and updates the topology table operation. The key-value pair is sent to all second management nodes 230, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information; after sending the topology table update operation key-value pair to all second management nodes 230, the first management node 220 updates the service node topology table in the first management node 220 to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, and generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes 230, so that each second management node 230 uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node 230 to a temporary service node topology table with blacklist information added. At this point, the distributed file system has completed the processing of the blacklist registration initiated by the user. When the user starts the distributed file system access client in the user terminal 210, the management service client 211 in the distributed file system access client will randomly subscribe to the service node topology table from any management node in the distributed file system. When the second management node 230 that receives the blacklist information sent by the user terminal 210 receives the subscription information sent by the user terminal 210 through the management service client 211, the second management node 230 will send the service node topology table with the blacklist information added to it to the user terminal 210; in response to receiving the service node topology table, the metadata service client 212 in the user terminal 210 will filter and determine the address information of the user terminal 210 according to the blacklist information in the service node topology table. When the blacklist information includes the address information of the user terminal 210, the distributed file system access client will exit the startup; when the blacklist information does not include the address information of the user terminal 210, the distributed file system access client will remain started. At this time, the user can access the distributed file system through the user terminal 210.

[0101] It should be noted that in various specific embodiments of the present invention, when it comes to the need to perform relevant processing based on the attribute information or attribute information set of the target object (such as a user, etc.) and other data related to the characteristics of the target object, the permission or consent of the target object will be obtained first, and the collection, use and processing of these data will comply with relevant laws, regulations and standards. In addition, when the embodiment of the present invention needs to obtain the attribute information of the target object, it will obtain the separate permission or separate consent of the target object through a pop-up window or jump to a confirmation page, and after clearly obtaining the separate permission or separate consent of the target object, it will obtain the relevant data of the target object necessary for the normal operation of the embodiment of the present invention.

[0102] Figure 4 The present invention provides a flowchart of a blacklist processing method for a distributed file system provided by an embodiment of the present invention. The blacklist processing method can be executed by a first management node in a distributed file system, or by the first management node and the second management node in a distributed file system. In the embodiment of the present invention, the method is executed by the first management node in a distributed file system as an example for description. Figure 4 The blacklist processing method of the distributed file system may include but is not limited to steps 410 to 450.

[0103] Step 410: When the blacklist information is received, the blacklist information is added to the temporary service node topology table that records the address information of the metadata service node.

[0104] In one embodiment, in a distributed file system, access control can use a whitelist (allowed list) to define users, organizations, or entities with access rights, wherein the users or entities listed in the whitelist will be authorized to access specific files or directories in the system. Similarly, access control can also define users, organizations, or entities without access rights through a blacklist (not allowed list), wherein the users or entities listed in the blacklist will be prohibited from accessing specific files or directories in the system, and users in the blacklist will be considered to have no access rights, and the system will deny their access requests. Therefore, access control can be achieved in the distributed file system by establishing a blacklist. After the first management node receives the blacklist during the access control process, the blacklist information is added to the temporary service node topology table that records the address information of the metadata service node, so that the service node topology table in the first management node and all the second management nodes can be updated subsequently according to the temporary service node topology table with the blacklist information added. The temporary service node topology table also records the address information of the metadata service node. Subsequently, the address information of the metadata service node can be updated to the service node topology table in the first management node and all the second management nodes according to the temporary service node topology table, so as to achieve access to the metadata service node through the service node topology table in each node, that is, to achieve access to the distributed file system.

[0105] In one embodiment, the distributed file system may be CephFS, the first management node may be a MON node in CephFS, and the temporary service node topology table is a temporary Mdsmap in the MON node. In some distributed file systems, the MON node may participate in the access control process, and may be responsible for managing the metadata and coordination tasks of the cluster, and the metadata includes access permission information about files and directories. In an embodiment of the present invention, by setting appropriate access control policies in the distributed file system, the MON node may perform access control on the file system. It should be noted that the first management node may not be the only node that controls access control. The first management node may be one of multiple MON nodes. Other components and nodes in the distributed file system may also participate in the access control process to ensure the security of the entire system and the protection of data. The embodiment of the present invention does not impose specific restrictions.

[0106] In one embodiment, the blacklist information may be obtained from a command line client (Cmd Client). A command line client is an application that runs under a command line interface. It communicates with each node of a distributed file system by executing specific commands and options to manage, operate, and query file system functions, and thus may be connected to the first management node in communication. The command line client may configure the IP address of the first management node, and may send a blacklist registration command to the corresponding first management node through the IP address, and carry the blacklist information through the blacklist registration command. For example, Figure 5 A schematic diagram showing a communication connection between the command line client 502 and the first management node 501 is shown. Figure 5 As shown, the MON0 node set in the distributed file system is the first management node 501, and the command line client is connected to the MON0 node for communication. During operation, the command line client 502 selects the MON0 node to send the blacklist information, and the MON0 node has read and write functions, so the MON0 node executes subsequent steps based on the received blacklist information.

[0107] In one embodiment, the blacklist information may also be obtained from the second management node, for example, Figure 6 A schematic diagram showing the communication connection between the command line client 620 and the MON cluster 610 is shown. Figure 6 As shown, a distributed file system may be provided with multiple MON nodes, and the MON nodes are divided into master nodes and slave nodes. The first management node (MON0) 611 is the master node, and the second management node (MON1) 612 and the second management node (MON2) 613 are slave nodes. The master node can provide read and write capabilities, and the slave node can only provide read functions. The master node and at least one slave node can form a MON cluster 610, and the command line client 620 can obtain the IP list of the MON cluster 610, and the IP list contains the IP addresses of each MON node. Then the command line client can randomly select a MON node from the IP list of the MON cluster 610 to send a blacklist registration message and blacklist information. Figure 6 In the example, the command line client selects the MON1 node to send the blacklist information, and both the MON1 node and the MON2 node are the second management nodes (612 and 613), and MON0 is the first management node 611. Therefore, the MON1 node has only a read function. After reading the blacklist information, the MON1 node forwards the blacklist information to the MON0 node.

[0108] In one embodiment, taking the command line client sending the blacklist information directly to the first management node as an example, the command line client in the embodiment of the present invention can send the restricted network segment and the fixed IP address as the blacklist information to the first management node, the restricted network segment indicates that all addresses in this network segment cannot access the distributed file system, and the fixed IP address indicates that the distributed file system cannot be accessed under this fixed IP address. For example, if the restricted network segment is 10.0.1.0 / 24, where the network segment is 10.0.1.0 and the mask bit number is / 24, it means that all addresses between 10.0.1.0 and 10.0.1.255 cannot access the distributed file system, and the fixed IP address is 10.0.2.24, it means that the fixed IP address is restricted from accessing the distributed file system. Figure 5 As shown, the MON0 node set in the distributed file system is the first management node 501, and the command line client 502 is connected to the MON0 node for communication. During operation, the command line client 502 selects the MON0 node to send blacklist information, and the blacklist information includes the restricted network segment 10.0.1.0 / 24 and the fixed IP address 10.0.2.24. The MON0 node has read and write functions, so the MON0 node executes subsequent steps based on the received blacklist information containing the restricted network segment 10.0.1.0 / 24 and the fixed IP address 10.0.2.24.

[0109] Step 420: Generate topology table update operation information by using the temporary service node topology table with blacklist information added as the operation value.

[0110] In one embodiment, after adding the blacklist information into the temporary service node topology table recording the address information of the metadata service node, the temporary service node topology table with the blacklist information added can be used as the operation value to generate the topology table update operation information. Wherein, the temporary service node topology table with the blacklist information added is a temporary service node topology table, which contains the blacklist information, and the blacklist is usually used to specify some nodes or resources that are restricted or prohibited from access. Therefore, the blacklist information in the topology table may indicate that the blacklist list of certain nodes needs to be updated or modified. And using the temporary service node topology table with the blacklist information added as the operation value means that the update operation of the topology table will use the temporary service node topology table with the blacklist information added as input, that is, after obtaining the temporary service node topology table with the blacklist information, it is necessary to update the existing service node topology table, and the topology table is updated by adding the new blacklist information to the existing service node topology table. Therefore, after adding the temporary service node topology table with the blacklist information as the operation value, the topology table update operation information is generated to perform the update operation of the actual topology table.

[0111] In one embodiment, the temporary service node topology table is a temporary Mdsmap. The first management node stores a service node topology table, namely, Mdsmap. Mdsmap is used to manage the mapping of metadata service nodes (MDS) in the distributed file system. It contains information about the status, location, and connection of the MDS. Therefore, the specific process of generating the topology table update operation information with the temporary service node topology table added with blacklist information as the operation value can first obtain the Mdsmap in the current first management node. In this process, appropriate tools (such as Ceph command line tools or related APIs) can be used to obtain the current Mdsmap and save it as a copy of the Mdsmap so that it can be modified on its basis. Then, as needed, the existing copy of the Mdsmap is modified, which may include adding, deleting or modifying MDS instances, adjusting weights, changing positions, etc., ensuring that the modified target is consistent with the temporary Mdsmap with blacklist information added. Then, the topology table update operation information is generated, and according to the modification of the existing topology table copy, the operation information for performing the topology table update is generated, which may include the operation type (addition, deletion, modification), MDS instance identifier, location information, etc. The operation information can be an instruction list or a corresponding command line parameter. After the topology table update operation information is generated, the topology table update operation can be performed later, such as using the generated topology table update operation information, applying it to the corresponding node in the distributed file system, and performing the topology table update operation.

[0112] Step 430: Generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information.

[0113] In one embodiment, a distributed file system contains multiple nodes, including a first management node and a second management node, and there are multiple second management nodes. Further, the first management node and the second management node are both MON nodes, and multiple MON nodes form a MON cluster, and the MON cluster is divided into a master node and a slave node, the first management node is the master node, and the second management node is the slave node. The master node can provide read and write capabilities, and the slave node can only provide read functions. Therefore, after the topology table update operation information is generated in the first management node, it is not only necessary to store the topology table update operation information so that the service node topology table stored in the first management node can be updated later, but also the topology table update operation information needs to be sent to all second management nodes so that all second management nodes can store the same topology table update operation information together. The second management node that has stored the topology table update operation information can also complete the update operation of the service node topology table, and realize the synchronization between the first management node and each second management node for the service node topology table with blacklist information added, so as to ensure the consistency of all management nodes in the distributed file system for the service node topology table with blacklist information added. Therefore, the process of realizing synchronization of the service node topology table containing blacklist information in the embodiment of the present invention, that is, the process of blacklist consistency synchronization, is divided into two stages. In the first stage, the topology table update operation key-value pairs are first sent to all second management nodes and persistently stored. In the subsequent second stage, the blacklist update operation is performed on the second management node that successfully stores the topology table update operation information. This avoids the situation where some second management nodes fail to execute data updates due to network delays, node failures and other problems. The synchronization of the service node topology table with blacklist information added can be realized between the first management node and each second management node, thereby ensuring the consistency of the service node topology table with blacklist information added for all management nodes in the distributed file system.

[0114] In one embodiment, the topology table update operation information is stored by generating a topology table update operation key-value pair. After the topology table update operation information is generated, the first management node generates a topology table update operation key-value pair according to the topology table update operation information, and the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information, and then the first management node sends the topology table update operation key-value pair to all second management nodes in the distributed file system. The purpose of the first management node sending the topology table update operation key-value pair to all the second management nodes is to notify all the second management nodes to update the service node topology table. The topology table update operation key-value pair includes value data and key data, wherein the value data is obtained from the topology table update operation information, so the value data not only includes the status, location or other related information of the metadata service node originally contained, but also includes the temporary service node topology table containing blacklist information. Therefore, by sending the topology table update operation key-value pair, the information of the temporary service node topology table containing blacklist information can be transmitted. After receiving the topology table update operation key-value pair, the second management node does not directly update the service node topology table according to the topology table update operation key-value pair, but stores the topology table update operation key-value pair, so as to synchronize and update the service node topology table stored in itself based on the topology table update operation key-value pair in the future.

[0115] In one embodiment, the process of generating a topology table update operation key-value pair based on the topology table update operation information can first define an empty operation key-value pair dictionary or list to store the key-value pairs of the topology table update operation information, and then traverse the topology table update operation information list to generate corresponding key-value pairs based on the type and parameters of the topology table update operation information, and add the generated key-value pairs to the operation key-value pair dictionary or list. Finally, after the traversal is completed, the obtained operation key-value pairs are the results generated based on the topology table update operation information.

[0116] In one embodiment, the first management node is a MON node. After generating the topology table update operation key-value pair, if the MON node fails, such as hardware failure, operating system crash or network problem, or the MON node restarts, the node cannot work normally, and the generated topology table update operation key-value pair may be lost or damaged. Based on this, the first management node can be provided with a key-value (KV) database, such as Figure 7 As shown, Figure 7A schematic diagram of data storage through a KV database 711 in the first management node 710 is shown. In the KV database 711, the key data is used to uniquely identify a data entry, and the value data stores the data associated with the key. By specifying the key, the corresponding value can be quickly stored, updated or retrieved, thereby achieving efficient access to data. The KV database 711 of the embodiment of the present invention can be of various types, including Redis, RocksDB, Etcd, etc., which provide high-performance, low-latency data access capabilities for the embodiment of the present invention, and support a variety of data structures and functions, such as lists, hashes, sets, etc., which are not specifically limited by the embodiment of the present invention. Therefore, by setting the KV database 711, the first management node 710 can store the topology table update operation key-value pairs in the local KV database 711 for persistent storage. Even after the first management node 710 fails or restarts, the configuration information can still be reliably restored, which can enable the MON node to quickly restore the cluster state after failure, and perform necessary reconstruction operations to minimize service interruption time and achieve persistent storage of data.

[0117] In one embodiment, the second management node is also a MON node. Specifically, the first management node is the master node in the MON cluster, and the second management node is the slave node in the MON cluster. Therefore, the second management node also stores the topology table update operation key-value pair in the KV database in each second management node. The distributed file system is usually composed of multiple MON nodes. By setting a KV database on each MON node for persistent storage, even if a certain MON node fails, the topology table update operation key-value pair can be obtained from other nodes. Therefore, it can be ensured that even if a certain MON node fails, other MON nodes can still provide persistent storage services to ensure the availability of configuration information. In addition, the first management node and all second management nodes store the same topology table update operation key-value pair in the KV database to ensure the data consistency and integrity of the entire distributed system, and finally achieve high availability and fault tolerance. Finally, through the persistent storage of the KV database, it is also possible to realize that different MON nodes simultaneously update the service node topology table according to a certain topology table update operation key-value pair, so that the consistency of all management nodes in the distributed file system for the service node topology table with blacklist information added can be ensured.

[0118] In one embodiment, the topology table update operation key-value pair contains value data, which is specific data associated with the key data. It can be any data type, including strings, numbers, Boolean values, objects, etc. The value data can store various types of data. In the embodiment of the present invention, it is necessary to store the topology table update operation information through the value data. Therefore, when generating the topology table update operation key-value pair according to the topology table update operation information, it is necessary to determine the value data in the key-value pair according to the topology table update operation information. Specifically, the topology table update operation information can be serialized to obtain the serialized topology table update operation information; the serialized topology table update operation information is used as the value data to generate the topology table update operation key-value pair.

[0119] In one embodiment, serialization refers to converting an object or data structure into a format that can be stored or transmitted so as to be transferred, stored or reconstructed in different environments. The reason why the topology table update operation information is serialized in the embodiment of the present invention is that if the topology table update operation information is not serialized, it cannot be directly stored in a persistent medium such as a disk or a database. In this way, each time the program is closed or the system is restarted, the update operation needs to be rebuilt, resulting in the data being unable to remain persistent. The unserialized topology table update operation information cannot be directly transmitted over the network or transmitted across systems. It is necessary to convert it into a transmittable form to ensure that it is accurately applied to the target system to maintain data consistency. Different systems or programming languages ​​may have different data representations and storage methods. Unserialized update operations may not be shared or exchanged between different systems, and additional data conversion and processing are required. In addition, the serialized topology table update operation information is used as value data, and the main purpose of generating key-value pairs is to accurately associate the topology table update operation information and the corresponding key data during storage or transmission. Based on this, the embodiment of the present invention can perform serialization operations on the topology table update operation information to obtain the serialized topology table update operation information, and use the serialized topology table update operation information as value data to generate a topology table update operation key-value pair. Therefore, the embodiment of the present invention converts the topology table update operation information into a form that can be stored and transmitted through serialization, so as to facilitate persistent storage or cross-system transmission in different environments and maintain data consistency. In addition, by serializing the topology table update operation information, it can be associated with the corresponding key data as value data to form a key-value pair storage.

[0120] In one embodiment, the serialization process needs to convert the topology table update operation information into a byte stream or other storable format, and the specific serialization method depends on the programming language and serialization library used. For example, in the embodiment of the present invention, a suitable serialization library can be selected according to the programming language and requirements, such as pickle, JSON, etc., which are commonly used in Python, and then the data structure of the topology table update operation information is defined, and the relevant information of the update operation is organized into a specific data structure, such as a dictionary, a class object, etc., to ensure that all necessary update information is included, so that the topology table update operation information can be correctly restored after serialization. Then, the embodiment of the present invention calls the serialization method, and uses the method provided by the selected serialization library to serialize the topology table update operation information data structure into a byte stream or other storable format. These methods usually accept the topology table update operation information data structure as input and return the serialized result. Finally, the serialization result is stored, and the serialized update operation result is associated with the corresponding key data to form a key-value pair, that is, a topology table update operation key-value pair, which can be stored in a file, a database, or transmitted to other systems via a network. In the embodiment of the present invention, storage in a KV database is taken as an example, but it is not intended to be a limitation on the embodiment of the present invention. It should be noted that when serializing update operations, it is necessary to ensure that the serialized operations are reversible, that is, the serialized topology table update operation information can be restored back to the original topology table update operation information through deserialization, so that the topology table update operation information can be correctly reapplied to the data when needed.

[0121] In one embodiment, the topology table update operation key-value pair contains key data, which is a unique identifier used to identify data entries. It can be a string, an integer or other data type, depending on the implementation and requirements of the database. Each key data is unique, and different key data corresponds to different data entries. In the embodiment of the present invention, the version of the blacklist information needs to be represented by key data. Therefore, when the serialized topology table update operation information is used as the value data to generate the topology table update operation key-value pair, it is necessary to determine the value data in the key-value pair according to the topology table update operation information, and specifically, the key value number corresponding to the serialized topology table update operation information can be determined; the key value number is used as the key data, and the serialized topology table update operation information is used as the value data to generate the topology table update operation key-value pair.

[0122] In one embodiment, the blacklist information in the distributed file system can be continuously updated. This is because new security threats and malicious behaviors may continue to emerge over time, so the blacklist information needs to be dynamically updated to identify and block these new threats. In addition, the system usually needs to continuously monitor and detect potential threats. By dynamically updating the blacklist, new suspicious IP addresses, URLs, domain names, etc. can be added to strengthen the monitoring and defense capabilities against potential threats. Moreover, sometimes certain IP addresses, etc. may be mistakenly included in the blacklist information, or some information that was once considered a threat may no longer be valid. By dynamically updating the blacklist information, false alarms can be corrected and outdated information can be deleted, reducing false alarms and unnecessary blocking. Since the blacklist information is constantly updated and transformed, the temporary service node topology table containing the blacklist information is also constantly updated and transformed, so the topology table update operation information is also constantly updated and transformed. In order to ensure the consistency of all management nodes in the distributed file system in executing the topology table update operation information, it is necessary to clarify the key data of the topology table update operation key-value pair. The embodiment of the present invention can determine the key number corresponding to the serialized topology table update operation information, use the key number as the key data, and use the serialized topology table update operation information as the value data, and finally generate the topology table update operation key-value pair, so as to determine the uniqueness of each topology table update operation key-value pair. The unique topology table update operation key-value pair is sent to the second management node. Therefore, when the second management node triggers the execution of the same topology table update, it can be the same as the first management node and all the second management nodes. For the service node topology table with the blacklist information added, it can be synchronized and updated, thereby ensuring the consistency of all management nodes in the distributed file system for the service node topology table with the blacklist information added.

[0123] In one embodiment, the key value number corresponding to the serialized topology table update operation information needs to be determined, and the following aspects need to be considered. For example, it is necessary to ensure that each topology table update operation information has a unique key value number, and a self-increasing integer can be used as the number, or a unique number can be generated by a globally unique identifier (UUID) or the like. In addition, the determined key value number is easy to index, and the selected key value number should be able to be easily indexed and retrieved, so a continuous integer number can be used, which makes it easier to implement fast indexing and positioning operations, and in order to facilitate data management, according to specific needs, a specific number allocation strategy can be considered, such as sequential numbering, hash numbering, etc., which depends on the scale of the data, the access mode, and the use scenario of the numbering, and the embodiment of the present invention does not make specific restrictions. In addition, if it is necessary to consider the relevance of the topology table update operation information with other data, its number can be associated with the key value of other data to establish a connection between the data. Based on this, according to the above considerations, the embodiment of the present invention uses a self-increasing integer as the key value number. For example, the initial number is 1. Each time a new serialized topology table update operation information is generated, the number is increased by 1, which can ensure that each topology table update operation information has a unique key value number. This numbering method is convenient for indexing, retrieval and management. When you need to find a specific topology table update operation information, you only need to use the corresponding number to search. It should be noted that in the embodiment of the present invention, when forming a corresponding temporary service node topology table according to different blacklist information, a corresponding version number will be added to the temporary service node topology table, and the temporary service node topology table with the blacklist information added is the operation value of the topology table update operation information. Therefore, the version number corresponding to the temporary service node topology table can be used as the key value number of the topology table update operation information.

[0124] In one embodiment, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair, and the first management node will subsequently send a blacklist update instruction to each second management node, so it is necessary to receive feedback from each second management node on the completion of persistent storage of the topology table update operation key-value pair. The blacklist processing method of the distributed file system provided by the embodiment of the present invention may also include but is not limited to the following steps:

[0125] Receiving response information sent by each second management node after persistently storing the key-value pairs of the topology table update operation;

[0126] A response mark corresponding to each second management node is set according to the response information, wherein the response mark is used to indicate that persistent storage of the key-value pair of the topology table update operation is completed.

[0127] In one embodiment, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair, and the first management node will subsequently send a blacklist update instruction to each second management node, so that all second management nodes update the service node topology table in each second management node to a temporary service node topology table with blacklist information added based on the stored topology table update operation key-value pair. The steps of the above embodiment describe that in the first stage, the topology table update operation key-value pair is first sent to all second management nodes and persistently stored. Then, the start of the second stage requires receiving feedback on the completion of the first stage.

[0128] In one embodiment, each second management node persistently stores the topology table update operation key-value pairs, which is implemented by storing the topology table update operation key-value pairs in a KV database established in each second management node. The second management node is a MON node. If a MON node fails, such as a hardware failure, an operating system crash, or a network problem, or a MON node restarts, the node cannot work properly, and the received topology table update operation key-value pairs may be lost or damaged. Based on this, each second management node can be provided with a KV database, such as Figure 8 As shown, Figure 8A schematic diagram of data storage through a KV database 811 in a second management node 810 is shown. In a KV database, key data is used to uniquely identify a data entry, and value data stores data associated with the key. By specifying a key, the corresponding value can be quickly stored, updated, or retrieved, thereby achieving efficient access to data. The KV database 811 of the embodiment of the present invention can be of various types, including Redis, RocksDB, Etcd, etc., providing high-performance, low-latency data access capabilities for the embodiment of the present invention, and supporting a variety of data structures and functions, such as lists, hashes, sets, etc., which are not specifically limited by the embodiment of the present invention. Therefore, by setting up a KV database 811, each second management node 810 can store the topology table update operation key-value pairs in the local KV database 811 for persistent storage. Even after each second management node 810 fails or restarts, the configuration information can still be reliably restored, which can enable the MON node to quickly restore the cluster state after a failure, and perform necessary reconstruction operations to minimize service interruption time and achieve persistent storage of data. On the premise of meeting the requirements of the embodiments of the present invention, other databases for permanent storage of key-value pairs of topology table update operations can also be selected, such as relational databases (such as MySQL, PostgreSQL), non-relational databases (such as MongoDB), etc., and corresponding data tables are created in the selected database to store key-value pair data. It is necessary to ensure that the structure of the data table matches the format of the key-value pairs, and design appropriate fields and indexes according to actual needs. In the process of storing in the database, the key-value pairs are serialized and converted into a format suitable for storage, such as JSON, XML, binary, etc., and when the data needs to be read, the deserialization operation is performed to restore the data to the key-value pair form. The embodiments of the present invention do not make specific restrictions on this. The embodiments of the present invention only take the provision of KV database 811 for storage as an example for illustration, but it does not constitute a limitation on the present invention.

[0129] In one embodiment, after each second management node persistently stores the topology table update operation key-value pair, it will send response information to the first management node respectively. After receiving the response information, the first management node sets the response mark corresponding to each second management node according to the response information. The response mark is used to indicate that the persistent storage of the topology table update operation key-value pair is completed. The response mark can be used to determine that the second management node of the sent information has received the topology table update operation key-value pair. For the second management nodes that have previously sent the topology table update operation key-value pair but have not received the response information, it is judged that these nodes have problems such as network delays and node failures. When the blacklist is updated subsequently, the relevant instructions will no longer be sent to these nodes, and the blacklist update instructions will only be sent to the second management nodes that sent the response information. Therefore, the embodiment of the present invention tells all the second management nodes to update the operation records in the first stage, and updates the data in the second stage. This ensures that the data on all the second management nodes are consistent, and all management nodes in the distributed file system can achieve consistency in the execution topology table update operation information and consistency in the service node topology table with blacklist information added. Therefore, all management nodes in the distributed file system can have the same service node topology table with blacklist information added.

[0130] In one embodiment, if Fig. 9 As shown, Fig. 9The schematic diagram of the first management node 910 performing the topology table update operation key-value pair transmission in the first stage is shown, wherein the first management node (MON0) 910 is the master node, the second management node (MON1) 920 and the second management node (MON2) 930 are slave nodes, MON0 first serializes the update operation of the Mdsmap registered with the blacklist information, and uses the number 1 as the key data (the subsequent update operation numbers are incremented in sequence), and the serialized update Mdsmap operation is used as the value data to obtain the topology table update operation key-value pair, and the topology table update operation key-value pair is stored in the local KV database 911 for persistent storage, and then the topology table update operation key-value pair numbered 1 is sent to MON1 and MON2 respectively. MON1 and MON2 receive the topology table. After the operation key-value pairs are updated, they are also stored in the local KV database (921 and 931) for persistent storage. Then, after MON1 and MON2 complete the storage, response information is generated and sent to the MON0 node. After receiving the response data sent by each second management node (920 and 930), MON0 generates response tags 912 based on its own storage situation. These tags respectively indicate which MONs have sent responses to complete the first stage. The management nodes that have stored the topology table update operation key-value pairs are recorded through the response tags 912. Subsequently, the service node topology table can be updated for these management nodes. After MON0 receives all responses from the first stage of MON, it enters the second stage of blacklist consistency synchronization.

[0131] In one embodiment, the first management node receives response information sent by each second management node, indicating that these second management nodes have received the topology table update operation key-value pairs in the first stage. Therefore, in the subsequent second stage, these second management nodes can be instructed to update the blacklist. The following introduces the process of blacklist update in the second stage of blacklist consistency synchronization.

[0132] Step 440: according to the value data in the topology table update operation key-value pair, the service node topology table in the first management node is updated to a temporary service node topology table with blacklist information added, and a corresponding blacklist update instruction is generated.

[0133] In one embodiment, after receiving the response information sent by each second management node, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the key-value pair of the topology table update operation. The first management node itself stores a service node topology table, which can be a topology table after the blacklist information is updated in history, but there is currently an updated version of the temporary service node topology table containing blacklist information, so the current service node topology table can be updated to a temporary service node topology table containing blacklist information, so that the blacklist information is added to the updated service node topology table, and since the service node topology table with the blacklist information added records the address information of the metadata service node, the distributed file system can be accessed through the service node topology table, so when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with the blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests.

[0134] In one embodiment, after the service node topology table in the first management node is updated to a temporary service node topology table with blacklist information added, the first management node completes the update of the service node topology table, and the updated service node topology table contains the blacklist information, and then the first management node generates a blacklist update instruction. The first management node in the embodiment of the present invention is the master node in the MON cluster, so it is necessary to send a response control instruction to each second management node that has successfully stored the topology table update operation key-value pair. This process is implemented through the blacklist update instruction, that is, the blacklist update instruction generated by the first management node can be sent to the second management node that has successfully stored the topology table update operation key-value pair, so that these second management nodes can subsequently update and synchronize the service node topology table.

[0135] In one embodiment, if Fig.10 As shown, Fig.10FIG. 1 is a schematic diagram showing the first management node 1010 updating the service node topology table in the second phase, wherein the first management node (MON0) 1010 is the master node, the second management node (MON1) 1020 and the second management node (MON2) 1030 are slave nodes, and the blacklist information includes the restricted network segment 10.0.1.0 / 24. After MON0, which is the master MON node, receives the response information of each slave node in the first phase, MON0 takes out the topology table update operation key-value pair from the KV database 1011, and the value data of the topology table update operation key-value pair is the serialized topology table update operation information. Therefore, MON0 can deserialize the serialized topology table update operation information and then obtain the topology table update operation information, and the topology table update operation information The operation value of the information is a temporary service node topology table with blacklist information added, so MON0 executes the KV operation of updating the service node topology table, and takes the service node topology table containing the blacklist information as the value data, and updates the service node topology table in MON0 to a temporary service node topology table with the blacklist information added. In this way, the service node topology table in the memory of the master MON0 contains the blacklist information with the restricted network segment of 10.0.1.0 / 24. After the update, MON0 sends a blacklist update instruction to the slave nodes MON1 and MON2, MON1 extracts data from the KV database 1021 to execute the KV operation of updating the service node topology table, and MON2 extracts data from the KV database 1031 to execute the KV operation of updating the service node topology table, which will not be repeated here.

[0136] In one embodiment, in order to flexibly manage and record different versions of service node topology tables while ensuring that blacklist information is correctly applied and protected, a temporary service node topology table containing blacklist information is required as a key-value pair of value data for updating the service node topology table in the first management node. Therefore, according to the value data in the topology table update operation key-value pair, the service node topology table in the first management node is updated to a temporary service node topology table with the blacklist information added. According to the value data in the topology table update operation key-value pair, a topology table update key-value pair can be generated, wherein the value data of the topology table update key-value pair is the temporary service node topology table with the blacklist information added; according to the topology table update key-value pair, the service node topology table in the first management node is updated to a temporary service node topology table with the blacklist information added.

[0137] In one embodiment, after receiving the response information sent by the second management node, the first management node enters the second stage of blacklist consistency synchronization. The embodiment of the present invention needs to generate a new key-value pair and use the new key-value pair to update the service node topology table in the first management node. Because the key-value pair provides a convenient way to transmit and store data, by storing the relevant information of the topology table update in the key-value pair, it can be transmitted as a whole, and the update operation can be uniquely identified according to the key. This method can simplify the data transmission and processing process, making the update operation easier to control. Therefore, in the embodiment of the present invention, a topology table update key-value pair can be generated according to the value data in the topology table update operation key-value pair, and the value data in the topology table update operation key-value pair is a temporary service node topology table with blacklist information added, so the value data in the newly generated topology table update key-value pair is also a temporary service node topology table with blacklist information added. Then the first management node obtains the previously stored service node topology table, which does not contain the new blacklist information and therefore needs to be discarded. Subsequently, if the service node topology table needs to be updated, the service node topology table in the first management node can be updated to a temporary service node topology table with blacklist information added according to the topology table update key-value pair, that is, the value data in the newly generated topology table update key-value pair is used. According to the temporary service node topology table with blacklist information added, as the updated service node topology table in the first management node, the subsequent work can be based on the updated service node topology table containing blacklist information. Since the service node topology table with blacklist information added will record the address information of the metadata service node, the access to the distributed file system can be realized through the service node topology table. Therefore, when the first management node in the distributed file system receives an access request from a client, it can filter the client according to the service node topology table with blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process of the client is realized according to the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for distributed file systems of different operating systems, the management node will store the service node topology table, so the embodiment of the present invention can enable distributed file systems of different operating systems to realize client filtering based on blacklist information, that is, the embodiment of the present invention can be applicable to distributed file systems of different operating systems, so it has good universal applicability.

[0138] In one embodiment, the value data in the topology table update operation key-value pair is the serialized topology table update operation information. The extracted serialized topology table update operation information needs to be deserialized before it can be used as the value data in the topology table update operation key-value pair, and then the topology table update key-value pair is generated in combination with the key data. Therefore, according to the value data in the topology table update operation key-value pair, the process of generating the topology table update key-value pair can specifically include deserializing the serialized topology table update operation information to obtain the topology table update operation information; obtaining the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added; determining the version number corresponding to the temporary service node topology table with the blacklist information added; using the version number as the key data and the temporary service node topology table with the blacklist information added as the value data to generate the topology table update key-value pair.

[0139] In one embodiment, deserialization is the process of converting serialized (or encoded) data back to its original form or object. In the data processing process, serialization refers to the operation of converting a data structure or object into a linear format for storage, transmission or interaction between different systems. Through deserialization, the serialized data can be restored to the original data structure or object for further processing and use. In the second stage of blacklist consistency synchronization in an embodiment of the present invention, the value data in the stored topology table update operation key-value pair can be extracted from the KV database, that is, the serialized topology table update operation information is extracted, and the serialized topology table update operation information is deserialized to obtain the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added. Therefore, the temporary service node topology table with the blacklist information added can be used as the value data of the newly generated topology table update key-value pair. The key data in the newly generated topology table update key-value pair is used to represent the unique identifier of the topology table update key-value pair. Therefore, in an embodiment of the present invention, when forming a corresponding temporary service node topology table according to different blacklist information, a corresponding version number will be added to the temporary service node topology table. Therefore, the version number can be used as the key data. Finally, a topology table update key-value pair can be generated with the version number as the key data and the temporary service node topology table with the blacklist information added as the value data.

[0140] In one embodiment, serialization and deserialization are a pair of corresponding processes, and the data can be restored to the original format using the corresponding deserialization method according to the serialization method selected during serialization. When serializing the topology table update operation information, the type of the first opcode can be recorded first, such as represented by a 1, and then the length of the opcode can be checked, and then the value of the opcode can be written to form data containing the type, length and opcode value after the serial number, and then spliced ​​together to form a string as the serialized topology table update operation information. Then, deserialization is also parsed in this way. After receiving a string, first look at what type, length and opcode value it is, and then read such a long byte data with this length, which is the value, and finally obtain the deserialized topology table update operation information. It should be noted that common serialization methods include JSON, XML, etc. When deserialization is performed by other serialization methods, the topology table update operation information is parsed according to the deserialized data structure, for example, according to the data structure and field definition before serialization, the deserialized data is parsed to obtain the topology table update operation information, and this embodiment of the present invention is not specifically limited.

[0141] In one embodiment, if Fig.10 As shown, Fig.10 The figure shows a schematic diagram of the first management node updating the service node topology table in the second stage, wherein the first management node (MON0) is the master node, the second management node (MON1 and MON2) is the slave node, and the blacklist information contains the restricted network segment of 10.0.1.0 / 24. After MON0, which is the master MON node in the first stage, receives the response information of each slave node, MON0 takes out the topology table update operation key-value pair from the KV database, and the value data of the topology table update operation key-value pair is the serialized topology table update operation information. Therefore, MON0 can deserialize the serialized topology table update operation information and then obtain the topology table update operation information, and the operation value of the topology table update operation information is to add A temporary service node topology table with blacklist information is added. Therefore, when MON0 executes the KV operation of updating the service node topology table, a new KV pair is generated in KV, with the version number corresponding to the temporary service node topology table with blacklist information added as the key data and the service node topology table containing the blacklist information as the value data, that is, the topology table update key-value pair. According to the new KV pair, the service node topology table in MON0 is updated to the temporary service node topology table with blacklist information added. In this way, the service node topology table in the memory of the main MON0 contains the blacklist information with the restricted network segment of 10.0.1.0 / 24. After the update, MON0 sends a blacklist update instruction to the slave nodes MON1 and MON2.

[0142] Step 450: Send the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added.

[0143] In one embodiment, after the first management node completes the process of updating the service node topology table in the first management node to a temporary service node topology table with blacklist information added, a blacklist update instruction is generated and sent to all second management nodes. After all second management nodes receive the blacklist update instruction, they perform the update operation on the service node topology table together in the second stage of blacklist consistency synchronization. Specifically, after each second management node receives the blacklist update instruction, it is also necessary to generate a new key-value pair, and use the new key-value pair to update the service node topology table in each second management node, because the key-value pair provides a convenient way to transmit and store data. By storing the relevant information of the topology table update in the key-value pair, it can be transmitted as a whole, and the update operation can be uniquely identified according to the key. This method can simplify the process of data transmission and processing, making the update operation easier to control. Therefore, in the embodiment of the present invention, at each second management node, a topology table update key-value pair can be generated based on the value data in the topology table update operation key-value pair obtained in the first stage, and the value data in the topology table update operation key-value pair is a temporary service node topology table with blacklist information added, so the value data in the newly generated topology table update key-value pair is also a temporary service node topology table with blacklist information added. Then, each second management node obtains the previously stored service node topology table, which does not contain the new blacklist information and therefore needs to be discarded. Subsequently, if the service node topology table needs to be updated, the service node topology table in each second management node can be updated to a temporary service node topology table with blacklist information added based on the topology table update key-value pair, that is, the newly generated topology The value data in the table update key-value pair is added with a temporary service node topology table with blacklist information, which serves as the updated service node topology table in each second management node. Subsequent work can be performed based on the updated service node topology table containing the blacklist information. Since the address information of the metadata service node is recorded in the service node topology table with the blacklist information, access to the distributed file system can be achieved through the service node topology table. Therefore, when any second management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with the blacklist information added. This can not only avoid attacks on the distributed file system by a large number of useless access requests, but also reduce network bandwidth consumption and access processing pressure caused by processing useless access requests.In addition, since the filtering process for the client is implemented based on the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, and thus has good universal applicability.

[0144] In this embodiment, through the blacklist processing method of the distributed file system including the above steps 410 to 450, when receiving the blacklist information, the first management node first adds the blacklist information into the temporary service node topology table that records the address information of the metadata service node, and then uses the temporary service node topology table with the blacklist information added as the operation value to generate the topology table update operation information, and generates the topology table update operation key-value pair according to the topology table update operation information, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information, and then sends the topology table update operation key-value pair to all the second management nodes in the distributed file system; because the value data of the topology table update operation key-value pair is obtained by the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added, after sending the topology table update operation key-value pair to all the second management nodes in the distributed file system, the synchronization of the topology table update operation information between the first management node and each second management node can be achieved, thereby ensuring the consistency of all management nodes in the distributed file system in executing the topology table update operation information. After sending the topology table update operation key-value pair to all second management nodes in the distributed file system, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added; since the first management node will send the corresponding blacklist update instruction to all second management nodes to trigger each second management node to perform the same topology table update after updating its service node topology table to the temporary service node topology table with blacklist information added, it is possible to achieve synchronization between the first management node and each second management node for the service node topology table with blacklist information added, thereby ensuring the consistency of the service node topology table with blacklist information added by all management nodes in the distributed file system.In addition, since all management nodes in the distributed file system can achieve consistency in the execution of topology table update operation information and consistency in the service node topology table with blacklist information added, all management nodes in the distributed file system can have the same service node topology table with blacklist information added, and since the service node topology table with blacklist information added records the address information of the metadata service node, the distributed file system can be accessed through the service node topology table, so when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process of the client is implemented according to the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for distributed file systems of different operating systems, the management node will store the service node topology table, so the embodiment of the present invention can enable distributed file systems of different operating systems to implement client filtering based on blacklist information, that is, the embodiment of the present invention can be applicable to distributed file systems of different operating systems, so it has good universal applicability.

[0145] Fig.11 This is a flowchart of another blacklist processing method of a distributed file system provided by an embodiment of the present invention. The blacklist processing method can be executed by the second management node in the distributed file system, or by the first management node and the second management node in the distributed file system. In the embodiment of the present invention, the method is executed by the second management node in the distributed file system as an example for description. Fig.11 The blacklist processing method of the distributed file system may include but is not limited to steps 1110 to 1130.

[0146] Step 1110: Receive a topology table update operation key-value pair sent by the first management node in the distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to the topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as the operation value, and the temporary service node topology table records the address information of the metadata service node.

[0147] In one embodiment, a distributed file system contains multiple nodes, including a first management node and a second management node, and there are multiple second management nodes. Further, the first management node and the second management node are both MON nodes, and multiple MON nodes form a MON cluster, and the MON cluster is divided into a master node and a slave node, the first management node is the master node, and the second management node is the slave node. The master node can provide read and write capabilities, and the slave node can only provide read functions. Therefore, after the topology table update operation information is generated in the first management node, it is not only necessary to store the topology table update operation information so that the service node topology table stored in the first management node can be updated later, but also the topology table update operation information needs to be sent to all second management nodes so that all second management nodes can store the same topology table update operation information together. The second management node that has stored the topology table update operation information can also complete the update operation of the service node topology table, and realize the synchronization between the first management node and each second management node for the service node topology table with blacklist information added, so as to ensure the consistency of all management nodes in the distributed file system for the service node topology table with blacklist information added. Therefore, the process of realizing synchronization of the service node topology table containing blacklist information in the embodiment of the present invention, that is, the process of blacklist consistency synchronization, is divided into two stages. In the first stage, the topology table update operation key-value pairs are first sent to all second management nodes and persistently stored. In the subsequent second stage, the blacklist update operation is performed on the second management node that successfully stores the topology table update operation information. This avoids the situation where some second management nodes fail to execute data updates due to network delays, node failures and other problems. The synchronization of the service node topology table with blacklist information added can be realized between the first management node and each second management node, thereby ensuring the consistency of the service node topology table with blacklist information added for all management nodes in the distributed file system.

[0148] In one embodiment, the value data in the topology table update operation key-value pair is obtained according to the topology table update operation information, and the topology table update operation information is generated by the first management node using the temporary service node topology table with blacklist information added as the operation value, and the temporary service node topology table records the address information of the metadata service node. Specifically, after the first management node receives the blacklist during the access control process, the blacklist information is added to the temporary service node topology table with the address information of the metadata service node, so that the service node topology table in the first management node and all the second management nodes can be updated according to the temporary service node topology table with the blacklist information added, and the temporary service node topology table also records the address information of the metadata service node, and the address information of the metadata service node can be updated to the service node topology table in the first management node and all the second management nodes according to the temporary service node topology table, so as to achieve access to the metadata service node through the service node topology table in each node, that is, to achieve access to the distributed file system. After the blacklist information is added to the temporary service node topology table recording the address information of the metadata service node, the first management node can use the temporary service node topology table with the blacklist information added as the operation value to generate the topology table update operation information. Among them, the temporary service node topology table with the blacklist information added is a temporary service node topology table, which contains the blacklist information, and the blacklist is usually used to specify some nodes or resources that are restricted or prohibited from access. Therefore, the blacklist information in the topology table may indicate that the blacklist list of certain nodes needs to be updated or modified. And using the temporary service node topology table with the blacklist information added as the operation value means that the update operation of the topology table will use the temporary service node topology table with the blacklist information added as input, that is, after obtaining the temporary service node topology table with the blacklist information, it is necessary to update the existing service node topology table, and update the topology table by adding the new blacklist information to the existing service node topology table. Therefore, after adding the temporary service node topology table with the blacklist information as the operation value, the topology table update operation information is generated to perform the update operation of the actual topology table.

[0149] In one embodiment, the temporary service node topology table is a temporary Mdsmap. The first management node stores a service node topology table, namely, Mdsmap. Mdsmap is used to manage the mapping of metadata service nodes (MDS) in the distributed file system. It contains information about the status, location, and connection of the MDS. Therefore, the specific process of generating the topology table update operation information with the temporary service node topology table added with blacklist information as the operation value can first obtain the Mdsmap in the current first management node. In this process, appropriate tools (such as Ceph command line tools or related APIs) can be used to obtain the current Mdsmap and save it as a copy of the Mdsmap so that it can be modified on its basis. Then, as needed, the existing copy of the Mdsmap is modified, which may include adding, deleting or modifying MDS instances, adjusting weights, changing positions, etc., ensuring that the modified target is consistent with the temporary Mdsmap with blacklist information added. Then, the topology table update operation information is generated, and according to the modification of the existing topology table copy, the operation information for performing the topology table update is generated, which may include the operation type (addition, deletion, modification), MDS instance identifier, location information, etc. The operation information can be an instruction list or a corresponding command line parameter. After the topology table update operation information is generated, the topology table update operation can be performed later, such as using the generated topology table update operation information, applying it to the corresponding node in the distributed file system, and performing the topology table update operation.

[0150] In one embodiment, after receiving the topology table update operation key-value pair sent by the first management node in the distributed file system, the second management node also needs to persistently store the topology table update operation key-value pair and feed back corresponding information to the first management node. Therefore, the blacklist processing method of the distributed file system provided in the embodiment of the present invention may also include but is not limited to the following steps:

[0151] Persistently store the key-value pairs of topology table update operations and generate corresponding response information;

[0152] The response information is sent to the first management node, so that the first management node sets a response mark corresponding to the second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the key-value pair of the topology table update operation is completed.

[0153] In one embodiment, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair, and the first management node will subsequently send a blacklist update instruction to each second management node, so that all second management nodes update the service node topology table in each second management node to a temporary service node topology table with blacklist information added based on the stored topology table update operation key-value pair. The steps of the above embodiment describe that in the first stage, the topology table update operation key-value pair is first sent to all second management nodes and persistently stored. Then, when the second stage is started, the first management node needs to receive feedback from each second management node on the completion of the first stage.

[0154] In one embodiment, each second management node persistently stores the topology table update operation key-value pairs, which is implemented by storing the topology table update operation key-value pairs in a KV database established in each second management node. The second management node is a MON node. If a MON node fails, such as a hardware failure, an operating system crash, or a network problem, or a MON node restarts, the node cannot work properly, and the received topology table update operation key-value pairs may be lost or damaged. Based on this, each second management node can be provided with a KV database, such as Figure 8 As shown, Figure 8A schematic diagram of data storage through a KV database 811 in each second management node 810 is shown. In the KV database 811, the key data is used to uniquely identify a data entry, and the value data stores the data associated with the key. By specifying the key, the corresponding value can be quickly stored, updated or retrieved, thereby achieving efficient access to data. The KV database 811 of the embodiment of the present invention can be of various types, including Redis, RocksDB, Etcd, etc., which provide high-performance, low-latency data access capabilities for the embodiment of the present invention, and support a variety of data structures and functions, such as lists, hashes, sets, etc., which are not specifically limited by the embodiment of the present invention. Therefore, by setting up the KV database 811, each second management node 810 can store the topology table update operation key-value pairs in the local KV database 811 for persistent storage. Even after each second management node 810 fails or restarts, the configuration information can still be reliably restored, which can enable the MON node to quickly restore the cluster state after failure, and perform necessary reconstruction operations to minimize service interruption time and achieve persistent storage of data. On the premise of meeting the requirements of the embodiments of the present invention, other databases for permanent storage of key-value pairs of topology table update operations can also be selected, such as relational databases (such as MySQL, PostgreSQL), non-relational databases (such as MongoDB), etc., and corresponding data tables are created in the selected database to store key-value pair data. It is necessary to ensure that the structure of the data table matches the format of the key-value pairs, and design appropriate fields and indexes according to actual needs. In the process of storing in the database, the key-value pairs are serialized and converted into a format suitable for storage, such as JSON, XML, binary, etc., and when the data needs to be read, the deserialization operation is performed to restore the data to the key-value pair form. The embodiments of the present invention do not make specific restrictions on this. The embodiments of the present invention only take the provision of KV database 811 for storage as an example for illustration, but it does not constitute a limitation on the present invention.

[0155] In one embodiment, after each second management node persistently stores the topology table update operation key-value pair, it will send response information to the first management node respectively. After receiving the response information, the first management node sets the response mark corresponding to each second management node according to the response information. The response mark is used to indicate that the persistent storage of the topology table update operation key-value pair is completed. The response mark can be used to determine that the second management node of the sent information has received the topology table update operation key-value pair. For the second management nodes that have previously sent the topology table update operation key-value pair but have not received the response information, it is judged that these nodes have problems such as network delays and node failures. When the blacklist is updated subsequently, the relevant instructions will no longer be sent to these nodes, and the blacklist update instructions will only be sent to the second management nodes that sent the response information. Therefore, the embodiment of the present invention tells all the second management nodes to update the operation records in the first stage, and updates the data in the second stage. This ensures that the data on all the second management nodes are consistent, and all management nodes in the distributed file system can achieve consistency in the execution topology table update operation information and consistency in the service node topology table with blacklist information added. Therefore, all management nodes in the distributed file system can have the same service node topology table with blacklist information added.

[0156] In one embodiment, the second management node is a slave node, the first management node is a master node, and before the second management node receives the topology table update operation key-value pair sent by the first management node in the distributed file system, if the second management node receives the blacklist information sent by the client, the blacklist information needs to be forwarded to the first management node for processing. Therefore, the blacklist processing method of the distributed file system provided in the embodiment of the present invention may also include but is not limited to the following steps:

[0157] When the blacklist information is received, the blacklist information is forwarded to the first management node, so that the first management node adds the blacklist information into the temporary service node topology table.

[0158] In one embodiment, the distributed file system may be CephFS, the first management node may be a MON node in CephFS, and the temporary service node topology table is a temporary Mdsmap in the MON node. In some distributed file systems, the MON node may participate in the access control process, and may be responsible for managing the metadata and coordination tasks of the cluster, and the metadata includes access permission information about files and directories. In the embodiment of the present invention, by setting appropriate access control policies in the distributed file system, the MON node may perform access control on the file system. It should be noted that there are multiple second management nodes, which are the various MON nodes in the cluster, and other components and nodes in the distributed file system may also participate in the access control process to ensure the security of the entire system and the protection of data, which is not specifically limited in the embodiment of the present invention.

[0159] In one embodiment, the blacklist information may be obtained from the second management node, for example, Figure 6 The following is an example of the communication connection between the command line client and the MON cluster: Figure 6 As shown, a distributed file system may be provided with multiple MON nodes, and the MON nodes are divided into master nodes and slave nodes. The first management node (MON0) 611 is the master node, and the second management node (MON1) 612 and the second management node (MON2) 613 are slave nodes. The master node can provide read and write capabilities, and the slave node can only provide read functions. The master node and at least one slave node can form a MON cluster 610, and the command line client 620 can obtain the IP list of the MON cluster 610, and the IP list contains the IP addresses of each MON node. Then the command line client can randomly select a MON node from the IP list of the MON cluster 610 to send a blacklist registration message and blacklist information. Figure 6 In the example, the command line client selects the MON1 node to send the blacklist information, and both the MON1 node and the MON2 node are the second management nodes (612 and 613), and MON0 is the first management node 611. Therefore, the MON1 node has only a read function. After reading the blacklist information, the MON1 node forwards the blacklist information to the MON0 node.

[0160] In one embodiment, taking the command line client sending blacklist information to the second management node as an example, the command line client in the embodiment of the present invention can send the restricted network segment and the fixed IP address as blacklist information to any second management node. The restricted network segment indicates that all addresses in this network segment cannot access the distributed file system, and the fixed IP address indicates that the distributed file system cannot be accessed under this fixed IP address. For example, if the restricted network segment is 10.0.1.0 / 24, it means that all addresses with IP addresses between 10.0.1.0 and 10.0.1.255 cannot access the distributed file system. If the fixed IP address is 10.0.2.24, it means that the fixed IP address is restricted from accessing the distributed file system. Figure 6 In the example, the command line client 620 selects the MON1 node to send the blacklist information, which includes the restricted network segment 10.0.1.0 / 24 and the fixed IP address 10.0.2.24. The MON1 node and the MON2 node are both second management nodes (612 and 613), and MON0 is the first management node 611. Therefore, the MON1 node has only a read function. After reading the blacklist information, the MON1 node forwards the blacklist information including the restricted network segment 10.0.1.0 / 24 and the fixed IP address 10.0.2.24 to the MON0 node.

[0161] Step XX20: receiving a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair;

[0162] In one embodiment, after receiving the response information sent by each second management node, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the key-value pair of the topology table update operation. The first management node itself stores a service node topology table, which can be a topology table after the blacklist information is updated in history, but there is currently an updated version of the temporary service node topology table containing blacklist information, so the current service node topology table can be updated to a temporary service node topology table containing blacklist information, so that the blacklist information is added to the updated service node topology table, and since the service node topology table with the blacklist information added records the address information of the metadata service node, the distributed file system can be accessed through the service node topology table, so when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with the blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests.

[0163] In one embodiment, after the service node topology table in the first management node is updated to a temporary service node topology table with blacklist information added, the first management node completes the update of the service node topology table, and the updated service node topology table contains the blacklist information, and then the first management node generates a blacklist update instruction. The first management node in the embodiment of the present invention is the master node in the MON cluster, so it is necessary to send a response control instruction to each second management node that has successfully stored the topology table update operation key-value pair. This process is implemented through the blacklist update instruction, that is, the blacklist update instruction generated by the first management node can be sent to the second management node that has successfully stored the topology table update operation key-value pair, so that these second management nodes can subsequently update and synchronize the service node topology table.

[0164] In one embodiment, if Fig.10 As shown, Fig.10The schematic diagram of the first management node updating the service node topology table in the second stage is shown, wherein the first management node is the master node, the second management node is the slave node, and the blacklist information contains the restricted network segment of 10.0.1.0 / 24. After MON0, which is the master MON node in the first stage, receives the response information of each slave node, MON0 takes out the topology table update operation key-value pair from the KV database, and the value data of the topology table update operation key-value pair is the serialized topology table update operation information. Therefore, MON0 can deserialize the serialized topology table update operation information and then obtain the topology table update operation information. The table updates operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added. Therefore, MON0 executes the KV operation of updating the service node topology table, and uses the service node topology table containing the blacklist information as the value data to update the service node topology table in MON0 to the temporary service node topology table with the blacklist information added. In this way, the service node topology table in the main MON0 memory contains the blacklist information with the restricted network segment of 10.0.1.0 / 24. After the update, MON0 sends a blacklist update instruction to the slave nodes MON1 and MON2.

[0165] Step 1130: according to the blacklist update instruction, the service node topology table in the second management node is updated to a temporary service node topology table with blacklist information added thereto by using the value data in the topology table update operation key-value pair.

[0166] In one embodiment, after the first management node completes the process of updating the service node topology table in the first management node to a temporary service node topology table with blacklist information added, a blacklist update instruction is generated and sent to all second management nodes. After all second management nodes receive the blacklist update instruction, they perform the update operation of the service node topology table together in the second stage of blacklist consistency synchronization. Specifically, after receiving the blacklist update instruction, each second management node updates the service node topology table in each second management node to a temporary service node topology table with blacklist information added. The subsequent work can be performed based on the updated service node topology table containing blacklist information. Since the address information of the metadata service node is recorded in the service node topology table with blacklist information added, the access to the distributed file system can be realized through the service node topology table. Therefore, when any second management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process for the client is implemented based on the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, and thus has good universal applicability.

[0167] In one embodiment, in order to flexibly manage and record different versions of service node topology tables while ensuring that blacklist information is correctly applied and protected, a temporary service node topology table containing blacklist information is required as a key-value pair of value data for updating the service node topology table in the first management node. Therefore, in the process of the second management node updating the service node topology table in the second management node to a temporary service node topology table with blacklist information added by using the value data in the topology table update operation key-value pair according to the blacklist update instruction, the value data in the topology table update operation key-value pair can be used to generate a topology table update key-value pair according to the blacklist update instruction, wherein the value data of the topology table update key-value pair is the temporary service node topology table with blacklist information added; and according to the topology table update key-value pair, the service node topology table in the second management node is updated to the temporary service node topology table with blacklist information added.

[0168] In one embodiment, after receiving the blacklist update instruction, the second management node enters the second stage of blacklist consistency synchronization. The embodiment of the present invention needs to generate a new key-value pair and use the new key-value pair to update the service node topology table in the second management node. Because the key-value pair provides a convenient way to transmit and store data, by storing the relevant information of the topology table update in the key-value pair, it can be transmitted as a whole, and the update operation can be uniquely identified according to the key. This method can simplify the data transmission and processing process, making the update operation easier to control. Therefore, in the embodiment of the present invention, a topology table update key-value pair can be generated according to the value data in the topology table update operation key-value pair, and the value data in the topology table update operation key-value pair is a temporary service node topology table with blacklist information added, so the value data in the newly generated topology table update key-value pair is also a temporary service node topology table with blacklist information added, and then the second management node obtains the previously stored service node topology table, which does not contain the new blacklist information and therefore needs to be discarded. Subsequently, if the service node topology table needs to be updated, the service node topology table in the second management node can be updated to a temporary service node topology table with blacklist information added according to the topology table update key-value pair, that is, the value data in the newly generated topology table update key-value pair is used. According to the temporary service node topology table with blacklist information added, as the updated service node topology table in the second management node, the subsequent work can be based on the updated service node topology table containing blacklist information. Since the address information of the metadata service node is recorded in the service node topology table with blacklist information added, the access to the distributed file system can be realized through the service node topology table. Therefore, when the second management node in the distributed file system receives an access request from the client, it can perform client filtering according to the service node topology table with blacklist information added, which can not only avoid the attack of a large number of useless access requests on the distributed file system, but also reduce the network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering processing of the client is realized according to the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for distributed file systems of different operating systems, the management node will store the service node topology table, so the embodiment of the present invention can enable distributed file systems of different operating systems to realize client filtering based on blacklist information, that is, the embodiment of the present invention can be applicable to distributed file systems of different operating systems, so it has good universal applicability.

[0169] In one embodiment, the value data in the topology table update operation key-value pair is the serialized topology table update operation information. The extracted serialized topology table update operation information needs to be deserialized before it can be used as the value data in the topology table update operation key-value pair, and then the topology table update key-value pair is generated in combination with the key data. Therefore, when the second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to generate the topology table update key-value pair, it can specifically deserialize the serialized topology table update operation information according to the blacklist update instruction to obtain the topology table update operation information; obtain the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added; determine the version number corresponding to the temporary service node topology table with the blacklist information added; use the version number as the key data and the temporary service node topology table with the blacklist information added as the value data to generate the topology table update key-value pair.

[0170] In one embodiment, deserialization is the process of converting serialized (or encoded) data back to its original form or object. In the data processing process, serialization refers to the operation of converting a data structure or object into a linear format for storage, transmission or interaction between different systems. Through deserialization, the serialized data can be restored to the original data structure or object for further processing and use. In the second stage of blacklist consistency synchronization in an embodiment of the present invention, after the second management node receives the blacklist update instruction, it can extract the value data in the stored topology table update operation key-value pair from the KV database according to the blacklist update instruction, that is, extract the serialized topology table update operation information, and deserialize the serialized topology table update operation information to obtain the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added. Therefore, the temporary service node topology table with the blacklist information added can be used as the value data of the newly generated topology table update key-value pair. The key data in the newly generated topology table update key-value pair is used to represent the unique identifier of the topology table update key-value pair. Therefore, in an embodiment of the present invention, when forming a corresponding temporary service node topology table according to different blacklist information, a corresponding version number will be added to the temporary service node topology table. Therefore, the version number can be used as the key data. Finally, the topology table update key-value pair can be generated with the version number as the key data and the temporary service node topology table with the blacklist information added as the value data.

[0171] In one embodiment, serialization and deserialization are a pair of corresponding processes, and the data can be restored to the original format using the corresponding deserialization method according to the serialization method selected during serialization. When serializing the topology table update operation information, the type of the first opcode can be recorded first, such as represented by a 1, and then the length of the opcode can be checked, and then the value of the opcode can be written to form data containing the type, length and opcode value after the serial number, and then spliced ​​together to form a string as the serialized topology table update operation information. Then, deserialization is also parsed in this way. After receiving a string, first look at what type, length and opcode value it is, and then read such a long byte data with this length, which is the value, and finally obtain the deserialized topology table update operation information. It should be noted that common serialization methods include JSON, XML, etc. When deserialization is performed by other serialization methods, the topology table update operation information is parsed according to the deserialized data structure, for example, according to the data structure and field definition before serialization, the deserialized data is parsed to obtain the topology table update operation information, and this embodiment of the present invention is not specifically limited.

[0172] In this embodiment, through the blacklist processing method of the distributed file system including the previous steps 1110 to 1130, the second management node can receive the topology table update operation key-value pair sent by the first management node; since the value data of the topology table update operation key-value pair is obtained by the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added, after the topology table update operation key-value pair is sent to all second management nodes in the distributed file system, the synchronization of the topology table update operation information can be achieved between the first management node and each second management node, thereby ensuring the consistency of all management nodes in the distributed file system in executing the topology table update operation information. After receiving and storing the topology table update operation key-value pair, the second management node can receive the blacklist update instruction sent by the first management node, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added; because after the first management node updates its service node topology table to a temporary service node topology table with blacklist information added, it will send the corresponding blacklist update instruction to all second management nodes to trigger each second management node to perform the same topology table update, so the synchronization of the service node topology table with blacklist information added can be achieved between the first management node and each second management node, thereby ensuring the consistency of the service node topology table with blacklist information added for all management nodes in the distributed file system. In addition, since all management nodes in the distributed file system can achieve consistency in the execution of topology table update operation information and consistency in the service node topology table with blacklist information added, all management nodes in the distributed file system can have the same service node topology table with blacklist information added. Since the service node topology table with blacklist information added records the address information of the metadata service node, access to the distributed file system can be achieved through the service node topology table. Therefore, when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added. This can not only avoid attacks on the distributed file system by a large number of useless access requests, but also reduce network bandwidth consumption and access processing pressure caused by processing useless access requests.In addition, since the filtering process for the client is implemented based on the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, and thus has good universal applicability.

[0173] In one embodiment, the blacklist processing method of the distributed file system provided by the embodiment of the present invention can also be applied to the access control scenario of the distributed file system. After the management node updates the service node topology table with the blacklist information added, all management nodes in the distributed file system can have the same service node topology table with the blacklist information added. When the client requests to access the distributed file system, it can perform verification based on the service node topology table with the blacklist information added. Therefore, the blacklist processing method of the distributed file system provided by the embodiment of the present invention can also include but is not limited to the following steps:

[0174] Sending a service node topology table subscription request to the first management node or the second management node, so that the first management node or the second management node obtains the service node topology table based on the service node topology table subscription request;

[0175] Receive a service node topology table sent by the first management node or the second management node, and determine blacklist information from the service node topology table;

[0176] Determine the restricted network segment or fixed IP address from the blacklist information, and obtain the target IP address of the client. If the target IP address does not match the restricted network segment or fixed IP address, obtain the metadata service node address information from the service node topology table, and send an access request to the metadata service node based on the address information.

[0177] In one embodiment, before sending an access request to the metadata service node, the client needs to perform a blacklist check first. First, it needs to send a service node topology table subscription request to the first management node or the second management node. It should be noted that in the node cluster composed of the first management node and the second management node, each management node can provide a blacklist check service. The client can obtain the IP list of the management cluster, and the IP address of each management node can be parsed from the IP list, so as to randomly select a management node to send the service node topology table subscription request. After receiving the service node topology table subscription request, the first management node or the second management node will obtain the latest updated service node topology table from the stored data and return the service node topology table to the client. After the client receives the service node topology table, it performs IP verification. In this process, the main purpose is to confirm whether the client's own IP address matches the IP address prohibited from access in the blacklist information. Therefore, after receiving the service node topology table sent by the first management node or the second management node, the client can determine the blacklist information from the service node topology table, and the blacklist information contains at least one of a restricted network segment or a fixed IP address. The restricted network segment indicates that all addresses in this network segment cannot access the distributed file system, and the fixed IP address indicates that the distributed file system cannot be accessed under this fixed IP address. Subsequently, the client needs to obtain its own IP address, that is, obtain the target IP address, and match the target IP address with at least one of the restricted network segment or the fixed IP address, and determine whether the subsequent client can access the distributed file system based on the matching result. If the target IP address belongs to the range of the restricted network segment, or the target IP address is the same as the fixed IP address, it means that the target IP address is an address with restricted access, indicating that the current client belongs to the blacklist device represented by the blacklist information, and therefore cannot access the distributed file system subsequently; conversely, if the target IP address does not match the restricted network segment or the fixed IP address, it means that the target IP address is an address that is allowed to be accessed, and the client can subsequently communicate with the distributed file system, and the service node topology table contains the address information of the metadata service node that needs to be accessed. Therefore, based on the address information, the client can send an access request to the metadata service node, and the metadata service node establishes a communication connection with the client in response to the request. The client can subsequently send the required query request to the metadata service node, and the embodiments of the present invention do not make specific restrictions on this.The embodiment of the present invention uses a blacklist check. When the blacklist information includes the address information of the client, the distributed file system access client will exit the startup, which can ensure that there will be no network requests between the user terminal and the metadata service node, thereby ensuring the service capacity and network bandwidth of the metadata service node. When the blacklist information does not include the address information of the user terminal, the distributed file system access client will remain started. At this time, the user can access the metadata service node through the metadata service client in the user terminal.

[0178] In one embodiment, the client has multiple forms, and can be any user terminal that needs to query relevant information through a distributed file system, including but not limited to smart phones, tablet computers, laptops, desktop computers, smart watches, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. Optionally, these devices, as user terminals, can be installed with a distributed file system access client, and the interaction between the user terminal and the distributed file system can be achieved through the distributed file system access client. Among them, the distributed file system access client can include a management service client and a metadata service client. Through the management service client, the user terminal can interact with the first management node or each second management node; through the metadata service client, the user terminal can interact with each metadata service node.

[0179] In one embodiment, if Fig.12 As shown, Fig.12The schematic diagram of the blacklist verification process performed by the client 1220 is shown, wherein the client 1220 is provided with a management service client (Monclient) 1221 and a metadata service client (Mdsclient) 1222. Through Monclient, the client 1220 can interact with the first management node 1211 or each second management node (1212 and 1213), and through Mdsclient, the client 1220 can interact with each metadata service node (MDS). In addition, a plurality of management nodes are provided in the distributed file system, including the first management node (MON0) 1211, the second management node (MON1) 1212, and the second management node (MON2) 1213. 2 and the second management node (MON2) 1213, wherein the first management node 1211 is the master node, the second management nodes (1212 and 1213) are slave nodes, and a plurality of metadata service nodes are arranged in the distributed file system, forming a metadata service node cluster 1230, namely the first metadata service node 1231, the second metadata service node 1232 and the third metadata service node 1233. When the client 1220 is started, it will subscribe to the MON cluster 1210 to obtain the Mdsmap information, i.e., the service node topology table, because the Mdsmap contains the information of each MDS, including the IP address, etc., so that after having the Mdsmap information, it can communicate with the MDS. Specifically, during the verification process, the client 1220 randomly sends an Mdsmap subscription request to a MON (here MON2) through Monclient when it starts. After receiving the Mdsmap subscription request, MON2 obtains the latest updated Mdsmap and returns it to the client 1220. The Mdsclient in the client 1220 matches its own IP address with the obtained Mdsmap information, and determines whether the IP address of the client 1220 is a fixed IP address in the Mdsmap blacklist information, and whether the IP address of the client 1220 is a fixed IP address in the Mdsmap blacklist information. The IP address is calculated by the number of mask bits (such as / 24). If the IP address of the client 1220 matches the blacklist information, the startup is exited. This ensures that there will be no network request between the client 1220 and the MDS, ensuring the MDS service and network bandwidth. On the contrary, if the IP address of the client 1220 does not match the blacklist information, the address of a certain MDS is obtained from Mdsmap. For example, the address information of the second metadata service node 1232 is obtained. Subsequently, the client 1220 sends an access request to the second metadata service node 1232 through Mdsclient, and after establishing a communication connection, sends a related information query request to the second metadata service node 1232.

[0180] In one embodiment, if Fig.13 As shown, Fig.13 A schematic diagram showing a blacklist verification process for multiple clients is shown. Fig.13 In the figure, multiple clients are respectively communicated and connected with the management node 1310. The management node 1310 can be the first management node or the second management node. There is no specific limitation here. The multiple clients include a first client 1321, a second client 1322 and a third client 1323. Different clients represent different terminal devices. Each client has a unique IP address, and the IP addresses of different clients are not the same. In a multi-client scenario, each client can obtain Mdsmap information, i.e., a service node topology table, from the management node 1310. After the management node 1310 sends Mdsmap to each client, the Mdsclient in each client matches its own IP address with the obtained Mdsmap information, and determines whether the IP address of each client is a fixed IP address in the Mdsmap blacklist information, and determines whether the IP address of the client is an IP address calculated by the network segment (such as 10.0.1.0) and the number of mask bits (such as / 24) in the Mdsmap blacklist information. If the IP address of a client matches the blacklist information, the startup is exited, which ensures that the blacklisted client will not have a network request with the MDS, and ensures that the MDS service and network bandwidth are not affected. On the contrary, if the IP address of the client does not match the blacklist information, the address of a certain MDS is obtained from the Mdsmap. For example, Fig.13 The first client 1321 and the third client 1323 are both included in the blacklist, but the second client 1322 is not. Therefore, only the second client 1322 can send an access request to the metadata service node 1330 through Mdsclient, and after establishing a communication connection, send relevant information query requests to the metadata service node 1330.

[0181] Through the above Fig.12 and Fig.13 It can be seen from the example that since the filtering processing of the client is implemented according to the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, so it has good universal applicability.

[0182] The blacklist processing method of the distributed file system provided by the embodiment of the present invention is described in detail below with a specific example.

[0183] Reference Fig.14 As shown, Fig.14 The following is a flowchart of the steps of the blacklist processing method of the distributed file system provided by a specific example. Fig.14 In the blacklist processing method of the distributed file system, the blacklist processing method may include but is not limited to steps 1401 to 1412.

[0184] Step 1401: The first management node determines whether blacklist information is received, or whether subscription information for a service node topology table sent by a client is received. When blacklist information is received, step 1402 is executed; when subscription information is received, step 1408 is executed.

[0185] Step 1402: The first management node adds the blacklist information into the temporary service node topology table that records the address information of the metadata service node.

[0186] Step 1403: The first management node uses the temporary service node topology table with the blacklist information added as an operation value to generate topology table update operation information.

[0187] Step 1404: The first management node serializes the topology table update operation information to obtain the serialized topology table update operation information, and then determines the key value number corresponding to the serialized topology table update operation information, and then uses the key value number as the key data and the serialized topology table update operation information as the value data to generate a topology table update operation key-value pair.

[0188] Step 1405: The first management node sends the topology table update operation key-value pair to all second management nodes in the distributed file system.

[0189] In this step, after the first management node sends the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node will perform persistent storage of the topology table update operation key-value pair. After each second management node completes the persistent storage of the topology table update operation key-value pair, each second management node will send a response message to the first management node. When receiving the response message sent by each second management node after performing persistent storage of the topology table update operation key-value pair, the first management node will set a response mark corresponding to each second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the topology table update operation key-value pair is completed.

[0190] Step 1406: The first management node deserializes the serialized topology table update operation information to obtain the topology table update operation information, and then obtains the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added, and then determines the version number corresponding to the temporary service node topology table with the blacklist information added, and uses the version number as the key data and the temporary service node topology table with the blacklist information added as the value data to generate a topology table update key-value pair. Then the first management node updates the service node topology table in the first management node to the temporary service node topology table with the blacklist information added according to the topology table update key-value pair, and generates a corresponding blacklist update instruction.

[0191] Step 1407: The first management node sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction, updates the service node topology table in each second management node to a temporary service node topology table with the blacklist information added, and then executes step 1401.

[0192] In this step, when the second management node updates the service node topology table in each second management node to a temporary service node topology table with the blacklist information added by using the value data in the topology table update operation key-value pair according to the blacklist update instruction, the second management node will first deserialize the serialized topology table update operation information according to the blacklist update instruction to obtain the topology table update operation information, and then obtain the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added, and then determine the version number corresponding to the temporary service node topology table with the blacklist information added, and use the version number as the key data and the temporary service node topology table with the blacklist information added as the value data to generate a topology table update key-value pair, and then update the service node topology table in the second management node to the temporary service node topology table with the blacklist information added according to the topology table update key-value pair.

[0193] Step 1408: The first management node sends the service node topology table with blacklist information added to the client.

[0194] Step 1409: The client receives the service node topology table with blacklist information added thereto, which is sent by the first management node.

[0195] Step 1410: The client filters the address information of the client according to the blacklist information in the service node topology table to determine whether the blacklist information includes the address information of the client. If so, execute step 1411; if not, execute step 1412.

[0196] Step 1411: The client exits startup and then executes step 1401.

[0197] Step 1412: The client remains started and then executes step 1401.

[0198] Through the blacklist processing method of the distributed file system of the above steps 1401 to 1412, when receiving the blacklist information, the first management node first adds the blacklist information into the temporary service node topology table that records the address information of the metadata service node, and then uses the temporary service node topology table with the blacklist information added as the operation value to generate the topology table update operation information, and generates the topology table update operation key-value pair according to the topology table update operation information, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information, and then sends the topology table update operation key-value pair to all the second management nodes in the distributed file system; because the value data of the topology table update operation key-value pair is obtained by the topology table update operation information, and the operation value of the topology table update operation information is the temporary service node topology table with the blacklist information added, after sending the topology table update operation key-value pair to all the second management nodes in the distributed file system, the synchronization of the topology table update operation information between the first management node and each second management node can be achieved, thereby ensuring the consistency of all management nodes in the distributed file system in executing the topology table update operation information. After sending the topology table update operation key-value pair to all second management nodes in the distributed file system, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added; since the first management node will send the corresponding blacklist update instruction to all second management nodes to trigger each second management node to perform the same topology table update after updating its service node topology table to the temporary service node topology table with blacklist information added, it is possible to achieve synchronization between the first management node and each second management node for the service node topology table with blacklist information added, thereby ensuring the consistency of the service node topology table with blacklist information added by all management nodes in the distributed file system.In addition, since all management nodes in the distributed file system can achieve consistency in the execution of topology table update operation information and consistency in the service node topology table with blacklist information added, all management nodes in the distributed file system can have the same service node topology table with blacklist information added. Since the service node topology table with blacklist information added records the address information of the metadata service node, access to the distributed file system can be achieved through the service node topology table. Therefore, when any management node in the distributed file system receives an access request from a client, it can perform client filtering according to the service node topology table with blacklist information added. This can not only avoid attacks on the distributed file system by a large number of useless access requests, but also reduce network bandwidth consumption and access processing pressure caused by processing useless access requests. In addition, since the filtering process for the client is implemented based on the service node topology table in the management node, there is no need for the metadata service node to verify the access request, and for the distributed file systems of different operating systems, the service node topology table is stored in the management node. Therefore, the embodiment of the present invention can enable the distributed file systems of different operating systems to implement client filtering based on blacklist information. In other words, the embodiment of the present invention can be applicable to the distributed file systems of different operating systems, so it has good universal applicability and can improve the stability of the distributed file system.

[0199] The following are some practical examples to illustrate the application scenarios of the embodiments of the present invention.

[0200] It should be noted that the blacklist processing method for a distributed file system provided in an embodiment of the present invention can be applied to different application scenarios such as blacklist registration of a distributed file system, blacklist modification of a distributed file system, and access control of a distributed file system. The following takes the blacklist registration scenario of a distributed file system, the blacklist modification scenario of a distributed file system, and the access control scenario of a distributed file system as examples for explanation.

[0201] Scene 1

[0202] The blacklist processing method of a distributed file system provided by an embodiment of the present invention can be applied to a blacklist registration scenario of a distributed file system. For example, assuming that a user registers a blacklist in a distributed file system through a user terminal and sends blacklist information to a first management node in the distributed file system, when the first management node receives the blacklist information, the first management node first adds the blacklist information to a temporary service node topology table that records address information of a metadata service node, and then uses the temporary service node topology table that has the blacklist information added as an operation value to generate topology table update operation information, then generates a topology table update operation key-value pair according to the topology table update operation information, and sends the topology table update operation key-value pair to the first management node. All second management nodes, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information; after sending the topology table update operation key-value pair to all second management nodes, the first management node updates the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, and generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added. At this point, the distributed file system completes the processing of the blacklist registration initiated by the user.

[0203] Scene 2

[0204] The blacklist processing method of a distributed file system provided by an embodiment of the present invention can also be applied to a blacklist modification scenario of a distributed file system. For example, assuming that a user modifies a blacklist of a distributed file system through a user terminal and sends the modified blacklist information to a first management node in the distributed file system, when the first management node receives the modified blacklist information, the first management node first adds the blacklist information to a temporary service node topology table that records the address information of the metadata service node, and then uses the temporary service node topology table to which the blacklist information is added as an operation value to generate topology table update operation information, and then generates a topology table update operation key-value pair according to the topology table update operation information, and adds the topology table update operation key value pair to the topology table update operation key value pair. The pair is sent to all second management nodes, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information; after sending the topology table update operation key-value pair to all second management nodes, the first management node updates the service node topology table in the first management node to a temporary service node topology table with the blacklist information added thereto according to the value data in the topology table update operation key-value pair, and generates a corresponding blacklist update instruction, and then sends the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with the blacklist information added thereto. At this point, the distributed file system has completed the processing of the blacklist modification initiated by the user.

[0205] Scene 3

[0206] The blacklist processing method of the distributed file system provided by the embodiment of the present invention can also be applied to the access control scenario of the distributed file system. For example, assuming that the user terminal is provided with a distributed file system access client including a management service client and a metadata service client, after the user registers the blacklist or modifies the blacklist with the distributed file system through the user terminal, when the user starts the distributed file system access client in the user terminal, the management service client in the distributed file system access client will randomly subscribe to the service node topology table from any management node in the distributed file system. When the first management node in the distributed file system receives the subscription information sent by the user terminal through the management service client, the first management node will send the subscription information to the user terminal. The terminal sends a service node topology table with blacklist information added to it; when receiving the service node topology table sent by the first management node, the metadata service client in the user terminal will filter and judge the address information of the user terminal according to the blacklist information in the service node topology table. When the blacklist information includes the address information of the user terminal, the distributed file system access client will exit and start up, so as to ensure that there will be no network request between the user terminal and the metadata service node, thereby ensuring the service capacity and network bandwidth of the metadata service node; when the blacklist information does not include the address information of the user terminal, the distributed file system access client will remain started up, and at this time, the user can access the metadata service node through the metadata service client in the user terminal.

[0207] It is to be understood that, although the steps in the above-mentioned various flow charts are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in the present embodiment, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the above-mentioned flow charts can include a plurality of steps or a plurality of stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of the steps or stages in other steps or other steps.

[0208] Reference Fig.15 The embodiment of the present invention further discloses a blacklist processing device for a distributed file system. The blacklist processing device 1500 is arranged at a first management node in the distributed file system. The blacklist processing device 1500 can implement the blacklist processing method for the distributed file system in the previous embodiment. The blacklist processing device 1500 includes:

[0209] The blacklist adding unit 1510 is used to add the blacklist information into the temporary service node topology table recording the address information of the metadata service node when the blacklist information is received;

[0210] An operation information generating unit 1520 is used to generate topology table update operation information by using the temporary service node topology table with blacklist information added as an operation value;

[0211] The first information sending unit 1530 is used to generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information;

[0212] The first information updating unit 1540 is used to update the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair, and generate a corresponding blacklist update instruction;

[0213] The second information sending unit 1550 is used to send the blacklist update instruction to all second management nodes, so that each second management node uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each second management node to a temporary service node topology table with blacklist information added.

[0214] In one embodiment, the first information sending unit 1530 is further configured to:

[0215] Serializing the topology table update operation information to obtain serialized topology table update operation information;

[0216] The serialized topology table update operation information is used as value data to generate a topology table update operation key-value pair.

[0217] In one embodiment, the first information sending unit 1530 is further configured to:

[0218] Determine the key value number corresponding to the serialized topology table update operation information;

[0219] A topology table update operation key-value pair is generated using the key value number as key data and the serialized topology table update operation information as value data.

[0220] In one embodiment, the first information updating unit 1540 is further configured to:

[0221] Generate a topology table update key-value pair according to the value data in the topology table update operation key-value pair, wherein the value data of the topology table update key-value pair is a temporary service node topology table with blacklist information added;

[0222] According to the topology table, the key-value pairs are updated, and the service node topology table in the first management node is updated to a temporary service node topology table with blacklist information added thereto.

[0223] In one embodiment, the value data in the topology table update operation key-value pair is serialized topology table update operation information; the first information update unit 1540 is further used to:

[0224] Deserialize the serialized topology table update operation information to obtain the topology table update operation information;

[0225] Obtain the operation value in the topology table update operation information, and obtain the temporary service node topology table with the blacklist information added;

[0226] Determine the version number corresponding to the temporary service node topology table with blacklist information added;

[0227] A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with blacklist information added as value data.

[0228] In one embodiment, after sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair;

[0229] The blacklist processing device 1500 further includes:

[0230] A response information receiving unit, used to receive the response information sent by each second management node after persistently storing the key-value pairs of the topology table update operation;

[0231] The response information setting unit is used to set the response mark corresponding to each second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the key-value pair of the topology table update operation is completed.

[0232] It should be noted that, since the blacklist processing device 1500 of the distributed file system of this embodiment can implement the blacklist processing method of the distributed file system of the previous embodiment, the blacklist processing device 1500 of the distributed file system of this embodiment and the blacklist processing method of the distributed file system of the previous embodiment have the same technical principles and the same beneficial effects, and in order to avoid duplication of content, they are not repeated here.

[0233] Reference Fig.16The embodiment of the present invention further discloses a blacklist processing device for a distributed file system. The blacklist processing device 1600 is arranged in a second management node in the distributed file system. The blacklist processing device 1600 can implement the blacklist processing method for the distributed file system in the previous embodiment. The blacklist processing device 1600 includes:

[0234] The first information receiving unit 1610 is used to receive a topology table update operation key-value pair sent by a first management node in a distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as an operation value, and the temporary service node topology table records address information of a metadata service node;

[0235] The second information receiving unit 1620 is used to receive a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to a temporary service node topology table with blacklist information added according to the value data in the topology table update operation key-value pair;

[0236] The second information updating unit 1630 is used to update the service node topology table in the second management node to a temporary service node topology table with blacklist information added thereto by using the value data in the topology table update operation key-value pair according to the blacklist update instruction.

[0237] In one embodiment, the second information updating unit 1630 is further configured to:

[0238] According to the blacklist update instruction, the value data in the topology table update operation key-value pair is used to generate a topology table update key-value pair, wherein the value data of the topology table update key-value pair is a temporary service node topology table with blacklist information added;

[0239] The key-value pairs are updated according to the topology table, and the service node topology table in the second management node is updated to a temporary service node topology table with blacklist information added thereto.

[0240] In one embodiment, the value data in the topology table update operation key-value pair is serialized topology table update operation information; the second information update unit 1630 is further used to:

[0241] Deserialize the serialized topology table update operation information according to the blacklist update instruction to obtain the topology table update operation information;

[0242] Obtain the operation value in the topology table update operation information, and obtain the temporary service node topology table with the blacklist information added;

[0243] Determine the version number corresponding to the temporary service node topology table with blacklist information added;

[0244] A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with blacklist information added as value data.

[0245] In one embodiment, the blacklist processing device 1600 further includes:

[0246] An information processing unit, used to persistently store the key-value pairs of the topology table update operation and generate corresponding response information;

[0247] The third information sending unit is used to send response information to the first management node, so that the first management node sets a response mark corresponding to the second management node according to the response information, wherein the response mark is used to indicate that the persistent storage of the key-value pair of the topology table update operation is completed.

[0248] In one embodiment, the second management node is a slave node, and the first management node is a master node; the blacklist processing device 1500 further includes:

[0249] The fourth information sending unit is used to forward the blacklist information to the first management node when receiving the blacklist information, so that the first management node adds the blacklist information into the temporary service node topology table.

[0250] It should be noted that, since the blacklist processing device 1500 of the distributed file system of this embodiment can implement the blacklist processing method of the distributed file system of the previous embodiment, the blacklist processing device 1600 of the distributed file system of this embodiment and the blacklist processing method of the distributed file system of the previous embodiment have the same technical principles and the same beneficial effects, and in order to avoid duplication of content, they are not repeated here.

[0251] Reference Fig.17 The embodiment of the present invention further discloses a blacklist processing device for a distributed file system, the blacklist processing device 1700 comprising:

[0252] at least one processor 1701;

[0253] At least one memory 1702, used to store at least one program;

[0254] When at least one program is executed by at least one processor 1701, the blacklist processing method of the distributed file system as described above is implemented.

[0255] The embodiment of the present invention further discloses a computer-readable storage medium, in which a computer program executable by a processor is stored. When the computer program executable by the processor is executed by the processor, it is used to implement the blacklist processing method of the distributed file system as described above.

[0256] An embodiment of the present invention also discloses a computer program product, including a computer program or computer instructions, wherein the computer program or computer instructions are stored in a computer-readable storage medium, and a processor of a blacklist processing device of a distributed file system reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the blacklist processing device executes the blacklist processing method of the distributed file system as described above.

[0257] The terms "first", "second", "third", "fourth", etc. (if any) in the description of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0258] It should be understood that in the present invention, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0259] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0260] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0261] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0262] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0263] The step numbers in the above method embodiment are only provided for the convenience of explanation and description, and no limitation is imposed on the order of the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

Claims

1. A blacklist processing method for a distributed file system, characterized in that: The method is applied to a first management node in the distributed file system, and the method comprises the following steps: When receiving the blacklist information, adding the blacklist information into the temporary service node topology table recording the address information of the metadata service node; Using the temporary service node topology table with the blacklist information added as an operation value, generating topology table update operation information; Generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information; According to the value data in the topology table update operation key-value pair, the service node topology table in the first management node is updated to the temporary service node topology table with the blacklist information added, and a corresponding blacklist update instruction is generated; The blacklist update instruction is sent to all the second management nodes, so that each of the second management nodes uses the value data in the topology table update operation key-value pair according to the blacklist update instruction, and updates the service node topology table in each of the second management nodes to the temporary service node topology table with the blacklist information added.

2. The method according to claim 1, characterized in that The generating a topology table update operation key-value pair according to the topology table update operation information includes: Serializing the topology table update operation information to obtain serialized topology table update operation information; The serialized topology table update operation information is used as value data to generate a topology table update operation key-value pair.

3. The method according to claim 2, characterized in that The method of using the serialized topology table update operation information as value data to generate a topology table update operation key-value pair includes: Determine the key value number corresponding to the serialized topology table update operation information; A topology table update operation key-value pair is generated by using the key value number as key data and the serialized topology table update operation information as value data.

4. The method according to claim 1, characterized in that: The updating of the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation comprises: Generate a topology table update key-value pair according to the value data in the topology table update operation key-value pair, wherein the value data of the topology table update key-value pair is the temporary service node topology table with the blacklist information added thereto; The key-value pairs are updated according to the topology table, and the service node topology table in the first management node is updated to the temporary service node topology table with the blacklist information added thereto.

5. The method according to claim 4, characterized in that The value data in the topology table update operation key-value pair is the serialized topology table update operation information; The step of generating a topology table update key-value pair according to the value data in the topology table update operation key-value pair comprises: Deserialize the serialized topology table update operation information to obtain the topology table update operation information; Acquire the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added; Determine the version number corresponding to the temporary service node topology table to which the blacklist information is added; A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with the blacklist information added as value data.

6. The method according to claim 1, characterized in that After sending the topology table update operation key-value pair to all second management nodes in the distributed file system, each second management node persistently stores the topology table update operation key-value pair; The method further comprises: Receiving response information sent by each of the second management nodes after persistently storing the key-value pairs of the topology table update operation; A response mark corresponding to each second management node is set according to the response information, wherein the response mark is used to indicate that persistent storage of the key-value pair of the topology table update operation is completed.

7. A blacklist processing method for a distributed file system, characterized in that: The method is applied to the second management node in the distributed file system, and the method comprises the following steps: Receive a topology table update operation key-value pair sent by a first management node in the distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as an operation value, and the temporary service node topology table records address information of a metadata service node; Receive a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation; According to the blacklist update instruction, the service node topology table in the second management node is updated to the temporary service node topology table with the blacklist information added thereto by using the value data in the topology table update operation key-value pair.

8. The method according to claim 7, characterized in that The updating of the service node topology table in the second management node to the temporary service node topology table with the blacklist information added thereto by using the value data in the topology table update operation key-value pair according to the blacklist update instruction comprises: Generate a topology table update key-value pair using the value data in the topology table update operation key-value pair according to the blacklist update instruction, wherein the value data of the topology table update key-value pair is the temporary service node topology table with the blacklist information added thereto; The key-value pairs are updated according to the topology table, and the service node topology table in the second management node is updated to the temporary service node topology table with the blacklist information added thereto.

9. The method according to claim 8, characterized in that The value data in the topology table update operation key-value pair is the serialized topology table update operation information; The step of generating a topology table update key-value pair by using the value data in the topology table update operation key-value pair according to the blacklist update instruction includes: Deserialize the serialized topology table update operation information according to the blacklist update instruction to obtain the topology table update operation information; Acquire the operation value in the topology table update operation information to obtain the temporary service node topology table with the blacklist information added; Determine the version number corresponding to the temporary service node topology table to which the blacklist information is added; A topology table update key-value pair is generated using the version number as key data and the temporary service node topology table with the blacklist information added as value data.

10. The method according to claim 7, characterized in that After receiving the topology table update operation key-value pair sent by the first management node in the distributed file system, the method further includes: Persistently store the key-value pairs of the topology table update operation and generate corresponding response information; The response information is sent to the first management node, so that the first management node sets a response mark corresponding to the second management node according to the response information, wherein the response mark is used to indicate that persistent storage of the key-value pair of the topology table update operation is completed.

11. A blacklist processing device for a distributed file system, characterized in that: The blacklist processing device is arranged at the first management node in the distributed file system, and the blacklist processing device comprises: A blacklist adding unit, configured to, upon receiving blacklist information, add the blacklist information into a temporary service node topology table recording address information of a metadata service node; An operation information generating unit, configured to generate topology table update operation information by using the temporary service node topology table with the blacklist information added thereto as an operation value; A first information sending unit is used to generate a topology table update operation key-value pair according to the topology table update operation information, and send the topology table update operation key-value pair to all second management nodes in the distributed file system, wherein the value data of the topology table update operation key-value pair is obtained according to the topology table update operation information; A first information updating unit, configured to update the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation, and generate a corresponding blacklist update instruction; The second information sending unit is used to send the blacklist update instruction to all the second management nodes, so that each of the second management nodes uses the value data in the topology table update operation key-value pair according to the blacklist update instruction to update the service node topology table in each of the second management nodes to the temporary service node topology table with the blacklist information added.

12. A blacklist processing device for a distributed file system, characterized in that: The blacklist processing device is arranged at a second management node in the distributed file system, and the blacklist processing device comprises: A first information receiving unit is configured to receive a topology table update operation key-value pair sent by a first management node in the distributed file system, wherein the value data in the topology table update operation key-value pair is obtained according to topology table update operation information, and the topology table update operation information is generated by the first management node using a temporary service node topology table with blacklist information added as an operation value, and the temporary service node topology table records address information of a metadata service node; A second information receiving unit is used to receive a blacklist update instruction sent by the first management node, wherein the blacklist update instruction is generated by the first management node when updating the service node topology table in the first management node to the temporary service node topology table with the blacklist information added thereto according to the value data in the key-value pair of the topology table update operation; The second information updating unit is used to update the service node topology table in the second management node to the temporary service node topology table with the blacklist information added thereto by using the value data in the topology table update operation key-value pair according to the blacklist update instruction.

13. A blacklist processing device for a distributed file system, characterized in that: include: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the blacklist processing method for the distributed file system as described in any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium, characterized in that: A computer program executable by a processor is stored therein, and when the computer program executable by the processor is executed by the processor, it is used to implement the blacklist processing method of the distributed file system as described in any one of claims 1 to 10.

15. A computer program product comprising a computer program or computer instructions, characterized in that The computer program or the computer instruction is stored in a computer-readable storage medium, and a processor of a blacklist processing device of a distributed file system reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the blacklist processing device executes the blacklist processing method of a distributed file system according to any one of claims 1 to 10.

Images