A resource access method and related device of a distributed system

Abstract

The present application discloses a resource access method for a distributed system, comprising: a node determines whether the lock-holding node in a local distributed lock file is the node; if so, a resource access operation is performed; if not, a lock request is sent to a data node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node; the step of determining whether the lock-holding node in the local distributed lock file is the node is performed, until the lock-holding node is the node, a service access resource operation is performed. By implementing a lock operation in a node, multiple nodes are prevented from accessing data at the same time, thereby improving data consistency. The present application also discloses a resource access device, a server, and a computer-readable storage medium for a distributed system, which have the above beneficial effects.

Description

Technical Field

[0001] The present application relates to the field of distributed technology, and in particular to a resource access method, a resource access device, a server, and a computer-readable storage medium of a distributed system. Background Art

[0002] With the continuous development of information technology, distributed systems have emerged. Distributed systems are software systems built on networks. It is precisely because of the characteristics of software that distributed systems have a high degree of cohesion and transparency. Therefore, the difference between networks and distributed systems lies more in high-level software rather than hardware. Distributed systems are loosely coupled systems composed of multiple processors interconnected by communication lines. From the perspective of a processor in the system, the rest of the processors and corresponding resources are remote, and only its own resources are local.

[0003] In related technologies, in order to access different data resources in a distributed system, it is necessary to coordinate different systems to synchronously access common resources. However, it is easy for different systems or different hosts in the same system to share one or a group of resources. When different systems or different hosts in the same system access the same resource at the same time, it is easy for the resource to be modified incorrectly, resulting in data consistency problems and reducing the reliability of the system.

[0004] Therefore, how to improve the consistency of data access in a distributed system is a key issue that technical personnel in this field are concerned about. Summary of the invention

[0005] The purpose of the present application is to provide a resource access method, a resource access device, a server and a computer-readable storage medium for a distributed system. By judging the distributed lock file, and when the lock-holding node is the node, the resource access operation can be performed. When it is not the lock-holding node, a lock request is sent to the data node for locking processing, and resource access is performed until the lock-holding node in the distributed lock file is the node, thereby realizing the locking processing of distributed resources and improving the consistency of data access.

[0006] To solve the above technical problems, the present application provides a resource access method for a distributed system, comprising:

[0007] The node determines whether the lock holding node in the local distributed lock file is the node;

[0008] If so, the resource access operation is performed;

[0009] If not, sending a lock request to the data node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node;

[0010] The step of determining whether the lock-taking node in the local distributed lock file is the node is executed, and when the lock-taking node is the node, a service access resource operation is executed.

[0011] Optionally, also include:

[0012] When the lock node in the distributed lock file is empty, the unique identifier of the node is written into the distributed lock file, and the business resource operation is performed.

[0013] Optionally, also include:

[0014] When the business resource operation is completed, determining whether the node is a data node;

[0015] When the node is not a data node, sending a data node identity request to obtain data node identity authority;

[0016] When the node is a data node, the lock node of the distributed lock file is modified to be empty.

[0017] Optionally, the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, including:

[0018] The data node receives the lock request;

[0019] Determine whether the lock node in the distributed lock file of the data node has been processed;

[0020] If yes, modify the lock holding node in the distributed lock file of the data node to the node corresponding to the lock request, and send the distributed lock file to the node;

[0021] If not, the distributed lock file is sent to the node.

[0022] The present application also provides a resource access device for a distributed system, comprising:

[0023] A lock determination module, used to determine whether the lock holding node in the local distributed lock file is the node;

[0024] A resource access module, used for executing a resource access operation when the lock-taking node is the node;

[0025] A lock request module, used for sending a lock request to a data node when the lock holding node is not the node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node;

[0026] A loop judgment module is used to execute the step of judging whether the lock-taking node in the local distributed lock file is the node, and when the lock-taking node is the node, a business access resource operation is performed.

[0027] Optionally, also include:

[0028] The lock node writing module is used to write the unique identifier of the node into the distributed lock file and perform business resource operations when the lock node in the distributed lock file is empty.

[0029] Optionally, also include:

[0030] The access completion operation module is used to determine whether the node is a data node when the business resource operation is completed; when the node is not a data node, send a data node identity request to obtain data node identity authority; when the node is a data node, modify the lock node of the distributed lock file to empty.

[0031] Optionally, the data node includes:

[0032] A lock request receiving unit, configured to receive the lock request;

[0033] A processing status judgment unit, used to judge whether the lock node in the distributed lock file of the data node has been processed;

[0034] A modification unit, configured to modify the lock holding node in the distributed lock file of the data node to the node corresponding to the lock request when the processing is completed, and send the distributed lock file to the node;

[0035] The sending unit is used to send the distributed lock file to the node when the processing is not completed.

[0036] The present application also provides a server, comprising:

[0037] Memory for storing computer programs;

[0038] A processor is used to implement the steps of the resource access method described above when executing the computer program.

[0039] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the resource access method described above are implemented.

[0040] A resource access method for a distributed system provided in the present application includes: a node determines whether a lock-holding node in a local distributed lock file is the node; if so, a resource access operation is performed; if not, a lock request is sent to a data node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node; the step of determining whether the lock-holding node in the local distributed lock file is the node is performed, until the lock-holding node is the node, and a business access resource operation is performed.

[0041] By judging the distributed lock file, resource access operations can be performed when the lock-holding node is this node. When it is not the lock-holding node, a lock request is sent to the data node for locking processing. Resource access is performed until the lock-holding node in the distributed lock file is this node. This realizes the locking processing of distributed resources and improves the consistency of data access.

[0042] The present application also provides a resource access device, a server and a computer-readable storage medium for a distributed system, which have the above beneficial effects and are not elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 A flowchart of a resource access method for a distributed system provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of the structure of a resource access device for a distributed system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The core of the present application is to provide a resource access method, a resource access device, a server and a computer-readable storage medium for a distributed system. By judging the distributed lock file, and when the lock-holding node is the node, the resource access operation can be performed. When it is not the lock-holding node, a lock request is sent to the data node for locking processing, and resource access is performed until the lock-holding node in the distributed lock file is the node, thereby realizing the locking processing of distributed resources and improving the consistency of data access.

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] In related technologies, in order to access different data resources in a distributed system, it is necessary to coordinate different systems to synchronously access common resources. However, it is easy for different systems or different hosts in the same system to share one or a group of resources. When different systems or different hosts in the same system access the same resource at the same time, it is easy for the resource to be modified incorrectly, resulting in data consistency problems and reducing the reliability of the system.

[0049] Therefore, the present application provides a resource access method for a distributed system, which judges the distributed lock file and performs resource access operations when the lock-holding node is this node. When it is not the lock-holding node, a lock request is sent to the data node for locking processing, until the lock-holding node in the distributed lock file is this node, resource access is performed, thereby realizing locking processing of distributed resources and improving the consistency of data access.

[0050] The following describes a distributed system resource access method provided by the present application through an embodiment.

[0051] Please refer to Figure 1 , Figure 1 A flowchart of a resource access method for a distributed system provided in an embodiment of the present application.

[0052] In this embodiment, the method may include:

[0053] S101, the node determines whether the lock-taking node in the local distributed lock file is a node; if so, execute S102; if not, execute S103;

[0054] This step is intended to determine whether the lock-holding node recorded in the locally stored distributed lock file is the node. The lock-holding node is the node with resource access rights in this embodiment. In other words, when node A is recorded as a lock-holding node in the distributed lock file, the node can access the corresponding resource, while other nodes cannot access the node. At any time, only one node can access the resource, ensuring the consistency of data resource access at this time.

[0055] Among them, the distributed lock file refers to a file used to store lock records in a distributed environment. Each node stores a distributed lock file. When the node needs to perform a resource access operation, it can determine whether to perform the resource access operation by judging whether the lock-holding node in the distributed lock file is the current node.

[0056] The distributed lock file can be updated in real time through the cluster management server, or through the relay node to update the distributed lock file in each node, or by actively acquiring the local distributed lock file. It can be seen that the method of updating the distributed lock file in this step is not unique and is not specifically limited here.

[0057] Further, in order to reduce the bandwidth occupied by the distributed lock file update and improve the network performance utilization rate in the cluster, the distributed lock file in this embodiment adopts a passive update method. When the distributed lock file is modified, the node that requests the modification applies to become a data node and modifies the distributed lock file, and the requested node receives the modified distributed lock file. In other words, the distributed lock file in this embodiment is not completely updated as a whole. When the node judges the local distributed lock file, the content of the distributed lock file may not be the latest, and the state of the lock-taking node recorded in the file is also uncertain. When the lock-taking node is the current node, it means that no other node has requested to modify the distributed lock file, that is, it is the latest distributed lock file. When the lock-taking node is not the current node, it means that the distributed lock file has been modified by other nodes and may not be the latest distributed lock file. It is necessary to request other nodes to obtain the latest distributed lock file.

[0058] S102, if yes, then perform resource access operation;

[0059] Based on S101, this step aims to perform a resource access operation, that is, once the lock holding node recorded in the distributed lock file is determined to be the current node, the resource access operation can be performed.

[0060] During the operation of the distributed system, the local distributed lock file may not be the latest file, and the content of the file may be modified by other nodes. Therefore, in this embodiment, when access to resources is required, it is necessary to determine whether the lock holding node in the local distributed lock file is the current node, and further determine whether to perform the resource access operation.

[0061] The resource access operation performed in this step may adopt any resource access operation provided by the prior art, and is not specifically limited here.

[0062] S103, if not, sending a lock request to the data node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node;

[0063] Based on S101, this step aims to send a lock request to the data node. The lock request may be accepted by the data node or not. The data node will return the latest distributed lock file to the node.

[0064] The data node refers to the file that has the permission to modify the distributed lock file in the distributed cluster. The data node can be a lock node or not.

[0065] The process of the data node processing the lock request may refer to the following optional solutions.

[0066] Furthermore, the operations performed by the data node in this step may include:

[0067] Step 1: The data node receives the lock request;

[0068] Step 2: Determine whether the lock node in the distributed lock file of the data node has been processed;

[0069] Step 3: If yes, modify the lock node in the distributed lock file of the data node to the node corresponding to the lock request, and send the distributed lock file to the node;

[0070] Step 4: If not, send the distributed lock file to the node.

[0071] It can be seen that this optional solution mainly explains how the data node processes the lock request. In this optional solution, the data node first receives the lock request; then, it determines whether the lock-taking node in the distributed lock file of the data node has been processed; if so, the lock-taking node in the distributed lock file of the data node is modified to the node corresponding to the lock request, and the distributed lock file is sent to the node; if not, the distributed lock file is sent to the node. Obviously, regardless of whether the corresponding lock-taking node in this data node has been processed, the latest distributed lock file is returned to the node so that the node can determine whether it can perform the resource access operation or not.

[0072] S104, executing the step of determining whether the lock-taking node in the local distributed lock file is a node, and when the lock-taking node is a node, executing the business access resource operation.

[0073] Based on S103, this step aims to cyclically determine whether the lock-taking node in the distributed lock file is the node, and if so, the corresponding resource access operation can be performed. That is, when the lock-taking node in the received distributed lock file is not the current node, the lock request is cyclically sent and determined until it is the current node.

[0074] In addition, the lock node in the latest distributed lock file returned by the data node may also be empty, in which case the identifier of the node may be directly written into the lock node.

[0075] When the lock node in the distributed lock file is empty, the unique identifier of the node is written into the distributed lock file and the business resource operation is performed.

[0076] Furthermore, this embodiment may also include:

[0077] Step 1: When the business resource operation is completed, determine whether the node is a data node;

[0078] Step 2: When the node is not a data node, send a data node identity request to obtain data node identity authority;

[0079] Step 3: When the node is a data node, change the lock node of the distributed lock file to empty.

[0080] It can be seen that this optional solution mainly explains how to operate when the execution is completed. In this optional solution, when the business resource operation is completed, it is determined whether the node is a data node, that is, whether the distributed lock file can be modified. If it is not a data node, a data node identity request is sent to obtain the data node identity authority, and the latest distributed lock file can also be obtained. When the node is a data node, the lock node of the distributed lock file is modified to empty. So that other nodes can obtain the new distributed lock file and modify it.

[0081] In summary, this embodiment judges the operation of the distributed lock file, and when the lock-holding node is this node, the resource access operation can be performed. When it is not the lock-holding node, a lock request is sent to the data node for locking processing, until the lock-holding node in the distributed lock file is this node and resource access is performed, thereby realizing locking processing of distributed resources and improving the consistency of data access.

[0082] The following is a specific example to further illustrate a distributed system resource access method provided by the present application.

[0083] In this embodiment, a distributed lock file is implemented through a key-value database based on hash storage. Each node has a corresponding distributed lock file, thereby realizing the locking requirements between different distributed nodes.

[0084] In this embodiment, the nodes in the distributed cluster have a permanent unique identifier, which is used to indicate the identity of the lock owner in the database. The distributed lock file itself is a separate db file, and the storage format of the distributed lock file content uses hash bucket technology. Keys with the same hash value will be stored in a chained manner, and chain access is achieved by storing the offset of the next record in the specific implementation. In addition, for deleted keys, after deleting the corresponding record, they are mounted on the free list in the freelist manner, and adjacent records are merged as much as possible to achieve a larger free storage area.

[0085] This embodiment introduces the concepts of data node and relay node for a single key, that is, a single node. The record of the data node is always the latest, and the relay node always knows the identifier of the data node. The relay node is selected from the current active node. If the active node does not change, the relay node of a single key remains unchanged. A single record records the unique identifier of the data node and the unique identifier of the existing lock node of the key.

[0086] When a single node tries to lock a key, it will lock the hash bucket to which the key belongs on this node through the locking interface of the above database; then it will obtain the record information stored by the key; if it is found that the data node recorded in the record is not this node, that is, the data recorded in the record is not the latest, it is necessary to initiate a lock request to the relay node and forward it to the data node through the relay node; after the data node receives the lock request, it will upgrade the initial initiator of the request to a new data node and record it in the record of its own node, and then send a data node change request to the relay node, carrying the new data node and the lock node recorded in the record; after receiving the request, the relay node will also record the new data node in the record of its own node, and finally send the response upgraded to the data node to the initial node of the request; after receiving the request, the initial node updates the data node and lock node information recorded in the record of this node.

[0087] After the node is upgraded to the data node of the corresponding key, check whether the locking node recorded in the record is the current node. If it is the current node, you can perform the business operations after locking. If it is not the current node, you need to release the lock of the key in the database of the current node and try the above steps again until you find that the locking node recorded in the latest record no longer exists. Then write the unique identifier of the current node into the record, indicating that the lock is successfully acquired.

[0088] In the system of this embodiment, there is also a database management cluster. When a node failure occurs, the current master node of the cluster can handle the data consistency between the surviving nodes. The specific operation is that the master node initiates the overall locking operation of the database and freezes the database on each surviving node. After freezing, all surviving nodes are notified to release the previous lock and retry locking. Then all the lock records in the record are cleared, the data node is modified into a new relay node, and synchronized to each surviving node. After clearing the record, the database of all surviving nodes is unfrozen and waits for a new locking operation.

[0089] It can be seen that this embodiment judges the operation of the distributed lock file, and when the lock-holding node is this node, the resource access operation can be performed. When it is not the lock-holding node, a lock request is sent to the data node for locking processing, until the lock-holding node in the distributed lock file is this node, resource access is performed, thereby realizing locking processing of distributed resources and improving the consistency of data access.

[0090] The following is an introduction to a resource access device of a distributed system provided in an embodiment of the present application. The resource access device of the distributed system described below and the resource access method of the distributed system described above can be referenced to each other.

[0091] Please refer to Figure 2 , Figure 2 A schematic diagram of the structure of a resource access device for a distributed system provided in an embodiment of the present application.

[0092] In this embodiment, the device may include:

[0093] The lock determination module 100 is used to determine whether the lock node in the local distributed lock file is a node;

[0094] The resource access module 200 is used to perform resource access operations when the lock node is a node;

[0095] The lock request module 300 is used to send a lock request to the data node when the lock holding node is not a node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node;

[0096] The loop judgment module 400 is used to execute the step of judging whether the lock-taking node in the local distributed lock file is a node, and when the lock-taking node is a node, the service access resource operation is performed.

[0097] Optionally, the device may further include:

[0098] The lock node writing module is used to write the unique identifier of the node into the distributed lock file and perform business resource operations when the lock node in the distributed lock file is empty.

[0099] Optionally, the device may further include:

[0100] The access completion operation module is used to determine whether the node is a data node when the business resource operation is completed; when the node is not a data node, send a data node identity request to obtain the data node identity authority; when the node is a data node, modify the lock node of the distributed lock file to empty.

[0101] Optionally, the data node includes:

[0102] A lock request receiving unit, used for receiving a lock request;

[0103] A processing status judgment unit is used to judge whether the lock node in the distributed lock file of the data node has been processed;

[0104] A modification unit, used for modifying the lock holding node in the distributed lock file of the data node to the node corresponding to the lock request when the processing is completed, and sending the distributed lock file to the node;

[0105] The sending unit is used to send the distributed lock file to the node when the processing is not completed.

[0106] The present application also provides a server, including:

[0107] Memory for storing computer programs;

[0108] The processor is used to implement the steps of the resource access method described in the above embodiment when executing the computer program.

[0109] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the resource access method described in the above embodiment are implemented.

[0110] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0111] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0112] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0113] The resource access method, resource access device, server and computer-readable storage medium of a distributed system provided by the present application are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A resource access method for a distributed system, characterized in that: include: The node determines whether the lock holding node in the local distributed lock file is the node; If so, the resource access operation is performed; If not, sending a lock request to the data node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node; Execute the step of determining whether the lock-taking node in the local distributed lock file is the node, and when the lock-taking node is the node, execute the service access resource operation; The method further includes: when the lock node in the distributed lock file is empty, writing the unique identifier of the node into the distributed lock file and performing a business resource operation; The method further includes: when the business resource operation is completed, determining whether the node is a data node; when the node is not a data node, sending a data node identity request to obtain data node identity authority; when the node is a data node, modifying the lock node of the distributed lock file to be empty; The distributed lock file is implemented based on a key-value database stored in a hash and is updated in a passive manner; the passive manner includes real-time updates through a cluster management server or updates of the distributed lock file in each node through a relay node; The data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, including: the data node receives the lock request; determines whether the lock-taking node in the distributed lock file of the data node has been processed; if so, modifies the lock-taking node in the distributed lock file of the data node to a node corresponding to the lock request, and sends the distributed lock file to the node; if not, sends the distributed lock file to the node.

2. A resource access device for a distributed system, characterized in that: include: A lock determination module, used to determine whether the lock holding node in the local distributed lock file is the node; A resource access module, used for executing a resource access operation when the lock-taking node is the node; A lock request module, used for sending a lock request to a data node when the lock holding node is not the node, so that the data node processes the distributed lock file of the data node according to the lock request to obtain a new distributed lock file, and sends the new distributed lock file to the node; A loop judgment module, used to execute the step of judging whether the lock-taking node in the local distributed lock file is the node, until the lock-taking node is the node, and then perform a business access resource operation; The device further includes: a lock node writing module, which is used to write the unique identifier of the node into the distributed lock file and perform a business resource operation when the lock node in the distributed lock file is empty; The access completion operation module is used to determine whether the node is a data node when the business resource operation is completed; when the node is not a data node, send a data node identity request to obtain data node identity authority; when the node is a data node, modify the lock node of the distributed lock file to empty; The distributed lock file is implemented based on a key-value database stored in a hash and is updated in a passive manner; the passive manner includes real-time updates through a cluster management server or updates of the distributed lock file in each node through a relay node; The data node includes: a lock request receiving unit, used to receive the lock request; a processing status judgment unit, used to judge whether the lock node in the distributed lock file of the data node has been processed; a modification unit, used to modify the lock node in the distributed lock file of the data node to the node corresponding to the lock request when the processing is completed, and send the distributed lock file to the node; a sending unit, used to send the distributed lock file to the node when the processing is not completed.

3. A server, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the resource access method as claimed in claim 1 when executing the computer program.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the resource access method according to claim 1 are implemented.

Images