Consistency read-write method supporting single-node transactions in distributed data

Abstract

The invention discloses a consistent read-write method supporting single node transactions in distributed data, which comprises the following steps: a request node judges whether a matched node lock is held or not according to an access demand, if not, a locking request is sent to an affiliation node and a serial number is given, the affiliation node checks whether tuple data is migrated or not, and if yes, the affiliation node sends the locking request to the affiliation node; if not, updating the lock table and informing the request node of successful locking; and if the new master node is migrated, notifying the new master node of information, and adding a transaction lock after the request node receives a successful request, or sending a locking request to the new master node and processing the locking request after receiving the new master node information. According to the technical scheme, based on a two-layer lock structure, the consistency and integrity of data are ensured, and meanwhile the concurrency performance of the system is improved. According to the method, the serial number is endowed to the request of the same resource, and a serial number mechanism and a mapping table are combined, so that a new Master node can correctly process the request after tuple migration, and the consistency and fairness of data are maintained.

Description

Technical Field

[0001] The present invention belongs to the technical field of data concurrency control, and in particular relates to a consistent reading and writing method supporting single-node transactions in distributed data. Background Art

[0002] Distributed storage systems are key components in modern computing environments. They provide data storage, management and access capabilities to support large-scale data processing and analysis tasks. With the rapid development of cloud computing and big data technologies, the demand for distributed storage systems is growing. They need to process massive amounts of data from all over the world while ensuring high availability, consistency and reliability of the data. In this context, it is particularly important to study the concurrency control of distributed storage.

[0003] Distributed storage systems need to have good scalability so that system performance does not degrade when storage capacity and computing resources are increased. Concurrency control mechanism is a key technology to ensure that the system can maintain high performance under high load. It involves how to effectively manage multiple clients' access to the same data resource to avoid data inconsistency and system overload. In distributed systems, data consistency is a core issue. The CAP theorem points out that in a distributed system, consistency, availability, and partition tolerance cannot be met at the same time. Concurrency control mechanism needs to find a balance between these factors to ensure data consistency and reliability. Concurrency control of distributed storage faces many technical challenges, including how to design efficient concurrency control protocols, how to optimize lock and transaction management, and how to achieve consistency across multiple data centers. Summary of the invention

[0004] The purpose of the present invention is to provide a consistent reading and writing method that supports single-node transactions in distributed data, so as to solve the problems existing in the above-mentioned prior art.

[0005] To achieve the above object, the present invention provides a consistent reading and writing method supporting single-node transactions in distributed data, comprising:

[0006] Obtaining access requirements of the requesting node for the tuple data to be accessed, and judging whether the requesting node holds a node lock matching the access requirements based on the access requirements, wherein the node lock includes a node shared lock and a node exclusive lock;

[0007] Determine whether the node lock held by the requesting node matches the access requirement, and based on the determination result, add a transaction lock or send a lock request to the home node corresponding to the tuple data to be accessed; wherein the transaction lock includes a transaction shared lock and a transaction exclusive lock;

[0008] When the lock request is sent to the home node, a serial number is assigned to the lock request, and the lock request after the serial number is assigned is processed;

[0009] The specific process of processing the lock request after the serial number is assigned includes: by querying the mapping table, judging whether the tuple data to be accessed has been migrated, if the tuple data to be accessed has not been migrated, adding a lock request in the lock table and updating the lock table, after the lock table is updated, sending a lock success request to the requesting node, if the tuple data to be accessed has been migrated, sending new master node information to the requesting node;

[0010] When the requesting node receives the successful lock request, a transaction lock is added by using a lock proxy module;

[0011] When the requesting node receives the new master node information, it sends a lock request to the new master node, assigns a serial number to the lock request, and processes the lock request after the serial number is assigned.

[0012] Optionally, the determining whether the node lock held by the requesting node matches the access requirement specifically includes:

[0013] When the requesting node holds a node lock that matches the access requirement, a transaction lock is added in the requesting node; when the requesting node does not hold a node lock that matches the access requirement, a lock request is sent to the home node corresponding to the tuple data to be accessed.

[0014] Optionally, the process of selecting the home node corresponding to the tuple data to be accessed specifically includes:

[0015] The home node to which the tuple data to be accessed originally belongs is determined based on the consistent hashing algorithm.

[0016] Optionally, the node shared lock allows multiple transactions to read data of a node simultaneously, but does not allow write transactions; the node exclusive lock allows exclusive access for write operations, preventing other transactions from reading or writing.

[0017] Optionally, the transaction shared lock allows multiple transactions to concurrently read the same data item; the transaction exclusive lock allows exclusive access for write operations, preventing other transactions from reading or writing.

[0018] Optionally, adding a lock request in a lock table and updating the lock table specifically includes:

[0019] The lock table is locked, and it is determined whether there is a conflict between the lock request and the original lock in the lock table. If there is a conflict, the conflict is processed. If there is no conflict, the lock request is considered successful. After the conflict judgment is completed, a lock is added to the lock table based on the lock request, and the status of each lock in the lock table is updated. After the lock table update is completed, the lock table is released.

[0020] Optionally, the specific process of handling the conflict includes:

[0021] If the node does not release the transaction exclusive lock, the current request thread will be blocked; when the node write operation is completed, the transaction exclusive lock will be released.

[0022] Optionally, the requesting node, the home node and the new master node are connected to communicate with each other via RDMA technology.

[0023] The technical effects of the present invention are:

[0024] The present invention provides a consistent read and write method that supports single-node transactions in distributed data. Based on a two-layer lock structure, the consistency and integrity of data are ensured, while improving the concurrent performance of the system. The present invention proposes to assign sequence numbers to requests for the same resource, combining a sequence number mechanism and a mapping table, thereby ensuring that after tuple migration, the new Master node can correctly process the request, thereby maintaining the consistency and fairness of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0027] Figure 1 A flowchart of a concurrency control method for reading / writing a tuple in an embodiment of the present invention;

[0028] Figure 2 An example diagram of conflict handling in a concurrency control method according to an embodiment of the present invention;

[0029] Figure 3 is a flowchart of node role conversion in an embodiment of the present invention;

[0030] Figure 4 This is a design diagram of a node module in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as limiting the present invention, but should be understood as a more detailed description of certain aspects, features, and embodiments of the present invention.

[0032] It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present invention description without departing from the scope or spirit of the present invention. Other embodiments derived from the present invention description will be apparent to those skilled in the art. The present application description and examples are exemplary only.

[0033] The words “include,” “including,” “have,” “contain,” etc. used in this article are open-ended terms, meaning including but not limited to.

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] Embodiment 1

[0036] like Figure 1 - Figure 4 As shown, this embodiment provides a consistent reading and writing method supporting single-node transactions in distributed data, including:

[0037] Obtaining access requirements of the requesting node for the tuple data to be accessed, and judging whether the requesting node holds a node lock matching the access requirements based on the access requirements, wherein the node lock includes a node shared lock and a node exclusive lock;

[0038] Determine whether the node lock held by the requesting node matches the access requirement, and based on the determination result, add a transaction lock or send a lock request to the home node corresponding to the tuple data to be accessed; wherein the transaction lock includes a transaction shared lock and a transaction exclusive lock;

[0039] When the lock request is sent to the home node, a serial number is assigned to the lock request, and the lock request after the serial number is assigned is processed;

[0040] The specific process of processing the lock request after the serial number is assigned includes: by querying the mapping table, judging whether the tuple data to be accessed has been migrated, if the tuple data to be accessed has not been migrated, adding a lock request in the lock table and updating the lock table, after the lock table is updated, sending a lock success request to the requesting node, if the tuple data to be accessed has been migrated, sending new master node information to the requesting node;

[0041] When the requesting node receives the successful lock request, a transaction lock is added by using a lock proxy module;

[0042] When the requesting node receives the new master node information, it sends a lock request to the new master node, assigns a serial number to the lock request, and processes the lock request after the serial number is assigned.

[0043] This embodiment provides a concurrency control method for a new hardware distributed storage system based on RDMA interconnection. This embodiment is applicable to a distributed storage system that requires high concurrent data processing, namely, TLCP (TupLock Concurrency Protocol). TLCP can effectively improve data access efficiency and ensure data consistency, specifically including:

[0044] A two-layer lock structure is used to ensure data consistency during multi-node transaction concurrency;

[0045] Improve the affinity of data access by implementing real-time migration of tuples through mapping tables;

[0046] By assigning sequence numbers to requests for the same resource, data inconsistency caused by communication delays is guaranteed;

[0047] By reducing communication through conflict handling, nodes do not actively release locks, ensuring full utilization of node locks.

[0048] This embodiment adopts a novel lock mechanism, and realizes precise control of read and write operations in the distributed storage system through a two-layer lock structure design. The first layer is the "node lock", which is responsible for coordinating concurrent access between different nodes to ensure that each node in the distributed system can share data efficiently and securely. The second layer is the "transaction lock", which manages the concurrent execution of transactions within a single node, further improving the concurrent processing capability of the system.

[0049] The method of ensuring data consistency during multi-node transaction concurrency through a two-layer lock structure specifically includes: each physical machine in a distributed system is regarded as a node, and there are multiple transaction threads on each node for read and write operations. Therefore, there are two layers of concurrency control between multiple transactions within a node and between multiple nodes in the system. Based on the above characteristics, a two-layer lock structure is designed for distributed locks, namely the "node lock" for controlling concurrency between nodes and the "transaction lock" for transaction concurrency within a node. The transaction lock can only be obtained after the node where the current thread is located obtains the node lock. The node-level write lock can support two types of transaction locks, but the node-level read lock can only support transaction read locks.

[0050] In order to enhance data affinity and real-time performance, this embodiment also introduces a mapping table to realize the real-time migration function of data. This mechanism enables data to be dynamically migrated within the system, thereby optimizing the storage location of data, reducing access delays, and improving the overall performance of the system. In addition, through a carefully designed lock mechanism, this embodiment not only ensures data consistency, but also significantly improves the throughput and response speed of the system. The method of improving the affinity of data access by implementing tuple migration through a mapping table specifically includes: the owner of a tuple determined by consistent hashing when the system is initialized is called the Home node, but in order to ensure affinity, once a node applies for a write lock on the tuple, the data of the tuple and the authority responsible for locking will be transferred to the node requesting the write lock, and the node needs to record where the tuple has been migrated, so the Home node records the node (i.e., Master) currently responsible for storing the tuple data and processing the lock request through a mapping table persisted in NVM.

[0051] The distributed lock protocol of this embodiment is particularly suitable for distributed storage systems that have strict requirements for high-concurrency data access and processing. It effectively improves the efficiency of data access by optimizing lock management and data migration strategies, while ensuring strong consistency of data in a distributed environment. In high-concurrency application scenarios, this embodiment can provide a reliable, efficient and high-performance solution to meet the dual needs of modern data centers for data processing speed and consistency.

[0052] The method of ensuring data inconsistency caused by communication delays by assigning a serial number to the same request specifically includes: the mapping table is updated in real time, and each time a tuple is accessed, the corresponding Home node will be queried for the mapping table. If it is a write lock request, the previous Master node will be queried and updated to the new Master, thereby achieving a logical order at the Home node. This requires that the order in which shared lock requests and exclusive lock requests arrive at the Master node must ensure the order in which the mapping table is queried at the Home. Therefore, this embodiment implements a serial number for all requests when they arrive at the Home node. Whenever an exclusive request arrives at the Home node, the Master will change, so the serial number will start again from 0, thereby ensuring that requests at the same Master node can be processed correctly.

[0053] The method of reducing communication through conflict handling and ensuring full utilization of node locks is as follows: In a single-machine nvmdb, a tuple can release the corresponding spin lock after access is completed. This is because the cost of releasing and relocking the lock on a single machine is relatively low. However, in a distributed environment, actively releasing the lock means that additional communication is required. Moreover, since it is a node lock, after a single thread in the node releases the lock, there may be other threads that need the lock of the tuple. After actively releasing the lock, it may be necessary to lock it immediately, resulting in a large amount of repeated communication and performance degradation. Therefore, whether it is a node read lock or a node write lock, the distributed lock protocol stipulates passive lock release, that is, the lock is released only when there is a conflict. When a conflict occurs, the node lock that owns the lock will be notified to prepare to release the node lock, so that each node lock obtained can be fully utilized, and a node will not be starved to death because it cannot obtain the lock all the time.

[0054] The two-layer lock structure refers to the "node lock" for controlling concurrency between nodes and the "transaction lock" for controlling transaction concurrency within a node.

[0055] Node Locks: Node locks are used to control access to individual nodes in a distributed system. Their main purpose is to coordinate concurrent operations between different nodes to prevent data inconsistencies and conflicts. There are two types of node locks:

[0056] Node-Shared (NS): This lock allows multiple transactions to read a node's data at the same time, but does not allow any transaction to write. When a transaction wants to read node data, it requests a Node-Shared lock. If there is no Node-Exclusive lock on the node, it can successfully obtain the NS lock and perform the read operation.

[0057] Node-Exclusive (NX): This lock is used for write operations to ensure that no other transactions can read or write to the node while the transaction is modifying the node data. If a transaction wants to write node data, it must first acquire a Node-Exclusive lock. This prevents other transactions from acquiring any type of node lock until the NX lock is released.

[0058] Transaction Locks: Once a transaction successfully acquires a node lock, it can perform further read and write operations within the node. Transaction locks are used to control concurrency within the node to ensure that data operations on a single node do not conflict with each other. Transaction locks are also divided into two types:

[0059] Shared (S): Read lock, which allows multiple transactions to read the same data item at the same time, but does not allow writing. When a transaction needs to read data, it requests a Shared lock.

[0060] Exclusive (E): Write lock, used for write operations, ensures that when a transaction writes data, no other transaction can read or write the same data item. When a transaction needs to modify data, it requests an Exclusive lock.

[0061] The mapping table is used to record which node each tuple data is currently stored on and which node is the master node responsible for processing the lock request of the tuple. The mapping table uses a suffix dictionary tree to implicitly store the LogicalRowid (logical row identifier) ​​of the tuple, which is an efficient data management method. When tuples migrate from one node to another, their (physical identifier) ​​PhysicalRowid on the new node is recorded in the mapping table. For the operation process of different LogicRowid (logical row identifier) ​​in the mapping table (mainly for the writing of different LogicRowid (logical row identifier)): determine whether Offset has a value, if so, lock it; after the lock is successfully locked, determine again whether Offset has a value, if so, release the lock, otherwise allocate space and insert the tree node.

[0062] According to the requirements of real-time migration, after the Requester obtains the exclusive lock of the node, the right to process the lock is also transferred. This will cause the requests that were previously waiting for the lock to be processed by the Master to be unable to be processed. These requests are either rejected or the requesting node is informed to request a new Master. No matter how it is solved, it will cause additional communication and time overhead. Therefore, the distributed lock adopts a mapping table instant update mechanism. When the Requester requests a write lock from the Home node, it will be recorded as the Master. All requests after the current Requester will be sent to the current Requester. After the current Requester obtains the lock, it becomes the Master and can process all waiting lock requests.

[0063] Among them, the method of assigning serial numbers to requests for the same resource is that by assigning serial numbers to requests for the same resource, the system can record the order of requests for the same tuple when they arrive at the Home node. This method not only ensures the order consistency of requests after they arrive at the Home node and before they are successfully locked, but also ensures the fairness of the lock. When a request arrives at the Home node, the system assigns a serial number to the request, which is an incrementing serial number before the next mapping table is updated. The sequence number ensures that even in high concurrency situations, requests can be processed by the Master node in the order of arrival.

[0064] Among them, conflict handling reduces communication, and the method of not actively releasing the lock by the node is that in a single-machine nvmdb, a tuple can release the corresponding spin lock after the access is completed. This is because the cost of releasing and relocking the lock on a single machine is relatively low. However, in a distributed environment, actively releasing the lock means that additional communication is required, and because it is a node lock, after a single thread in the node releases the lock, there may be other threads that need the lock of the tuple. After actively releasing the lock, it may be necessary to lock it immediately, resulting in a large amount of repeated communication and performance degradation. Therefore, whether it is a node read lock or a node write lock, the distributed lock protocol stipulates passive lock release, that is, the lock is released only when there is a conflict, so that each node lock obtained can be fully utilized.

[0065] This embodiment provides a concurrency control method for a new hardware distributed storage system based on RDMA interconnection, based on a two-layer lock structure, to ensure data consistency and integrity, while improving the concurrent performance of the system. This embodiment proposes to assign serial numbers to requests for the same resource, implicitly store LogicalRowid in the suffix dictionary tree, and combine the serial number mechanism to ensure that after tuple migration, the new Master node can correctly process the request, maintaining data consistency and fairness.

[0066] like Figure 1 As shown, the specific implementation steps of this embodiment include:

[0067] Step 1: The Requester node initiates a lock request to the Home node;

[0068] Step 1.1: Initiate a lock request;

[0069] When a Requester node needs to access a specific tuple data, it first checks itself to confirm whether it already holds a node lock that matches the access intention. "Match" here means that if the Requester node wants to perform a read operation, it needs to hold a shared node lock; if it wants to perform a write operation, it needs an exclusive node lock.

[0070] Lock matching check: If the Requester node already holds the corresponding type of node lock, it does not need to apply for the node lock again. In this case, the Requester node can directly perform transaction-level locking operations in its node through the lock proxy module.

[0071] Lock mismatch or missing: If the Requester node does not hold a node lock of the corresponding type, or does not hold any node lock at all, it must first apply for a node lock. To do this, the Requester node uses a consistent hashing algorithm to determine the node to which the tuple originally belongs, that is, the Home node. Consistent hashing is a special hashing algorithm that minimizes data migration when nodes are added or removed, and maintains system load balancing.

[0072] Step 1.2: Locate the Home node and send a request;

[0073] Once the Home node is determined, the Requester node will send a lock request to the Home node through Remote Direct Memory Access (RDMA) technology. RDMA is an efficient network communication technology that allows one computer to send data directly to the memory of another computer without involving the kernel space of the operating system, thereby reducing latency and system overhead.

[0074] Step 2: Home node message processing flow;

[0075] Step 2.1: Check the mapping table;

[0076] When the Home node receives a lock request from the Requester node, its message processing module first queries the mapping table. The mapping table is a key data structure used to record the current location of tuple data and the information of the Master node responsible for managing this data.

[0077] Tuple migration check: The Home node will check whether the requested tuple has been migrated to other nodes. If the tuple has not been migrated, the Home node will continue to process the request; if the tuple has been migrated, the Home node will need to adopt a different processing strategy.

[0078] Step 2.2: Update the lock table;

[0079] Step 2.2.1: No migration;

[0080] If the tuple has not been migrated, the Home node will check its lock table, which records all currently active lock requests. The Home node will add the information of the new lock request to the lock table and query and update the lock table according to the sequence number (seq) of the request. The sequence number ensures that the requests are processed in the order they arrive at the Home node, thereby maintaining the fairness of the lock.

[0081] Step 2.2.2: Process the request;

[0082] Locking the lock table: When the Master node is ready to process a lock request, it first needs to lock the lock table. This step ensures the atomicity of operations when querying and updating the lock table, that is, these operations are either fully executed or not executed at all, with no intermediate state. Through the locking mechanism, the Master node ensures that only one request can modify the lock table at the same time, thereby avoiding data inconsistency problems that may be caused by multiple requests modifying the lock table at the same time.

[0083] Query the lock table to check conflicts: After the lock table is locked, the Master node will query the lock table to check whether the current request conflicts with the existing locks.

[0084] Conflict detection: The Master node checks whether the requested lock type (shared or exclusive) is compatible with the locks recorded in the lock table. For example, if the request is an exclusive lock and any type of lock already exists in the lock table, or if the request is a shared lock and an exclusive lock exists in the lock table, these are considered conflicts.

[0085] Conflict handling: If a conflict is detected, the Master node will execute the conflict handling strategy, such as Figure 2 As shown;

[0086] Requester node:

[0087] 1) The requesting node cannot find the tuple to be written locally, so it prepares an idle ProwID for the tuple to be written, applies an S lock to the extent where the ProwID is located, creates a lock table for the tuple, and sets itself as the X lock owner.

[0088] 2) Send an X-lock request to the remote node (the Home node calculated by logicalRowid).

[0089] 3) After receiving the response message from the Home node, send a request to the Master node (the Master node calculated by ProwID);

[0090] 4) After receiving the response message and data from the Master node, when the transaction processing in the node is completed, the X lock of its own node is downgraded to S lock, and the extent lock is stat-1.

[0091] Home node:

[0092] 1) As the Home node, when receiving a TRX request, it first checks the mapping table to determine whether the data has been migrated, and increases the sequence number distributor by one.

[0093] 2) Update the mapping table immediately, record the ProwID found, and update the mapping entry to the ProwID prepared by the Requester.

[0094] 3) Send the recorded ProwID and sequence number to the Requester.

[0095] Master node: When the Master receives a TRX / TRS request, it first adds an S lock to the extent lock table and adds Stat to the counter + 1. It checks the lock table to determine if a conflict occurs. The Master node has not released the X lock, blocking the thread processing the current request.

[0096] 3) The write operation of the Master node is completed, and the X lock and extent lock stat-1 are released.

[0097] 4) Continue to process TRX / TRS requests. If there is no conflict, add X / S lock successfully and transfer the data to the requesting node.

[0098] 5) (When the requesting node is TRX) Delete the lock table and data, stat-1 of extent.

[0099] (When the requesting node is TRS) stat-1 of extent.

[0100] No conflict: If no conflict is detected, the Master node will consider the lock request successful and continue to perform the lock operation.

[0101] Update the lock table: Once a lock request is successfully processed, whether by resolving conflicts or locking directly, the Master node needs to update the lock table to reflect this change.

[0102] Update operation: The Master node will update the lock table based on the results of the lock request, which may include adding new lock records, updating the status of existing locks, or deleting outdated lock records.

[0103] Release the lock table: After the update is completed, the Master node will release the lock table lock, allowing other requests to continue processing.

[0104] Step 2.3: Respond to the requesting node;

[0105] Non-migration situation: Once the current request is processed, the Home node will send a response message to the Requester node to inform it of the result of the lock request.

[0106] Migration: If the tuple has been migrated and a write lock is requested, the Home node will record the queried Master node information and immediately update the requested node information to the new Master node. This update ensures that subsequent requests can recognize changes to the Master node.

[0107] Step 2.4: Forward Master information;

[0108] Migration: After confirming that the tuple has been migrated, the Home node sends the queried Master node information to the Requester node. This step allows the Requester node to understand the current Master node and communicate with it directly to complete the lock request.

[0109] Through the message processing flow designed in this embodiment, the Home node can efficiently manage access requests to tuple data, ensuring data consistency and fairness of lock operations. At the same time, by timely updating and maintaining the mapping table and lock table, the Home node can adapt to data migration and node changes in the system, maintaining high availability and high performance of the system.

[0110] Step 3: The Requester node processes the message according to the type of message replied by the Home node;

[0111] Step 3.1: Receive the reply from the Home node;

[0112] After sending the lock request, the Requester node will wait for the reply from the Home node. This reply will contain key information about the result of the lock request, which may be a confirmation of successful lock or a notification indicating that the tuple has been migrated to the new Master node.

[0113] Step 3.2: Process the reply of successful locking;

[0114] Successful lock: If the Requester node receives a reply that the lock is successful, it will proceed to the next step. In this case, the lock agent module of the Requester node will be responsible for performing the lock operation at the transaction level. This means that the Requester node will ensure that access to specific tuples is controlled during the transaction execution to maintain data consistency and integrity.

[0115] Step 3.3: Perform transaction-level locking in the lock proxy module;

[0116] Transaction-level locking: The lock proxy module of the Requester node will apply for the corresponding lock (such as shared lock or exclusive lock) according to the transaction requirements. This process ensures that the transaction can safely access or modify data according to the established isolation level when executing, while preventing conflicts with other concurrent transactions.

[0117] Step 3.4: Process new Master information;

[0118] Migrate to the new Master: If the Requester node receives a new Master node message, it will need to redirect its request. In this case, the Requester node will send a node lock request to the new Master node to continue its transaction operation. This step is necessary because the management and locking permissions of the tuple have been transferred to the new Master node.

[0119] Step 3.5: Send a node lock request;

[0120] Request a new Master node: The Requester node will send a request to add a node lock to the new Master node based on the new Master information provided by the Home node. This request will follow the same process as before to ensure that the new Master node can handle the lock request and maintain the consistency and stability of the entire system.

[0121] Step 4: The Requester node initiates a lock request to the Master node;

[0122] Once the status of the Master node is confirmed, the Requester node will construct and send a specific request, such as a read lock or write lock request.

[0123] Step 5: The Master node processes the lock request.

[0124] This step is consistent with step 2.2.2.

[0125] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A consistent reading and writing method supporting single-node transactions in distributed data, characterized in that: include: Obtaining access requirements of the requesting node for the tuple data to be accessed, and judging whether the requesting node holds a node lock matching the access requirements based on the access requirements, wherein the node lock includes a node shared lock and a node exclusive lock; Determine whether the node lock held by the requesting node matches the access requirement, and based on the determination result, add a transaction lock or send a lock request to the home node corresponding to the tuple data to be accessed; wherein the transaction lock includes a transaction shared lock and a transaction exclusive lock; When the lock request is sent to the home node, a serial number is assigned to the lock request, and the lock request after the serial number is assigned is processed; The specific process of processing the lock request after the serial number is assigned includes: by querying the mapping table, judging whether the tuple data to be accessed has been migrated, if the tuple data to be accessed has not been migrated, adding a lock request in the lock table and updating the lock table, after the lock table is updated, sending a lock success request to the requesting node, if the tuple data to be accessed has been migrated, sending new master node information to the requesting node; When the requesting node receives the successful lock request, a transaction lock is added by using a lock proxy module; When the requesting node receives the new master node information, it sends a lock request to the new master node, assigns a serial number to the lock request, and processes the lock request after the serial number is assigned.

2. The consistent reading and writing method according to claim 1, characterized in that: The determining whether the node lock held by the requesting node matches the access requirement specifically includes: When the requesting node holds a node lock that matches the access requirement, a transaction lock is added in the requesting node; when the requesting node does not hold a node lock that matches the access requirement, a lock request is sent to the home node corresponding to the tuple data to be accessed.

3. The consistent reading and writing method according to claim 2, characterized in that: The process of selecting the home node corresponding to the tuple data to be accessed specifically includes: The home node to which the tuple data to be accessed originally belongs is determined based on the consistent hashing algorithm.

4. The consistent reading and writing method according to claim 1, characterized in that: The node shared lock allows multiple transactions to read data from a node at the same time, but does not allow write transactions; the node exclusive lock allows exclusive access for write operations, preventing other transactions from reading or writing.

5. The consistent reading and writing method according to claim 1, characterized in that: The transaction shared lock allows multiple transactions to read the same data item concurrently; the transaction exclusive lock allows exclusive access for write operations, preventing other transactions from reading or writing.

6. The consistent reading and writing method according to claim 1, characterized in that: The adding of a lock request in the lock table and updating the lock table specifically includes: The lock table is locked, and it is determined whether there is a conflict between the lock request and the original lock in the lock table. If there is a conflict, the conflict is processed. If there is no conflict, the lock request is considered successful. After the conflict judgment is completed, a lock is added to the lock table based on the lock request, and the status of each lock in the lock table is updated. After the lock table update is completed, the lock table is released.

7. The consistent reading and writing method according to claim 6, characterized in that: The specific process of handling the conflict includes: If the node does not release the transaction exclusive lock, the current request thread will be blocked; when the node write operation is completed, the transaction exclusive lock will be released.

8. The consistent reading and writing method according to claim 1, characterized in that: The requesting node, the home node and the new master node are connected to each other through RDMA technology.

Images