Method for managing remote lock waits in distributed database, and distributed database

Abstract

The present application provides a method for managing remote lock waits in a distributed database. The method comprises: executing a remote data processing task sent by a control node; if the remote data processing task encounters a lock conflict on first data, sending lock conflict information corresponding to the first data to the control node; in response to a received lock wait message corresponding to the first data and sent by the control node, adding the remote data processing task to a lock wait queue corresponding to the first data; and if the first data is released and the remote data processing task is the first task in the lock wait queue corresponding to the first data, sending a retry message corresponding to the remote data processing task to the control node, so that the control node sends the remote data processing task to at least one execution node again.

Description

Remote lock waiting management method in distributed database and distributed database TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of database, and in particular to a remote lock waiting management method in distributed database and distributed database. BACKGROUND

[0002] In today's high-speed development of the Internet and informatization, the generation of data is growing explosively, so the requirements for database and its management are getting higher and higher. In the process of executing SQL requests and other data processing tasks, if there is concurrent access to data, it is often necessary to add locks to the accessed data; for example, adding shared locks, exclusive locks, etc. In this way, the consistency of data and the isolation of transactions can be maintained when concurrent access to data is performed.

[0003] This makes different data processing tasks may have the need to lock the same data at the same time, resulting in lock conflicts. In actual application, in order to maintain the consistency, integrity and accuracy of data, the database management system usually does not allow multiple data processing tasks to perform lock processing on the same data at the same time. The so-called lock conflict is specifically the conflict caused by the attempt to perform lock processing on the same data at the same time due to the operation (such as reading, writing) of multiple data processing tasks on the data itself.

[0004] In the face of the above lock conflict problem, for a single database instance, a lock waiting queue can often be introduced to manage lock waiting, so that different data processing tasks can perform lock on data based on the lock waiting queue.

[0005] However, for a distributed database, since it may involve cross-node task execution, when facing the problem of lock conflict involving multiple nodes, how to efficiently manage lock waiting is a technical challenge. SUMMARY

[0006] Therefore, one or more embodiments of the present specification provide a remote lock waiting management method in distributed database, electronic equipment and storage medium.

[0007] To achieve the above purpose, one or more embodiments of the present specification provide technical solutions as follows:

[0008] According to a first aspect of one or more embodiments of the present specification, a remote lock waiting management method in distributed database is provided, the method comprising:

[0009] executing the remote data processing task sent by the control node; wherein the remote data processing task is a distributed task sent by the control node to at least one execution node for remote execution;

[0010] if a lock conflict occurs on the first data for the remote data processing task, sending lock conflict information corresponding to the first data to the control node;

[0011] in response to the lock waiting message corresponding to the first data sent by the control node, adding the remote data processing task to a lock waiting queue corresponding to the first data;

[0012] if the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data, sending a retry message corresponding to the remote data processing task to the control node, so that the control node re-sends the remote data processing task to the at least one execution node.

[0013] Optionally, the method further comprises:

[0014] in response to the rollback message corresponding to the remote data processing task sent by the control node, performing task rollback processing on the remote data processing task; wherein the rollback message is a rollback message sent by the control node to the at least one execution node after receiving the lock conflict information.

[0015] Optionally, the response to the lock waiting message corresponding to the first data sent by the control node and adding the remote data processing task to the lock waiting queue corresponding to the first data comprises:

[0016] in response to the lock waiting message corresponding to the first data sent by the control node, determining whether the first data has been released; if the first data has not been released, further adding the remote data processing task to the lock waiting queue corresponding to the first data.

[0017] Optionally, adding the remote data processing task to the lock waiting queue corresponding to the first data comprises:

[0018] creating a false task corresponding to the remote data processing task that has been rolled back;

[0019] adding the false task to the lock waiting queue corresponding to the first data.

[0020] Optionally, the method further comprises:

[0021] if the first data has been released, sending the retry message to the control node, so that the control node re-sends the remote data processing task to the at least one execution node.

[0022] Optionally, the remote data processing task is a data processing task performed on a plurality of data maintained by the execution node; the execution node maintains a plurality of lock waiting queues corresponding to the plurality of data one by one;

[0023] The method further comprises:

[0024] executing the remote data processing task re-sent by the control node;

[0025] if a lock conflict occurs on second data in the process of executing the remote data processing task, sending lock conflict information corresponding to the second data to the control node, so that the control node adds the remote data processing task to a lock waiting queue corresponding to the second data and maintained by the control node locally; wherein the first data and the second data are different;

[0026] in response to the lock waiting message corresponding to the second data sent by the control node and received, adding the remote data processing task to the lock waiting queue corresponding to the second data; and,

[0027] removing the remote data processing task from the lock waiting queue corresponding to the first data.

[0028] Optionally, removing the remote data processing task from the lock waiting queue corresponding to the first data comprises:

[0029] actively removing the remote data processing task from the lock waiting queue corresponding to the first data; or,

[0030] in response to the lock waiting termination message corresponding to the first data sent by the control node and received, removing the remote data processing task from the lock waiting queue corresponding to the first data.

[0031] Optionally, the method further comprises:

[0032] if the first data is released, and the remote data processing task is the first task in the lock waiting queue corresponding to the first data, adding a retry waiting time length for a second task in the lock waiting queue;

[0033] If, after sending the retry message to the control node, the remote data processing task is not received from the control node within the retry waiting time, the remote data processing task is removed from the lock waiting queue corresponding to the first data, and a retry message corresponding to the second task is sent to the control node, so that the control node resends the second task to at least one execution node.

[0034] Optionally, the method further includes:

[0035] If, after sending the retry message to the control node, the remote data processing task is received again by the control node within the retry waiting time, the retry waiting time added for the second task is removed.

[0036] According to a second aspect of one or more embodiments of this specification, a remote lock wait management method in a distributed database is also proposed, the method comprising:

[0037] A remote data processing task is sent to at least one execution node so that the at least one execution node executes the remote data processing task; wherein the remote data processing task is a distributed task sent by a control node to at least one execution node for remote execution;

[0038] In response to receiving lock conflict information corresponding to the first data sent by any of the at least one execution node, the remote data processing task is added to the lock waiting queue corresponding to the first data, and a lock waiting message corresponding to the first data is sent to the execution node, so that the execution node adds the remote data processing task to the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0039] In response to a retry message sent by the execution node corresponding to the remote data processing task, the remote data processing task is resent to the at least one execution node; wherein the retry message is a retry message sent by the execution node to the control node when the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0040] Optionally, the method further includes:

[0041] In response to a lock conflict message corresponding to the first data sent by any of the at least one execution node, a rollback message is sent to the at least one execution node so that the at least one execution node performs a task rollback process on the remote data processing task.

[0042] Optionally, in response to the retry message corresponding to the remote data processing task sent by the execution node, the remote data processing task is re-sent to the at least one execution node, including:

[0043] In response to the retry message corresponding to the remote data processing task sent by the execution node, the remote data processing task is removed from the lock waiting queue corresponding to the first data, and after the remote data processing task is removed from the lock waiting queue corresponding to the first data, the remote data processing task is re-sent to the at least one execution node.

[0044] Optionally, the remote data processing task is a data processing task performed on a plurality of data maintained by the execution node; the control node maintains a plurality of lock waiting queues corresponding to the plurality of data one by one;

[0045] The method further includes:

[0046] After the remote data processing task is re-sent to the at least one execution node, in response to the lock conflict information corresponding to the second data sent by any one of the at least one execution node received, the remote data processing task is added to the lock waiting queue corresponding to the second data;

[0047] The lock waiting message corresponding to the second data is sent to the execution node, so that the execution node adds the remote data processing task to the lock waiting queue corresponding to the second data maintained locally by the execution node.

[0048] Optionally, the method further includes:

[0049] In response to the lock conflict information corresponding to the second data sent by any one of the at least one execution node received, the lock waiting termination message corresponding to the first data is sent to the execution node, so that the execution node removes the remote data processing task from the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0050] Optionally, the method further includes:

[0051] The state of the at least one execution node is periodically probed;

[0052] If any one of the at least one execution node is detected to be in an abnormal state, the remote data processing task is removed from the lock waiting queue, and after the remote data processing task is removed from the lock waiting queue, the remote data processing task is re-sent to the at least one execution node.

[0053] The technical solutions provided by the embodiments of the present specification can include the following beneficial effects:

[0054] The remote lock waiting management method provided by the embodiments of the present specification can achieve remote lock waiting management by means of the execution node actively reporting lock conflict information to the control node and the timing of task retry, so that remote lock waiting management can be performed on remote data processing tasks in the distributed database, thereby avoiding the waste of computing resources and bandwidth between database instances caused by continuous task retry after lock conflict occurs in the remote data processing task, and improving the data processing effect of the distributed database and the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0055] FIG. 1 is a structural diagram of a distributed database provided by an example embodiment.

[0056] FIG. 2 is a flowchart of a remote lock waiting management method in a distributed database applied to an execution node according to an example embodiment.

[0057] FIG. 3 is a flowchart of a remote lock waiting management method in a distributed database applied to a control node according to an example embodiment.

[0058] FIG. 4 is a structural diagram of a device according to an example embodiment.

[0059] FIG. 5 is a block diagram of a remote lock waiting management apparatus in a distributed database applied to an execution node according to an example embodiment.

[0060] FIG. 6 is a block diagram of a remote lock waiting management apparatus in a distributed database applied to a control node according to an example embodiment. DETAILED DESCRIPTION

[0061] The example embodiments will be described in detail herein, with examples shown in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following example embodiments do not represent all implementations consistent with one or more embodiments of the present specification. Instead, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present specification, as detailed in the appended claims.

[0062] It should be noted that the steps of the respective methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the steps included in its method can be more or less than described in this specification. In addition, a single step described in this specification can be broken down into multiple steps for description in other embodiments; while multiple steps described in this specification can be combined into a single step for description in other embodiments.

[0063] Based on the technical problems mentioned in the background, at least one embodiment of the specification provides a remote lock waiting management method in a distributed database, which can efficiently manage the remote lock waiting of a remote data processing task in a distributed database after the lock conflict of the remote data processing task on a data resource, so as to avoid the waste of computing resources and bandwidth between database instances caused by continuous task retries after the lock conflict of the remote data processing task, and improve the data processing effect of the distributed database and the user experience.

[0064] The method can be applied to each database instance in the distributed database as shown in FIG. 1. The database instances in these database instances can act as both a control node (i.e. the source end of the task) and an execution node (i.e. the destination end of the task) when executing a remote data processing task.

[0065] For example, assuming that the database instance 1 shown in FIG. 1 receives a remote data processing task initiated by a user, which is specifically used for remotely processing the specified target data in a target data table commonly maintained by the database instances 2 and 3, at this time as the receiver of the task, the database instance 1 can act as the control node of the remote data processing task, and the database instances 2 and 3 will act as the execution node of the remote data processing task.

[0066] Among them, the above-mentioned remote data processing task can be a distributed task that needs to be executed remotely. The distributed task can be split into coordination logic that needs to be executed on the control node and computing logic that needs to be executed on at least one execution node, so as to avoid a single node from bearing all the tasks. The control node can be responsible for the coordination and management of the task (such as task distribution, lock conflict detection, waiting queue management, etc.), while the execution node can be responsible for the actual execution of the task (such as locking data, computing locked data, etc.).

[0067] The above-mentioned distributed database can include any form of distributed database, which is not limited in this specification. For example, in one example, the above-mentioned distributed database can be a distributed database based on LSM-Tree storage engine.

[0068] It should be understood that when performing a remote data processing task, the control node can send the remote data processing task to one or more execution nodes, which perform the task on the data resources maintained locally by the execution nodes involved in the task, and return the execution results to the control node after the execution is completed, so that the control node determines the final execution result of the task according to the execution results reported by all execution nodes. Of course, some execution nodes may encounter lock conflicts when executing the task, and the method is to manage lock waiting in this scenario.

[0069] Next, the remote lock waiting management method provided in the present specification will be described in detail from the execution node and the control node respectively.

[0070] Please refer to FIG. 2, which exemplarily shows the flow of the remote lock waiting management method applied to the execution node, including steps S201 to S203.

[0071] In step S201, the execution node executes the remote data processing task sent by the control node; wherein the remote data processing task is a distributed task sent by the control node to at least one execution node for remote execution.

[0072] The above-mentioned remote data processing task can specifically include a distributed task sent by the control node to at least one execution node for remote execution.

[0073] For example, in actual application, the above-mentioned remote data task can be a remote SQL (Structured Query Language, structured query language) task. After the control node receives a SQL request input by a user, if the SQL request is used to remotely process the specified target data in a target data table commonly maintained by multiple execution nodes, the control node can remotely issue the SQL request to the above-mentioned multiple execution nodes for distributed execution.

[0074] In step S202, if the remote data processing task encounters a lock conflict on the first data, the lock conflict information corresponding to the first data is sent to the control node;

[0075] The above-mentioned first data can be part or all of the data in the local data resource involved by the execution node when executing the remote data processing task; for example, it can be one or more rows of data in the local data resource involved by the execution node.

[0076] The control node allocates an execution thread for the remote data processing task when processing or retrying the remote data processing task, and the execution thread is used to process operations related to the remote data processing task. The execution node allocates an execution thread for the remote data processing task after receiving the remote data processing task, and the execution thread executes operations related to the remote data processing task.

[0077] For example, when the execution node executes the task based on the locally maintained data resource related to the remote data task, if a lock conflict occurs on the first data maintained locally, such as the first data has been locked by another task in advance, the execution node can report lock conflict information corresponding to the first data to the control node.

[0078] The lock conflict information reported by the execution node to the control node corresponding to the first data can include the data identifier of the first data. For example, the data identifier can include the key value of the data row, the hash value of the data row, and the like.

[0079] Of course, in actual application, in addition to the data identifier of the first data, the lock conflict information can also include other lock conflict related information, such as a lock conflict identifier that can uniquely identify the lock conflict.

[0080] In step S203, in response to the lock waiting message corresponding to the first data sent by the control node, the remote data processing task is added to the lock waiting queue corresponding to the first data.

[0081] When the control node processes or retries the remote data processing task for the first time, the control node sends the remote data processing task to one or more execution nodes related to the remote data processing task, and these execution nodes can have lock conflicts and report lock conflict information to the control node in the manner of step S202.

[0082] If the control node receives lock conflict information corresponding to the first data reported by an execution node, the control node can add the remote data processing task to the lock waiting queue corresponding to the first data maintained by the control node.

[0083] Of course, if the control node receives multiple lock conflict information sent by multiple execution nodes, and the multiple lock conflict information corresponds to multiple data, the control node can locally maintain multiple lock waiting queues corresponding to the multiple data, and add the remote data processing task to the multiple lock waiting queues respectively.

[0084] In some embodiments, after receiving the lock conflict information reported by any of the at least one execution node, the control node can further send a rollback message to the at least one execution node (i.e., each execution node performing the remote data processing task) to make each execution node perform transaction rollback on the remote data processing task according to the rollback message and release the thread of the remote data processing task, so as to ensure that when a lock conflict occurs in part of the execution nodes, the local thread can be released in time by other execution nodes, thereby avoiding waste of computing resources and improving data processing efficiency on the execution nodes.

[0085] In this way, the remote data processing task can be collectively rolled back on all execution nodes performing the remote data processing task as an atomic transaction.

[0086] In some embodiments, after adding the remote data processing task to the lock waiting queue corresponding to the first data maintained locally, the control node can further send a lock waiting message corresponding to the first data to the execution node reporting the lock conflict information corresponding to the first data. After receiving the lock waiting message, the execution node can also add the remote data processing task to the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0087] Subsequently, the execution node can actively determine whether the remote data processing task added to the lock waiting queue meets the task retry condition based on the lock waiting queue. In some embodiments, after receiving the lock waiting message corresponding to the first data sent by the control node, the execution node can further determine whether the first data has been released; if the first data has not been released, the remote data processing task is added to the lock waiting queue corresponding to the first data maintained locally.

[0088] For example, the execution node can determine whether the first data has been released by comparing whether the seq value of the lock waiting queue corresponding to the first data when the lock conflict occurs and the seq value of the lock waiting queue corresponding to the first data when the lock waiting message is received have changed, i.e., if the values have changed, the first data has been released, and if the values have not changed, the first data has not been released.

[0089] In some embodiments, after receiving the lock waiting message corresponding to the first data sent by the control node, if the first data has been released, the remote data processing task can no longer be added to the lock waiting queue corresponding to the first data maintained locally, but a retry message can be directly sent to the control node to make the control node resend the remote data processing task to the at least one execution node.

[0090] In some embodiments, after the execution node receives the lock waiting message corresponding to the first data sent by the control node, if the first data has been released at this time, the remote data processing task can no longer need to be added to the lock waiting queue corresponding to the first data maintained locally, and a retry message can be directly sent to the control node to make the control node resend the remote data processing task to the at least one execution node.

[0091] Since there is a time interval between the occurrence of the lock conflict and the receipt of the lock waiting message, the first data can have been released in this time interval. Therefore, the execution node confirms again whether the first data has been released when the lock waiting message is received. On the one hand, if the first data is confirmed again to have not been released, the remote data processing task can be added to the lock waiting queue corresponding to the first data maintained locally. In this way, local queue resources can be saved.

[0092] On the other hand, if the first data is confirmed again to have been released, the remote data processing task can be prevented from being added to the lock waiting queue corresponding to the first data maintained locally, and a retry message can be sent to the control node at the first time. In this way, the problem of the first data being idle due to the need to continue lock waiting although the first data has been released can be avoided.

[0093] In some embodiments, when the execution node adds the remote data processing task to the lock waiting queue corresponding to the first data, since the execution node has performed rollback processing on the remote data processing task and released the thread according to the rollback message issued by the control node before this time, the execution node can create a fake remote data processing task corresponding to the remote data processing task that has been rolled back, and add the fake remote data processing task to the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0094] The fake remote data processing task can specifically include a task simulated by the execution node after the remote data processing task has been rolled back, which is the same as the remote data processing task that has been rolled back.

[0095] For example, taking the remote SQL task as the remote data processing task, the execution node can create a simulated SQL request corresponding to the rolled-back SQL task, and add the simulated SQL request to the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0096] In step S204, if the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data, a retry message corresponding to the remote data processing task is sent to the control node, so that the control node re-sends the remote data processing task to the at least one execution node.

[0097] In actual application, when the execution node determines whether the remote data processing task added in the lock waiting queue meets the task retry condition based on the lock waiting queue maintained locally, the execution node can make a comprehensive determination by referring to the specific state of the first data and the position of the remote data processing task in the lock waiting queue.

[0098] In some embodiments, the execution node can determine whether the first data has been released and whether the remote data processing task is the first task in the lock waiting queue corresponding to the first data, to determine whether the remote data processing task meets the task retry condition. If the first data has been released and the remote data processing task has become the first task in the lock waiting queue corresponding to the first data, the execution node can send a retry message corresponding to the remote data processing task to the control node, so that the control node re-sends the remote data processing task to the at least one execution node.

[0099] In this way, the execution node can actively report a retry message to the control node to trigger the control node to retry the remote data processing task, so that the task retry for the remote data processing task can be triggered accurately and timely.

[0100] For example, in actual application, the first task in the lock waiting queue corresponding to the first data maintained by the control node is usually set with a default retry waiting time. Only when the retry waiting time is exceeded, the control node actively retries the first task. If the execution node reports a retry message to the control node in the first time, the control node can trigger the retry of the first task before the retry waiting time is exceeded, which can obviously trigger the retry of the first task more accurately and timely.

[0101] Of course, if the first data is not released or the remote data processing task is not the first task in the lock waiting queue corresponding to the first data, the execution node can continue to wait for the lock, and when the first data is released and the remote data processing task becomes the first task in the lock waiting queue corresponding to the first data, the execution node actively reports a retry message to the control node in the above manner.

[0102] In some embodiments, after receiving the retry message reported by the execution node, the control node can first remove the remote data processing task from the lock waiting queue corresponding to the first data maintained locally by the control node in response to the retry message, and then re-send the remote data processing task to the at least one execution node after successfully removing the remote data processing task from the lock waiting queue corresponding to the first data.

[0103] In some embodiments, the remote data processing task can specifically include a data processing task performed on a plurality of data maintained locally by each of the at least one execution node.

[0104] For example, a data processing task performed on a plurality of data rows maintained locally by the execution node; in this case, each of the at least one execution node can also maintain a plurality of lock waiting queues corresponding to the plurality of data locally.

[0105] In this case, when the execution node receives the remote data processing task re-sent by the control node again, the execution node can perform the remote data processing task again based on the plurality of data maintained locally.

[0106] If the execution node encounters a lock conflict on a second data during the execution of the remote data processing task again, the execution node can send lock conflict information corresponding to the second data to the control node; wherein the first data and the second data are different.

[0107] For example, the last time the remote data processing task was executed, a lock conflict occurred on the first data. After the control node re-issues the remote data processing task, the execution node encounters a lock conflict on the second data during the execution of the remote data processing task again.

[0108] After receiving the lock conflict information corresponding to the second data sent by the execution node, the control node can perform the same operation of adding the remote data processing task to the lock conflict queue corresponding to the second data maintained locally by the control node, and sending a lock waiting message corresponding to the second data to the execution node.

[0109] In this case, when the execution node receives the lock waiting message corresponding to the second data sent by the control node, on one hand, the remote data processing task can be added to the lock waiting queue corresponding to the second data maintained locally by the execution node; on the other hand, since the first data has been released at this time, and the first data does not conflict, the execution node can also remove the above-mentioned remote data processing task from the lock waiting queue corresponding to the above-mentioned first data maintained locally.

[0110] For example, in one example, the above-mentioned execution node can actively remove the above-mentioned remote data processing task from the lock waiting queue corresponding to the above-mentioned first data maintained locally.

[0111] In another example, the control node can send a lock waiting termination message corresponding to the first data to the execution node in response to the received lock conflict information corresponding to the second data sent by the execution node. And the execution node can remove the remote data processing task from the lock waiting queue corresponding to the above-mentioned first data maintained locally by the execution node after receiving the lock waiting termination message. In this way, the remote data processing task can be removed from the lock waiting queue corresponding to the first data in time, and thus the problem that other tasks that need to access the first data cannot access the first data all the time can be avoided.

[0112] Of course, when the above-mentioned execution node executes the remote data processing task re-sent by the control node when retrying the remote data processing task, there is no lock conflict and the remote data processing task is completed, the remote data processing task can be removed from the lock waiting queue corresponding to the first data maintained locally immediately, and the next task in the lock waiting queue corresponding to the first data can be woken up, and so on. In this way, the tasks in the queue can be executed in strict accordance with the order of the tasks in the lock waiting queue.

[0113] In some embodiments, if the first data has been released, and the remote data processing task is the first task in the lock waiting queue corresponding to the first data, the above-mentioned execution node can also add a retry waiting time length for the second task in the lock waiting queue (that is, the next task of the first task).

[0114] The specific length of the retry waiting time length can be flexibly set based on specific needs. For example, the retry waiting time length can be set based on the life cycle of the above-mentioned remote data processing task. For example, the retry waiting time length is set to a time length that is not greater than the life cycle of the above-mentioned remote data processing task.

[0115] Next, if the execution node sends a retry message corresponding to the remote data processing task to the control node, and if the remote data processing task re-sent by the control node is not received after the retry waiting duration (i.e., the retry waiting duration expires), the remote data processing task can be removed from the locally maintained lock waiting queue corresponding to the first data, and a retry message corresponding to the second task is sent to the control node to make the control node re-send the second task to the at least one execution node.

[0116] In this way, in the case that the remote data processing task does not respond for a long time, the next task waiting for the first data can be processed in time, so that the situation that the subsequent tasks cannot be executed in time due to long waiting for the remote data processing task can be avoided.

[0117] In some embodiments, if the execution node receives the remote data processing task re-sent by the control node within the retry waiting duration after sending the retry message to the control node, the retry waiting duration added for the second task can be removed. Thus, the waste of computing resources caused by redundant retries of the second task can be avoided, because the first data has lock conflict, and the retry of the second task cannot lock the first data.

[0118] Please refer to FIG. 3, which exemplarily shows the flow of the remote lock waiting management method applied to the control node, including steps S301 to S303.

[0119] In step S301, a remote data processing task is sent to at least one execution node to make the at least one execution node execute the remote data processing task; wherein the remote data processing task is a distributed task sent by the control node to the at least one execution node for remote execution;

[0120] In step S302, in response to the received lock conflict information corresponding to the first data sent by any execution node of the at least one execution node, the remote data processing task is added to the lock waiting queue corresponding to the first data, and a lock waiting message corresponding to the first data is sent to the execution node to make the execution node add the remote data processing task to the lock waiting queue corresponding to the first data maintained locally by the execution node; the lock conflict information is the lock conflict information sent by the execution node in the case that the remote data processing task has lock conflict on the first data;

[0121] In step S303, in response to the retry message corresponding to the remote data processing task sent by the execution node, the remote data processing task is re-sent to the at least one execution node; wherein the retry message is a retry message sent by the execution node to the control node in the case that the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data maintained locally by the execution node.

[0122] It should be noted that steps S301 to S303 are method processes with the control node as the execution subject. The implementation details involved in these processes are the same as those of the method processes of steps S201 to S203. For details, refer to the previous embodiments, which will not be described herein again.

[0123] In this embodiment, in addition to the manner of actively reporting lock conflict information by the execution node to the control node and the timing of task retry, a periodic active detection of the state of each execution node can be introduced on the control node to actively initiate the task retry mechanism.

[0124] In some embodiments, after the control node issues the remote data processing task to the at least one execution node, before receiving the lock conflict information and the retry message reported by any of the at least one execution node, the control node can periodically actively detect the state of the at least one execution node.

[0125] If any of the at least one execution node is detected to be in an abnormal state (such as a crash, network anomaly, etc.), the remote data processing task can be directly removed from the locally maintained lock waiting queue at this time, and after the remote data processing task is removed from the lock waiting queue, the remote data processing task is re-sent to the at least one execution node.

[0126] By combining the mechanism of actively reporting the timing of task retry by the execution node and the mechanism of actively initiating task retry by the control node by actively detecting the state of each execution node, the control node can efficiently perform remote lock waiting management on the remote data processing task, ensure that the task retry for the remote data processing task can be triggered accurately and timely, and improve the sensitivity of task retry for the remote data processing task.

[0127] For example, when the execution node executing the remote data processing task is in an abnormal state, the control node can avoid waiting for the remote data processing task for a long time, thereby causing the remote data processing task to have a large task processing delay.

[0128] After receiving the remote data processing task, the at least one execution node can execute the remote data processing task. On one hand, if the control node receives a lock conflict on a certain data reported by one of the execution nodes again (for example, a lock conflict reported by an execution node other than the execution node in the abnormal state), the above process can be repeated.

[0129] On the other hand, if the control node receives the execution result corresponding to the remote data processing task returned by each of the execution nodes other than the execution node in the abnormal state, and the state of the execution node in the abnormal state has not been restored, the control node can return a response message of task execution failure to the user who initiates the remote data processing task. Of course, if the state of the execution node in the abnormal state is restored, and the control node receives the execution result corresponding to the remote data processing task returned by all the execution nodes, the control node can return a response message of task execution success to the user who initiates the remote data processing task, and return the final execution result of the remote data processing task to the user.

[0130] FIG. 4 is a schematic structural diagram of an electronic device according to an example embodiment. Referring to FIG. 4, at the hardware level, the device includes a processor 402, an internal bus 404, a network interface 406, a memory 408, a non-volatile memory 410, and can further include other hardware required by tasks. One or more embodiments of the present specification can be implemented in a software manner, for example, by reading a corresponding computer program from the non-volatile memory 410 into the memory 408 by the processor 402 and then running. Of course, in addition to the software implementation manner, one or more embodiments of the present specification do not exclude other implementation manners, such as a logic device or a combination of software and hardware, and the like, that is, the execution subject of the following process flow is not limited to the logic unit, but can also be hardware or a logic device.

[0131] Referring to FIG. 5, FIG. 5 is a block diagram of a remote lock waiting management apparatus in a distributed database according to an example embodiment of the present specification. The apparatus can be applied to the electronic device shown in FIG. 4 to implement the technical solutions of the present specification. The apparatus can be applied to the electronic device shown in FIG. 4 to implement the technical solutions of the present specification. When the apparatus shown in FIG. 5 is an execution node, the apparatus can include:

[0132] An execution module 501, which executes a remote data processing task sent by a control node; wherein the remote data processing task is a distributed task sent by the control node to at least one execution node for remote execution;

[0133] The first sending module 502 sends lock conflict information corresponding to the first data to the control node if a lock conflict occurs on the first data for the remote data processing task;

[0134] The adding module 503 adds the remote data processing task to a lock waiting queue corresponding to the first data in response to the lock waiting message corresponding to the first data sent by the control node, wherein the lock waiting message is a lock waiting message sent by the control node to the execution node after the remote data processing task is added to the lock waiting queue corresponding to the first data maintained locally by the control node;

[0135] The second sending module 504 sends a retry message corresponding to the remote data processing task to the control node if the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data, so that the control node re-sends the remote data processing task to the at least one execution node.

[0136] Please refer to FIG. 6, which is a block diagram of a remote lock waiting management apparatus in another distributed database according to an exemplary embodiment of the present specification. The apparatus can also be applied to the electronic device shown in FIG. 4 to implement the technical solutions of the present specification. When the apparatus shown in FIG. 6 is a control node, the apparatus can include:

[0137] The third sending module 604 sends a remote data processing task to at least one execution node, so that the at least one execution node executes the remote data processing task, wherein the remote data processing task is a distributed task sent by a control node to at least one execution node for remote execution;

[0138] The fourth sending module 605 adds the remote data processing task to a lock waiting queue corresponding to the first data and sends a lock waiting message corresponding to the first data to the execution node in response to the lock conflict information corresponding to the first data sent by any execution node of the at least one execution node, so that the execution node adds the remote data processing task to the lock waiting queue corresponding to the first data maintained locally by the execution node, wherein the lock conflict information is lock conflict information sent by the execution node if a lock conflict occurs on the first data for the remote data processing task;

[0139] The retry module 6, 6 re-sends the remote data processing task to the at least one execution node in response to a retry message corresponding to the remote data processing task sent by the execution node; wherein the retry message is a retry message sent by the execution node to the control node in the case that the first data is released, and the remote data processing task is the first task in a lock waiting queue corresponding to the first data and maintained locally by the execution node.

[0140] One or more embodiments of the present specification also provide a computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method provided by any of the above embodiments.

[0141] One or more embodiments of the present specification also provide a computer readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method provided by any of the above embodiments.

[0142] According to one or more embodiments of the present specification, a distributed database is also provided, comprising:

[0143] The at least one execution node is configured to execute the method flow in steps S201 to S204.

[0144] The control node is configured to execute the method flow in steps S301 to S303.

[0145] The system, device, module or unit illustrated in the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0146] In a typical configuration, a computer includes one or more processors (CPU), input / output interface, network interface, and memory.

[0147] The memory can include non-persistent memory in computer readable medium, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer readable media.

[0148] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, disk storage, quantum memory, graphene-based storage medium or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0149] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also other elements not explicitly listed, or other elements inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0150] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0151] The terms used in one or more embodiments of the present specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present specification. The singular forms "a", "an" and "the" used in one or more embodiments of the present specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0152] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the specification are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0153] It should be understood that, although one or more embodiments of the specification can use the terms first, second, third, etc. to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the specification, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to determining".

[0154] The above only describes the preferred embodiments of one or more embodiments of the specification, and does not limit one or more embodiments of the specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of the specification shall be included in the protection scope of one or more embodiments of the specification.

Claims

1. A method for remote lock waiting management in a distributed database, the method comprising: executing a remote data processing task sent by a control node; wherein the remote data processing task is a distributed task sent by the control node to at least one execution node for remote execution; sending lock conflict information corresponding to a first data to the control node if a lock conflict occurs on the first data in the remote data processing task; adding the remote data processing task to a lock waiting queue corresponding to the first data in response to a lock waiting message corresponding to the first data sent by the control node; sending a retry message corresponding to the remote data processing task to the control node to make the control node resend the remote data processing task to the at least one execution node if the first data is released and the remote data processing task is the first task in the lock waiting queue corresponding to the first data. 2.The method of claim 1, further comprising: performing task rollback processing on the remote data processing task in response to a rollback message corresponding to the remote data processing task sent by the control node; wherein the rollback message is a rollback message sent by the control node to the at least one execution node after receiving the lock conflict information. 3.The method of claim 1, wherein the adding the remote data processing task to the lock waiting queue corresponding to the first data in response to the lock waiting message corresponding to the first data sent by the control node comprises: determining whether the first data has been released in response to the lock waiting message corresponding to the first data sent by the control node; further adding the remote data processing task to the lock waiting queue corresponding to the first data if the first data has not been released. 4.The method of claim 3, wherein the adding the remote data processing task to the lock waiting queue corresponding to the first data comprises: creating a fake task corresponding to the remote data processing task that has been rolled back; and adding the fake task to the lock waiting queue corresponding to the first data. 5.The method of claim 3, further comprising: sending the retry message to the control node to make the control node resend the remote data processing task to the at least one execution node if the first data has been released.

6. The method of claim 1, the remote data processing task being a data processing task performed on a plurality of data maintained for the execution node; the execution node maintains a plurality of lock waiting queues corresponding to the plurality of data one by one; the method further comprises: executing the remote data processing task resent by the control node; sending lock conflict information corresponding to a second data to the control node to make the control node add the remote data processing task to a lock waiting queue corresponding to the second data maintained by the control node locally if a lock conflict occurs on the second data in the process of executing the remote data processing task; wherein the first data and the second data are different. adding the remote data processing task to a lock waiting queue corresponding to the second data in response to a lock waiting message corresponding to the second data received from the control node; and removing the remote data processing task from the lock waiting queue corresponding to the first data.

7. The method of claim 6, wherein removing the remote data processing task from the lock waiting queue corresponding to the first data comprises: actively removing the remote data processing task from the lock waiting queue corresponding to the first data; or removing the remote data processing task from the lock waiting queue corresponding to the first data in response to a lock waiting termination message corresponding to the first data received from the control node.

8. The method of claim 1, further comprising: adding a retry waiting duration for a second task in the lock waiting queue corresponding to the first data if the first data is released and the remote data processing task is a first task in the lock waiting queue corresponding to the first data; removing the remote data processing task from the lock waiting queue corresponding to the first data and sending a retry message corresponding to the second task to the control node to make the control node resend the second task to at least one execution node if the remote data processing task is not received from the control node within the retry waiting duration after the retry message is sent to the control node.

9. The method of claim 8, further comprising: removing the retry waiting duration added for the second task if the remote data processing task is received from the control node within the retry waiting duration after the retry message is sent to the control node.

10. A remote lock waiting management method, the method comprising: sending a remote data processing task to at least one execution node to make the at least one execution node execute the remote data processing task; wherein the remote data processing task is a distributed task sent by a control node to at least one execution node for remote execution; adding the remote data processing task to a lock waiting queue corresponding to first data in response to lock conflict information corresponding to the first data received from any of the at least one execution node and sending a lock waiting message corresponding to the first data to the execution node to make the execution node add the remote data processing task to a lock waiting queue corresponding to the first data maintained locally by the execution node; resending the remote data processing task to the at least one execution node in response to a retry message corresponding to the remote data processing task sent by the execution node; wherein the retry message is a retry message sent by the execution node to the control node in a case that the first data is released and the remote data processing task is a first task in a lock waiting queue corresponding to the first data maintained locally by the execution node. 11.The method of claim 10, further comprising: in response to the received lock conflict information corresponding to the first data sent by any of the at least one execution node, sending a rollback message to the at least one execution node to cause the at least one execution node to perform task rollback processing on the remote data processing task. 12.The method of claim 11, in response to the received retry message corresponding to the remote data processing task sent by the execution node, re-sending the remote data processing task to the at least one execution node, comprising: in response to the received retry message corresponding to the remote data processing task sent by the execution node, removing the remote data processing task from the lock waiting queue corresponding to the first data, and after removing the remote data processing task from the lock waiting queue corresponding to the first data, re-sending the remote data processing task to the at least one execution node.

13. The method of claim 12, the remote data processing task being a data processing task performed on a plurality of data maintained for the execution node; the control node maintains a plurality of lock waiting queues corresponding to the plurality of data one by one; the method further comprises: after re-sending the remote data processing task to the at least one execution node, in response to the received lock conflict information corresponding to the second data sent by any of the at least one execution node, adding the remote data processing task to the lock waiting queue corresponding to the second data; sending a lock waiting message corresponding to the second data to the execution node to cause the execution node to add the remote data processing task to the lock waiting queue corresponding to the second data maintained locally by the execution node. 14.The method of claim 13, further comprising: in response to the received lock conflict information corresponding to the second data sent by any of the at least one execution node, sending a lock waiting termination message corresponding to the first data to the execution node to cause the execution node to remove the remote data processing task from the lock waiting queue corresponding to the first data maintained locally by the execution node. 15.The method of claim 14, further comprising: periodically probing the state of the at least one execution node; if any of the at least one execution node is detected to be in an abnormal state, removing the remote data processing task from the lock waiting queue, and after removing the remote data processing task from the lock waiting queue, re-sending the remote data processing task to the at least one execution node. 16.A computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 15. 17.An electronic device comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the method of any one of claims 1 to 15 by running the executable instructions.

18. A computer readable storage medium having stored thereon computer instructions, the instructions, when executed by a processor, implementing the steps of the method of any one of claims 1 to 15.

19. A distributed database comprising: at least one execution node, the execution node being configured to perform the method of any one of claims 1 to 9; a control node, the control node being configured to perform the method of any one of claims 10 to 15.

Images