Distributed database system, request processing method, device, storage medium and program product

Abstract

The embodiment of the invention provides a distributed database system, a request processing method, computing equipment, a computer storage medium and a computer program product. The distributed database system comprises a master node, at least one slave node and a server, the target slave node in the at least one node is used for receiving a first transaction started by a user; forwarding the write operation request in the first transaction to the main node; the target slave node is any slave node; the master node is used for processing the write operation request in the proxy transaction in response to the write operation request forwarded by the target slave node so as to generate a pre-write log; sending the pre-writing log to a server; and the server is used for updating the corresponding database data based on the pre-write log and providing the database data for the master node and the at least one slave node. According to the technical scheme provided by the embodiment of the invention, any node in the distributed database system can respond to the write operation request of the client.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of database, and particularly relate to a distributed database system, a request processing method, a computing device, a computer storage medium and a computer program product. BACKGROUND

[0002] With the development of computer technology, in order to support massive computing load, a distributed database system emerges as the times require.

[0003] In a current distributed database system architecture, mainly composed of a master node (Master Node) and at least one slave node (Slave Node), in order to ensure the data consistency of the master node and the at least one slave node, the master node is usually used as a read-write node, that is, the master node can process read data requests and write data requests, and the slave node is used as a read-only node and can only process read data requests.

[0004] Therefore, the user needs to explicitly distinguish the nodes connected by the write operation request and the read operation request, and the operation is not convenient enough. SUMMARY

[0005] Embodiments of the present application provide a distributed database system, method, device, computing device, computer storage medium and computer program product.

[0006] In a first aspect, a distributed database system is provided in embodiments of the present application, comprising a master node, at least one slave node and a server;

[0007] The target slave node in the at least one node is configured to receive a first transaction opened by a user, and forward a write operation request in the first transaction to the master node; the target slave node is any slave node;

[0008] The master node is configured to, in response to the write operation request forwarded by the target slave node, process the write operation request in a proxy transaction to generate a prewrite log, and send the prewrite log to the server;

[0009] The server is configured to update corresponding database data based on the prewrite log, and provide the database data to the master node and the at least one slave node.

[0010] In a second aspect, a request processing method is provided in embodiments of the present application, applied to a master node in a distributed database system, the distributed database system further comprising at least one slave node and a server, and the method comprises:

[0011] Receiving a write operation request forwarded by a target slave node;

[0012] handling the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request;

[0013] sending the prewrite log to the server; the prewrite log is used to update corresponding database data; a log sequence number of the prewrite log is used to be sent to the at least one slave node to obtain database data after update.

[0014] In a third aspect, an embodiment of the present application provides a request processing method, applied to a slave node in a distributed database system, the distributed database system further comprising a master node and a server, and the method comprises:

[0015] obtaining a first transaction opened by a user;

[0016] forwarding a write operation request in the first transaction to the master node, so that the master node handles the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request, and sends the prewrite log to the server; the prewrite log is used to update corresponding database data;

[0017] receiving a log sequence number of the prewrite log sent by the server; the log sequence number is used to obtain database data after update.

[0018] In a fourth aspect, an embodiment of the present application provides a request processing method, applied to a server in a distributed database system, the distributed database system further comprising a master node and at least one slave node, and the method comprises:

[0019] receiving a prewrite log sent by the master node, the prewrite log being generated by the master node by handling a write operation request in a proxy transaction, the write operation request being forwarded to the master node by a target slave node;

[0020] updating corresponding database data by using the prewrite log;

[0021] sending a log sequence number of the prewrite log to the at least one slave node, so that the at least one slave node obtains database data after update based on the log sequence number.

[0022] In a fifth aspect, an embodiment of the present application provides a request processing apparatus, applied to a master node in a distributed database system, the distributed database system further comprising at least one slave node and a server, and the apparatus comprises:

[0023] a first receiving module, configured to receive a write operation request forwarded by a target slave node;

[0024] The request processing module is configured to process the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request.

[0025] The log sending module is configured to send the prewrite log to the server, where the prewrite log is used to update corresponding database data, and a log sequence number of the prewrite log is used to be sent to the at least one slave node to obtain the database data after the update.

[0026] In a sixth aspect, an embodiment of the present application provides a request processing apparatus applied to a slave node in a distributed database system, the distributed database system further comprising a master node and a server, and the apparatus comprises:

[0027] The transaction receiving module is configured to obtain a first transaction opened by a user.

[0028] The request sending module is configured to forward a write operation request in the first transaction to the master node, so that the master node processes the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request, and sends the prewrite log to the server, where the prewrite log is used to update corresponding database data.

[0029] The information receiving module is configured to receive a log sequence number of the prewrite log sent by the server, where the log sequence number is used to obtain the database data after the update.

[0030] In a seventh aspect, an embodiment of the present application provides a request processing apparatus applied to a server in a distributed database system, the distributed database system further comprising a master node and at least one slave node, and the apparatus comprises:

[0031] The log receiving module is configured to receive a prewrite log sent by the master node, where the prewrite log is generated by the master node by processing a write operation request in a proxy transaction, and the write operation request is forwarded to the master node by a target slave node.

[0032] The data updating module is configured to update corresponding database data by using the prewrite log.

[0033] The information sending module is configured to send a log sequence number of the prewrite log to the at least one slave node, so that the at least one slave node obtains the database data after the update based on the log sequence number.

[0034] In an eighth aspect, an embodiment of the present application provides a computing device comprising a processing component and a storage component.

[0035] The storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component, and implement the request processing method provided in the embodiments of the present application.

[0036] In a ninth aspect, the embodiments of the present application provide a computer storage medium, which stores a computer program. When the computer program is executed by a computer, the request processing method provided in the embodiments of the present application is implemented.

[0037] In a tenth aspect, the embodiments of the present application provide a computer program product, which includes computer program code. When the computer program code is executed by a computer, the request processing method provided in the embodiments of the present application is implemented.

[0038] In the embodiments of the present application, a user can connect any one node to request to start a transaction. If the connected node is any target slave node, the target slave node can forward the write operation request in the first transaction started by the user to the master node, and the master node executes the write operation request in the proxy transaction, so that from the perspective of the user, any node in the distributed database system can respond to the write operation request, without the user distinguishing, and the operation is more convenient. However, actually all the write operation requests are executed by the master node, and the prewrite log generated by the master node in processing the write operation request is sent to the server. The server updates the corresponding database data according to the prewrite log, and then distributes the updated database data to the master node and at least one slave node, thereby avoiding the multiple write conflicts of multiple nodes, ensuring that the node executing the write operation request has a unified transaction view, and thus ensuring the data consistency between multiple nodes.

[0039] These aspects or other aspects of the present application will be more apparent in the following description of the embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort.

[0041] Figure 1 A schematic diagram of a distributed database system provided by an embodiment of the present application is schematically shown;

[0042] Figure 2 An interaction schematic diagram in a distributed database system provided by another embodiment of the present application is schematically shown;

[0043] Figure 3A schematic diagram of a distributed database system according to another embodiment of the present application is shown;

[0044] Figure 4 A flowchart of a request processing method according to an embodiment of the present application is shown;

[0045] Figure 5 A flowchart of a request processing method according to another embodiment of the present application is shown;

[0046] Figure 6 A flowchart of a request processing method according to another embodiment of the present application is shown;

[0047] Figure 7 A block diagram of a computing device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0048] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application.

[0049] In some of the processes described in the specification and claims of the present application and the accompanying drawings, multiple operations are described in a specific order. However, it should be clear that these operations can be performed in another order or in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and the operations can be performed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this paper are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do "first" and "second" represent different types.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0051] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0052] With the development of computer technology, in order to support massive computing load, a distributed database system emerges as the times require.

[0053] In a current distributed database system architecture, mainly composed of a master node (Master Node) and at least one slave node (Slave Node), in order to ensure the data consistency of the master node and the at least one slave node, the master node is usually used as a read-write node, that is, the master node can process read data requests and write data requests, and the slave node is used as a read-only node and can only process read data requests. In another distributed database system architecture, also composed of a master node and at least one slave node, users can only connect to the master node, and the slave node is used to expand the computing performance from the distributed level, and users cannot connect to the slave node.

[0054] The inventor found that in the above-mentioned first distributed database system, on the one hand, since all write operations need to be executed through the master node, if the concurrent write operations are too many, the performance of the master node may be affected due to the limited processing capacity of the master node, thereby causing the overall write performance of the system to decrease. On the other hand, although the slave node can process read operations, the data update of the slave node depends on the write operation of the master node, so the update of the master node needs to be copied to the slave node, and this process often has a delay. If the delay is too large, the data on the slave node may not be consistent with the data on the master node. In addition, if the master node fails, the write operation of the entire system will be unable to be executed, resulting in the system being unavailable. Thirdly, users need to explicitly distinguish the nodes connected by write operation requests and read operation requests, which is not convenient for operation. In the above-mentioned second distributed database system, since users can only be connected to the master node, the performance of the master node will be affected, so the number of connections to the master node is limited, and therefore the horizontal expansion is limited.

[0055] In order to solve the above-mentioned technical problems, the inventor realizes a distributed database system based on an optimistic lock, in which any node can execute both read operations and write operations.

[0056] In the optimistic lock-based distributed database system, a write operation received by any node is first attempted to be locally executed, and after the local execution, the write operation is attempted to be submitted to a central management node of the distributed database system through an optimistic lock conflict. In the submission process of the write operation, if the optimistic lock conflict causes the submission to fail, an internal retry without user awareness is triggered, that is, the write operation of the user is not rejected or reported as an error in the process of the retry until the submission is successful.

[0057] After the write operation is successfully submitted, the remaining nodes can synchronize the submitted write operation from the central management node, and replay the obtained write operation locally.

[0058] The inventors have found in the implementation of the concept of the present application that in the above-described optimistic lock-based distributed database system, since each node can simultaneously process locally submitted write operations and replay operations, multiple nodes can simultaneously attempt to modify the same resource, and such multi-write conflict can cause data inconsistency and trigger logical conflict. The logical conflict refers to that all nodes should obtain the same result when executing the same transaction. However, due to the multi-write conflict, the processing order of each node when processing the same transaction can be different, thereby causing logical inconsistency.

[0059] In order to ensure data consistency between multiple nodes while any node of the distributed database system can process write operations, the present application embodiment provides a distributed database system, Figure 1 The present application provides a distributed database system, and a schematic diagram of an embodiment of the distributed database system is shown in Figure 1 As shown in the figure, the distributed database system can include a master node 101, at least one slave node 102, and a server 103.

[0060] A target slave node 104 in the at least one node 102 is configured to receive a first transaction opened by a user, and forward a write operation request in the first transaction to the master node 101.

[0061] The master node 101 is configured to, in response to the write operation request forwarded by the target slave node 104, process the write operation request in a proxy transaction to generate a prewrite log, and send the prewrite log to the server 103.

[0062] The server 103 is configured to update corresponding database data based on the prewrite log, and provide the database data to the master node 101 and the at least one slave node 102.

[0063] The target slave node 104 can be any one of the at least one slave node 102 of the distributed database system, and is named as the target slave node and labeled as 104 for ease of description.

[0064] In the embodiments of the present application, the master node and the at least one slave node can be connected to the user corresponding client, that is, any one of the slave nodes in the distributed database system can receive the transaction opened by the user request, and forward the write operation request in the transaction to the master node. The first transaction can be sent by the user to the target slave node through the client.

[0065] In the embodiments of the present application, the first transaction can be an explicit transaction or an implicit transaction. The implicit transaction can be any one of the database operation requests (write operation request or read operation request) sent by the user, and the system can automatically create a transaction for any one of the query requests sent by the user. The explicit transaction can be a transaction block, and the transaction boundary is manually defined by the user, which allows more fine-grained control and error handling. The user can use specific naming such as BEGIN TRANSACTION in SQL (Structured Query Language) to open a transaction, use specific command such as COMMIT in SQL to submit the transaction, or use specific command such as ROLLBACK to roll back the transaction. The explicit transaction can include multiple database operation requests (write operation request or read operation request). The write operation request can be any one of the write operation requests involved in the multiple database operation requests.

[0066] The target slave node can forward the write operation request in the first transaction to the master node. The read operation request in the first transaction can be executed locally by the target slave node.

[0067] The write operation request can be an operation request for updating the database data, such as insertion, modification or deletion, etc. The database request included in the first transaction can be a SQL statement, and the write operation request can be, for example, an INSERT statement, an UPDATE statement, or a DELETE statement, and the read operation request can be, for example, a Select statement, etc. The write operation request or the read operation request can be distinguished by different types of SQL statements.

[0068] In addition, the write operation request can be, for example, a DDL (Data Definition Language) statement, a DML (Data Manipulation Language) statement, a DCL (Data Control Language) statement, etc. SQL statement to modify the database or data table or data record, etc.

[0069] Write-Ahead Logging (WAL) is a widely used technique in database systems to ensure atomicity, consistency, and durability of transaction processing. When database data is updated, these updates are first recorded in a special log, which is the pre-write log, rather than being directly written to the database data. The pre-write log records the specific content of the update, such as the data location being updated and the data values before and after the update. Therefore, in the embodiments of the present application, after the master node receives the write operation request forwarded by the target slave node, the processing operation of the write operation request can be to generate the pre-write log corresponding to the write operation request, and the pre-write log is used to record the update operation of the write operation request on the database data. In practical applications, the distributed database system can organize the database data into data pages, and each data page can contain one or more data records. The pre-write log can record the update operation on the data page.

[0070] After generating the pre-write log, the master node can send the pre-write log to the server, and the server can update the corresponding database data based on the pre-write log after receiving the pre-write log. For example, the server can apply the operation record recorded in the pre-write log to the database data to perform the corresponding update on the database data. When the database data is organized into data pages, the server updates the corresponding data pages.

[0071] The pre-write log can include, for example, transaction identification (Transaction ID), operation type (Operation Type), affected data (Affected Data), old value and new value (Old Value and New Value), and other information, where the transaction identification is used to identify the first transaction, the operation type is used to describe the specific operation type, such as insertion, modification, or deletion, the affected data can refer to the specific data or data row identification affected by the write operation, and the old value and new value (Old Value and New Value) can refer to the old value and new value of the updated field for the update operation, so that the uncompleted transaction can be restored when the transaction is rolled back, to ensure data consistency.

[0072] The server can update the corresponding database data based on the above information recorded in the pre-write log.

[0073] The server can be a copy of the database data in the cache persistent storage medium, such as a data page copy, etc. After receiving the pre-write log, the server can update the cache data, and provide the database data after the update to the master node and at least one slave node. The server can decide whether to materialize the cache data after the update to the persistent storage medium according to actual application, which is not limited in the present application.

[0074] The server can send the database data after the update to the master node or at least one slave node when needed. However, it is not limited thereto, and the server can send the database data after the update to the master node or at least one slave node, which will be described in detail in the following embodiments.

[0075] In the embodiments of the present application, a user can connect any one node to request to start a transaction. If the connected node is any target slave node, the target slave node can forward the write operation request in the first transaction started by the user to the master node, and the master node executes the write operation request in the proxy transaction, so that any node in the distributed database system can respond to the write operation request from the user's perspective, without the user's distinction, and the operation is more convenient. However, all write operation requests are executed by the master node, and the pre-write log generated by the master node in processing the write operation request is sent to the server. The server updates the corresponding database data according to the pre-write log, and then distributes the database data after the update to the master node and at least one slave node, thereby avoiding the multi-write conflict of multiple nodes, ensuring that the node executing the write operation request has a unified transaction view, and ensuring the data consistency between multiple nodes.

[0076] In actual application, the distributed database system provided by the embodiments of the present application can be a database product realized by distributed expansion of PostgreSQL (an open source object-relational database management system). However, the present application is not limited thereto.

[0077] In some embodiments, the server is further configured to synchronize the log sequence number of the pre-write log to the at least one slave node.

[0078] The slave node is configured to obtain the corresponding database data from the server according to the log sequence number.

[0079] In some embodiments, the master node is further configured to generate a response request according to the log sequence number of the pre-write log, and send the response request to the target slave node.

[0080] The target slave node is further configured to obtain the corresponding database data from the server according to the log sequence number.

[0081] Each pre-write log is assigned a unique log sequence number, which is used to uniquely identify the pre-write log and ensure consistency of the pre-write log and the data.

[0082] When the server applies the database operation to the data page based on the pre-write log, the log sequence number on the data page is also updated to reflect the latest update operation, at which time the log sequence number can be used to identify the version of the data page. Thus, the unique correspondence between the pre-write log and the log sequence number will be transformed into a unique correspondence between the log sequence number and the data page version, and the corresponding data page after the update can be queried through the log sequence number.

[0083] For the target slave node, i.e., the sender of the write operation request, the master node can write the log sequence number into the response request after generating the pre-write log and assigning the log sequence number to it, and send the response request to the target slave node, so that the target slave node can obtain the log sequence number from the response request. The target slave node can obtain the updated database data from the server based on the log sequence number when needed after obtaining the log sequence number.

[0084] The server can synchronize the log sequence number of the pre-write log to at least one slave node. In one possible implementation, the server can synchronize the pre-write log to each slave node of the distributed database system, so that each slave node can obtain the updated database data from the server based on the log sequence number.

[0085] In another implementation, as described above, the master node can write the log sequence number into the response request after generating the pre-write log, and send the response request to the target slave node, so that the server can synchronize the log sequence number of the pre-write log to the slave nodes other than the target slave node when synchronizing the log sequence number of the pre-write log to the slave nodes.

[0086] In some embodiments, the master node is further configured to obtain a lock resource of a data object to which the write operation request is directed; process the write operation request to generate a pre-write log corresponding to the write operation request; and release the lock resource of the data object in the case of proxy transaction submission.

[0087] The master node can first obtain a lock resource of a data object to which the write operation request is directed after receiving the write operation request forwarded by the target slave node. The lock resource is obtained to ensure that other transactions cannot concurrently modify the data object during the execution of the write operation, thereby ensuring data consistency and atomicity of the operation.

[0088] In embodiments of the present application, the master node can maintain lock resource information of different data objects, which can include, for example, lock status, waiting queue, current holder of the lock, lock type, etc.

[0089] After obtaining the lock resource, the master node can process the write operation request and generate a corresponding pre-written log.

[0090] After the write operation request is processed and the corresponding pre-written log is generated, the master node needs to perform a transaction commit operation to make the server persistently store the pre-written log and apply the modification operation to the data page based on the pre-written log. After the transaction is committed, the master node releases the lock resource obtained previously, and the release of the lock resource allows other transactions to continue to access and modify these data objects. Through the above pessimistic lock mechanism, the master node can arbitrate and sequence transactions, where arbitration can refer to determining by the master node which write operation request can be executed first to avoid conflicts between write operation requests, and sequencing can refer to globally ordering transactions to ensure that transactions are executed in order, maintaining data consistency.

[0091] In some embodiments, if the lock resource is already occupied by other transactions, the master node cannot immediately obtain the lock resource and needs to enter a waiting state. The master node can place the write operation request in the waiting queue of the lock resource, which can be arranged according to the first-in-first-out principle. During the waiting period, the master node can also periodically check whether the lock resource is released until the lock resource is obtained.

[0092] During the waiting process, the master node also needs to detect possible deadlock situations, which can refer to a situation where two or more transactions wait for each other to hold a lock resource, thereby causing the transactions to be unable to continue execution. The master node can detect and handle deadlocks through, for example, a timeout mechanism, waiting graph analysis, etc. Once a deadlock is detected, one or more of the tasks can be selected to break the deadlock.

[0093] In some embodiments, the target slave node is also configured to obtain lock status of different data objects from the master node.

[0094] The target slave node forwarding the write operation request in the first transaction to the master node includes: according to the lock status corresponding to the data object involved in the write operation request, forwarding the write operation request to the master node in the case that the data object is not locked.

[0095] The target slave node can periodically or on-demand obtain the lock status of different data objects from the master node. The lock status can include, for example, information such as whether the data object is currently locked, the type of the lock, the holder of the lock, etc.

[0096] The lock status can be saved in the cache of the target slave node to facilitate quick access to the lock status.

[0097] The target slave node can first check the lock state of the data object involved in the write operation request after receiving the write operation request, and decide whether to forward the write operation request to the master node according to the lock state. For example, if the target slave node detects that the data object is not locked, the write operation request can be forwarded to the master node; if the target slave node detects that the data object is locked, the target slave node can wait for the release of the lock until the target slave node detects that the lock is released, and then forwards the write operation request to the master node.

[0098] By first determining the lock state of the data object corresponding to the write operation request before the target slave node forwards the write operation request to the master node, the target slave node can effectively decide whether to forward the write operation request to the master node to ensure that the write operation request is performed when the data object is not locked, thereby improving the concurrency and performance of the distributed database system.

[0099] In an embodiment of the present application, a unified lock conflict view can be implemented in the master node, and through the lock conflict view, it can be ensured that all nodes in the distributed database system have consistent understanding of the lock state and conflict information, avoiding concurrency conflicts and maintaining consistency.

[0100] In some embodiments, the target slave node is further configured to acquire, according to the read operation request in the first transaction, a lock resource of a data object to which the read operation request is directed; and process the read operation request to perform a corresponding data reading operation.

[0101] Optionally, the target slave node can also send the lock state of the data object to the master node.

[0102] For the read operation request in the first transaction, the target slave node can acquire a lock resource of a data object to which the read operation request is directed.

[0103] After successfully acquiring the lock resource of the data object, the target slave node can perform a corresponding data reading operation to read the data object.

[0104] After processing the read operation request, the target slave node can send the lock state of the data object to the master node. In this way, the master node can update the lock state of the data object, facilitating the coordination of processing database operation requests in the master node, and ensuring the consistency and concurrency of the distributed database system.

[0105] In some embodiments, sending the lock state of the data object to the master node can be implemented by determining whether to send the lock state to the master node according to the processing requirement of the target slave node for the transaction.

[0106] In a possible implementation of the present application, the target slave node can determine whether to send the lock state of the data object to the master node according to different processing requirements of the transaction. For example, if the slave node needs to ensure that the data state is consistent with the master node when reading data, the read lock can be propagated to the master node, and for another example, propagating the read lock to the master node can increase the delay and load of the distributed database system, especially in the case of high concurrency or large data volume, in order to improve the performance of the system, the slave node can choose to maintain the read lock locally.

[0107] In some embodiments, the target slave node is further configured to, in response to the first transaction, send a transaction proxy request to the master node in a case where the first transaction involves a write operation request.

[0108] The master node is further configured to receive the transaction proxy request sent by the target slave node, and in response to the transaction proxy request, create a proxy transaction.

[0109] For the write operation request in the first transaction, the master node can be requested to create a proxy transaction by sending a transaction proxy request to the master node, and the write operation request can be processed by using the proxy transaction.

[0110] The proxy transaction can refer to a node, which in the embodiments of the present application can refer to the target slave node, and when receiving a transaction request, the specific operation of the transaction can be delegated to another node, which in the embodiments of the present application can refer to the master node, for execution.

[0111] The target slave node can send a transaction proxy request to the master node in a case where the first transaction involves a write operation request, and the master node can execute the write operation request involved in the first transaction by creating a corresponding proxy transaction. The target slave node can save the mapping relationship between the first transaction and the corresponding proxy transaction, so that the write operation request in the first transaction can be forwarded to the corresponding proxy transaction in the master node.

[0112] In some embodiments, after the target slave node sends the transaction proxy request to the master node, the target slave node is further configured to send a heartbeat message to the master node.

[0113] The master node is further configured to, based on the heartbeat information, determine that the target slave node is in an abnormal state, and roll back the proxy transaction.

[0114] The heartbeat information can be sent periodically to achieve the purpose of periodic lease renewal to the master node, so as to extend the transaction life cycle, and indirectly extend the life cycle of the held lock. The master node can determine that the target slave node is in an abnormal state in a case where the heartbeat message is not received within a predetermined time. In addition, the heartbeat message can further include a user connection state. For example, the master node can determine that the target slave node is in an abnormal state in a case where the corresponding user connection is disconnected according to the user connection state.

[0115] In practical applications, the corresponding operations can be performed by the corresponding processes created within the nodes. Figure 2 The illustration shows an interactive diagram in a distributed database system provided in an embodiment of this application.

[0116] like Figure 2 As shown, a user can connect to any node through client 200. Assuming that client 200 connects to target slave node 104, target slave node 104 may include database master process (PostMaster) 2011 and child process created by database master process 2011: MetaTunnel process 2012. Master node 101 may include database master process (PostMaster) 2021 and child process created by database master process 2021: MetaHub process 2022.

[0117] The database master process 2011 of the target node 104 can receive the first transaction initiated by the user. The database master process 2011 can establish a session connection with the meta tunnel process 2012 through the connection process, so that the meta tunnel process 2012 can establish a connection with the database master process 2021 of the master node 101, and can send transaction proxy requests and forward write operation requests in the first transaction to the meta hub process 2022 in the master node 101.

[0118] Furthermore, the meta-tunnel 2012 can also be used to send heartbeat information to the master node 101 to report the health status of the database master process 2011, and to periodically renew the connection lease between the target slave node 104 and the master node 101, thereby extending the lifecycle of the transaction processing proxies by the master node 101 on behalf of the target slave node 104, and also indirectly renewing the lifecycle of the locks held by the lock. In addition, it can also report the transaction status of the local session connection to the master node 101.

[0119] The meta-hub process 2022 can receive transaction proxy requests sent from the meta-channel process 2012 of the target slave node 104. It can establish a connection session with the database master process 2021, perform arbitration, ordering, and rate limiting, create proxy transactions, and execute write operations within those transactions, as well as commit and rollback them. Arbitration refers to the meta-hub process 2022 determining which write operation request can be executed first to avoid conflicts between them. Ordering refers to globally sorting transactions to ensure they are executed in sequence. Rate limiting refers to controlling the system's write operation request processing rate.

[0120] The meta-hub process 2022 is also used to generate a response request to the write operation request of the target slave node 104, and to write the log sequence number of the write-ahead log corresponding to the write operation request into the response request.

[0121] The meta hub process 2022 is also used to clean up all inactive transactions and transactions left over from node failures.

[0122] All slave nodes registered with the proxy transaction need to periodically report a heartbeat to the meta hub process through the meta channel process, so as to automatically roll back the related garbage transactions when the lease expires.

[0123] It should be noted that, Figure 2 Taking a slave node (target slave node) as an example, the interaction process with the master node is described, and it can be understood that the same operation can be performed between each slave node and the master node to realize forwarding the write operation request in the transaction received by each slave node to the master node for processing.

[0124] In some embodiments, the target slave node is also configured to send a transaction commit request to the master node in response to the commit operation of the first transaction; and send a transaction rollback request to the master node in response to the rollback operation of the first transaction.

[0125] The master node is also configured to commit the proxy transaction according to the transaction commit request; and roll back the proxy transaction according to the transaction rollback request of the slave node.

[0126] By coordinating and executing the commit and rollback operations of the transaction by the master node, the transaction consistency and data integrity in the distributed database system can be ensured.

[0127] In some embodiments, the master node is also configured to, in response to a read operation request in a second transaction initiated by a user, obtain an updated data page from the server according to a latest log sequence number of the data page accessed by the request; in response to a write operation request in the second transaction, generate a prewritten log corresponding to the write operation request; and send the prewritten log to the server. The server can also update the corresponding database data based on the prewritten log of the master node.

[0128] When the user initiates a read operation request in the second transaction, the master node can obtain the latest data page from the server according to the latest log sequence number of the data page accessed by the request, so that the read operation request accesses the latest committed data.

[0129] When the user initiates a write operation request in the second transaction, the master node can send the generated prewritten log to the server after generating the prewritten log. After receiving the prewritten log, the server can apply the prewritten log to the local storage to maintain the consistency and integrity of the data.

[0130] In some embodiments, the server can include a data service node and a log service node.

[0131] The master node is specifically configured to send the prewrite log to the log service node; the prewrite log can be generated by the master node in a second transaction or a proxy transaction.

[0132] The log service node is configured to provide a log sequence number of the prewrite log to at least one slave node and provide the prewrite log to a data service node.

[0133] The data service node is configured to perform a replay operation based on the prewrite log to update corresponding database data.

[0134] In an implementation of the present application, for a read operation request in the first transaction, the target slave node can obtain the data page corresponding to the read operation request from the data service node according to the obtained log sequence number. Therefore, in some embodiments, the target slave node is further configured to obtain the corresponding data page from the data service node and cache the data page according to the latest log sequence number obtained from the log service node; and find the corresponding data page from the local cache according to the read operation request in the first transaction.

[0135] In another implementation of the present application, the target slave node can pre-cache the data page according to the read operation request based on the log sequence number, and then find the corresponding data page from the local cache according to the read operation request in the first transaction and return the data page to the client. Therefore, in some embodiments, the target slave node is further configured to obtain the data page updated after the latest log sequence number according to the read operation request in the first transaction.

[0136] In an embodiment of the present application, the master node is further configured to: in response to a read operation request in a second transaction initiated by a user, obtain the data page updated after the latest log sequence number from the data service node.

[0137] In another embodiment of the present application, the master node is further configured to: obtain the corresponding data page from the data service node according to the latest log sequence number and cache the data page; and find the corresponding data page from the local cache in response to a read operation request in a second transaction initiated by a user.

[0138] In an embodiment of the present application, the master node can also locally process the read operation request in the second transaction. When the user initiates the read operation request in the second transaction, the master node can first find the corresponding data page from the local cache. If the data page exists in the local cache, the cached data page can be directly returned to avoid repeated access to the data service node. If the data page does not exist in the cache, the master node can obtain the data page from the data service node according to the latest log sequence number and cache the data page, and then return the data page to the user.

[0139] Figure 3A schematic diagram of a distributed database system according to another embodiment of the present application is shown.

[0140] As shown in Figure 3 , the distributed database system includes a master node 101, at least one slave node 102, a log service node 301, and a data service node 302.

[0141] In the distributed database system according to the embodiments of the present application, one master node 101 and a plurality of slave nodes 102 supporting horizontal expansion can be included.

[0142] A user can establish a user connection with any node through a client, i.e., a transaction can be started through the user connection. Assuming that a user connection is established with any slave node, the slave node 102 can receive a first transaction sent by the client and forward a write operation request in the first transaction to the master node 101. Assuming that a user connection is established with the master node 101, the master node 101 can receive a second transaction sent by the client.

[0143] The slave node 102 can only perform locally after receiving the first transaction, if the first transaction does not involve a write operation request, so as to fully enjoy the advantages of horizontal expansion of the distributed cluster without worrying about excessive performance burden on a single point in the cluster. If the first transaction involves a write operation request, the slave node 102 can send a transaction proxy request to the master node 101 to request to start a remote proxy transaction and forward the write operation request to the master node 101. The master node 101 can allocate a globally unified transaction ID for the proxy transaction and generate a prewrite log by processing the write operation request, if the proxy transaction is created successfully. The master node 101 generates a response request according to a log sequence number required by the slave node 102 to be synchronized and sends the response request to the slave node 102. The slave node 102 can synchronize the log sequence number in the response request to a local log sequence number. In addition, the master node 101 can synchronize lock states of different data objects to the slave node 102, so that the nodes in the system have a unified lock view.

[0144] The slave node 102 can continue to forward the write operation request of the first transaction to the master node 101 and continue to synchronize the log sequence number fed back by the master node 101.

[0145] In addition, the slave node 102 can send a transaction commit request or a transaction rollback request to the master node 101 according to a commit operation or a rollback operation of the first transaction, and the master node 101 can commit or rollback the proxy transaction. The master node 101 can be responsible for write operation requests of the entire distributed database system.

[0146] Referring to Figure 2, the client can request the meta channel process of the slave node 102 to start a remote transaction session, and then the meta channel process can initiate a transaction proxy request to the meta hub process of the master node 101. The meta hub process can create a proxy transaction, process a write operation request, generate a write-ahead log corresponding to the write operation request, and return a response request containing a log sequence number of the write-ahead log to the slave node 102.

[0147] In addition, the write-ahead log generated by the master node 101 can be sent to the log service node 301, and the slave node 102 can also subscribe to the write-ahead log from the log service node 301, for example, can obtain the latest log sequence number from the log service node 301 through stream replication, to synchronize the log sequence number in the master node 101.

[0148] The log service node 301 can send the write-ahead log to the data service node 302.

[0149] The data service node 302 can perform an actual data page update operation based on the received write-ahead log to update the locally cached data page, at which time the log sequence number of the write-ahead log can be used to identify the data page after the update, to distinguish different versions of the data page. When the slave node 102 performs transaction submission, a corresponding submission log can also be generated, and the data service node 302 can end the update operation according to the submission log.

[0150] The master node 101 and the slave node 102 can obtain the corresponding version data page from the data service node 302 according to the locally cached latest log sequence number.

[0151] In the embodiments of the present application, the master node is responsible for the write operation request of the entire distributed database system, can perform transaction partial order arbitration and sequencing through pessimistic locking in the master node, can perform transaction conflict, application and submission in the master node, and can synchronize the lock state to the slave node. The slave node unidirectionally replicates and observes the write result of the entire distributed database system, so that the distributed write conflict and consistency problem can be solved. At the same time, since the conflict processing of the distributed write operation is based on pessimistic locking, and the conflict granularity can be minimized, compared with the optimistic locking retry conflict resolution scheme, the performance is better in a high load and high concurrency conflict environment.

[0152] The embodiments of the present application can present any node supporting concurrent read and write transaction characteristics to the user, but the underlying still maintains a clear single-write multi-read architecture, effectively avoiding consistency risks.

[0153] In addition, on the basis of forwarding the write operation request, transaction and lock semantics forwarding is also ensured, so that the global distributed node executing the write operation request has a unified transaction view and lock conflict view.

[0154] And the user can not only perform a semantic complete read-write transaction at any distributed node, each node shows complete read-write characteristics, operation is more convenient, and improves user experience.

[0155] Figure 4 A flow chart of a request processing method provided by an embodiment of the present application is schematically shown, the request processing method can be applied to a master node in a distributed database system, the distributed database system further includes at least one slave node and a server, as shown in the figure, the request processing method can specifically include the following steps: Figure 4

[0156] 401, receiving a write operation request forwarded by a target slave node;

[0157] 402, processing the write operation request in the proxy transaction to generate a prewritten log corresponding to the write operation request;

[0158] 403, sending the prewritten log to the server; the prewritten log is used to update the corresponding database data; the log sequence number of the prewritten log is used to send to at least one slave node to obtain the database data after the update.

[0159] Among them, the target slave node can be any one of at least one slave node of the distributed database system. Any slave node in the distributed database system can receive the write operation request sent by the client and forward the write operation request to the master node.

[0160] Among them, the first transaction can refer to a logical work unit executed in the distributed database system, including a plurality of database operations executed together in the distributed database system. The above write operation request can be one operation in the plurality of database operations.

[0161] The target slave node can receive the plurality of database operations in the first transaction sent by the client, and then forward the write operation request in the plurality of database operations to the master node. The read operation request in the plurality of database operations can be executed locally by the target slave node.

[0162] The write operation request can be an operation request for modifying the database data. The write operation request can be sent by, for example, a SQL (Structured Query Language) statement such as DDL (Data Definition Language), DML (Data Manipulation Language), and DCL (Data Control Language).

[0163] ​After the master node receives the write operation request forwarded by the target slave node, instead of processing the write operation request, the master node generates a prewrite log corresponding to the write operation request, and the prewrite log is used to record the modification operation of the write operation request on the database data. In an embodiment of the present application, the distributed database system can organize the database data into data pages, and the data page can be a fixed-size data block, and each data page can contain one or more data records.

[0164] After generating the prewrite log, the master node can send the prewrite log to the server, and after receiving the prewrite log, the server can update the corresponding database data based on the prewrite log. For example, the server can apply the operation record recorded in the prewrite log to the database data to modify the database data correspondingly.

[0165] The prewrite log can include transaction identification (Transaction ID), operation type (Operation Type), affected data (Affected Data), old value and new value (Old Value and New Value), and the like, wherein the transaction identification is used to identify the first transaction, the operation type is used to describe the specific operation type, such as insertion, update or deletion, the affected data can refer to the specific data or data row identification affected by the write operation, and the old value and new value (Old Value and New Value) can refer to the old value and new value of the modified field recorded for the update operation.

[0166] The server can update the corresponding database data based on the above information recorded in the prewrite log.

[0167] After receiving the prewrite log, the server can first persistently store the prewrite log on the disk or other persistent storage medium, so as to ensure that even if the distributed database system crashes or is powered off, the prewrite log will not be lost.

[0168] After updating the corresponding database data based on the prewrite log, the server can send the updated database data to the master node or at least one slave node when the master node or at least one slave node needs. However, it is not limited thereto, and the server can send the updated database data to the master node or at least one slave node after updating the corresponding database data based on the prewrite log.

[0169] In the embodiments of the present application, by forwarding the write operation request to the master node after the target slave node receives the write operation request, any node in the distributed database system can respond to the write operation request of the client. Moreover, after receiving the write operation request sent by the target slave node, the master node does not perform locally, but generates a corresponding prewrite log, and sends the prewrite log to the server, so that the write operation request is uniformly processed in the server, the updated data is generated, and then the updated database data is distributed to the master node and at least one slave node by the server, avoiding multiple write conflicts among multiple nodes, and realizing data consistency among multiple nodes.

[0170] In some embodiments, the request processing method further includes:

[0171] According to the log sequence number of the prewrite log, a response request is generated, and the response request is sent to the target slave node.

[0172] In some embodiments, processing the write operation request in the proxy transaction to generate the prewrite log corresponding to the write operation request can be implemented as:

[0173] The lock resource of the data object to which the write operation request is directed is acquired;

[0174] The write operation request is processed to generate a prewrite log corresponding to the write operation request;

[0175] In some embodiments, the request processing method further includes:

[0176] In the case of proxy transaction submission, the lock resource of the data object is released.

[0177] In some embodiments, the request processing method further includes:

[0178] In the case that the lock resource of the data object is locked, the lock resource is waited.

[0179] In some embodiments, the request processing method further includes:

[0180] Receiving a transaction proxy request sent by the target slave node; the transaction proxy request is sent by the target slave node in the case that the first transaction opened by the user involves a write operation request;

[0181] Creating a proxy transaction.

[0182] In some embodiments, the request processing method further includes:

[0183] Receiving heartbeat information from the target slave node;

[0184] Based on the heartbeat information, in the case that the target slave node is in an abnormal state, the proxy transaction is rolled back.

[0185] In some embodiments, the server comprises a log service node and a data service node.

[0186] In some embodiments, sending the prewrite log to the server can be implemented as follows:

[0187] sending the prewrite log to the log service node, so that the log service node synchronizes the log sequence number of the prewrite log to at least one slave node, and provides the prewrite log to the data service node; the data service node is used to perform a replay operation based on the prewrite log to update the corresponding database data.

[0188] In some embodiments, the request processing method further comprises:

[0189] receiving a transaction commit request sent by the target slave node, committing the proxy transaction;

[0190] receiving a transaction rollback request sent by the target slave node, rolling back the proxy transaction.

[0191] In some embodiments, the request processing method further comprises:

[0192] In response to a read operation request in the second transaction initiated by the user, obtaining the data page after the update from the server according to the latest log sequence number of the data page accessed by the request.

[0193] In some embodiments, obtaining the data page after the update from the server can be implemented by obtaining the data page after the update from the data service node.

[0194] In some embodiments, the request processing method further comprises:

[0195] In response to a write operation request in the second transaction, generating a prewrite log corresponding to the write operation request; and sending the prewrite log to the server; the prewrite log is used to update the corresponding database data; the log sequence number of the prewrite log is used to send to the at least one slave node for obtaining the database data after the update.

[0196] When the user initiates a read operation request in the second transaction, the master node can obtain the latest data page from the server according to the latest log sequence number of the data page accessed by the request, so that the read operation request accesses the latest committed data.

[0197] After generating the prewrite log, the master node can send the generated prewrite log to the server. After receiving the prewrite log, the server can apply the prewrite log to the local storage to maintain the consistency and integrity of the data.

[0198] In some embodiments, the request processing method further comprises:

[0199] According to the latest log sequence number, the corresponding data page is obtained from the server and cached.

[0200] In response to a read operation request in a second transaction initiated by a user, the corresponding data page is searched from the local cache.

[0201] In which, obtaining the corresponding data page from the server and caching the data page can be specifically implemented as obtaining the corresponding data page from the data service node and caching the data page.

[0202] Figure 4 The specific implementation process of the request processing method shown can be referred to Figure 1 The description of the distributed database system shown is not repeated here.

[0203] Figure 5 The flowchart of the request processing method provided by another embodiment of the application is schematically shown, which can be applied to the slave node in the distributed database system, and the distributed database system further includes a master node and a server, such as Figure 5 The request processing method shown can specifically include the following steps:

[0204] 501, obtaining a first transaction initiated by a user;

[0205] 501, forwarding the write operation request in the first transaction to the master node, so that the master node processes the write operation request in a proxy transaction to generate a prewritten log corresponding to the write operation request, and sends the prewritten log to the server, which is used to update the corresponding database data;

[0206] 502, receiving the log sequence number of the prewritten log sent by the server; the log sequence number is used to obtain the database data after the update.

[0207] In some embodiments, the server is further configured to synchronize the log sequence number of the prewritten log to the at least one slave node.

[0208] In some embodiments, the request processing method further includes: obtaining the corresponding database data from the server according to the log sequence number.

[0209] In some embodiments, the request processing method further includes:

[0210] In response to a commit operation of the first transaction, a transaction commit request is sent to the master node; in response to a rollback operation of the first transaction, a transaction rollback request is sent to the master node.

[0211] In some embodiments, the server comprises a data service node and a log service node; the master node is configured to send the pre-write log to the log service node; the log service node is configured to provide a log sequence number of the pre-write log to the at least one slave node, and provide the pre-write log to the data service node; the data service node is configured to perform a replay operation based on the pre-write log to update corresponding database data, and provide the database data to the master node and the at least one slave node.

[0212] In some embodiments, the request processing method further comprises: according to a read operation request in the first transaction, obtaining a data page updated after the data page from the data service node according to a latest log sequence number of the data page accessed by the request.

[0213] In some embodiments, the request processing method further comprises: according to the latest log sequence number obtained from the log service node, obtaining a corresponding data page from the data service node and caching the data page; and according to a read operation request in the first transaction, searching for a corresponding data page from the local cache.

[0214] Figure 5 The specific implementation process of the request processing method shown can be referred to Figure 1 The description of the distributed database system shown is not repeated here.

[0215] Figure 6 An illustrative flowchart of a request processing method provided by another embodiment of the present application is shown, which can be applied to a server in a distributed database system, and the distributed database system further comprises a master node and at least one slave node, as shown in Figure 6 The request processing method can specifically include the following steps:

[0216] 601, receiving a pre-write log sent by a master node, the pre-write log being generated by the master node in processing a write operation request in a proxy transaction, the write operation request being forwarded to the master node by a target slave node;

[0217] 602, updating corresponding database data using the pre-write log;

[0218] 603, sending a log sequence number of the pre-write log to at least one slave node, so that the at least one slave node obtains database data updated after the database data based on the log sequence number.

[0219] In some embodiments, the request processing method further comprises: synchronizing the log sequence number of the pre-write log to the at least one slave node; so that the slave node obtains corresponding database data from the server according to the log sequence number.

[0220] In some embodiments, the server comprises a data service node and a log service node; and the master node is configured to send the prewrite log to the log service node.

[0221] The log service node is configured to provide a log sequence number of the prewrite log to the at least one slave node, and provide the prewrite log to the data service node.

[0222] The data service node is configured to perform a replay operation based on the prewrite log to update corresponding database data, and provide the database data to the master node and the at least one slave node.

[0223] Figure 6 The specific implementation process of the request processing method shown can be referred to Figure 1 The description of the distributed database system shown is not repeated here.

[0224] In the embodiments of the present application, a request processing apparatus is also provided, which is applied to a master node in a distributed database system, and the distributed database system further comprises at least one slave node and a server. The request processing apparatus can specifically comprise:

[0225] The first receiving module is configured to receive a write operation request forwarded by a target slave node.

[0226] The request processing module is configured to process the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request.

[0227] The log sending module is configured to send the prewrite log to the server. The prewrite log is used to update corresponding database data. A log sequence number of the prewrite log is used to be sent to the at least one slave node to obtain the database data after the update.

[0228] The request processing apparatus can perform Figure 4 The request processing method of the embodiments shown has no need to be repeated in terms of implementation principles and technical effects. The specific operation modes of each module and unit of the request processing apparatus in the above embodiments have been described in detail in the embodiments related to the method, and thus will not be described in detail here.

[0229] In another embodiment of the present application, a request processing apparatus is provided, which is applied to a slave node in a distributed database system, and the distributed database system further comprises a master node and a server. The request processing apparatus can specifically comprise:

[0230] The transaction receiving module is configured to obtain a first transaction opened by a user.

[0231] The request sending module is configured to forward the write operation request to the master node, so that the master node processes the write operation request in a proxy transaction to generate a write-ahead log corresponding to the write operation request, and sends the write-ahead log to the server, where the write-ahead log is used to update the corresponding database data.

[0232] The information receiving module is configured to receive a log sequence number of the write-ahead log sent by the server, where the log sequence number is used to obtain the database data after the update.

[0233] The request processing apparatus can perform Figure 7 The request processing method of the embodiments shown above will not be described again in detail. The specific manners in which the modules and units in the request processing apparatus described above perform operations have been described in detail in the embodiments related to the method, and thus will not be described again in detail here.

[0234] In yet another embodiment, a request processing apparatus is provided, which is applied to a server in a distributed database system, and the distributed database system further includes a master node and at least one slave node. The request processing apparatus can specifically include:

[0235] The log receiving module is configured to receive a write-ahead log sent by the master node, where the write-ahead log is generated by the master node when processing a write operation request in a proxy transaction, and the write operation request is forwarded to the master node by a target slave node.

[0236] The data updating module is configured to update the corresponding database data by using the write-ahead log.

[0237] The information sending module is configured to send a log sequence number of the write-ahead log to the at least one slave node, so that the at least one slave node obtains the database data after the update based on the log sequence number.

[0238] The request processing apparatus can perform Figure 6 The request processing method of the embodiments shown above will not be described again in detail. The specific manners in which the modules and units in the request processing apparatus described above perform operations have been described in detail in the embodiments related to the method, and thus will not be described again in detail here.

[0239] In one possible design, the request processing apparatus provided by the embodiments of the present application can be implemented as a computing device, such as a server. Figure 7 As shown in the figure, the computing device can include a storage component 701 and a processing component 702.

[0240] The storage component 701 stores one or more computer instructions, where the one or more computer instructions are called and executed by the processing component 702 to implement the request processing method provided by the embodiments of the present application.

[0241] Of course, the computing device can also necessarily include other components, such as an input / output interface, a communication component, and the like. The input / output interface provides an interface between the processing component and peripheral interface modules, which can be output devices, input devices, and the like. The communication component is configured to facilitate communication between the computing device and other devices in a wired or wireless manner, and the like.

[0242] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform, and in this case, the computing device can refer to a cloud server, and the processing component, the storage component, and the like can be basic server resources rented or purchased from the cloud computing platform.

[0243] When the computing device is a physical device, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device.

[0244] The embodiments of the present application also provide a computer readable storage medium storing a computer program, and the computer program can implement the request processing method provided by the embodiments of the present application when executed by a computer.

[0245] The embodiments of the present application also provide a computer program product including a computer program, and the computer program can implement the request processing method provided by the embodiments of the present application when executed by a computer.

[0246] The processing component in the foregoing embodiments can include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component can also be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic elements, for executing the above method.

[0247] The storage component is configured to store various types of data to support operations in the device. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0248] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0249] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0250] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software plus the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software products, and the computer software products can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and include a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0251] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed database system, characterized in that, The system comprises a master node, at least one slave node and a server; The target slave node in the at least one node is configured to receive a first transaction initiated by a user, and forward a write operation request in the first transaction to the master node; The target slave node is any slave node; The master node is configured to, in response to the write operation request forwarded by the target slave node, process the write operation request in a proxy transaction to generate a prewrite log; The prewrite log is sent to the server; The server is configured to update corresponding database data based on the prewrite log, and provide the database data to the master node and the at least one slave node.

2. The system of claim 1, wherein, The server is further configured to synchronize a log sequence number of the prewrite log to the at least one slave node; The at least one slave node is respectively configured to acquire corresponding database data from the server according to the log sequence number.

3. The system of claim 1, wherein, The master node is further configured to generate a response request according to a log sequence number of the prewrite log, and send the response request to the target slave node; The target slave node is further configured to acquire corresponding database data from the server according to the log sequence number.

4. The system of claim 1, wherein, The master node processes the write operation request in a proxy transaction to generate a prewrite log, comprising: acquiring a lock resource of a data object to which the write operation request is directed; processing the write operation request to generate a prewrite log corresponding to the write operation request; The master node is further configured to release the lock resource of the data object in the case of committing the proxy transaction.

5. The system of claim 1, wherein, The target slave node is further configured to acquire a lock state of different data objects from the master node; The target slave node forwards the write operation request in the first transaction to the master node, comprising: According to the lock state corresponding to the data object involved in the write operation request, the write operation request is forwarded to the master node in the case that the data object is not locked.

6. The system of claim 1, wherein, The target slave node is further configured to acquire a lock resource of a data object to which a read operation request in the first transaction is directed, process the read operation request to execute a corresponding data reading operation, and send the lock state of the data object to the master node.

7. The system of claim 1, wherein, The target slave node is further configured to, in response to the first transaction, send a transaction proxy request to the master node in the case that the first transaction involves a write operation request. The master node is further configured to receive the transaction proxy request sent by the target slave node, and create the proxy transaction in response to the transaction proxy request.

8. The system of claim 7, wherein, After the target slave node sends the transaction proxy request to the master node, the target slave node is further configured to send a heartbeat message to the master node. The master node is further configured to, in the case that the target slave node is in an abnormal state based on the heartbeat information, roll back the proxy transaction.

9. The system of claim 1, wherein The target slave node is further configured to send a transaction commit request to the master node in response to a commit operation of the first transaction, and send a transaction rollback request to the master node in response to a rollback operation of the first transaction. The master node is further configured to commit the proxy transaction according to the transaction commit request, and roll back the proxy transaction according to the transaction rollback request of the slave node.

10. The system of claim 1, wherein, The master node is further configured to, in response to a read operation request in a second transaction initiated by a user, obtain a data page after update from the server according to a latest log sequence number of the data page requested to be accessed, in response to a write operation request in the second transaction, generate a prewrite log corresponding to the write operation request, and send the prewrite log to the server.

11. The system of claim 1 or 10, wherein, The server comprises a data service node and a log service node; The master node is specifically configured to send the prewrite log to the log service node; The log service node is configured to provide a log sequence number of the prewrite log to the at least one slave node, and provide the prewrite log to the data service node; The data service node is configured to perform a replay operation based on the prewrite log to update corresponding database data, and provide the database data to the master node and the at least one slave node.

12. The system of claim 11, wherein, The target slave node is further configured to, in response to a read operation request in the first transaction, obtain a data page after update from the data service node according to a latest log sequence number of the data page requested to be accessed.

13. The system of claim 11, wherein, The target slave node is further configured to, according to the latest log sequence number obtained from the log service node, obtain a corresponding data page from the data service node and cache the data page, and in response to a read operation request in the first transaction, find the corresponding data page from the local cache.

14. A request processing method characterized by comprising: A master node applied to a distributed database system, the distributed database system further comprising at least one slave node and a server, the method comprising: receiving a write operation request forwarded by a target slave node; processing the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request; sending the prewrite log to the server; the prewrite log is used to update corresponding database data; and a log sequence number of the prewrite log is used to send to the at least one slave node to obtain database data after update.

15. The method of claim 14, wherein, The method further comprises: in response to a read operation request in a second transaction initiated by a user, obtaining a data page after update from the server according to a latest log sequence number of the data page requested to be accessed; in response to a write operation request in the second transaction, generating a prewrite log corresponding to the write operation request, and sending the prewrite log to the server; the prewrite log is used to update corresponding database data; and a log sequence number of the prewrite log is used to send to the at least one slave node to obtain database data after update.

16. The method of claim 15, wherein, The method further comprises: obtaining a corresponding data page from the server according to the latest log sequence number and caching the data page; in response to a read operation request in a second transaction initiated by a user, finding the corresponding data page from the local cache.

17. A request processing method characterized by comprising: The application discloses a method applied to a slave node in a distributed database system, the distributed database system further comprising a master node and a server, the method comprising: obtaining a first transaction initiated by a user; forwarding a write operation request in the first transaction to the master node, so that the master node processes the write operation request in a proxy transaction to generate a prewrite log corresponding to the write operation request, and sends the prewrite log to the server, the prewrite log being used to update corresponding database data; receiving a log sequence number of the prewrite log sent by the server, the log sequence number being used to obtain the database data after update.

18. A request processing method characterized by comprising: The application discloses a method applied to a server in a distributed database system, the distributed database system further comprising a master node and at least one slave node, the method comprising: receiving a prewrite log sent by the master node, the prewrite log being generated by the master node when processing a write operation request in a proxy transaction, the write operation request being forwarded to the master node by a target slave node; updating corresponding database data by using the prewrite log; sending a log sequence number of the prewrite log to the at least one slave node, so that the at least one slave node obtains database data after update based on the log sequence number.

19. A computing device, comprising: comprising a processing component and a storage component; the storage component stores one or more computer instructions, the one or more computer instructions being used to be called and executed by the processing component, and realizing the request processing method according to any one of claims 14 to 16 or the request processing method according to claim 17 or the request processing method according to claim 18.

20. A computer storage medium, comprising, a computer program is stored, and the computer program is executed by a computer to realize the request processing method according to any one of claims 14 to 16 or the request processing method according to claim 17 or the request processing method according to claim 18.

21. A computer program product, characterised in that, the computer program product comprises computer program code, and the computer program code is executed by a computer to realize the request processing method according to any one of claims 14 to 16 or the request processing method according to claim 17 or the request processing method according to claim 18.

Images