Database processing method and device

Abstract

The present specification provides a database processing method and device, wherein the database processing method includes: receiving a read request, and in response to the read request, obtaining the latest operation of the master node from the master node; determining that all slave nodes in at least one slave node have been updated for the latest operation, and then sending the read request to any slave node in the at least one slave node. The database processing method provided in the present specification sends a read request to the slave node after determining that all slave nodes have been updated for the latest operation of the master node through a proxy node, thereby avoiding the influence of replication delay between the master node and the slave node on data consistency, and realizing that the data read at any read node is the latest global data, thereby realizing global consistency reading.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a database processing method, a database processing device, a computing device, and a computer-readable storage medium. Background Art

[0002] In Internet database solutions, MySQL (relational database) is a relatively commonly used database. When the data access concurrency is high, MySQL generally adopts read-write separation technology to implement data reading and writing operations. A master node (master database) and at least one slave node (slave database) are set up in the database. The data in the master node is synchronized to the slave node in real time. The master node is responsible for write operations, and the slave node is responsible for read operations.

[0003] Under read-write separation, there are causal dependencies between the connections (sessions) of many applications, but due to the replication delay under the read-write separation architecture, the correctness of the causal relationship, that is, the consistency, cannot be guaranteed. For example, an application first establishes connection A with the master node, performs data updates, and then establishes connection B with the slave node to query the data corresponding to the update. Under read-write separation, because the write operation is for the master node and the read operation is for the slave node, and there is a replication delay between the master node and the slave node, when querying data through connection B, the data previously updated through connection A may not be queried, resulting in the consistency of the transaction logic being affected.

[0004] Existing technologies usually ensure consistency through distributed (XA) transactions, but doing so means locking multiple read nodes and increasing the number of interactions, which may result in the performance of using multiple nodes being lower than that of a single node, and thus requires a higher-performance method to ensure consistent operations or processing. Summary of the invention

[0005] In view of this, the embodiment of this specification provides a database processing method. This specification also relates to a database processing device, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0006] According to a first aspect of an embodiment of this specification, a database processing method is provided, which is applied to a proxy node, including:

[0007] receiving a read request, and obtaining, in response to the read request, a latest operation of the master node from the master node;

[0008] It is determined that all slave nodes in at least one slave node are updated for the latest operation, and then the read request is sent to any slave node in the at least one slave node.

[0009] Optionally, obtaining the latest operation of the master node from the master node includes:

[0010] A query request is sent to the master node and information representing the latest operation of the master node is received from the master node.

[0011] Optionally, determining that all slave nodes in at least one slave node are updated for the latest operation includes:

[0012] verifying a number of update reports received from the at least one slave node with a number of the at least one slave node;

[0013] When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

[0014] Optionally, receiving a read request includes:

[0015] The read request is received by a first thread, and the read request is added to a request queue.

[0016] Optionally, sending the query request to the master node includes:

[0017] Sending a query request to the master node through a second thread;

[0018] The receiving the information returned by the master node and representing the latest operation of the master node includes: receiving the information returned by the master node through the second thread.

[0019] Optionally, sending the read request to any slave node of the at least one slave node includes:

[0020] When sending an inquiry request to the master node, transferring the read request in the request queue to the first waiting queue;

[0021] When receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue;

[0022] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read requests in the second waiting queue to the sending queue;

[0023] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0024] Optionally, sending the read request to any slave node of the at least one slave node includes:

[0025] When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue;

[0026] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue;

[0027] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0028] Optionally, the information includes a log sequence number of a log record representing the latest operation returned by the master node, and receiving the information includes receiving the log sequence number.

[0029] According to a second aspect of an embodiment of this specification, a database processing device is provided, which is applied to a proxy node, including:

[0030] An acquisition module is configured to: receive a read request, and in response to the read request, acquire the latest operation of the master node from the master node;

[0031] The sending module is configured to: determine that all slave nodes in the at least one slave node have been updated for the latest operation, and then send the read request to any slave node in the at least one slave node.

[0032] Optionally, the acquisition module is further configured to send an inquiry request to the master node and receive information returned by the master node that represents the latest operation of the master node.

[0033] Optionally, the sending module is further configured to:

[0034] verifying a number of update reports received from the at least one slave node with a number of the at least one slave node;

[0035] When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

[0036] Optionally, the acquisition module is further configured to:

[0037] The read request is received by a first thread, and the read request is added to a request queue.

[0038] Optionally, the acquisition module is further configured to:

[0039] Sending a query request to the master node through a second thread;

[0040] The information returned by the master node is received through the second thread.

[0041] Optionally, the sending module is further configured to:

[0042] When sending an inquiry request to the master node, transferring the read request in the request queue to the first waiting queue;

[0043] When receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue;

[0044] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read requests in the second waiting queue to the sending queue;

[0045] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0046] Optionally, the sending module is further configured to:

[0047] When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue;

[0048] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue;

[0049] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0050] Optionally, the information includes a log sequence number of a log record representing the latest operation returned by the master node, and the acquisition module is further configured to receive the log sequence number.

[0051] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0052] Memory and processor;

[0053] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions:

[0054] receiving a read request, and obtaining, in response to the read request, a latest operation of the master node from the master node;

[0055] It is determined that all slave nodes in at least one slave node are updated for the latest operation, and then the read request is sent to any slave node in the at least one slave node.

[0056] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of any of the database processing methods are implemented.

[0057] The database processing method provided in this specification utilizes a proxy node to receive a read request, and in response to the read request, obtains the latest operation of the master node from the master node, and sends a read request to any slave node in at least one slave node only after determining that all slave nodes in at least one slave node have been updated for the latest operation. The database processing method provided in this specification sends a read request to the slave node after the proxy node determines that all slave nodes have been updated for the latest operation of the master node, thereby avoiding the influence of replication delay between the master node and the slave node on data consistency, and realizing that the data read at any read node is the latest global data, thereby realizing global consistency reading. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flowchart of a database processing method provided according to an embodiment of this specification is shown;

[0059] Figure 2 A schematic diagram showing an operation flow between a proxy node, a master node, and a slave node of a database processing method provided according to an embodiment of the present specification;

[0060] Figure 3 A flowchart of a database processing method provided according to another embodiment of the present specification is shown;

[0061] Figure 4 A schematic diagram showing a queue division method of a database processing method provided according to another embodiment of the present specification is shown;

[0062] Figure 5 A flowchart of a database processing method provided according to another embodiment of the present specification is shown;

[0063] Figure 6 A schematic diagram showing a queue division method of a database processing method provided according to another embodiment of the present specification is shown;

[0064] Figure 7 A schematic diagram of the structure of a database processing device provided in an embodiment of this specification is shown;

[0065] Figure 8 A structural block diagram of a computing device provided according to an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0066] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0067] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0068] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0069] First, the terms involved in one or more embodiments of this specification are explained.

[0070] Read-write separation: a distributed expansion architecture that uses the characteristics of transactions with more reads and fewer writes to concentrate writes on a write node, namely the master node (usually called the master database), and then replicates the data of the write node to multiple read nodes, namely slave nodes (usually called slave nodes), and then reads data from the read nodes, that is, let the write node handle transactional addition, modification, and deletion operations (INSERT, UPDATE, DELETE), while the read node handles query (SELECT) operations.

[0071] Global consistency read: In the read-write separation architecture, there are multiple read nodes. Global consistency read ensures that the data read at any read node is the latest data globally (the entire cluster).

[0072] SQL (Structured Query Language): is a special purpose programming language, a database query and programming language, used to access data and query, update and manage relational database systems. SQL is a high-level non-procedural programming language that allows users to work on high-level data structures. It does not require users to specify the storage method for data, nor does it require users to understand the specific data storage method. Therefore, different database systems with completely different underlying structures can use the same SQL as the interface for data input and management. SQL can be nested, which makes it extremely flexible and powerful.

[0073] MySQL: It is a relational database management system. As MySQL continues to mature, it is gradually used in more large-scale websites and applications.

[0074] Binary log (binlog): A binary log file used to record changes to the database within MySQL (only data modification operations are recorded). It is mainly used for master-slave replication and incremental recovery of the database. MySQL officially supports three binlog data formats, corresponding to the parameter binlog_format. The optional values ​​are ROW, STATEMENT, and MIXED. The corresponding default value is MIXED, which needs to be adjusted to ROW mode. The characteristics of this mode are: every SQL added, modified, or deleted will record the values ​​of the rows affected by the SQL before and after the change in the binlog.

[0075] Redo log (REDO log): can be used to restore the database and generate data pages at any point in time. When writing to the database, data pages will be modified. Each modification of a data page will generate a corresponding REDO log. REDO log does not record the user's SQL, but only records the final result, that is, the data page changes caused by the execution of this SQL. Therefore, relying on this replication architecture, SQL parsing and SQL optimization are not required. MySQL directly finds the data page in the corresponding file, locates to the specified offset, and directly updates it.

[0076] Next, the basic concept of this manual is briefly described:

[0077] Under read-write separation, there are causal dependencies between the connections (sessions) of many applications, but due to the delay of replication under the read-write separation architecture, the correctness of the causal relationship, i.e., consistency, cannot be guaranteed. For example, in one embodiment, a certain application first establishes connection A with the master node, executes the data update, and then establishes connection B with the slave node to query the data corresponding to the update. Under read-write separation, because the write operation is for the master node and the read operation is for the slave node, and there is a replication delay between the master node and the slave node, when querying the data through connection B, the data updated through connection A may not be queried, resulting in the consistency of the transaction logic being affected. For example, in the example where user a transfers 100 yuan to user b through an electronic banking application, the electronic banking application establishes connection A with the master node, executes the transaction of subtracting 100 yuan from user a's account and adding 100 yuan to user b's account, and then establishes connection B with the slave node. When querying the balance of user b's account, due to the replication delay, the balance of user b's account may not be queried through connection B.

[0078] This specification provides a database processing method, which adds a proxy node, i.e., a proxy server. The proxy node establishes a connection with the application program, and parses each SQL sent in. If it is a write operation such as modification (UPDATE), deletion (DELETE), addition (INSERT), creation (CREATE), etc., it is directly sent to the master node. If it is a query (SELECT) read request, in response to the read request, the latest operation of the master node is obtained from the master node, and a read request is sent to any slave node in the at least one slave node only after it is determined that all slave nodes in at least one slave node have been updated for the latest operation. The database processing method provided in this specification sends a read request to the slave node after the proxy node determines that all slave nodes have been updated for the latest operation of the master node, thereby avoiding the influence of replication delay between the master node and the slave node on data consistency, and realizing that the data read at any read node is the latest global data, thereby realizing global consistency reading.

[0079] In this specification, a database processing method is provided. This specification also relates to a database processing device, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0080] Figure 1 A flowchart of a database processing method provided according to an embodiment of the present specification is shown, which specifically includes steps 102 to 104.

[0081] Step 102: Receive a read request, and in response to the read request, obtain the latest operation of the master node from the master node.

[0082] In the read-write separation architecture, write operations are concentrated on a write node, namely the master node, and the data of the master node is copied to multiple read nodes, namely the slave nodes. The data is then read from the read nodes. That is, the master node handles transactional addition, modification, and deletion operations (INSERT, UPDATE, DELETE), while the slave nodes handle query (SELECT) operations.

[0083] The read request may be a SQL (Structured Query Language) query (SELECT) request. The proxy node receives the read request sent by the application. In response to the read request, the proxy node obtains the latest operation of the master node from the master node. The latest operation of the master node is a transactional addition, modification or deletion operation (INSERT, UPDATE, DELETE).

[0084] In one embodiment of the present specification, obtaining the latest operation of the master node from the master node may be achieved by sending a query request to the master node and receiving information returned by the master node that represents the latest operation of the master node.

[0085] Figure 2 A schematic diagram showing an operation flow between a proxy node, a master node, and a slave node of a database processing method provided according to an embodiment of the present specification is shown. Figure 2 As shown, after receiving the read request, the proxy node sends an ask request to the master node. After receiving the ask request, the master node sends information representing the latest operation of the master node to the proxy node. The information may include SQL statements related to the latest operation (write operation) of the master node, binary log records recording the SQL statements, or redo log records recording the results after the SQL statements are executed.

[0086] In one embodiment of the present specification, the information may include a log sequence number (LSN) of a log record representing the latest operation returned by the master node, such as an LSN representing a binary log record representing the latest operation or a redo log record representing the latest operation. LSN represents the addressing address of the log file, and there is an LSN for each log record of an operation, which represents the addressing position of the log record in the log file.

[0087] Step 104: if it is determined that all slave nodes in at least one slave node have been updated for the latest operation, the read request is sent to any slave node in the at least one slave node.

[0088] Take the binary log record as an example to illustrate the latest operation of the master node. The master node records the latest operation in the binary log record, and the slave node copies the master node's binary log to its relay log. Specifically, the slave node starts a worker thread (I / O), the I / O thread reads events from the master node's binary log, and the I / O thread writes these events to the relay log. The slave node starts the SQL slave thread, reads events from the relay log, and replays the events to update the slave node data to make it consistent with the data in the master node. Figure 2 As shown, after the slave node completes the data update, it sends an update report to the proxy node. Figure 2 It is exemplarily shown that the number of slave nodes is 2, which is only an example and is not specifically limited.

[0089] Taking the redo log record, which is the information representing the latest operation of the master node, as an example, the redo log record of the master node records the data page changes caused by the latest operation, rather than recording the SQL statements related to the latest operation. MySQL directly finds the data page in the corresponding file, locates to the specified offset, and directly updates it. After the slave node completes the data update, it sends an update report to the proxy node.

[0090] The information representing the latest operation of the master node is taken as an example of the LSN of the log record representing the latest operation returned by the master node. After completing the update of the LSN of the log record, the slave node sends an update report to the proxy node.

[0091] In practical applications, determining that all slave nodes in at least one slave node are updated for the latest operation may be implemented as follows:

[0092] verifying a number of update reports received from the at least one slave node with a number of the at least one slave node;

[0093] When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

[0094] The proxy node compares the number of update reports from the slave nodes with the number of at least one slave node. If the two are confirmed to be the same, it is determined that update reports from all slave nodes have been received, and that all slave nodes have been updated for the latest operation. A read request is then sent to any slave node, thereby providing a specific method for determining that all slave nodes have been updated for the latest operation, thereby concretizing the determination process.

[0095] The database processing method provided in this specification determines through the proxy node that all slave nodes have been updated according to the latest operation of the master node, and then sends a read request to the slave node, thereby avoiding the impact of replication delay between the master node and the slave node on data consistency, and ensuring that the data read at any read node is the latest global data, thereby achieving global consistency reading.

[0096] The following describes the database processing method provided in this specification based on the specific scenario described above where user a transfers money to user b through an electronic banking application.

[0097] User a transfers 100 yuan to user b through the electronic banking application. The electronic banking application establishes a connection with the master node and executes transactions to the master node to subtract 100 yuan from user a's account and add 100 yuan to user b's account. After the transaction is committed, it is written into the log record of the master node. If the electronic banking application directly sends a query request to the slave node to query the balance of user b's account, it may fail to query the increase of 100 yuan in user b's account balance due to replication delay.

[0098] Therefore, in the database processing method provided in this specification, a proxy node is added, and the electronic banking application sends a query (SELECT) request (querying the balance of user b's account) to the proxy node. The proxy node sends a query request to the master node, and the master node returns information representing its latest operation (user b's account increases by 100 yuan) to the proxy node, such as the SQL statement in the binary log record related to the increase of 100 yuan in user b's account.

[0099] The slave node copies the above binary log records of the master node to the relay log of the slave node, reads the event (user b's account increases by 100 yuan) from the relay log, replays the event to update the slave node data, and sends an update report to the proxy node after completing the data update. After the proxy node confirms that it has received the update reports of all slave nodes, it sends a read request to any slave node to query the account balance of user b. This ensures that the account balance of user b read by any slave node increases by 100 yuan, thereby achieving global consistency reading.

[0100] In one implementation of the present specification, the receiving the read request includes: receiving the read request through a first thread, and adding the read request to a request queue.

[0101] The sending of the query request to the master node includes: sending the query request to the master node through a second thread; the receiving of the information returned by the master node representing the latest operation of the master node includes: receiving the information returned by the master node through the second thread.

[0102] The first thread is an independent queuing thread. The proxy node starts the queuing thread to add the received read requests to the request queue in order. The second thread is an independent inquiry thread different from the first thread. The proxy node starts the inquiry thread to send an inquiry request to the master node and receive information returned by the master node. Using different threads to receive read requests and send inquiry requests to the master node can reduce waiting time and reduce the time overhead of the entire operation.

[0103] Since the above database processing method needs to obtain the latest operation of the master node and confirm that all slave nodes are updated for the latest operation after receiving each read request, the time overhead is relatively large. In order to further reduce the time overhead of the entire operation, batch processing is performed by dividing the queue.

[0104] Figure 3 A flowchart of a database processing method provided according to another embodiment of the present specification is shown, including steps 302 to 310.

[0105] Step 302: receiving a read request, adding the read request to a request queue, and sending a query request to the master node in response to the read request;

[0106] Step 304: When sending a query request to the master node, transfer the read request in the request queue to the first waiting queue;

[0107] Step 306: upon receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue;

[0108] Step 308: When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, the read requests in the second waiting queue are transferred to the sending queue;

[0109] Step 310: Send the read request in the sending queue to any slave node among the at least one slave node.

[0110] Combine the following Figure 4 right Figure 3 The operation flow of each step in is described.

[0111] like Figure 4As shown, according to the operation flow for the read request, the four queues are divided into a request queue 402, a first waiting queue 404, a second waiting queue 406, and a sending queue 408. After receiving the read request, it is added to the request queue 402. When a query request is sent to the master node, the read request is transferred from the request queue 402 to the first waiting queue 404. When the latest operation returned by the master node is received, the read request is transferred from the first waiting queue 404 to the second waiting queue 406. When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, the read request is transferred from the second waiting queue 406 to the sending queue 408. The four queues are processed in parallel, the read request waits in each queue, and the read request that reaches the top of the corresponding queue is processed in each queue, which is explained in detail below.

[0112] As described above, a read request is received by the first thread and added to the request queue 402. The request queue 402 stores multiple read requests in sequence. For a specific read request, it is first determined whether the read request reaches the top of the request queue 402. If it does not reach the top of the request queue 402, it waits for other read requests to be processed. If it reaches the top of the request queue 402, it is determined whether an inquiry request is sent to the master node for the read request. If it is determined to be yes, Figure 4 The first arrow indicates that the read request is transferred to the first waiting queue 404. If the judgment is no, the queue waits for sending an inquiry request to the master node for the read request.

[0113] Determine whether the read request in the first waiting queue 404 reaches the top of the first waiting queue 404. If not, wait for other read requests to be processed. If it reaches the top of the first waiting queue 404, determine whether the latest operation returned by the master node is received. If so, Figure 4 The second arrow indicates that the read request is transferred to the second waiting queue 406. If not, the read request is waited for receiving the latest operation returned by the master node. Next, when it is determined that all slave nodes in at least one slave node have been updated for the latest operation, as shown in FIG. Figure 4 As shown by the third arrow, the read request is transferred from the second waiting queue 406 to the sending queue 408. Next, it is determined whether the read request reaches the top of the sending queue 408. If not, wait for other read requests to be sent. If it reaches the top of the sending queue 408, send the read request to any slave node of the at least one slave node.

[0114] According to the database processing method provided in this embodiment, the operation flow of all received read requests is divided into four queues for parallel processing. There is no need to wait for all operations for the previous read request to be completed before executing the current read request. Instead, the read requests of the corresponding operation flow stage are processed in each queue, thereby sending the read requests to the slave nodes more quickly while achieving global consistency reads, thereby reducing the time overhead of achieving global consistency reads.

[0115] The above describes an embodiment in which four queues are divided. This specification does not specifically limit the specific number of queues, and the following describes an embodiment in which three queues are divided.

[0116] Figure 5 A flowchart of a database processing method provided according to another embodiment of the present specification is shown, including steps 502 to 508.

[0117] Step 502: receiving a read request, and adding the read request to a request queue, in response to the read request, sending a query request to the master node and receiving the latest operation returned by the master node;

[0118] Step 504: When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue;

[0119] Step 506: When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, the read request in the waiting queue is transferred to the sending queue;

[0120] Step 508: Send the read request in the sending queue to any slave node among the at least one slave node.

[0121] Combine the following Figure 6 right Figure 5 The operation flow of each step in is described.

[0122] like Figure 6 As shown, according to the operation flow for the read request, the three queues are divided into a request queue 402, a waiting queue 604, and a sending queue 408. After receiving the read request, it is added to the request queue 402. When an inquiry request is sent to the master node and the latest operation returned by the master node is received, the read request is transferred from the request queue 402 to the waiting queue 604. When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, the read request is transferred from the waiting queue 604 to the sending queue 408. The three queues are processed in parallel, the read request waits in each queue, and the read request that reaches the top of the corresponding queue is processed in each queue, which is explained in detail below.

[0123] As described above, a read request is received by the first thread and added to the request queue 402, and multiple read requests are sequentially stored in the request queue 402. For a specific read request in the request queue 402, it is first determined whether the read request has reached the top of the request queue 402. If it has not reached the top of the request queue 402, it waits for other read requests to be processed. If it has reached the top of the request queue 402, it is determined whether a query request has been sent to the master node for the read request and the latest operation returned by the master node has been received. If it is determined to be yes, it is determined as follows. Figure 4 The first arrow indicates that the read request is transferred to the waiting queue 604. If the judgment is no, the queue waits for sending an inquiry request to the master node for the read request and receiving the latest operation returned by the master node.

[0124] It is determined whether the read request in the waiting queue 604 reaches the top of the waiting queue 604. If not, wait for other read requests to be processed. If it reaches the top of the waiting queue 604, it is determined that all slave nodes in at least one slave node have been updated for the latest operation. Figure 4 The second arrow shows that the read request is transferred from the waiting queue 604 to the sending queue 408. Next, it is determined whether the read request reaches the top of the sending queue 408. If not, wait for other read requests to be sent. If it reaches the top of the sending queue 408, send the read request to any slave node of the at least one slave node.

[0125] Figure 5 The database processing method shown is divided into three parallel processing queues according to the operation flow of all received read requests. There is no need to wait for all operations on the previous read request to be completed before executing the current read request. Instead, each queue processes the read request of the corresponding operation flow stage, thereby quickly sending the read request to the slave node under the premise of achieving global consistency read, reducing the time overhead of achieving global consistency read. The database processing method provided by this embodiment is Figure 3 The database processing method in the embodiment shown simplifies the queue division, but the time overhead is reduced to less than Figure 3 The time overhead of the database processing method of the illustrated embodiment is reduced.

[0126] Corresponding to the above method embodiment, this specification also provides a database processing device embodiment, Figure 7 FIG. 1 shows a schematic diagram of the structure of a database processing device provided in an embodiment of the present specification. Figure 7 As shown, the device comprises:

[0127] The acquisition module 702 is configured to: receive a read request, and in response to the read request, acquire the latest operation of the master node from the master node;

[0128] The sending module 704 is configured to: determine that all slave nodes in the at least one slave node are updated for the latest operation, and then send the read request to any slave node in the at least one slave node.

[0129] The read request can be a SQL query (SELECT) request. The proxy node receives the read request sent by the application. In response to the read request, the proxy node obtains the latest operation of the master node from the master node. The latest operation of the master node is a transactional addition, modification or deletion operation (INSERT, UPDATE, DELETE).

[0130] The database processing device provided in this specification determines through a proxy node that all slave nodes have been updated according to the latest operation of the master node, and then sends a read request to the slave node, thereby avoiding the impact of replication delays between the master node and the slave node on data consistency, and ensuring that the data read at any read node is the latest global data, thereby achieving global consistency reading.

[0131] Optionally, the acquisition module is further configured to send an inquiry request to the master node and receive information returned by the master node that represents the latest operation of the master node.

[0132] Optionally, the sending module is further configured to:

[0133] verifying a number of update reports received from the at least one slave node with a number of the at least one slave node;

[0134] When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

[0135] Optionally, the acquisition module is further configured to:

[0136] The read request is received by a first thread, and the read request is added to a request queue.

[0137] Optionally, the acquisition module is further configured to:

[0138] Sending a query request to the master node through a second thread;

[0139] The information returned by the master node is received through the second thread.

[0140] Optionally, the sending module is further configured to:

[0141] When sending an inquiry request to the master node, transferring the read request in the request queue to the first waiting queue;

[0142] When receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue;

[0143] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read requests in the second waiting queue to the sending queue;

[0144] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0145] The database processing device provided in this embodiment divides the operation flow of all received read requests into four parallel processing queues. There is no need to wait for all operations on the previous read request to be completed before executing the current read request. Instead, the read requests of the corresponding operation flow stage are processed in each queue, thereby sending the read requests to the slave nodes more quickly while achieving global consistency reads, thereby reducing the time overhead of achieving global consistency reads.

[0146] Optionally, the sending module is further configured to:

[0147] When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue;

[0148] When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue;

[0149] The read request in the sending queue is sent to any slave node of the at least one slave node.

[0150] The database processing device provided in this embodiment divides the operation flow of all received read requests into three parallel processing queues. There is no need to wait for all operations on the previous read request to be completed before executing the current read request. Instead, the read requests of the corresponding operation flow stage are processed in each queue, thereby quickly sending the read requests to the slave nodes under the premise of achieving global consistency read, reducing the time overhead of achieving global consistency read.

[0151] Optionally, the information includes a log sequence number of a log record representing the latest operation returned by the master node, and the acquisition module is further configured to receive the log sequence number.

[0152] The above is a schematic scheme of a database processing device of this embodiment. It should be noted that the technical scheme of the database processing device and the technical scheme of the above database processing method belong to the same concept, and the details not described in detail in the technical scheme of the database processing device can be referred to the description of the technical scheme of the above database processing method.

[0153] Figure 8 The structure block diagram of a computing device 800 provided according to an embodiment of the present specification is shown. The components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0154] The computing device 800 also includes an access device 840 that enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0155] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0156] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 800 may also be a mobile or stationary server.

[0157] The processor 820 is used to execute the following computer executable instructions:

[0158] receiving a read request, and obtaining, in response to the read request, a latest operation of the master node from the master node;

[0159] It is determined that all slave nodes in at least one slave node are updated for the latest operation, and then the read request is sent to any slave node in the at least one slave node.

[0160] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above database processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above database processing method.

[0161] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which are used when executed by a processor to:

[0162] receiving a read request, and obtaining, in response to the read request, a latest operation of the master node from the master node;

[0163] It is determined that all slave nodes in at least one slave node are updated for the latest operation, and then the read request is sent to any slave node in the at least one slave node.

[0164] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above database processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above database processing method.

[0165] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0167] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0168] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0169] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A database processing method, applied to a proxy node, wherein the proxy node is used to parse a received request, comprising: When receiving a write request, forwarding the write request to the master node; In the case of receiving a read request, in response to the read request, sending an inquiry request to the master node and receiving information returned by the master node representing the latest operation of the master node, wherein the read request is processed in batches by dividing the queues, and the queues are divided based on the operation flow of the read request; Determining that all slave nodes in at least one slave node have been updated for the latest operation, sending the read request to any slave node in the at least one slave node; The sending the read request to any slave node of the at least one slave node comprises: When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue; When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue; The read request in the sending queue is sent to any slave node of the at least one slave node.

2. The database processing method according to claim 1, wherein: Determining that all slave nodes in at least one slave node are updated for the latest operation includes: verifying a number of update reports received from the at least one slave node with a number of the at least one slave node; When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

3. The database processing method according to claim 1, wherein: Receiving a read request includes: The read request is received by a first thread, and the read request is added to a request queue.

4. The database processing method according to claim 1, wherein: The sending of the query request to the master node comprises: Sending a query request to the master node through a second thread; The receiving the information returned by the master node and representing the latest operation of the master node includes: receiving the information returned by the master node through the second thread.

5. The database processing method according to claim 3, wherein: The sending the read request to any slave node of the at least one slave node comprises: When sending an inquiry request to the master node, transferring the read request in the request queue to the first waiting queue; When receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue; When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read requests in the second waiting queue to the sending queue; The read request in the sending queue is sent to any slave node of the at least one slave node.

6. The database processing method according to claim 1, wherein: The information includes a log sequence number of a log record representing the latest operation returned by the master node, and receiving the information includes receiving the log sequence number.

7. A database processing device, applied to a proxy node, the proxy node being used to parse a received request, comprising: The acquisition module is configured to: forward the write request to the master node when receiving the write request; In the case of receiving a read request, in response to the read request, sending an inquiry request to the master node and receiving information returned by the master node representing the latest operation of the master node, wherein the read request is processed in batches by dividing the queues, and the queues are divided based on the operation flow of the read request; The sending module is configured to: determine that all slave nodes in at least one slave node are updated for the latest operation, and then send the read request to any slave node in the at least one slave node; The sending module is further configured to: when sending a query request to the master node and receiving the latest operation returned by the master node, transfer the read request in the request queue to the waiting queue; When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue; The read request in the sending queue is sent to any slave node of the at least one slave node.

8. The database processing device according to claim 7, wherein: The sending module is further configured as: verifying a number of update reports received from the at least one slave node with a number of the at least one slave node; When the update reports of all slave nodes are received, it is determined that all slave nodes are updated for the latest operation.

9. The database processing device according to claim 7, wherein: The acquisition module is further configured to: The read request is received by a first thread, and the read request is added to a request queue.

10. The database processing device according to claim 7, wherein: The acquisition module is further configured to: Sending a query request to the master node through a second thread; The information returned by the master node is received through the second thread.

11. The database processing device according to claim 9, wherein: The sending module is further configured as: When sending an inquiry request to the master node, transferring the read request in the request queue to the first waiting queue; When receiving the latest operation returned by the master node, transferring the read requests in the first waiting queue to the second waiting queue; When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read requests in the second waiting queue to the sending queue; The read request in the sending queue is sent to any slave node of the at least one slave node.

12. The database processing device according to claim 7, wherein: The information includes a log sequence number of a log record representing the latest operation returned by the master node, and the acquisition module is further configured to receive the log sequence number.

13. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: When receiving a write request, forwarding the write request to the master node; In the case of receiving a read request, in response to the read request, sending an inquiry request to the master node and receiving information returned by the master node representing the latest operation of the master node, wherein the read request is processed in batches by dividing the queues, and the queues are divided based on the operation flow of the read request; Determining that all slave nodes in at least one slave node have been updated for the latest operation, sending the read request to any slave node in the at least one slave node; The sending the read request to any slave node of the at least one slave node comprises: When sending a query request to the master node and receiving the latest operation returned by the master node, transferring the read request in the request queue to the waiting queue; When it is determined that all slave nodes in at least one slave node have been updated for the latest operation, transferring the read request in the waiting queue to the sending queue; The read request in the sending queue is sent to any slave node of the at least one slave node.

14. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the database processing method according to any one of claims 1 to 6.

Images