Read-write method of shared page, database system and computer equipment

Abstract

The embodiment of the invention is suitable for the technical field of databases, and provides a read-write method of a shared page, a database system and computer equipment. By applying the method, the owner mode of the shared page is marked as the shared memory mode in the page directory, so that each computing node can unilaterally access the shared page in the memory when a plurality of nodes share the read-write page under the data sharing architecture, the phenomenon that the shared page is frequently transferred through remote procedure calling is avoided, and the data sharing efficiency is improved. And the performance of multi-node conflict access to the shared page is effectively improved. When the read-write method is adopted to read and write the shared page, the performance of a database system can be improved by more than 30%.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of database technology, and in particular to a shared page reading and writing method, a database system, and a computer device. Background Art

[0002] Data sharing is a technology that allows data to be exchanged or shared between multiple applications, services, and devices. In a data sharing architecture, the storage and computing of data in the system are separated from each other. Each computing node in the system can access read and write pages, and the page buffer pool is shared between multiple computing nodes, and each computing node can access the full amount of data.

[0003] like Figure 1 As shown, it is a schematic diagram of data sharing in the prior art. Figure 1 , a distributed page buffer pool is shown in FIG. 1 , in which cached pages are from a shared storage and can be shared by multiple nodes. For example, the page buffer pool can be Figure 1 Node 1, Node 2 and Node 3 in the page buffer pool are shared. Node 1, Node 2 and Node 3 can all read and write pages in the page buffer pool. In the prior art, when the page buffer pool is shared among multiple computing nodes, the buffer page can be transmitted through a remote procedure call (RPC). However, the overhead of obtaining pages through RPC is relatively large. How to adopt a new method to reduce the overhead of nodes when reading and writing shared pages is a technical problem that urgently needs to be solved. Summary of the invention

[0004] The embodiments of the present application provide a shared page reading and writing method, a database system and a computer device, which can be used to solve the problem of high overhead of buffering pages transmitted through RPC in the prior art.

[0005] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0006] In a first aspect, a method for reading and writing a shared page is provided, comprising: determining a shared page to be read and written, wherein the address of the shared page can be shared among multiple computing nodes. According to the address of the shared page, the computing node can obtain page information of the shared page from a page buffer pool, wherein the page information at least includes lock information of the shared page and a log sequence number stored in the shared page. The computing node can read and write the shared page based on the acquired lock information and log sequence number.

[0007] It should be understood that under the data sharing architecture, multiple computing nodes can access the page buffer pool and obtain shared pages from the page buffer pool. The embodiment of the present application stores the lock information and log sequence number of the shared page in the page buffer pool, so that each computing node can directly obtain the shared page and its corresponding page information from the page buffer pool based on the address of the shared page when reading and writing the shared page, without the need to pass the shared page to be read and written and its page information in multiple computing nodes through processes such as RPC. In this way, the communication overhead caused by multiple computing nodes passing shared pages and the overhead during thread calling can be reduced, the computing performance of each computing node in the database system can be improved, and the normal operation of the database system can be guaranteed.

[0008] In a possible implementation of the first aspect, shared pages are read and written based on lock information and log sequence numbers, including: determining an operation type for reading and writing shared pages, where the operation type includes a read operation or a modify operation; setting lock information for the shared pages according to the operation type; and after setting the lock information, performing read and write operations on the shared pages that match the operation type based on the log sequence number.

[0009] It should be understood that the operation type can be determined according to the specific transaction requirements, including read operations or modification operations. Among them, the read operation can refer to reading the data or information in the shared page, that is, the read page operation; the modification operation can refer to modifying the data or information in the shared page, that is, the write page operation. After determining the type of operation on the shared page, the computing node can set the lock information of the shared page, for example, when the page needs to be written, the shared page is locked. Then, based on the set lock information, the corresponding read and write operations are performed.

[0010] In a possible implementation of the first aspect, the lock information of the shared page is set according to the operation type, including: when the operation type is a modification operation, the lock information of the shared page is set to a write lock, and the holder of the write lock is set to the current computing node.

[0011] It should be understood that when the operation type is a modification operation, that is, when the computing node needs to perform a write page process on a shared page, the computing node should hold the write lock of the shared page. When a computing node holds the write lock of a shared page, the computing node can modify the data in the shared page, or write data to the page, or delete part or all of the data in the page. At the same time, after the computing node holds the write lock of the shared page, other computing nodes cannot modify, delete or write data in the shared page.

[0012] In a possible implementation of the first aspect, a read and write operation matching the operation type is performed on the shared page based on the log sequence number, including: when the operation type is a modification operation, checking whether the shared page has been persisted based on the log sequence number; if the shared page has been persisted, performing a modification operation on the shared page; if the shared page has not been persisted, waiting for the shared page to be persisted before performing a modification operation on the shared page.

[0013] It should be understood that the persistence processing of the shared page can refer to the previous node of the current computing node writing the processed data in the shared page to disk. Only after the previous node writes the data to disk, the current computing node can read and write the shared page; otherwise, it is necessary to wait for the previous node to complete the persistence processing of the page before reading and writing the shared page.

[0014] In a possible implementation of the first aspect, checking whether a shared page has been persistently processed is performed based on a log sequence number, including: obtaining a log sequence number formed by burning a log of a previous computing node that last modified the shared page; comparing the log sequence number formed by burning the log of the previous computing node with the log sequence number of the shared page; if the log sequence number formed by burning the log of the previous computing node is greater than or equal to the log sequence number of the shared page, determining that the shared page has been persistently processed; otherwise, determining that the shared page has not been persistently processed.

[0015] In one implementation, it is possible to determine whether the previous computing node has persisted the shared page by verifying whether the log sequence number formed by the previous computing node burning the log is greater than or equal to the log sequence number of the shared page. The log sequence number of the shared page includes a head sequence number and a tail sequence number. When comparing the log sequence number formed by the previous computing node burning the log with the log sequence number of the shared page, the log sequence number formed by the previous computing node burning the log can be compared with the head sequence number of the shared page, or the log sequence number formed by the previous computing node burning the log can be compared with the tail sequence number of the shared page.

[0016] In a possible implementation of the first aspect, after the shared page is modified, the method further includes: sequentially writing the header sequence number, the data in the modified shared page, and the tail sequence number into a remote page buffer pool.

[0017] In another possible implementation of the first aspect, performing read and write operations on a shared page that match the operation type based on the log sequence number also includes: when the operation type is a read operation, comparing the header sequence number and the tail sequence number in the log sequence number of the shared page; if the header sequence number and the tail sequence number are the same, reading the data in the shared page; if the header sequence number and the tail sequence number are different, reading the shared page by locking.

[0018] It should be understood that the read and write operations may refer to the page reading process for a shared page. Before reading the page, the computing node needs to verify the data in the shared page. For example, the verification can be performed by comparing whether the header sequence number and the tail sequence number in the log sequence number of the shared page are the same. If the two are the same, it means that the shared page verification has passed, and the computing node can read the data in the page. If one verification fails, the computing node can perform multiple verifications according to the above method. If multiple verifications fail, it means that the shared page has not passed the verification in the lock-free verification process. At this time, the computing node can try to lock the data in the page.

[0019] In a possible implementation of the first aspect, a shared page is read by locking, including: setting the lock information of the shared page to a read lock, and setting the holder of the read lock to the current computing node; after setting the read lock, reading the data in the shared page.

[0020] It should be understood that after adding a read lock to a shared page and holding the read lock, the current computing node can prevent other computing nodes from modifying the data in the page. In this way, only the current computing node can read and write data in the page. Therefore, the computing node can read the data in the page while holding the read lock of the page.

[0021] In a possible implementation of the first aspect, the method further includes: when a failure occurs in reading and writing a shared page based on lock information and a log sequence number, obtaining at least one cache page of the shared page from an associated node; and determining the latest version page of the shared page from the at least one cache page.

[0022] It should be understood that the associated node may be a computing node that shares the above-mentioned shared page, and the number of associated nodes may include multiple. After reading and writing the shared page, the multiple computing nodes may store the cache page of the shared page locally. In this way, when a computing node fails to read and write a page, the computing node may obtain the cache page from other associated nodes that cache the shared page, and determine the latest version page of the shared page from the cache page.

[0023] In a possible implementation of the first aspect, determining the latest version page of the shared page from at least one cache page includes: obtaining the log sequence number stored in each cache page; and determining the cache page corresponding to the maximum log sequence number as the latest version page of the shared page.

[0024] It should be understood that the number of associated nodes may include multiple, the number of buffered pages obtained by the computing node from the associated nodes may also include multiple, and the computing node needs to determine the latest version page from multiple buffered pages. In one implementation, the computing node can compare the log sequence numbers of each buffered page and determine the buffered page corresponding to the maximum log sequence number as the latest version page of the shared page.

[0025] In a possible implementation of the first aspect, obtaining at least one cache page of a shared page from an associated node includes: if any associated node fails, then after waiting for the failed associated node to return to normal, obtaining the cache page of the shared page from each associated node. That is, when a certain associated node fails, causing the computing node to be unable to obtain the cache page from the associated node, the computing node needs to wait until the failure of the associated node is eliminated before obtaining the cache page from the node. In this way, it can be ensured that the cache page that needs to be determined by comparing the logs is the real latest version of the page.

[0026] In a possible implementation of the first aspect, the page buffer pool exists in a shared memory based on a low-latency host interconnection protocol. The low-latency host interconnection protocol may include a remote direct memory access (RDMA) protocol, a CXL (compute express link, a cache coherent interconnection protocol for processors, memory extensions, and accelerators) protocol, and the like.

[0027] In a second aspect, a shared page reading and writing device is provided, comprising: a determination module, an acquisition module and a reading and writing module. The determination module is used to determine the shared page to be read and written, and the address of the shared page is shared among multiple computing nodes; the acquisition module is used to obtain the page information of the shared page from the page buffer pool according to the address, and the page information at least includes the lock information of the shared page and the log sequence number stored in the shared page; the reading and writing module is used to read and write the shared page based on the lock information and the log sequence number.

[0028] In a third aspect, a shared page reading and writing device is provided, which may be a computer device or a functional module in a computer device. The device may be used to implement the shared page reading and writing method described in the first aspect.

[0029] In a fourth aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for reading and writing a shared page as described in the first aspect is implemented.

[0030] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for reading and writing a shared page as described in the first aspect is implemented.

[0031] In a sixth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer can execute the shared page reading and writing method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of data sharing in the prior art;

[0033] Figure 2 It is a schematic diagram of a page reading and writing process in the prior art;

[0034] Figure 3 It is a schematic diagram of another page reading and writing process in the prior art;

[0035] Figure 4 It is a schematic diagram of the architecture of a database system to which a shared page reading and writing method provided in an embodiment of the present application is applicable;

[0036] Figure 5 is a schematic diagram of an auxiliary structure for recording page information of a shared page provided by an embodiment of the present application;

[0037] Figure 6 It is a schematic diagram of a fault handling process in the reading and writing process of a shared page provided by an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of the steps of a method for reading and writing a shared page provided in an embodiment of the present application;

[0039] Figure 8 It is a schematic diagram of a fault handling process in the reading and writing process of a shared page provided in an embodiment of the present application;

[0040] Fig. 9 It is a structural block diagram of a shared page reading and writing device provided in an embodiment of the present application;

[0041] Fig.10 It is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and the like are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first computing node, the second computing node, etc. are only used to distinguish different nodes in the system, and their number and execution order are not limited.

[0043] It should be noted that, in the embodiments of the present application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0044] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0045] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0046] The steps involved in the shared page reading and writing method provided in the embodiment of the present application are only examples. Not all steps must be executed, or not all contents in each step are mandatory. They can be increased or decreased as needed during use.

[0047] The same step or steps or contents with the same functions in different embodiments of the present application can be referenced to each other.

[0048] When the distributed page buffer pool is shared among multiple computing nodes, the overhead of obtaining pages through RPC is relatively large. These overheads include not only the overhead during network communication, but also the scheduling overhead of waiting and waking up threads. For example, Figure 2 The figure is a schematic diagram of a page reading and writing process. Specifically, Figure 2 The following is an example of a process for reading and writing pages in a relational database Oracle. Figure 2 (a) in the figure shows a process example of reading a page through RPC, that is, reading a page; Figure 2 (b) in the figure shows an example of the process of modifying a page through RPC, that is, writing a page.

[0049] Figure 2 The database system shown in (a) and (b) includes at least one storage unit, namely Figure 2 The shared storage 200 in (a) and (b), and multiple computing nodes, such as Figure 2 Node A, Node B, Node C, Node D in (a) and (b) of FIG. It should be understood that in the above database system, it may also include Figure 2 For other storage units and computing nodes not shown in (a) and (b), the embodiments of the present application do not limit the number of storage units and computing nodes included in a database system.

[0050] exist Figure 2 In (a), node D is a directory node, which can be used to manage the information of each shared page in the database. Figure 2 When node B in (a) requests to read a page, node B can send a lock request to the directory node. By sending a lock request, node B can inform the directory node of its request to read a certain page. This process is also the process of node B looking up the directory node to determine the actual current location of the page it wants to read. Assuming that the page that node B currently wants to read is located at node C, the directory node can send a ping command to node C to inform node C of node B's request to read the page. Then, node C can send the page that node B wants to read to node B. After obtaining the page, node B can send the segment addressing pseudo-instruction assume to the directory node to set the relationship between the page and node B. From Figure 2 It can be seen from the page reading process shown in (a) that when reading a page across nodes through RPC, processes such as page locking request and page cross-node transmission are involved.

[0051] exist Figure 2In (b), node D is the directory node, and node C is the page that initiates the modification request for a certain page. When node C expects to modify a certain page, node C can send a lock request to the directory node. The lock request is used to inform the directory node that node C expects to modify a certain page. By searching the directory node, the page that node C expects to modify is currently located at node A. The directory node can send a ping command to node A to inform node A of node C's request to modify the page. Then, node A can send the page that node C expects to modify to node C. After obtaining the page, node C sends an assume instruction to the directory node. From Figure 2 It can be seen from the page reading process shown in (b) that when modifying a page across nodes through RPC, it involves processes such as page lock requests and page owner transfers, which have a large overhead.

[0052] Depend on Figure 2 It can be seen from the page read and write processes shown in (a) and (b) that when shared pages are transmitted between multiple computing nodes through RPC, shared pages need to be frequently transferred between various computing nodes, which will bring a large communication overhead to the database system. In addition, frequent transfer of shared pages will also reduce the performance of the database system and affect the normal operation of the system.

[0053] The industry has explored a variety of ways to reduce the network and thread scheduling overhead when reading and writing pages. For example, reading and writing shared pages through unilateral operations. Figure 3 , which is a schematic diagram of another page reading and writing process. Specifically, Figure 3 The figure shows an example of a process of reading and writing pages in a FaRM database. The FaRM database is a database based on a shared nothing architecture, and can read and write remote memory data unilaterally based on a remote direct memory access (RDMA) protocol. Figure 3 The figure shows three nodes in the FaRM database, namely node 1, node 2 and node 3. Node 1 and node 3 can read and write pages in node 2. Since the FaRM database is designed based on the optimistic concurrency control (OCC) read and write method at the record level, in the case of a large number of conflicting write pages, the transaction will roll back and cannot be persisted to the disk. In addition, because the one-sided read and write technology of the FaRM database can only be used in the shared nothing architecture, the read and write process will generate distributed transactions, and there is a two-phase commit problem.

[0054] In response to the above problems, the embodiment of the present application provides a method for reading and writing shared pages, which can adopt low-latency host interconnection protocols such as RDMA. When multiple nodes in a data sharing architecture share read and write pages, each computing node unilaterally accesses the shared pages in the memory, avoiding frequent transfer of shared pages through RPC operations, and effectively improving the performance of multi-node conflict access to shared pages. In practice, using the read and write method provided in the embodiment of the present application to read and write shared pages can improve the performance of the database system by more than 30%.

[0055] like Figure 4 , which is a schematic diagram of the architecture of a database system to which a shared page reading and writing method provided in an embodiment of the present application is applicable. Figure 4 In the architecture shown, the distributed buffer includes page directory and page buffer pool management, so that each computing node can read and write the full amount of data in the shared memory. For example, Figure 4 Compute Node 1 and Compute Node 2 in the example can both read and write shared pages based on the distributed buffer. Figure 4 In the system architecture shown, the distributed buffer is equivalent to the support of the storage engine. The (structured query language, SQL) layer of each computing node is just a single-machine execution plan, which can avoid frequent transmission of shared pages between computing nodes through RPC and reduce the overhead of communication and thread scheduling.

[0056] In a database system, each computing node may include a SQL engine for responding to transactions initiated by an application. For example, an application initiates a transaction request, and the SQL engine in each computing node may process the transaction request. The SQL engine's processing of the transaction may involve reading and writing shared pages. For example, Figure 4 The application in initiates transactions 1, 2, 3, and 4, where computing node 1 can respond to and process transactions 1 and 2, and computing node 2 can respond to and process transactions 3 and 4. Computing node 1 and computing node 2 access the shared page through the heap (HEAP) and index (INDEX) in their own nodes.

[0057] Based on Figure 4 When the system architecture shown in the figure implements the reading and writing of shared pages, the owner mode of each shared page can be marked. Figure 4 In the system architecture shown, the owner mode of a page shared by multiple computing nodes can be marked as a shared memory mode, and the owner mode of each shared page can be recorded in the page directory of the page, and the page directory can also record information of multiple computing nodes that the page is allowed to be shared. Figure 5, which is a schematic diagram of an auxiliary structure for recording page information of a shared page provided by an embodiment of the present application. Figure 5 The information of pages in the database system that are allowed to be shared by multiple nodes for reading and writing is shown in FIG. 1 , that is, the information of shared page 1. The page directory of shared page 1 records that the owner mode of the page is the shared memory mode, and which computing nodes can share the page for reading and writing. For example, Figure 5 The shared page 1 whose owner mode is the shared memory mode can be shared and written by the computing node 1 and the computing node 2. Therefore, the address of the shared page 1 can be shared between the computing node 1 and the computing node 2.

[0058] In one possible implementation, Figure 5 As shown, Figure 4 The shared storage in can be an RDMA shared memory based on a low-latency host interconnection protocol. Therefore, the address of shared page 1 can represent the storage address of shared page 1 in the RDMA shared memory. In another possible implementation, Figure 4 The shared storage in can also be shared memory based on the CXL (Compute Express Link, a cache coherent interconnect protocol for processors, memory extensions, and accelerators) protocol.

[0059] like Figure 5 As shown, taking RDMA shared memory as an example, each shared page stored in the RDMA shared memory corresponds to a page information (page_info). Figure 5 The page information of the shared page 1 in the shared page includes the lock information (lock_bit), page ID (page_id), write ahead log (WAL) stream information of the modified page, etc. Among them, the WAL log stream information may include the write ahead log owner node ID (WAL_owner_node_id), the start timeline ID (WAL_owner_node_timeline_id) of the write ahead log owner node, and other information.

[0060] Each shared page stored in the RDMA shared memory also stores a log sequence number (LSN) in the page, and each log sequence number includes a head sequence number Head_LSN and a tail sequence number Tail_LSN.

[0061] based on Figure 5 The auxiliary structure shown, the shared page reading and writing method provided by the embodiment of the present application may include a page reading process and a page writing process. The page reading process and the page writing process are introduced below respectively.

[0062] Read page process:

[0063] 1. Read the header sequence number of the shared page: RDMA read page->tailLsn;

[0064] 2. Read the data stored in the shared page: RDMA read page content;

[0065] 3. Read the tail number of the shared page: RDMA read page->headerLsn;

[0066] 4. Compare the head number and the tail number:

[0067] if(page->headerLsn==page->tailLsn)

[0068] {

[0069] Read page OK;

[0070] }

[0071] In this step, comparing the header sequence number and the tail sequence number of the shared page can be regarded as a verification of the shared page. If the header sequence number and the tail sequence number are the same, it means that the page verification has passed, and the computing node has completed reading the data stored in the page. The above process of reading a page is also the process of reading a page without lock.

[0072] If the head sequence number and tail sequence number read by the computing node are different, it means that the lock-free read verification has failed. You can repeat the lock-free read multiple times to perform multiple verifications. When the verification passes, the computing node can read the data stored in the page. If multiple verifications still fail, it means that the lock-free read page verification has failed. In the case of a lock-free read verification failure, the computing node can try to perform a locked read.

[0073] 5. Lock and read;

[0074] When reading with lock, the computing node can set the lock information of the shared page to be read, that is, set the lock information of the shared page to read lock, and hold the read lock. Then, the computing node can read the data stored in the page again. After the reading is completed, the computing node can release the read lock. The process can be expressed as:

[0075] {

[0076] RDMA_Lock_page();

[0077] RDMA read page();

[0078] RDMA unlock_page();

[0079] }

[0080] Write page process:

[0081] 1. Set the lock information of the shared page: RDMA atomic lock bits and set lock owner (lockpage);

[0082] In this step, the computing node can set the lock information of the shared page to be read, that is, set the lock information to a write lock and hold the write lock. When a computing node holds the write lock of a page, the page can only be modified by the node holding the write lock, and cannot be modified by other nodes. In one possible implementation, setting the lock information of a shared page to a write lock can refer to setting the lock bit of the shared page to 1; correspondingly, setting the lock information of a shared page to a read lock can refer to setting the lock bit of the shared page to 0.

[0083] 2. Read the shared page and store it locally: RDMA read page_info, page and copy to local mem;

[0084] In this step, the computer reading the shared page may include reading the page information of the shared page and the data stored in the shared page. Then, the computing node may store the read data in the local memory.

[0085] 3. Check whether the page log has been flushed to disk: Check whether page owner node flush xlogdone;

[0086] In this step, checking whether the page log has been written to the disk means checking whether the previous node that modified the shared page has persisted the modified page. Only after the previous node completes the persistence of the page data, the next node can modify the data stored in the page.

[0087] Verifying whether the page log has been written to disk can be achieved by comparing the log sequence numbers:

[0088]

[0089] Specifically, the current computing node can compare the log sequence number xlog_owner_node->flush_lsn formed by the previous computing node that modified the shared page when burning the log with the log sequence number headerLsn of the shared page. If xlog_owner_node->flush_lsn is greater than or equal to headerLsn, it means that the shared page has been persisted and the current computing node can continue the subsequent page writing process. Otherwise, it means that the shared page has not been persisted and the current computing node needs to wait for the previous computing node to complete the persistence process before continuing the subsequent page writing process.

[0090] 4. Modify the shared page: Modify local mem;

[0091] In this step, the computing node can modify the data stored in the shared page according to actual needs. The above modification process can be completed in the local storage.

[0092] 5. Write the modified shared page header number to the remote end: RDMA write page->headerLsn;

[0093] 6. Write the modified shared page to the remote end: RDMA write page to remote page;

[0094] 7. Write the modified tail number of the shared page to the remote end: RDMA write page->taillsn;

[0095] After completing the modification of the shared page, the computing node may sequentially write the header sequence number, data in the shared page, and the tail sequence number of the modified shared page to the remote end, that is, write the above information into the page buffer pool in the shared memory.

[0096] 8. Set the lock information of the shared page: RDMA atomic reset lock bits and clean lockowner (unlock page).

[0097] After completing the write process of the shared page, the computing node can set the lock information of the page again. Setting the lock information this time may mean that the computing node releases the write lock it holds. Exemplarily, the computing node releasing the write lock of the shared page may mean changing the lock bit of the shared page from 1 to 0. After the write lock is released, the shared page can be read and written by other computing nodes.

[0098] It can be seen from the above read and write processes that in a data sharing architecture where storage and computing are separated, the distributed buffer pool in the database system can use a hybrid method of local memory and shared memory to optimize page reading and writing, improve performance, and eliminate the need to frequently transmit shared pages to be read and written between computing nodes through RPC.

[0099] Based on the above-mentioned shared page reading and writing process, the present application embodiment also provides a fault handling mechanism during the reading and writing process. Figure 6 , is a schematic diagram of a fault handling process in the reading and writing process of a shared page provided by an embodiment of the present application. When a computing node fails to access a shared page in a shared memory, the computing node can obtain the corresponding cache page from other nodes that cache the shared page. For example, Figure 6 When computing node 1 in accesses the RDMA shared memory, the RDMA shared memory reports an error. The computing node can obtain the cache page from other computing nodes that have the shared page cached. For example, Figure 6 Compute node 2 in the . By comparing the log sequence numbers of the cache pages obtained from other compute nodes, compute node 1 can determine the cache page corresponding to the maximum log sequence number as the latest version page of the shared page.

[0100] In a possible implementation of the embodiment of the present application, if a shared memory error occurs when a computing node accesses the shared memory, and other nodes that cache the shared pages to be read also fail, the current computing node should wait until the failures of other nodes are eliminated, and then obtain the pages cached by other nodes according to the above-mentioned fault handling mechanism. In this way, it can be ensured that the shared page finally determined according to the log sequence number of the cached page is the real latest version of the page in each computing node.

[0101] In combination with the various embodiments described above, Figure 7 FIG. 1 is a schematic diagram showing the steps of a method for reading and writing a shared page provided in an embodiment of the present application. The method may specifically include the following steps S701-S703:

[0102] S701. Determine a shared page to be read or written, where the address of the shared page is shared among multiple computing nodes.

[0103] In the embodiment of the present application, the shared page to be read or written may refer to a page shared by multiple computing nodes, on which a computing node currently expects to perform a read or write operation. Figure 4 , Figure 5 and Figure 6 Shared page 1 in.

[0104] The address of the shared page is shared among multiple computing nodes, which means that the address of the shared page in the shared memory is known by multiple computing nodes. For example, the computing nodes that can share the shared page 1 in the above example include Figure 4 , Figure 5 and Figure 6 Therefore, compute nodes 1 and 2 can know the address of shared page 1 in the shared memory. This address can be Figure 5 or Figure 6 The address in the RDMA shared memory.

[0105] S702: Obtain page information of the shared page from the page buffer pool according to the address of the shared page, where the page information at least includes lock information of the shared page and a log sequence number stored in the shared page.

[0106] According to the address of the shared page, the computing node can obtain the page information of the page from the page buffer pool. The page buffer pool can exist in the shared memory based on the low-latency host interconnection protocol. For example, the page buffer pool can exist in Figure 5 or Figure 6 In the RDMA shared memory.

[0107] In an embodiment of the present application, the page information of a shared page may include the lock information of the page and the log sequence number stored in the page. Figure 5 The page information (page_info) and the log sequence number LSN in the above log sequence number LSN may include the head sequence number Head_LSN and the tail sequence number Tail_LSN in the above embodiments.

[0108] S703: Read and write the shared page based on the lock information and the log sequence number.

[0109] The reading and writing method for a shared page provided in the embodiment of the present application can be implemented based on the acquired lock information and log sequence number of the shared page.

[0110] In a possible implementation, the type of operation of reading and writing a shared page may be determined first, and the operation type may include a read operation or a modify operation. The read operation of a shared page may refer to the operation of reading a shared page in the aforementioned embodiments; the modify operation of a shared page may refer to the operation of writing a shared page in the aforementioned embodiments.

[0111] Then, the computing node can set the lock information of the shared page according to the operation type. When the operation type is a modification operation, the computing node can set the lock information of the shared page to a write lock, and set the holder of the write lock to the current computing node. Specifically, the computing node can set the lock_bit in the page information to 1 in the manner described in the above embodiments.

[0112] After setting the lock information, the computing node performs a read or write operation on the shared page that matches the above operation type based on the log sequence number of the shared page, for example, performing a read process or a write process on the shared page.

[0113] In an embodiment of the present application, when the operation type for a shared page is a modification operation, the computing node can verify whether the shared page has been persisted based on the log sequence number. If the shared page has been persisted, the computing node can perform a modification operation on the shared page; if the shared page has not been persisted, the computing node needs to wait for the shared page to be persisted before performing a modification operation on it. Verifying whether a shared page has been persisted can refer to the verification introduced in the aforementioned embodiments, that is, whether the previous computing node that modified the shared page has written the log formed after modifying the page to the disk.

[0114] In a possible implementation, the computing node may verify whether the shared page has been persisted by comparing the log sequence numbers as described in the above embodiments.

[0115] Exemplarily, the computing node may obtain the log sequence number xlog_owner_node->flush_lsn formed by the last computing node burning log that last modified the shared page, and compare xlog_owner_node->flush_lsn with the log sequence number headerLsn of the shared page. If xlog_owner_node->flush_lsn is greater than or equal to headerLsn, it can be determined that the shared page has been persistently processed; otherwise, it is determined that the shared page has not been persistently processed.

[0116] After determining that the shared page has been persisted, the computing node can modify the data in the shared page. After completing the modification of the shared page, the computing node can sequentially write the header sequence number, data in the shared page, and tail sequence number of the modified shared page into the remote page buffer pool. Figure 5 or Figure 6 In the RDMA shared memory.

[0117] In the embodiment of the present application, when the operation type for the shared page is a read operation, the computing node shall verify the shared page according to the steps in the page read process described in the above embodiments. The above verification process can be implemented by comparing the header sequence number headerLsn and the tail sequence number tailLsn in the log sequence number of the shared page.

[0118] If the header sequence number headerLsn and the tail sequence number tailLsn are the same, it means that the shared page verification has passed, and the computing node can read the data in the shared page. If the header sequence number headerLsn and the tail sequence number tailLsn are different, it means that the shared page verification has not passed, and the computing node can continue to verify in the above manner. If it fails after multiple verifications, the computing node can try to read the shared page by locking it.

[0119] When reading a shared page by locking, the computing node can set the lock information of the shared page to a read lock and set the holder of the read lock to the current computing node. Then, after setting the read lock, read the data in the shared page. When the computing node completes reading the shared page, the computing node can release the read lock.

[0120] In a possible implementation, a fault may occur in the process of reading and writing a shared page using the methods in the above embodiments. The fault may occur in the shared memory or in one or more computing nodes that share a certain page. The present application also provides a method for handling the fault. Figure 8 FIG. 8 is a schematic diagram showing a fault handling process in a shared page reading and writing process provided by an embodiment of the present application. The process may specifically include the following steps S801-S802:

[0121] S801. When a failure occurs in reading and writing a shared page based on lock information and a log sequence number, at least one cache page of the shared page is obtained from an associated node, where the associated node is a computing node that shares the shared page.

[0122] In one possible implementation, the fault may occur in the shared memory, e.g. Figure 6 The RDMA shared memory shown in FIG. 1 fails. At this time, a computing node that needs to read a shared page from the shared memory can obtain at least one cache page of the shared page from an associated node. The above-mentioned associated node can be other computing nodes in the database system that share the same page with the computing node. The number of associated nodes can include one or more.

[0123] When the shared memory fails, the compute nodes cannot fetch shared pages from the shared memory. For example, Figure 6After the RDMA shared memory in the RAID fails, compute node 1 cannot obtain shared pages from the RDMA shared memory. At this time, compute node 1 can obtain cache pages from associated nodes such as compute node 2.

[0124] In another possible implementation, the associated node may also fail. For example, the computing node that currently needs to read the shared page is Figure 6 Compute node 1 in the figure, and compute node 2 is the associated node of compute node 1. Figure 6 The computing node 2 in the network may also fail. In this case, it is necessary to wait for the associated nodes to recover before obtaining the cached pages of the shared pages from computing node 2. That is, if any associated node fails, it is necessary to wait for the failed associated node to recover before obtaining the cached pages of the shared pages from each associated node.

[0125] S802: Determine the latest version page of the shared page from at least one cached page.

[0126] In an embodiment of the present application, determining the latest version page of a shared page from a cache page can be achieved by comparing the log sequence numbers of each cache page as described in the aforementioned embodiments.

[0127] Specifically, after obtaining the cache pages cached in each associated node, the computing node can obtain the log sequence number LSN stored in each cache page, and then determine the cache page corresponding to the maximum value of the log sequence number LSN as the latest version page of the shared page.

[0128] In a possible implementation, the log sequence number LSN includes a header sequence number headerLsn and a tail sequence number tailLsn, and the header sequence number headerLsn and the tail sequence number tailLsn in the cache page should be the same. Therefore, when comparing the log sequence number LSNs of each cache page, either the header sequence number headerLsn or the tail sequence number tailLsn can be compared.

[0129] After determining the latest version of the shared page, the computing node can read and write the latest version of the page according to the reading and writing process described above.

[0130] Corresponding to the above embodiments, refer to Fig. 9 , shows a structural block diagram of a shared page reading and writing device provided in an embodiment of the present application, the device can be used to implement the steps in the above-mentioned various method embodiments, and the device can specifically include the following modules: a determination module 901, an acquisition module 902 and a reading and writing module 903, wherein:

[0131] A determination module 901 is used to determine a shared page to be read or written, where the address of the shared page is shared among multiple computing nodes;

[0132] An acquisition module 902 is used to acquire page information of the shared page from a page buffer pool according to the address, wherein the page information at least includes lock information of the shared page and a log sequence number stored in the shared page;

[0133] The read / write module 903 is used to read and write the shared page based on the lock information and the log sequence number.

[0134] In a possible implementation of an embodiment of the present application, the read-write module 903 can be specifically used to: determine the operation type of reading and writing the shared page, the operation type including a read operation or a modify operation; set the lock information of the shared page according to the operation type; after setting the lock information, perform a read-write operation on the shared page that matches the operation type based on the log sequence number.

[0135] In a possible implementation of an embodiment of the present application, the read-write module 903 can also be used to: when the operation type is the modification operation, set the lock information of the shared page to a write lock, and set the holder of the write lock to the current computing node.

[0136] In a possible implementation method of an embodiment of the present application, the read-write module 903 can also be used for: when the operation type is the modification operation, verifying whether the shared page has been persisted based on the log sequence number; if the shared page has been persisted, performing a modification operation on the shared page; if the shared page has not been persisted, waiting for the shared page to be persisted before performing a modification operation on the shared page.

[0137] In a possible implementation method of an embodiment of the present application, the read-write module 903 can also be used to: obtain the log sequence number formed by the previous computing node burning log that last modified the shared page; compare the log sequence number formed by the previous computing node burning log with the log sequence number of the shared page; if the log sequence number formed by the previous computing node burning log is greater than or equal to the log sequence number of the shared page, it is determined that the shared page has been persisted; otherwise, it is determined that the shared page has not been persisted.

[0138] In a possible implementation of the embodiment of the present application, the read-write module 903 may also be used to: write the header sequence number, the modified data in the shared page, and the tail sequence number into the remote page buffer pool in sequence.

[0139] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used for: when the operation type is the read operation, comparing the header sequence number and the tail sequence number in the log sequence number of the shared page; if the header sequence number and the tail sequence number are the same, reading the data in the shared page; if the header sequence number and the tail sequence number are different, reading the shared page by locking.

[0140] In another possible implementation method of the embodiment of the present application, the read-write module 903 can also be used to: set the lock information of the shared page to a read lock, and set the holder of the read lock to the current computing node; after setting the read lock, read the data in the shared page.

[0141] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used for: when a failure occurs in reading and writing the shared page based on the lock information and the log sequence number, obtaining at least one cache page of the shared page from an associated node, where the associated node is a computing node that shares the shared page; and determining the latest version page of the shared page from at least one of the cache pages.

[0142] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used to: obtain the log sequence number stored in each of the cache pages; and determine the cache page corresponding to the maximum value of the log sequence number as the latest version page of the shared page.

[0143] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used for: if any of the associated nodes fails, then after waiting for the failed associated node to recover to normal, obtaining the cache page of the shared page from each of the associated nodes.

[0144] In an embodiment of the present application, the page buffer pool exists in a shared memory based on a low-latency host interconnection protocol.

[0145] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0146] Reference Fig.10 , shows a schematic diagram of a computer device provided by an embodiment of the present application. Fig.10As shown, the computer device 1000 in the embodiment of the present application includes: a processor 1010, a memory 1020, and a computer program 1021 stored in the memory 1020 and executable on the processor 1010. When the processor 1010 executes the computer program 1021, the steps in each embodiment of the above-mentioned shared page reading and writing method are implemented, for example Figure 7 Alternatively, when the processor 1010 executes the computer program 1021, the functions of each module / unit in the above-mentioned device embodiments are realized, for example Fig. 9 The functions of the determination module 901, the acquisition module 902 and the reading and writing module 903 are shown.

[0147] Exemplarily, the computer program 1021 may be divided into one or more modules / units, which are stored in the memory 1020 and executed by the processor 1010 to complete the present application. The one or more modules / units may be a series of computer program instruction segments that can complete specific functions, and the instruction segments may be used to describe the execution process of the computer program 1021 in the computer device 1000. For example, the computer program 1021 may be divided into Fig. 9 The determination module 901, the acquisition module 902 and the reading and writing module 903 are shown.

[0148] The computer device 1000 may be a device for implementing each step in the above-mentioned method embodiments, and the computer device 1000 may be a computing device such as a desktop computer, a cloud server, etc. The computer device 1000 may include, but is not limited to, a processor 1010 and a memory 1020. Those skilled in the art will understand that Fig.10 This is only an example of the computer device 1000 and does not constitute a limitation of the computer device 1000. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device 1000 may also include input and output devices, network access devices, buses, etc.

[0149] The processor 1010 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0150] The memory 1020 may be an internal storage unit of the computer device 1000, such as a hard disk or memory of the computer device 1000. The memory 1020 may also be an external storage device of the computer device 1000, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device 1000. Further, the memory 1020 may also include both an internal storage unit of the computer device 1000 and an external storage device. The memory 1020 is used to store the computer program 1021 and other programs and data required by the computer device 1000. The memory 1020 may also be used to temporarily store data that has been output or is to be output.

[0151] Finally, it should be noted that the above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application.

Claims

1. A method for reading and writing a shared page, characterized in that: include: Determine a shared page to be read and written, where an address of the shared page is shared among multiple computing nodes; According to the address, obtaining page information of the shared page from a page buffer pool, the page information at least including lock information of the shared page and a log sequence number stored in the shared page; Based on the lock information and the log sequence number, the shared page is read and written.

2. The method according to claim 1, characterized in that The reading and writing of the shared page based on the lock information and the log sequence number includes: Determine an operation type of reading and writing the shared page, the operation type including a read operation or a modify operation; Setting lock information of the shared page according to the operation type; After the lock information is set, a read and write operation matching the operation type is performed on the shared page based on the log sequence number.

3. The method according to claim 2, characterized in that The setting the lock information of the shared page according to the operation type includes: When the operation type is the modification operation, the lock information of the shared page is set to a write lock, and the holder of the write lock is set to the current computing node.

4. The method according to claim 2 or 3, characterized in that: The performing a read and write operation matching the operation type on the shared page based on the log sequence number includes: When the operation type is the modification operation, verifying whether the shared page is persistently processed based on the log sequence number; If the shared page has been persisted, modify the shared page; If the shared page has not been processed persistently, the shared page is modified after waiting for the shared page to be processed persistently.

5. The method according to claim 4, characterized in that The checking whether the shared page is persistently processed based on the log sequence number includes: Obtaining a log sequence number formed by a previous computing node burning log that last modified the shared page; Compare the log sequence number formed by burning the log of the previous computing node with the log sequence number of the shared page; If the log sequence number formed by the previous computing node burning log is greater than or equal to the log sequence number of the shared page, it is determined that the shared page has been persistently processed; otherwise, it is determined that the shared page has not been persistently processed.

6. The method according to claim 4 or 5, characterized in that: The log sequence number of the shared page includes a head sequence number and a tail sequence number. After the shared page is modified, the method further includes: The head sequence number, the modified data in the shared page and the tail sequence number are sequentially written into the page buffer pool at the remote end.

7. The method according to claim 2, characterized in that The performing a read and write operation matching the operation type on the shared page based on the log sequence number includes: When the operation type is the read operation, comparing the head sequence number and the tail sequence number in the log sequence number of the shared page; If the head sequence number and the tail sequence number are the same, reading the data in the shared page; If the head sequence number and the tail sequence number are different, the shared page is read in a locked manner.

8. The method according to claim 7, characterized in that The reading of the shared page by locking the shared page includes: Set the lock information of the shared page to a read lock, and set the holder of the read lock to the current computing node; After setting the read lock, the data in the shared page is read.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: When a failure occurs in reading and writing the shared page based on the lock information and the log sequence number, obtaining at least one cache page of the shared page from an associated node, the associated node being a computing node that shares the shared page; Determine the latest version page of the shared page from at least one of the cached pages.

10. The method according to claim 9, characterized in that Determining the latest version page of the shared page from at least one of the cached pages includes: Obtaining the log sequence number stored in each of the cache pages; The cache page corresponding to the maximum value of the log sequence number is determined as the latest version page of the shared page.

11. The method according to claim 9 or 10, characterized in that: The acquiring at least one cache page of the shared page from the associated node comprises: If any of the associated nodes fails, then after waiting for the failed associated node to recover, the cached pages of the shared page are obtained from each of the associated nodes.

12. The method according to any one of claims 1 to 11, characterized in that: The page buffer pool exists in a shared memory based on a low-latency host interconnection protocol.

13. A database system, comprising a database and a plurality of computing nodes, wherein the plurality of computing nodes share pages in the database, characterized in that: Any of the computing nodes reads and writes pages in the database using the shared page reading and writing method as described in any of claims 1-12.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for reading and writing a shared page as described in any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the shared page reading and writing method according to any one of claims 1 to 12.

Images