Read-write method for shared page, database system and computer device

Abstract

Embodiments of the present application are applicable to the technical field of databases, and provide a read-write method for a shared page, a database system and a computer device. By applying the method, an owner mode of the shared page is marked as a shared memory mode in a page directory, so that when a plurality of nodes under a data sharing architecture share a read-write page, each computing node unilaterally accesses the shared page in a memory, which avoids the frequent transfer of the shared page by means of remote procedure call, and effectively improves the performance of multi-node conflict access to the shared page. Using the read-write method to read and write the shared page can improve the performance of the database system by more than 30%.

Description

Shared page reading and writing method, database system and computer equipment

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on August 29, 2023, with application number 202311101888.3 and entitled “Shared Page Reading and Writing Method, Database System and Computer Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of database technology, and in particular to a shared page reading and writing method, a database system, and a computer device. Background Art

[0003] Data sharing is a technology that enables data to be exchanged or shared across multiple applications, services, and devices. In a data sharing architecture, data storage and computation are separated. Every compute node in the system has access to read and write pages. The page buffer pool is shared across multiple compute nodes, giving each compute node access to the full data set.

[0004] As shown in Figure 1, it is a schematic diagram of data sharing in the prior art. Figure 1 shows a distributed page buffer pool, in which the cached pages are derived from shared storage and can be shared by multiple nodes. For example, the page buffer pool can be shared by node 1, node 2, and node 3 in Figure 1. Node 1, node 2, and node 3 can all read and write pages in the page buffer pool. In the prior art, when a page buffer pool is shared among multiple computing nodes, buffer pages can be transmitted via remote procedure call (RPC). However, the overhead of obtaining pages via RPC is relatively high. How to adopt a new method to reduce the overhead of nodes when reading and writing shared pages is a technical problem that urgently needs to be solved.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a shared page reading and writing method, a database system, and a computer device, which can be used to solve the problem of high overhead of buffering pages transmitted through RPC in the prior art.

[0007] To achieve the above objectives, the present invention adopts the following technical solutions:

[0008] In a first aspect, a method for reading and writing a shared page is provided, comprising: determining a shared page to be read or written, where the address of the shared page can be shared among multiple computing nodes. Based on the address of the shared page, the computing node can obtain page information of the shared page from a page buffer pool, where the page information includes at least lock information of the shared page and a log sequence number stored in the shared page. The computing node can read and write the shared page based on the obtained lock information and log sequence number.

[0009] It should be understood that under the data sharing architecture, multiple computing nodes can access the page buffer pool and obtain shared pages from the page buffer pool. The embodiment of the present application stores the lock information and log sequence number and other page information of the shared page in the page buffer pool, so that each computing node can directly obtain the shared page and its corresponding page information from the page buffer pool based on the address of the shared page when reading and writing the shared page, without the need to transfer the shared page to be read and written and its page information among multiple computing nodes through processes such as RPC. In this way, the communication overhead caused by the transmission of shared pages by multiple computing nodes and the overhead of thread calls can be reduced, the computing performance of each computing node in the database system can be improved, and the normal operation of the database system can be guaranteed.

[0010] In a possible implementation of the first aspect, shared pages are read and written based on lock information and log sequence numbers, including: determining an operation type for reading and writing the shared page, where the operation type includes a read operation or a modify operation; setting lock information for the shared page according to the operation type; and after setting the lock information, performing read and write operations on the shared page that match the operation type based on the log sequence number.

[0011] It should be understood that the operation type can be determined based on specific transaction requirements, including read operations or modify operations. Among them, a read operation can refer to reading data or information in a shared page, that is, a read page operation; a modify operation can refer to modifying data or information in a shared page, that is, a write page operation. After determining the operation type on the shared page, the computing node can set the lock information of the shared page. For example, when a page needs to be written, the shared page is locked. Then, based on the set lock information, the corresponding read and write operations are performed.

[0012] In a possible implementation of the first aspect, the lock information of the shared page is set according to the operation type, including: when the operation type is a modification operation, setting the lock information of the shared page to a write lock, and setting the holder of the write lock to the current computing node.

[0013] It should be understood that when the operation type is a modification operation, that is, when a compute node needs to perform a write operation on a shared page, the compute node must hold the write lock on the shared page. When a compute node holds the write lock on a shared page, it can modify the data in the shared page, write data to the page, or delete some or all of the data in the page. At the same time, after the compute node holds the write lock on the shared page, other compute nodes cannot modify, delete, or write data in the shared page.

[0014] In a possible implementation of the first aspect, read and write operations that match the operation type are performed on the shared page based on the log sequence number, including: when the operation type is a modification operation, checking whether the shared page has been persisted based on the log sequence number; if the shared page has been persisted, performing a modification operation on the shared page; if the shared page has not been persisted, waiting for the shared page to be persisted before performing a modification operation on the shared page.

[0015] It should be understood that persisting a shared page can mean that the previous node of the current compute node writes the processed data in the shared page to disk. Only after the previous node writes the data to disk can the current compute node read and write the shared page; otherwise, it must wait until the previous node completes the persistence processing before reading and writing the shared page.

[0016] In a possible implementation of the first aspect, verifying whether a shared page has been persisted is performed based on a log sequence number, including: obtaining a log sequence number formed by burning a log of a previous computing node that last modified the shared page; comparing the log sequence number formed by burning the log of the previous computing node with the log sequence number of the shared page; if the log sequence number formed by burning the log of the previous computing node is greater than or equal to the log sequence number of the shared page, determining that the shared page has been persisted; otherwise, determining that the shared page has not been persisted.

[0017] In one implementation, it is possible to determine whether the previous computing node has persisted the shared page by verifying whether the log sequence number formed by the previous computing node burning the log is greater than or equal to the log sequence number of the shared page. The log sequence number of the shared page includes a head sequence number and a tail sequence number. When comparing the log sequence number formed by the previous computing node burning the log with the log sequence number of the shared page, the log sequence number formed by the previous computing node burning the log can be compared with the head sequence number of the shared page, or the log sequence number formed by the previous computing node burning the log can be compared with the tail sequence number of the shared page.

[0018] In a possible implementation of the first aspect, after modifying the shared page, the method further includes: sequentially writing the header sequence number, the data in the modified shared page, and the tail sequence number into a remote page buffer pool.

[0019] In another possible implementation of the first aspect, performing read and write operations on a shared page that match the operation type based on the log sequence number further includes: when the operation type is a read operation, comparing the header sequence number and the tail sequence number in the log sequence number of the shared page; if the header sequence number and the tail sequence number are the same, reading the data in the shared page; if the header sequence number and the tail sequence number are different, reading the shared page by locking.

[0020] It should be understood that the read and write operations may refer to the page reading process for a shared page. Before reading the page, the computing node needs to verify the data in the shared page. For example, the verification can be performed by comparing whether the header sequence number and the tail sequence number in the log sequence number of the shared page are the same. If the two are the same, it means that the shared page verification has passed, and the computing node can read the data in the page. If the verification fails once, the computing node can perform multiple verifications according to the above method. If multiple verifications fail, it means that the shared page has failed the verification in the lock-free verification process. At this time, the computing node can try to read the data in the page by locking.

[0021] In a possible implementation of the first aspect, a shared page is read by locking it, including: setting the lock information of the shared page to a read lock, and setting the holder of the read lock to the current computing node; after setting the read lock, reading the data in the shared page.

[0022] It should be understood that after applying a read lock on a shared page and holding the read lock, the current compute node can prevent other compute nodes from modifying the data in the page. Thus, only the current compute node can read and write data in the page. Therefore, a compute node can read data in a page while holding the read lock on the page.

[0023] In a possible implementation of the first aspect, the above method also includes: when a failure occurs in reading and writing a shared page based on lock information and log sequence number, obtaining at least one cache page of the shared page from an associated node; and determining the latest version page of the shared page from the at least one cache page.

[0024] It should be understood that the associated nodes may be computing nodes that share the aforementioned shared page, and the number of associated nodes may include multiple. After reading and writing a shared page, the multiple computing nodes may locally store cached pages of the shared page. In this way, if a computing node encounters a failure in reading or writing a page, the computing node can obtain the cached page from another associated node that has the shared page cached, and determine the latest version of the shared page from the cached page.

[0025] In a possible implementation of the first aspect, determining the latest version page of the shared page from at least one cache page includes: obtaining the log sequence number stored in each cache page; and determining the cache page corresponding to the maximum log sequence number as the latest version page of the shared page.

[0026] It should be understood that there may be multiple associated nodes, and the number of cached pages retrieved by the computing node from the associated nodes may also include multiple, and the computing node needs to determine the latest version of the page from the multiple cached pages. In one implementation, the computing node may compare the log sequence numbers of the cached pages and determine the cached page corresponding to the maximum log sequence number as the latest version of the shared page.

[0027] In one possible implementation of the first aspect, obtaining at least one cached page of a shared page from associated nodes includes: if any associated node fails, then after waiting for the failed associated node to recover, obtaining cached pages of the shared page from each associated node. That is, if a failure occurs in an associated node, preventing a computing node from obtaining a cached page from that associated node, the computing node must wait until the failure of the associated node is resolved before obtaining a cached page from that node. This ensures that the cached page to be determined through log comparison is the true latest version of the page.

[0028] In one possible implementation of the first aspect, the page buffer pool exists in a shared memory based on a low-latency host interconnect protocol. The low-latency host interconnect protocol may include a remote direct memory access (RDMA) protocol, a compute express link (CXL) protocol, a cache coherent interconnect protocol for processors, memory extensions, and accelerators, and the like.

[0029] In a second aspect, a shared page reading and writing device is provided, comprising: a determination module, an acquisition module, and a reading and writing module. The determination module is configured to determine a shared page to be read or written, where the address of the shared page is shared among multiple computing nodes; the acquisition module is configured to obtain page information of the shared page from a page buffer pool based on the address; the page information includes at least lock information of the shared page and a log sequence number stored in the shared page; and the reading and writing module is configured to read and write the shared page based on the lock information and the log sequence number.

[0030] In a third aspect, a device for reading and writing a shared page is provided, which may be a computer device or a functional module in the computer device. The device can be used to implement the method for reading and writing a shared page as described in the first aspect.

[0031] In a fourth aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for reading and writing shared pages as described in the first aspect above is implemented.

[0032] In a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for reading and writing a shared page as described in the first aspect is implemented.

[0033] In a sixth aspect, a computer program product is provided. When the computer program product is run on a computer, the computer can execute the shared page reading and writing method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram of data sharing in the prior art;

[0035] FIG2 is a schematic diagram of a page reading and writing process in the prior art;

[0036] FIG3 is a schematic diagram of another page reading and writing process in the prior art;

[0037] FIG4 is a schematic diagram of the architecture of a database system to which a shared page reading and writing method provided in an embodiment of the present application is applicable;

[0038] FIG5 is a schematic diagram of an auxiliary structure for recording page information of a shared page provided in an embodiment of the present application;

[0039] FIG6 is a schematic diagram of a fault handling process during the reading and writing of a shared page provided by an embodiment of the present application;

[0040] FIG7 is a schematic diagram of the steps of a method for reading and writing a shared page provided in an embodiment of the present application;

[0041] FIG8 is a schematic diagram of a fault handling process during the reading and writing of a shared page provided by an embodiment of the present application;

[0042] FIG9 is a structural block diagram of a shared page reading and writing device provided in an embodiment of the present application;

[0043] FIG10 is a schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] To facilitate a clear description of the technical solutions of the embodiments of the present application, the terms "first" and "second" are used in the embodiments of the present application to distinguish between identical or similar items with substantially the same functions and effects. For example, the terms "first computing node," "second computing node," and so on are merely used to distinguish between different nodes in the system and do not limit their number or execution order.

[0045] It should be noted that in the embodiments of this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplarily" or "for example" is intended to present the relevant concepts in a concrete manner.

[0046] The business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field will know that with the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0047] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0048] The steps involved in the shared page reading and writing method provided in the embodiment of the present application are only examples. Not all steps must be executed, or not all contents in each step are mandatory. They can be increased or decreased as needed during use.

[0049] The same step or steps or contents with the same function in the embodiments of the present application can be referenced to each other in different embodiments.

[0050] When a distributed page buffer pool is shared among multiple computing nodes, the overhead of obtaining pages through RPC is relatively large. These overheads include not only the overhead during network communication, but also the scheduling overhead of thread waiting and waking up. For example, as shown in FIG2 , it is a schematic diagram of a page reading and writing process. Specifically, FIG2 shows an example of a process for reading and writing pages in a relational database Oracle. Among them, FIG2 (a) shows an example of a process of reading a page through RPC, that is, a process of reading a page; FIG2 (b) shows an example of a process of modifying a page through RPC, that is, a process of writing a page.

[0051] The database system shown in (a) and (b) of FIG2 includes at least one storage unit, namely, the shared storage 200 in (a) and (b) of FIG2 , and multiple computing nodes, such as Node A, Node B, Node C, and Node D in (a) and (b) of FIG2 . It should be understood that the above database system may further include other storage units and computing nodes not shown in (a) and (b) of FIG2 , and the embodiments of the present application do not limit the number of storage units and computing nodes included in a database system.

[0052] In Figure 2 (a), node D is a directory node, which manages information about shared pages in the database. When node B in Figure 2 (a) requests to read a page, node B can send a lock request to the directory node. By sending a lock request, node B informs the directory node of its request to read a specific page. This process involves node B searching the directory node to determine the actual current location of the page it wishes to read. Assuming that the page node B currently wishes to read is located on node C, the directory node can send a ping command to node C, informing node C of node B's request to read the page. Node C can then send the page node B to node B. After obtaining the page, node B can send the segment addressing pseudo-instruction "assume" to the directory node to establish the relationship between the page and node B. The page read process shown in Figure 2 (a) shows that cross-node page reads via RPC involve processes such as page lock requests and cross-node page transfers.

[0053] In (b) of Figure 2, node D is a directory node, and node C is the page that initiates a modification request for a certain page. When node C expects to modify a certain page, node C can send a lock request to the directory node. The lock request is used to inform the directory node that node C expects to modify a certain page. By searching the directory node, the page that node C expects to modify is currently located at node A. The directory node can then send a ping command to node A, thereby informing node A of node C's request to modify the page. Node A can then send the page that node C expects to modify to node C. After obtaining the page, node C sends an assume instruction to the directory node. It can be seen from the page reading process shown in (b) of Figure 2 that when modifying a page across nodes through RPC, processes such as page lock requests and page owner transfers are involved, which have a large overhead.

[0054] As shown in Figure 2 (a) and (b), the page read and write processes indicate that when shared pages are transferred between multiple compute nodes via RPC, they need to be frequently transferred between the nodes, which incurs significant communication overhead for the database system. Furthermore, frequent transfers of shared pages can degrade database system performance and affect normal system operation.

[0055] The industry has explored various ways to reduce network and thread scheduling overhead when reading and writing pages. For example, shared pages can be read and written through unilateral operations. Figure 3 shows a schematic diagram of another page read and write process. Specifically, Figure 3 shows an example of the process for reading and writing pages in the FaRM database. The FaRM database is a database based on a share-nothing architecture that can unilaterally read and write remote memory data based on the remote direct memory access (RDMA) protocol. Figure 3 shows three nodes in the FaRM database, namely node 1, node 2, and node 3. Node 1 and node 3 can read and write pages in node 2. Because the FaRM database is designed based on record-level optimistic concurrency control (OCC) read and write methods, in the case of a large number of conflicting page writes, the transaction will roll back and disk persistence will not be achieved. In addition, because the above-mentioned unilateral read and write technology based on the FaRM database can only be used in a share-nothing architecture, the read and write process will generate distributed transactions, and there is a two-phase commit problem.

[0056] To address the above issues, embodiments of the present application provide a method for reading and writing shared pages. This method can utilize low-latency host interconnection protocols such as RDMA. When multiple nodes in a data sharing architecture share read and write pages, each computing node unilaterally accesses the shared pages in memory, avoiding frequent transfers of shared pages through RPC operations and effectively improving the performance of conflicting multi-node access to shared pages. In practice, using the read and write method provided in embodiments of the present application to read and write shared pages can improve database system performance by over 30%.

[0057] As shown in Figure 4, it is a schematic diagram of the architecture of a database system to which a shared page reading and writing method provided in an embodiment of the present application is applicable. In the architecture shown in Figure 4, the distributed buffer includes a page directory and a page buffer pool management, so that each computing node can read and write the full amount of data in the shared memory. For example, computing node 1 and computing node 2 in Figure 4 can both read and write shared pages based on the distributed buffer. Based on the system architecture shown in Figure 4, the distributed buffer is equivalent to being supported by the storage engine, and the (structured query language, SQL) layer of each computing node is only a stand-alone execution plan, which can avoid frequently transmitting shared pages between computing nodes through RPC, reducing the overhead of communication and thread scheduling.

[0058] In a database system, each compute node may include a SQL engine for responding to transactions initiated by an application. For example, an application initiates a transaction request, and the SQL engine in each compute node may process the transaction request. The SQL engine's transaction processing may involve reading and writing shared pages. For example, the application in Figure 4 initiates transactions 1, 2, 3, and 4, where compute node 1 can respond to and process transactions 1 and 2, and compute node 2 can respond to and process transactions 3 and 4. Compute nodes 1 and 2 access shared pages through the heap and index in their respective nodes.

[0059] When implementing the reading and writing of shared pages based on the system architecture shown in Figure 4, the owner mode of each shared page can be marked. In the system architecture shown in Figure 4, the owner mode of the page shared by multiple computing nodes can be marked as shared memory mode, and the owner mode of each shared page can be recorded in the page directory of the page. The page directory can also record information about the multiple computing nodes that the page is allowed to be shared. As shown in Figure 5, a schematic diagram of an auxiliary structure for recording page information of a shared page provided by an embodiment of the present application is shown. Figure 5 shows the information of a page in the database system that is allowed to be shared by multiple nodes for reading and writing, that is, the information of shared page 1. The page directory of shared page 1 records that the owner mode of the page is shared memory mode, and which computing nodes can share the page for reading and writing. For example, the shared page 1 in Figure 5, whose owner mode is shared memory mode, can be shared by computing node 1 and computing node 2 for reading and writing. Therefore, the address of shared page 1 can be shared between the above-mentioned computing node 1 and computing node 2.

[0060] In one possible implementation, as shown in Figure 5 , the shared storage in Figure 4 can be RDMA shared memory based on the low-latency host interconnect protocol. Therefore, the address of shared page 1 can represent the storage address of shared page 1 in the RDMA shared memory. In another possible implementation, the shared storage in Figure 4 can also be shared memory based on the CXL (Compute Express Link) protocol, a cache coherent interconnect protocol for processors, memory extensions, and accelerators.

[0061] As shown in Figure 5, taking RDMA shared memory as an example, each shared page stored in the RDMA shared memory corresponds to a corresponding page information (page_info). For example, the page information of shared page 1 in Figure 5 includes the shared page's lock information (lock_bit), page ID (page_id), and write-ahead log (WAL) stream information for the modified page. The WAL log stream information can include information such as the write-ahead log owner node ID (WAL_owner_node_id) and the startup timeline ID of the write-ahead log owner node (WAL_owner_node_timeline_id).

[0062] Each shared page stored in the RDMA shared memory also stores a log sequence number (LSN). Each log sequence number includes a head sequence number Head_LSN and a tail sequence number Tail_LSN.

[0063] Based on the auxiliary structure shown in Figure 5, the shared page reading and writing method provided by the embodiment of the present application may include a page reading process and a page writing process. The following describes the page reading process and the page writing process respectively.

[0064] Read page process:

[0065] 1. Read the header sequence number of the shared page: RDMA read page->tailLsn;

[0066] 2. Read the data stored in the shared page: RDMA read page content;

[0067] 3. Read the tail sequence number of the shared page: RDMA read page->headerLsn;

[0068] 4. Compare the head number and tail number:

[0069] In this step, comparing the shared page's header and tail numbers can be considered a verification of the shared page. If the header and tail numbers are identical, the page verification has passed, and the compute node has completed reading the data stored in the page. The above page reading process is also the process of lock-free page reading.

[0070] If the header and tail sequence numbers read by the compute node differ, the lock-free read verification has failed. Repeating the lock-free read verification can be repeated. Once the verification passes, the compute node can read the data stored on the page. If the verification fails after multiple attempts, the lock-free read page verification has failed. If the lock-free read verification fails, the compute node can attempt a locked read.

[0071] 5. Lock and read;

[0072] During a locked read, the compute node can set the lock information for the shared page to be read, i.e., set the lock information for the shared page to a read lock and hold the read lock. The compute node can then read the data stored in the page again. After the read is complete, the compute node can release the read lock. This process can be represented as:

[0073] Write page process:

[0074] 1. Set the lock information of the shared page: RDMA atomic lock bits and set lock owner (lock page);

[0075] In this step, the computing node can set the lock information of the shared page to be read, that is, set the lock information to a write lock and hold the write lock. When a computing node holds the write lock of a page, the page can only be modified by the node holding the write lock and cannot be modified by other nodes. In one possible implementation, setting the lock information of a shared page to a write lock can refer to setting the lock bit of the shared page to 1; correspondingly, setting the lock information of a shared page to a read lock can refer to setting the lock bit of the shared page to 0.

[0076] 2. Read the shared page and store it locally: RDMA read page_info, page and copy to local mem;

[0077] In this step, the computer reading the shared page may include reading the page information of the shared page and the data stored in the shared page. Then, the computing node may store the read data in the local memory.

[0078] 3. Check whether the page log has been flushed to disk: Check whether page owner node flush xlog done;

[0079] In this step, verifying whether the page log has been written to disk means verifying whether the previous node that modified the shared page has persisted the modified page. Only after the previous node has completed the persistence of the page data can the next node modify the data stored in the page.

[0080] Verifying that the page log has been written to disk can be achieved by comparing the log sequence numbers:

[0081] Specifically, the current computing node can compare the log sequence number xlog_owner_node->flush_lsn generated by the previous computing node that modified the shared page when burning the log with the log sequence number headerLsn of the shared page. If

[0082] If xlog_owner_node->flush_lsn is greater than or equal to headerLsn, the shared page has been persisted and the current compute node can continue writing the page. Otherwise, the shared page has not been persisted and the current compute node must wait for the previous compute node to complete the persistence process before continuing writing the page.

[0083] 4. Modify the shared page: Modify local mem;

[0084] In this step, the computing node can modify the data stored in the shared page according to actual needs. The above modification process can be completed in the local storage.

[0085] 5. Write the modified shared page header number to the remote end: RDMA write page->headerLsn;

[0086] 6. Write the modified shared page to the remote end: RDMA write page to remote page;

[0087] 7. Write the modified tail sequence number of the shared page to the remote end: RDMA write page->taillsn;

[0088] After completing the modification of the shared page, the computing node can sequentially write the header sequence number, data in the shared page, and tail sequence number of the modified shared page to the remote end, that is, write the above information into the page buffer pool in the shared memory.

[0089] 8. Set the lock information of the shared page: RDMA atomic reset lock bits and clean lock owner (unlock page).

[0090] After completing the write process for a shared page, the compute node can set the lock information for that page again. This time, setting the lock information may mean that the compute node releases the write lock it holds. For example, releasing the write lock for a shared page by the compute node may mean changing the lock bit of the shared page from 1 to 0. After the write lock is released, the shared page can be read and written by other compute nodes.

[0091] As can be seen from the above read and write processes, in a data sharing architecture with separated storage and computing, the distributed buffer pool in the database system can use a hybrid approach of local memory and shared memory to optimize page reading and writing, improving performance without the need to frequently transmit shared pages to be read and written between various computing nodes through RPC.

[0092] Based on the above-mentioned shared page reading and writing process, an embodiment of the present application also provides a fault handling mechanism during the reading and writing process. As shown in Figure 6, it is a schematic diagram of fault handling during the reading and writing process of a shared page provided by an embodiment of the present application. When a computing node fails to access a shared page in the shared memory, the computing node can obtain the corresponding cache page from other nodes that have cached the shared page. For example, when computing node 1 in Figure 6 accesses the RDMA shared memory, the RDMA shared memory reports an error, then the computing node can obtain the cache page from other computing nodes that have cached the shared page, such as computing node 2 in Figure 6. By comparing the log sequence numbers of the cache pages obtained from other computing nodes, computing node 1 can determine the cache page corresponding to the maximum log sequence number as the latest version page of the shared page.

[0093] In one possible implementation of the present invention, if a compute node encounters a shared memory error while accessing shared memory, and other nodes that cache the shared page to be read also experience a fault, the current compute node should wait until the fault in the other nodes is resolved before retrieving the cached pages from the other nodes using the fault handling mechanism described above. This ensures that the shared page ultimately determined based on the cached page's log sequence number is the actual latest version of the page in each compute node.

[0094] In combination with the various embodiments described above, FIG7 shows a schematic diagram of the steps of a shared page reading and writing method provided by an embodiment of the present application. The method may specifically include the following steps S701-S703:

[0095] S701: Determine a shared page to be read or written, where the address of the shared page is shared among multiple computing nodes.

[0096] In the embodiment of the present application, the shared page to be read or written may refer to a page shared by multiple computing nodes, on which a computing node currently desires to perform a read or write operation, for example, shared page 1 in Figures 4, 5, and 6.

[0097] Sharing the address of a shared page across multiple compute nodes may mean that the address of the shared page in shared memory is known by multiple compute nodes. For example, the compute nodes that can share shared page 1 in the above example include compute node 1 and compute node 2 in Figures 4, 5, and 6. Therefore, compute node 1 and compute node 2 can know the address of shared page 1 in shared memory. This address may be the address in the RDMA shared memory in Figure 5 or 6.

[0098] S702 . Obtain page information of the shared page from the page buffer pool according to the address of the shared page. The page information at least includes lock information of the shared page and a log sequence number stored in the shared page.

[0099] Based on the address of the shared page, the computing node can obtain the page information of the page from the page buffer pool. The above-mentioned page buffer pool can exist in the shared memory based on the low-latency host interconnection protocol. Exemplarily, the page buffer pool can exist in the RDMA shared memory in Figure 5 or Figure 6.

[0100] In an embodiment of the present application, the page information of a shared page may include the lock information of the page and the log sequence number stored in the page. Exemplarily, the page information of a shared page may include the page information (page_info) and the log sequence number LSN in FIG5 . The above-mentioned log sequence number LSN may include the head sequence number Head_LSN and the tail sequence number Tail_LSN in the aforementioned embodiments.

[0101] S703: Read and write the shared page based on the lock information and the log sequence number.

[0102] The reading and writing method for a shared page provided in the embodiment of the present application can be implemented based on the acquired lock information and log sequence number of the shared page.

[0103] In one possible implementation, the type of operation for reading or writing a shared page may be determined first. The operation type may include a read operation or a modify operation. A read operation on a shared page may refer to an operation of reading a shared page in the aforementioned embodiments; a modify operation on a shared page may refer to an operation of writing a shared page in the aforementioned embodiments.

[0104] The compute node can then set the lock information for the shared page based on the operation type. Specifically, when the operation type is a modification operation, the compute node can set the lock information for the shared page to a write lock and set the write lock holder to the current compute node. Specifically, the compute node can set the lock_bit in the page information to 1 in the manner described in the aforementioned embodiments.

[0105] After setting the lock information, the computing node performs a read or write operation on the shared page based on the log sequence number of the shared page, such as a read process or a write process on the shared page.

[0106] In an embodiment of the present application, when the operation type for a shared page is a modification operation, the computing node can verify whether the shared page has been persisted based on the log sequence number. If the shared page has been persisted, the computing node can perform a modification operation on the shared page; if the shared page has not been persisted, the computing node needs to wait for the shared page to be persisted before performing a modification operation on it. Verifying whether a shared page has been persisted can refer to the aforementioned embodiments, which verify whether the previous computing node that modified the shared page has written the log generated after the modified page to the disk.

[0107] In a possible implementation, the computing node may verify whether the shared page has been persisted by comparing log sequence numbers as described in the aforementioned embodiments.

[0108] For example, the compute node can obtain the log sequence number xlog_owner_node->flush_lsn generated by the last compute node that last modified the shared page, and compare xlog_owner_node->flush_lsn with the log sequence number headerLsn of the shared page. If xlog_owner_node->flush_lsn is greater than or equal to headerLsn, it can be determined that the shared page has been persisted; otherwise, it is determined that the shared page has not been persisted.

[0109] After determining that the shared page has been persisted, the compute node can modify the data in the shared page. After completing the modification of the shared page, the compute node can sequentially write the header sequence number, data in the shared page, and tail sequence number of the modified shared page to the remote page buffer pool, that is, the RDMA shared memory in Figure 5 or Figure 6.

[0110] In the embodiments of the present application, when the operation type for a shared page is a read operation, the computing node shall verify the shared page according to the steps in the page read process described in the aforementioned embodiments. The above verification process can be implemented by comparing the header sequence number headerLsn and the tail sequence number tailLsn in the log sequence number of the shared page.

[0111] If the header sequence number (headerLsn) and the tail sequence number (tailLsn) match, the shared page has passed verification, and the compute node can read the data in the shared page. If the header sequence number (headerLsn) and the tail sequence number (tailLsn) differ, the shared page has failed verification, and the compute node can continue verification as described above. If multiple verifications fail, the compute node can attempt to read the shared page by locking it.

[0112] When reading a shared page using a lock, a compute node can set the shared page's lock information to a read lock and set the read lock holder to the current compute node. After setting the read lock, the compute node can read the data in the shared page. When the compute node completes reading the shared page, it can release the read lock.

[0113] In one possible implementation, a fault may occur during the process of reading and writing a shared page using the methods in the aforementioned embodiments. The fault may occur in the shared memory or in one or more computing nodes that share a certain page. The present application also provides a method for handling faults. As shown in FIG8 , a schematic diagram of a fault handling process during the reading and writing of a shared page provided by an embodiment of the present application is shown. The process may specifically include the following steps S801-S802:

[0114] S801. When a failure occurs in reading and writing a shared page based on lock information and a log sequence number, at least one cache page of the shared page is obtained from an associated node, where the associated node is a computing node that shares the shared page.

[0115] In one possible implementation, a fault may occur in shared memory, such as the RDMA shared memory fault shown in FIG6 . In this case, a compute node that needs to read a shared page from shared memory can obtain at least one cached page of the shared page from an associated node. The associated node may be another compute node in the database system that shares the same page with the compute node. The number of associated nodes may include one or more.

[0116] When a shared memory failure occurs, a compute node cannot retrieve shared pages from the shared memory. For example, in Figure 6, after an RDMA shared memory failure occurs, compute node 1 cannot retrieve shared pages from the RDMA shared memory. In this case, compute node 1 can retrieve cached pages from associated nodes, such as compute node 2.

[0117] In another possible implementation, an associated node may also fail. For example, if the compute node currently needing to read a shared page is compute node 1 in Figure 6, and compute node 2 is an associated node of compute node 1, compute node 2 in Figure 6 may also fail. In this case, it is necessary to wait for the associated node to recover before retrieving the cached page of the shared page from compute node 2. In other words, if any associated node fails, it is necessary to wait for the failed associated node to recover before retrieving the cached page of the shared page from each associated node.

[0118] S802: Determine the latest version page of the shared page from at least one cached page.

[0119] In an embodiment of the present application, determining the latest version page of a shared page from a cache page can be achieved by comparing the log sequence numbers of each cache page as described in the aforementioned embodiments.

[0120] Specifically, after obtaining the cache pages cached in each associated node, the computing node can obtain the log sequence number LSN stored in each cache page, and then determine the cache page corresponding to the maximum log sequence number LSN as the latest version page of the shared page.

[0121] In one possible implementation, the log sequence number LSN includes a header sequence number (headerLsn) and a tail sequence number (tailLsn). The header sequence number and tail sequence number in the cache page should be identical. Therefore, when comparing the log sequence number LSNs of cache pages, either the header sequence number (headerLsn) or the tail sequence number (tailLsn) can be compared.

[0122] After determining the latest version of the shared page, the computing node can read and write the latest version of the page according to the read and write process described above.

[0123] Corresponding to the above embodiments, referring to FIG9 , a structural block diagram of a shared page reading and writing device provided in an embodiment of the present application is shown. The device can be used to implement the steps in the above method embodiments. The device may specifically include the following modules: a determination module 901, an acquisition module 902, and a reading and writing module 903, wherein:

[0124] A determination module 901 is configured to determine a shared page to be read or written, where the address of the shared page is shared among multiple computing nodes;

[0125] An acquisition module 902 is configured to acquire page information of the shared page from a page buffer pool according to the address, wherein the page information includes at least lock information of the shared page and a log sequence number stored in the shared page;

[0126] The read / write module 903 is configured to read and write the shared page based on the lock information and the log sequence number.

[0127] In a possible implementation of an embodiment of the present application, the read-write module 903 can be specifically used to: determine the operation type of reading and writing the shared page, the operation type including a read operation or a modify operation; set the lock information of the shared page according to the operation type; after setting the lock information, perform a read-write operation on the shared page based on the log sequence number that matches the operation type.

[0128] In a possible implementation of an embodiment of the present application, the read-write module 903 can also be used to: when the operation type is the modification operation, set the lock information of the shared page to a write lock, and set the holder of the write lock to the current computing node.

[0129] In a possible implementation of an embodiment of the present application, the read-write module 903 can also be used for: when the operation type is the modification operation, verifying whether the shared page has been persisted based on the log sequence number; if the shared page has been persisted, performing a modification operation on the shared page; if the shared page has not been persisted, waiting for the shared page to be persisted before performing a modification operation on the shared page.

[0130] In a possible implementation of an embodiment of the present application, the read-write module 903 can also be used to: obtain the log sequence number formed by the previous computing node burning log that last modified the shared page; compare the log sequence number formed by the previous computing node burning log with the log sequence number of the shared page; if the log sequence number formed by the previous computing node burning log is greater than or equal to the log sequence number of the shared page, it is determined that the shared page has been persisted; otherwise, it is determined that the shared page has not been persisted.

[0131] In a possible implementation of the embodiment of the present application, the read-write module 903 may also be used to sequentially write the header sequence number, the modified data in the shared page, and the tail sequence number into the remote page buffer pool.

[0132] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used for: when the operation type is the read operation, comparing the header sequence number and the tail sequence number in the log sequence number of the shared page; if the header sequence number and the tail sequence number are the same, reading the data in the shared page; if the header sequence number and the tail sequence number are different, reading the shared page by locking.

[0133] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used to: set the lock information of the shared page to a read lock, and set the holder of the read lock to the current computing node; after setting the read lock, read the data in the shared page.

[0134] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used to: when a failure occurs in reading and writing the shared page based on the lock information and the log sequence number, obtain at least one cache page of the shared page from an associated node, and the associated node is a computing node that shares the shared page; determine the latest version page of the shared page from at least one of the cache pages.

[0135] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used to: obtain the log sequence number stored in each of the cache pages; and determine the cache page corresponding to the maximum value of the log sequence number as the latest version page of the shared page.

[0136] In another possible implementation of the embodiment of the present application, the read-write module 903 can also be used to: if any of the associated nodes fails, then after waiting for the failed associated node to return to normal, obtain the cache page of the shared page from each of the associated nodes.

[0137] In an embodiment of the present application, the page buffer pool exists in a shared memory based on a low-latency host interconnection protocol.

[0138] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiment part.

[0139] With reference to Figure 10, a schematic diagram of a computer device provided in an embodiment of the present application is shown. As shown in Figure 10, the computer device 1000 in the embodiment of the present application includes: a processor 1010, a memory 1020, and a computer program 1021 stored in the memory 1020 and executable on the processor 1010. When the processor 1010 executes the computer program 1021, the steps in each embodiment of the above-mentioned shared page reading and writing method are implemented, such as steps S701 to S703 shown in Figure 7. Alternatively, when the processor 1010 executes the computer program 1021, the functions of each module / unit in the above-mentioned device embodiments are implemented, such as the functions of the determination module 901, the acquisition module 902, and the read-write module 903 shown in Figure 9.

[0140] Exemplarily, the computer program 1021 may be divided into one or more modules / units, which are stored in the memory 1020 and executed by the processor 1010 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments may be used to describe the execution process of the computer program 1021 in the computer device 1000. For example, the computer program 1021 may be divided into the determination module 901, the acquisition module 902, and the read / write module 903 shown in FIG. 9 .

[0141] The computer device 1000 may be a device that implements each step of the aforementioned method embodiments. The computer device 1000 may be a computing device such as a desktop computer or a cloud server. The computer device 1000 may include, but is not limited to, a processor 1010 and a memory 1020. Those skilled in the art will appreciate that FIG10 is merely an example of a computer device 1000 and does not limit the computer device 1000. The computer device 1000 may include more or fewer components than shown, or may combine certain components or different components. For example, the computer device 1000 may also include input and output devices, network access devices, buses, and the like.

[0142] The processor 1010 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0143] The memory 1020 may be an internal storage unit of the computer device 1000, such as a hard disk or memory of the computer device 1000. The memory 1020 may also be an external storage device of the computer device 1000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the computer device 1000. Furthermore, the memory 1020 may include both an internal storage unit of the computer device 1000 and an external storage device. The memory 1020 is used to store the computer program 1021 and other programs and data required by the computer device 1000. The memory 1020 may also be used to temporarily store data that has been output or is about to be output.

[0144] Finally, it should be noted that the above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be covered by the protection scope of the present application.

Claims

1. A method for reading and writing a shared page, characterized in that: include: Determine a shared page to be read and written, where an address of the shared page is shared among multiple computing nodes; According to the address, obtaining page information of the shared page from a page buffer pool, the page information at least including lock information of the shared page and a log sequence number stored in the shared page; Based on the lock information and the log sequence number, the shared page is read and written.

2. The method according to claim 1, characterized in that The reading and writing of the shared page based on the lock information and the log sequence number includes: Determine an operation type of reading and writing the shared page, where the operation type includes a read operation or a modify operation; Setting lock information of the shared page according to the operation type; After the lock information is set, a read and write operation matching the operation type is performed on the shared page based on the log sequence number.

3. The method according to claim 2, characterized in that The setting the lock information of the shared page according to the operation type includes: When the operation type is the modification operation, the lock information of the shared page is set to a write lock, and the holder of the write lock is set to the current computing node.

4. The method according to claim 2 or 3, characterized in that: The performing a read and write operation matching the operation type on the shared page based on the log sequence number includes: When the operation type is the modification operation, verifying whether the shared page is persistently processed based on the log sequence number; If the shared page has been persisted, modify the shared page; If the shared page has not been processed persistently, the shared page is modified after waiting for the shared page to be processed persistently.

5. The method according to claim 4, characterized in that The checking whether the shared page is persistently processed based on the log sequence number includes: Obtaining a log sequence number formed by a previous computing node burning log that last modified the shared page; Compare the log sequence number formed by burning the log of the previous computing node with the log sequence number of the shared page; If the log sequence number formed by the previous computing node burning log is greater than or equal to the log sequence number of the shared page, it is determined that the shared page has been persistently processed; otherwise, it is determined that the shared page has not been persistently processed.

6. The method according to claim 4 or 5, characterized in that: The log sequence number of the shared page includes a head sequence number and a tail sequence number. After the shared page is modified, the method further includes: The head sequence number, the modified data in the shared page and the tail sequence number are sequentially written into the remote page buffer pool.

7. The method according to claim 2, characterized in that The performing a read and write operation on the shared page that matches the operation type based on the log sequence number includes: When the operation type is the read operation, comparing the head sequence number and the tail sequence number in the log sequence number of the shared page; If the head sequence number and the tail sequence number are the same, reading the data in the shared page; If the head sequence number and the tail sequence number are different, the shared page is read in a locked manner.

8. The method according to claim 7, characterized in that The reading of the shared page by locking the shared page includes: Set the lock information of the shared page to a read lock, and set the holder of the read lock to the current computing node; After setting the read lock, the data in the shared page is read.

9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: When a failure occurs in reading and writing the shared page based on the lock information and the log sequence number, obtaining at least one cache page of the shared page from an associated node, the associated node being a computing node that shares the shared page; Determine the latest version page of the shared page from at least one of the cached pages.

10. The method according to claim 9, characterized in that Determining the latest version page of the shared page from at least one of the cached pages includes: Obtaining the log sequence number stored in each of the cache pages; The cache page corresponding to the maximum value of the log sequence number is determined as the latest version page of the shared page.

11. The method according to claim 9 or 10, characterized in that: The acquiring at least one cache page of the shared page from the associated node comprises: If any of the associated nodes fails, then after waiting for the failed associated node to recover, the cached pages of the shared page are obtained from each of the associated nodes.

12. The method according to any one of claims 1 to 11, characterized in that: The page buffer pool exists in a shared memory based on a low-latency host interconnection protocol.

13. A database system, comprising a database and a plurality of computing nodes, wherein the plurality of computing nodes share pages in the database, characterized in that: Any of the computing nodes reads and writes pages in the database using the shared page reading and writing method as described in any of claims 1-12.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for reading and writing a shared page as described in any one of claims 1 to 12 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the shared page reading and writing method according to any one of claims 1 to 12.

Images