Dirty page brushing method and related product

Abstract

The invention provides a dirty page brushing method and a related product. The dirty page disk brushing method is applied to any node in a database cluster, and comprises the following steps: responding to a global check point process started by the database cluster; participating in the collection of redo sites by the database cluster; node-level dirty page disc brushing is carried out for multiple times; synchronizing dirty page disk brushing progress of the buffer area with other nodes; and performing one-time cluster-level dirty page disk brushing with other nodes. The method has the advantage that the dirty page brushing efficiency during the global check point period can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of dirty page disk flushing, and in particular to a dirty page disk flushing method and related products. Background Art

[0002] As databases have evolved, single-node databases have struggled to meet growing business demands due to performance and resource limitations. This has led to the emergence of database shared storage clusters, an innovative extension of single-node databases. Their core principle is to coordinate database instances on each node in the cluster, enabling them to read and write to the same data directory and provide unified external services.

[0003] To ensure secure access to shared resources within a database cluster, buffered consistency technology is widely used. This technology uses blocking and synchronization mechanisms to ensure that the entire cluster presents only one buffer to external logic, thus avoiding data access conflicts and inconsistencies.

[0004] Checkpoints in a database shared storage cluster are usually global checkpoints, meaning that all nodes in the cluster work together to flush all dirty pages in the cluster buffer before the cluster redo point to the shared disk. This shortens the recovery time after a database cluster crash, effectively controls the cluster I / O (read / write) load, and improves cluster stability.

[0005] However, to ensure cluster buffer consistency during a global checkpoint, the cluster-level flushing of a single dirty page involves several coordinated scheduling processes, such as inter-node communication, inter-process communication, and blocking. This results in low flushing efficiency for a single dirty page, and the impact of this inefficiency increases as business volume increases. Furthermore, inter-node communication during flushing increases the pressure on the communication process to maintain cluster buffer consistency, leading to performance jitter in the communication process and, in turn, increased business processing latency. Summary of the Invention

[0006] An object of the present invention is to improve the efficiency of dirty page flushing during a global checkpoint.

[0007] A further object of the present invention is to effectively coordinate the operations of multiple nodes on the same data page to ensure the consistency and timeliness of global dirty page data in the entire cluster.

[0008] In particular, according to a first aspect of the present invention, a dirty page flushing method is provided, which is applied to any node in a database cluster. The method comprises:

[0009] Respond to the global checkpoint process initiated by the database cluster;

[0010] Participate in the collection of redo sites by the database cluster;

[0011] Perform multiple node-level dirty page flushes;

[0012] Synchronize the dirty page flushing progress of the buffer with other nodes;

[0013] Perform a cluster-level dirty page flush with other nodes.

[0014] Optionally, the step of performing multiple node-level dirty page flushing steps includes:

[0015] Traverse its own buffer and identify all dirty pages to be written to disk;

[0016] For each dirty page identified, determine its corresponding role attribute in the global resource lock information;

[0017] Perform a round of node-level dirty page flushing based on the role attributes of the dirty page;

[0018] Calculate the ratio of the number of dirty pages flushed to disk in this round to the number of dirty pages flushed to disk in the previous round;

[0019] If the ratio is less than a preset proportional factor, stop traversing and brushing;

[0020] If the ratio is greater than or equal to the set scaling factor, the own buffer is traversed again to prepare for the next round of node-level dirty page flushing.

[0021] Optionally, the steps of performing a round of node-level dirty page flushing according to the role attribute of the dirty page include:

[0022] When the role attribute is a local role, the dirty page is directly flushed to disk;

[0023] When the role attribute is a global role, determine the type of buffer consistency lock held by the role;

[0024] If the node does not hold a lock, clear the dirty mark of the dirty page and do not flush the disk;

[0025] If the node holds an exclusive lock, the dirty page is flushed directly to disk;

[0026] If the node holds a shared lock, the latest write mark corresponding to the dirty page is checked in the global resource lock information. If the node identified by the latest write mark is the current node, the dirty page is flushed to disk.

[0027] Optionally, the dirty page flushing method further includes:

[0028] Transferring dirty pages with other nodes, including dirty page forwarding and dirty page reception;

[0029] If the dirty page forwarding is before the redo site collection phase, sending a first instruction to the receiving node, the first instruction being used to instruct the receiving node to process the dirty page on its own;

[0030] If the dirty page forwarding is in the redo site collection phase, a second instruction is sent to the receiving node, where the second instruction is used to instruct the receiving node to write the dirty page to the disk.

[0031] Optionally, if the dirty page forwarding is in the stage of multiple rounds of node-level dirty page disk flushing, determine the disk flushing status of the dirty page itself;

[0032] If the forwarding node has not written the dirty page to disk, sending a third instruction to the receiving node, wherein the third instruction is used to instruct the receiving node to write the dirty page to disk;

[0033] If the forwarding node has written the dirty page to the disk, a fourth instruction is sent to the receiving node, where the fourth instruction is used to instruct the receiving node that there is no need to process the dirty page.

[0034] Optionally, if the dirty page receiving is in the dirty page disk write stage of multiple rounds at the node level, when the first instruction, second instruction or third instruction of the sending node is received, the dirty page is marked and included in the scope that requires the global checkpoint disk write processing, waiting for the checkpoint process to perform statistical collection and processing.

[0035] Optionally, after performing multiple node-level dirty page flushing steps, the method further includes:

[0036] Participate in flushing all communication channels to prevent messages transmitted by dirty pages from remaining in the communication channels.

[0037] Optionally, the steps for performing a cluster-level dirty page flush with other nodes include:

[0038] Traverse its own buffer and identify all dirty pages to be written to disk;

[0039] Simultaneously with other nodes, each node writes each dirty page it identifies to disk at the cluster level.

[0040] According to a second aspect of the present invention, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements any one of the above-mentioned dirty page flushing methods.

[0041] According to a third aspect of the present invention, the present invention provides a computer program product, comprising a computer program, which implements any one of the above-mentioned dirty page flushing methods when executed by a processor.

[0042] According to the dirty page disk flushing method of the present invention, during the global checkpoint process, node-level dirty page disk flushing does not require inter-node communication, and only needs to write the latest version of the dirty page held by the node to the disk. This method reduces unnecessary communication interactions between nodes, improves the efficiency of dirty page disk flushing and the utilization efficiency of system resources, and also reduces the risk of data inconsistency caused by communication delays or errors to a certain extent. Multiple node-level dirty page disk flushing can minimize the amount of non-persistent data in the memory, and then combined with cluster-level dirty page disk flushing, the dirty pages of all node buffers are comprehensively checked and processed from the cluster level, ensuring that all dirty pages that should be persisted can be accurately written to the disk, further enhancing the consistency and integrity of the data.

[0043] Furthermore, the dirty page disk flushing method of the present invention directly flushes the dirty pages of local roles to ensure that the data modifications of each node can be persisted in time. For the dirty pages of global roles, the disk flushing strategy is determined by judging the buffer consistency lock type and the latest write mark, which effectively coordinates the operations of multiple nodes on the same data page. When holding an exclusive lock, the disk is directly flushed to ensure the uniqueness and integrity of data modifications and avoid data conflicts caused by concurrent modifications of multiple nodes. When holding a shared lock, the disk is flushed only when the current node is the latest write node to prevent the persistence of old data and ensure the consistency and timeliness of global dirty page data in the entire cluster.

[0044] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0046] Figure 1 is a schematic flow chart of a dirty page flushing method according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic flow chart of performing multiple node-level dirty page flushing according to an optional embodiment of the present invention;

[0048] Figure 3 This is a schematic flowchart of performing a round of node-level dirty page flushing based on the role attributes of the dirty pages according to one embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of a computer program product according to one embodiment of the present invention;

[0050] Figure 5 is a schematic diagram of a computer-readable storage medium according to one embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of a computer device according to one embodiment of the present invention. DETAILED DESCRIPTION

[0052] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0053] A database cluster, based on a single-node database, coordinates read and write access to the same data directory on each node in the cluster, providing unified external services. To ensure secure access to shared resources, the database cluster employs buffer consistency technology. This technology uses blocking and synchronization mechanisms to logically present the entire cluster as a single buffer.

[0054] In a database shared storage cluster, a global checkpoint is often used. A global checkpoint involves all nodes in the cluster coordinating and writing all dirty pages in the cluster buffer before the cluster redo point to the shared disk to ensure cluster data consistency and integrity.

[0055] In the prior art, during the global checkpoint dirty page write process, to ensure the consistency of the cluster buffer, the cluster-level write of each dirty page follows the following process:

[0056] Each page in the buffer has a corresponding page management node in the cluster. When a node initiates a dirty page flush request, it must first submit an application to the page management node.

[0057] The page management node will check the global resource lock information of the cluster, find the node holding the latest version of the page, designate it as the flushing node for this flush, and notify the flushing node to perform the dirty page flushing operation.

[0058] The disk flushing node performs the disk flushing task of a single node dirty page. After completing the disk flushing, it feeds back the flushing results to the page management node.

[0059] The page management node updates the global resource lock information based on the disk flushing results. At this point, the cluster-level dirty page disk flushing operation is completed.

[0060] However, after research, the inventors discovered that, under the global checkpoint mechanism, to ensure cluster buffer consistency, the cluster-level flushing process for a single dirty page involves a series of coordinated scheduling tasks, such as inter-node communication, inter-process communication, and blocking operations. This leads to two major problems:

[0061] 1. Inefficient disk flushing of individual dirty pages. Furthermore, during a global checkpoint, the number of dirty pages in the cluster is closely related to business volume. As business volume increases, the negative impact of inefficient disk flushing of individual dirty pages becomes increasingly significant.

[0062] 2. Inter-node communication during the disk flushing process puts significant pressure on the communication process to maintain cluster buffer consistency, causing performance fluctuations in the communication process. This performance fluctuation further prolongs business processing time and increases business processing latency.

[0063] To solve the above problem, an embodiment of the present invention provides a dirty page disk flushing method, which is applied to any node in a database cluster (particularly a database shared storage cluster).

[0064] Figure 1 FIG. 1 is a schematic flow chart of a dirty page flushing method according to an embodiment of the present invention. Figure 1 As shown, the present invention provides a dirty page disk flushing method, which at least includes steps S102 to S110.

[0065] Step S102: responding to the global checkpoint process initiated by the database cluster.

[0066] When a database cluster initiates a global checkpoint, each node senses the signal and responds. A global checkpoint is a key mechanism in database systems, ensuring that database data is consistent at a specific moment in time, providing a reliable foundation for subsequent data recovery operations.

[0067] During daily database operations, transactions continuously modify data. These modifications are first made in the memory buffer rather than immediately written to disk. Over time, inconsistencies may arise between the data in memory and the data on disk. The initiation of a global checkpoint process takes a comprehensive "snapshot" of the database, persisting the modifications of committed transactions in memory to disk to ensure data consistency.

[0068] Step S104: The participating database cluster collects redo sites.

[0069] A redo point is a key location in the redo log that records all database changes up to a certain point in time. Nodes cooperate with the database cluster by providing their own redo log information so that the cluster can collect and consolidate redo points from all nodes.

[0070] The redo log is a database file that records all changes to data pages. When a database failure requires recovery, the system can use the redo log to re-execute transactions that have been committed but not yet fully persisted to disk, starting from a specific redo point, thereby restoring the database to a consistent state. Collecting redo points clearly defines the starting point for recovery operations, reducing recovery time and effort.

[0071] Step S106: perform multiple node-level dirty page flushes.

[0072] As database operations continue, the number of dirty pages in the node buffer may fluctuate. By flushing dirty pages to disk multiple times at the node level, you can minimize the amount of non-persistent data in memory and improve data security and consistency.

[0073] Step S108: Synchronize the dirty page flushing progress of the buffer with other nodes.

[0074] In a database cluster, the load and performance of each node may vary, which can lead to inconsistent dirty page flushing progress across different nodes. Without synchronization, some nodes may have completed flushing while others are still processing a large number of dirty pages, affecting the data consistency and performance of the entire cluster. By synchronizing the flushing progress, all nodes can complete the dirty page flushing operation in roughly the same time.

[0075] Step S110: Perform a cluster-level dirty page flush with other nodes.

[0076] Although most dirty pages have been persisted to disk after multiple node-level dirty page flushes, some may still be missed or unprocessed. Cluster-level dirty page flushing performs a comprehensive check and processing of the buffers of all nodes across the entire cluster, ensuring that all dirty pages that should be persisted are correctly written to disk.

[0077] In this embodiment, during a global checkpoint, node-level dirty page flushing eliminates the need for inter-node communication and simply flushes the latest version of the dirty page held by the node to disk. This approach reduces unnecessary inter-node communication, improves dirty page flushing efficiency and system resource utilization, and also reduces the risk of data inconsistency caused by communication delays or errors.

[0078] Multiple node-level dirty page flushes can minimize the amount of non-persistent data in memory. Combined with cluster-level dirty page flushes, dirty pages in all node buffers can be comprehensively checked and processed at the cluster level, ensuring that all dirty pages that should be persisted can be accurately written to disk, further enhancing data consistency and integrity.

[0079] Figure 2 FIG. 1 is a schematic flow chart of performing multiple node-level dirty page flushing according to an optional embodiment of the present invention. Figure 2 As shown, performing multiple node-level dirty page flushing at least includes the following steps S202 to S212.

[0080] Step S202: traverse the own buffer and identify all dirty pages to be written to disk.

[0081] The node performs a comprehensive scan and check of its buffer. The buffer is a memory area used by the database to temporarily store data. During database operation, data modifications are first performed in the buffer. Data pages that have been modified but not yet persisted to disk are called dirty pages. The node can identify all dirty pages in the buffer that are pending disk writes based on specific markers or status information.

[0082] Step S204: For each identified dirty page, determine its corresponding role attribute in the global resource lock information.

[0083] Global resource lock information is used to record the status and information collection of page operation permissions for each node in the database cluster. Nodes query global resource lock information to determine the role attribute corresponding to the dirty page. The role attribute may indicate whether the dirty page affects only the local node (local role) or the entire database cluster (global role).

[0084] Step S206: Perform a round of node-level dirty page flushing according to the role attributes of the dirty pages.

[0085] Dirty pages associated with different role attributes may have different importance and handling methods within the database. Dirty pages associated with local roles are typically only relevant to operations within the node, while dirty pages associated with global roles may involve data consistency issues across multiple nodes. Therefore, flushing disks based on role attributes allows for more precise dirty page handling, ensuring data consistency and system operation.

[0086] Step S208: Calculate the ratio of the number of dirty pages written to disk in this round to the number of dirty pages written to disk in the previous round.

[0087] After completing a round of node-level dirty page flushing, the node will count the number of dirty pages successfully flushed to disk in this round, compare it with the number of dirty pages flushed to disk in the previous round, and calculate the ratio of the two.

[0088] This ratio reflects the effectiveness and trend of dirty page flushing. A larger ratio indicates a significant increase in the number of dirty pages flushed to disk compared to the previous round, possibly indicating that more dirty pages need to be processed. A smaller ratio indicates a gradual weakening of the flushing effect, suggesting that most dirty pages have already been flushed to disk.

[0089] By calculating the ratio, the node can dynamically evaluate the progress and effect of dirty page flushing, providing a basis for decision-making on whether to continue the subsequent flushing operation.

[0090] Step S210: If the ratio is less than the preset proportional factor, stop traversing and refreshing.

[0091] The preset scaling factor may be an empirical value or a threshold value set according to system performance and requirements.

[0092] If the ratio is less than the proportional factor, it means that the effect of this round of disk flushing is no longer obvious compared to the previous round, and most dirty pages may have been successfully written to the disk. At this time, the node will stop traversing the buffer and flushing dirty pages to the disk.

[0093] In step S212, if the ratio is greater than or equal to the set scaling factor, the own buffer is traversed again to prepare for the next round of node-level dirty page flushing.

[0094] If the ratio is greater than or equal to this scaling factor, the dirty page flushing is still effective and many dirty pages need to be persisted to disk. By traversing the buffer again and performing the next round of flushing, the number of dirty pages in memory that have not been flushed to disk can be further reduced, improving data persistence and consistency.

[0095] Figure 3 FIG. 1 is a schematic flow chart of performing a round of node-level dirty page flushing according to the role attributes of the dirty page according to an embodiment of the present invention. Figure 3 As shown, performing a round of node-level dirty page flushing according to the role attributes of the dirty pages may include the following steps S302 to S314.

[0096] Step S302: When the role attribute is a local role, the dirty pages are directly flushed to disk.

[0097] During the node-level dirty page flushing process, if a dirty page's role attribute is determined to be local, it means that the dirty page only affects the buffer of the current node and does not involve data consistency issues between other nodes. In this case, the node will directly write the dirty page data from the buffer to a persistent storage device such as disk.

[0098] Dirty pages in the local role are only related to the local operations of this node and do not affect other nodes. Direct disk flushing can promptly persist modified data, avoiding data loss due to node failures or system anomalies, and ensuring the consistency and persistence of data on this node.

[0099] Step S304: When the role attribute is a global role, determine the type of buffer consistency lock held by the role.

[0100] If the dirty page's role attribute is global, it indicates that the dirty page affects the data consistency of the entire database cluster. In this case, the node needs to determine the type of buffered consistency lock it currently holds for the dirty page. Buffered consistency locks are used to coordinate access to the same data page by multiple nodes.

[0101] In a multi-node database cluster, multiple nodes may access and modify the same data page simultaneously. By determining the lock type, a node can understand its own permissions and status when accessing the dirty page, and thus adopt appropriate disk flushing strategies based on different lock conditions to ensure data consistency.

[0102] Step S306: If the node does not hold a lock, clear the dirty mark of the dirty page and do not flush the disk.

[0103] When a node determines that it does not hold any cache consistency locks on a dirty page of a global role, it indicates that it currently does not have exclusive or shared access to the dirty page. At this point, the node clears the dirty mark of the dirty page (the dirty mark is used to indicate that the data page has been modified in memory but not yet persisted) and does not flush the dirty page to disk.

[0104] Not holding the lock means that the node may not have been the last to modify the dirty page or may not have the permission to operate on the dirty page. Clearing the dirty mark can prevent the page from being mistakenly treated as a dirty page that needs to be flushed to disk. At the same time, not flushing to disk can avoid unnecessary disk I / O operations and improve system performance.

[0105] Step S308: If the node holds an exclusive lock, the dirty page is directly flushed to disk.

[0106] If a node holds an exclusive lock on a global role dirty page, it indicates that it is the only node that can currently read and write the dirty page and has exclusive access to the dirty page. At this time, the node will directly write the dirty page data from the memory buffer to disk.

[0107] Exclusive locks ensure the uniqueness and integrity of a node's dirty page modifications. Directly flushing to disk can persist a node's latest dirty page modifications in a timely manner, ensuring the consistency of dirty page data in a cluster environment and preventing other nodes from reading inconsistent data.

[0108] Step S310: If the node holds a shared lock, check the latest write mark corresponding to the dirty page in the global resource lock information.

[0109] When a node holds a shared lock on a global role dirty page, multiple nodes can read the dirty page simultaneously. To determine whether to flush the dirty page to disk, the node checks the last write flag for the dirty page in the global resource lock information. The last write flag identifies the node that last wrote the dirty page.

[0110] Shared locks allow multiple nodes to read the same data page simultaneously, but they cannot guarantee the order in which each node modifies the page or its latest state. By checking the latest write marker, a node can understand the latest modification status of the dirty page, determine whether it holds the latest version of the data, and decide whether to flush the data to disk.

[0111] Step S312: If the node marked with the latest write flag is the current node, the dirty page is flushed to disk.

[0112] If the node marked with the latest write flag is found to be the current node after checking it in step S310, it means that the current node is the latest write node for the global role dirty page and holds the latest version of the dirty page. At this point, the node will write the data of the dirty page from the memory buffer to the disk.

[0113] As the latest write node, this node holds the latest and most accurate dirty page data. Flashing the disk can persist the latest data, ensuring the consistency and timeliness of the dirty page data in the database cluster and preventing other nodes from reading outdated versions of the data.

[0114] Step S314: If the node identified by the latest written mark is not the current node, the dirty mark of the dirty page is cleared without flushing the disk.

[0115] If the node with the latest write mark is not the current node, it means that the global role dirty page held by the current node is not the latest version, and other nodes have made the latest changes to the page. In this case, the node will clear the dirty mark of the dirty page and will not flush the dirty page to disk.

[0116] Because the node does not hold the latest version of data, flushing the disk will cause old data to be persisted, destroying the consistency of data in the database cluster. Clearing the dirty mark can prevent the old version of the page from being treated as a dirty page in the future, improving system accuracy and performance.

[0117] The dirty page flushing method of this embodiment directly flushes dirty pages of local roles to disk, ensuring that data modifications within each node are promptly and persistently persisted. Because these dirty pages only affect the local node, direct flushing avoids data loss caused by abnormal conditions such as node failures and system crashes, maintaining the consistency and persistence of the node's data and providing a solid foundation for stable node operation.

[0118] For dirty pages of global roles, the flushing strategy is determined by judging the buffer consistency lock type and the most recent write flag, effectively coordinating operations on the same data page across multiple nodes. Direct flushing to disk when holding an exclusive lock ensures the uniqueness and integrity of data modifications and avoids data conflicts caused by concurrent modifications on multiple nodes. When holding a shared lock, flushing to disk occurs only when the current node is the most recent write node, preventing the persistence of stale data and ensuring the consistency and timeliness of global dirty page data across the entire cluster.

[0119] During a global checkpoint, since dirty pages in the cluster are always stored on certain nodes, performing a node-level dirty page flush on each node ensures that all dirty pages in the cluster are written to disk and avoids the problem of inefficient dirty page flushing. Since each node in the cluster may have multiple versions of a dirty page, to avoid unnecessary flushing, only the latest version of a page in the cluster needs to be written to disk.

[0120] However, in a multi-node database cluster, multiple nodes may simultaneously read and write the same data page. To clearly record which node performed the most recent write to a page, thereby ensuring data consistency and integrity, this embodiment introduces a last write flag ("Last Write") in the global resource lock information to identify the node that last wrote to the page.

[0121] When a node requests a buffered consistency exclusive lock, it means it wants exclusive read and write access to a specific page. This means only that node can modify that page at any given time. At this point, the node is marked as the last write in the global resource lock information. Other nodes clear their own last write flags to avoid confusion and ensure only one node is marked as the last write in the global resource lock information, guaranteeing accurate data consistency judgment.

[0122] It is understood that during a global checkpoint, if database cluster services are still in progress, dirty pages will be transferred between nodes. To prevent pages from being missed during the checkpoint during dirty page transfer, the dirty page flushing method of this embodiment also includes transferring dirty pages between nodes. Dirty page transfer includes dirty page forwarding and dirty page receiving.

[0123] If the dirty page forwarding is before the redo site collection phase, the current node, as a forwarding node, sends a first instruction to the receiving node. The first instruction is used to instruct the receiving node to process the dirty page on its own.

[0124] Before the redo site collection phase, the database state is not yet fully determined, allowing for greater flexibility in dirty page handling. Receiving nodes can independently decide how to handle received dirty pages based on their own circumstances, such as memory usage and load. This fully leverages the autonomy of each node and improves overall processing efficiency.

[0125] If the dirty page forwarding is in the redo site collection phase, the current node, as the forwarding node, will send a second instruction to the receiving node. The second instruction is used to instruct the receiving node to write the dirty page to the disk.

[0126] The redo site collection phase is a critical period for determining database recovery points. It's crucial to ensure that all dirty pages are persisted to disk promptly to ensure data consistency and recoverability. Therefore, the current node requires the receiving node to immediately flush dirty pages to disk, effectively avoiding data loss or inconsistencies during subsequent recovery.

[0127] If the dirty page forwarding is in the stage of multiple rounds of node-level dirty page writes, then determine its own dirty page write status. If the forwarding node has not written the dirty page to disk, send a third instruction to the receiving node, the third instruction is used to instruct the receiving node to write the dirty page to disk; if the forwarding node has written the dirty page to disk, send a fourth instruction to the receiving node, the fourth instruction is used to instruct the receiving node not to process the dirty page.

[0128] During multiple rounds of node-level dirty page flushing, to avoid repeated flushing and resource waste, differentiated processing is performed based on the flushing status of the forwarding node. If the forwarding node has not flushed the dirty page, the dirty page processing task is assigned to the receiving node, ensuring that the dirty page is eventually flushed. If the forwarding node has already flushed the page, the receiving node is notified that no further processing is required, avoiding unnecessary operations on the receiving node and improving system efficiency.

[0129] If the dirty page receiving is in the stage of multiple rounds of node-level dirty page flushing, when the first instruction, second instruction or third instruction of the sending node is received, the dirty page is marked and included in the scope of global checkpoint flushing processing, waiting for the checkpoint process to perform statistical collection and processing.

[0130] The instructions sent by the sending node are based on its own dirty page processing status and the stage of the global checkpoint. By marking the dirty pages received by the receiving node, it can clearly track the source and processing requirements of these dirty pages, ensuring that these special dirty pages are not missed or incorrectly processed.

[0131] Marked dirty pages are included in the global checkpoint process to ensure data consistency across the entire database cluster. The global checkpoint process centrally counts and collects all dirty pages that need to be flushed to disk, avoiding data inconsistencies that might result from independent processing of dirty pages by each node. This centralized approach ensures that all dirty pages are persisted to disk at the appropriate time.

[0132] In an optional embodiment, after performing multiple node-level dirty page flushing steps, all communication channels may also be flushed to prevent messages transmitted by dirty pages from remaining in the communication channels.

[0133] In a database cluster, dirty pages are frequently transferred between nodes, and this information is transmitted over communication channels. During multiple node-level dirty page flushes, dirty page transfer messages are constantly flowing back and forth in the communication channels. If these messages are not processed promptly, they may remain in the communication channels.

[0134] After completing multiple node-level dirty page flushes, the node participates in flushing all communication channels to prevent residual dirty page messages from interfering with subsequent operations. This could, for example, cause dirty pages to be processed repeatedly or omitted, compromising the consistency and integrity of database data. Flushing communication channels clears these residual messages, creating a favorable environment for subsequent operations.

[0135] In an optional embodiment, when a node performs a cluster-level dirty page flush with other nodes, the node first traverses its own buffer to identify all dirty pages to be written to disk, and then simultaneously performs a cluster-level dirty page write for each dirty page identified by each node with other nodes.

[0136] After multiple node-level dirty page flushes, although most dirty pages have been processed, some may still need to be processed uniformly at the cluster level to ensure the consistency and integrity of the database cluster data. Therefore, a cluster-level dirty page flush is required.

[0137] The node first performs a comprehensive traversal of its own buffers to identify all dirty pages that are waiting to be flushed to disk. The purpose of this step is to clearly define the range of dirty pages that need to be flushed to disk at the cluster level on this node, preparing for the subsequent unified flushing operation.

[0138] After identifying dirty pages to be flushed to disk, a node synchronizes with other nodes in the cluster to perform a cluster-level flush for each dirty page identified. This simultaneous flushing process ensures data consistency across the entire cluster. If individual nodes perform flushing asynchronously, some nodes may complete the flush while others are still processing dirty pages, potentially leading to data inconsistencies. By performing a simultaneous cluster-level flush, all nodes can persist dirty pages to disk at the same time, maintaining data consistency across the database cluster.

[0139] The flowchart provided in this embodiment is not intended to indicate that the operations of the method will be performed in any particular order, or that all operations of the method are included in all every case. In addition, the method may include additional operations. Within the scope of the technical ideas provided by the method of this embodiment, additional changes can be made to the above method.

[0140] It should be understood that in some embodiments, each part can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system.

[0141] This embodiment also provides a computer program product 10 , a computer-readable storage medium 20 , and a computer device 30 . Figure 4 is a schematic diagram of a computer program product 10 according to one embodiment of the present invention, Figure 5 is a schematic diagram of a computer-readable storage medium 20 according to one embodiment of the present invention, Figure 6 is a schematic diagram of a computer device 30 according to one embodiment of the present invention. Computer program product 10 includes a computer program 11. When executed by a processor 32, computer program 11 implements the steps of any of the above-described dirty page flushing methods. Computer-readable storage medium 20 stores computer program 11. When executed by processor 32, computer program 11 implements the steps of any of the above-described dirty page flushing methods. Computer device 30 may include memory 31, processor 32, and computer program 11 stored in memory 31 and executed by processor 32.

[0142] The computer program 11 for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages ​​and procedural programming languages. The computer program 11 may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuit.

[0143] In the description of this embodiment, the computer program product 10 is a related product including the computer program 11 .

[0144] For the purposes of the description of this embodiment, the computer-readable storage medium 20 is a tangible device capable of retaining and storing the computer program 11, and can be any device that can contain, store, communicate, propagate, or transmit the program 11 for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable storage medium 20 include the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the foregoing.

[0145] The computer device 30 can be, for example, a server, a desktop computer, a laptop computer, a tablet computer, or a smartphone. In some examples, the computer device 30 can be a cloud computing node. The computer device 30 can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer device 30 can be implemented in a distributed cloud computing environment where remote processing devices linked via a communication network perform tasks. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media, including storage devices.

[0146] The computer device 30 may include a processor 32 adapted to execute stored instructions, and a memory 31 that provides temporary storage for the instructions during operation. The processor 32 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 31 may include random access memory (RAM), read-only memory, flash memory, or any other suitable storage system.

[0147] The computer device 30 may also include a network adapter / interface and an input / output (I / O) interface. The I / O interface allows data to be input and output with external devices that can be connected to the computer device. The network adapter / interface can provide communication between the computer device and a network, which is generally shown as a communication network.

[0148] At this point, those skilled in the art will recognize that, although a number of exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications consistent with the principles of the present invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

1. A dirty page flushing method, applied to any node in a database cluster, comprising: Respond to the global checkpoint process initiated by the database cluster; Participate in the collection of redo sites by the database cluster; Perform multiple node-level dirty page flushes; Synchronize the dirty page flushing progress of the buffer with other nodes; Perform a cluster-level dirty page flush with other nodes.

2. The dirty page flushing method according to claim 1, wherein: The steps of performing multiple node-level dirty page flushing steps include: Traverse its own buffer and identify all dirty pages to be written to disk; For each dirty page identified, determine its corresponding role attribute in the global resource lock information; Perform a round of node-level dirty page flushing based on the role attributes of the dirty page; Calculate the ratio of the number of dirty pages flushed to disk in this round to the number of dirty pages flushed to disk in the previous round; If the ratio is less than a preset proportional factor, stop traversing and brushing; If the ratio is greater than or equal to the set scaling factor, the own buffer is traversed again to prepare for the next round of node-level dirty page flushing.

3. The dirty page flushing method according to claim 2, wherein: The steps for performing a round of node-level dirty page flushing based on the role attributes of the dirty page include: When the role attribute is a local role, the dirty page is directly flushed to disk; When the role attribute is a global role, determine the type of buffer consistency lock held by the role; If the node does not hold a lock, clear the dirty mark of the dirty page and do not flush the disk; If the node holds an exclusive lock, the dirty page is flushed directly to disk; If the node holds a shared lock, the latest write mark corresponding to the dirty page is checked in the global resource lock information. If the node identified by the latest write mark is the current node, the dirty page is flushed to disk.

4. The dirty page flushing method according to claim 3, wherein: Also includes: Transferring dirty pages with other nodes, including dirty page forwarding and dirty page reception; If the dirty page forwarding is before the redo site collection phase, sending a first instruction to the receiving node, the first instruction being used to instruct the receiving node to process the dirty page on its own; If the dirty page forwarding is in the redo site collection phase, a second instruction is sent to the receiving node, where the second instruction is used to instruct the receiving node to write the dirty page to the disk.

5. The dirty page flushing method according to claim 4, wherein: If the dirty page forwarding is in the stage of multiple rounds of node-level dirty page disk flushing, determine the disk flushing status of the dirty page; If the forwarding node has not written the dirty page to disk, sending a third instruction to the receiving node, wherein the third instruction is used to instruct the receiving node to write the dirty page to disk; If the forwarding node has written the dirty page to the disk, a fourth instruction is sent to the receiving node, where the fourth instruction is used to instruct the receiving node that there is no need to process the dirty page.

6. The dirty page flushing method according to claim 5, wherein: If the dirty page receiving is in the dirty page disk write stage at the multi-round node level, when the first instruction, second instruction or third instruction of the sending node is received, the dirty page is marked and included in the scope of the global checkpoint disk write processing, waiting for the checkpoint process to perform statistical collection and processing.

7. The dirty page flushing method according to claim 6, wherein: After performing multiple node-level dirty page flushing steps, the following steps are also included: Participate in flushing all communication channels to prevent messages transmitted by dirty pages from remaining in the communication channels.

8. The dirty page flushing method according to claim 1, wherein: The steps for performing a cluster-level dirty page flush with other nodes include: Traverse its own buffer and identify all dirty pages to be written to disk; Simultaneously with other nodes, each node writes each dirty page it identifies to disk at the cluster level.

9. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the dirty page flushing method according to any one of claims 1 to 8 is implemented.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the dirty page flushing method according to any one of claims 1 to 8 is implemented.

Images