Transaction scheduling method and device for database and electronic equipment

Abstract

This application provides a database transaction scheduling method, apparatus, and electronic device, relating to the fields of cloud computing, big data, and edge computing technologies. The method includes: upon receiving a first transaction request, obtaining the first page data of the database that the first transaction request needs to access; if the management node of the first page data is a first node, accessing the first page data within a first time period and processing the transaction corresponding to the first transaction request based on the first page data; if the management nodes of the first page data include both a first node and a second node, accessing the first page data within a second time period and processing the transaction corresponding to the first transaction request based on the first page data. Thus, through a two-stage transaction scheduling mechanism, only intra-node transactions are processed in the first time period, and cross-node transactions are processed in the second time period, significantly reducing the frequency of page ownership transfers and lowering system overhead.

Description

Technical Field

[0001] This application relates to the fields of cloud computing, big data, and edge computing technologies, and in particular to a database transaction scheduling method, apparatus, and electronic device. Background Technology

[0002] In the development trend of cloud-native database systems, multi-primary databases based on shared-store architecture are gradually becoming a hot topic. For example... Figure 1 As shown, the core idea of ​​shared storage is to connect all master nodes to a unified shared storage system, enabling cross-node data sharing and multi-point writing without moving the physical location of the data. This architecture has good write scalability, disaster recovery capabilities, and resource utilization efficiency, and is therefore increasingly used in modern cloud-native scenarios with high availability and high load.

[0003] However, in a shared storage architecture, the existing First-Come, First-Served (FCFS) transaction scheduling strategy can easily lead to frequent and large-scale transfers of page ownership between different nodes, resulting in high system overhead. Summary of the Invention

[0004] This application provides a database transaction scheduling method, apparatus, and electronic device to address the problem of high system overhead.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a database transaction scheduling method, executed by a first node, the method comprising:

[0007] Upon receiving a first transaction request, the system retrieves the first page data of the database that the first transaction request needs to access. The database is divided into multiple partitions, each partition corresponding to a node. Each node manages the page data of one partition.

[0008] When the management node of the first page data is the first node, the first page data is accessed within the first time period, and the transaction corresponding to the first transaction request is processed based on the first page data;

[0009] In the case where the management node of the first page data includes the first node and the second node, the first page data is accessed during the second time period, and the transaction corresponding to the first transaction request is processed based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is the time period following the first time period.

[0010] Secondly, embodiments of this application provide a database transaction scheduling device applied to a first node, the device comprising:

[0011] The first acquisition module is used to acquire the first page data of the database that the first transaction request needs to access when a first transaction request is received. The database is divided into multiple partitions, and the multiple partitions correspond one-to-one with multiple nodes. Each of the multiple nodes is used to manage the page data of one partition.

[0012] The first processing module is used to access the first page data within a first time period when the management node of the first page data is the first node, and to process the transaction corresponding to the first transaction request based on the first page data.

[0013] The second processing module is configured to access the first page data during a second time period when the management nodes of the first page data include the first node and the second node, and process the transaction corresponding to the first transaction request based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is a time period located after the first time period.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, it implements the steps of the database transaction scheduling method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the database transaction scheduling method described in the first aspect.

[0016] Fifthly, a computer program product is provided, including computer instructions that, when executed by a processor, implement the steps of the database transaction scheduling method as described in the first aspect.

[0017] In this embodiment, upon receiving a first transaction request, the first page data of the database to be accessed by the first transaction request is obtained. The database is divided into multiple partitions, each corresponding to a specific node. Each node manages the page data of one partition. If the first node is the managing node for the first page data, the first page data is accessed within a first time period, and the transaction corresponding to the first transaction request is processed based on the first page data. If the first node and the second node are the managing nodes for the first page data, the first page data is accessed within a second time period, and the transaction corresponding to the first transaction request is processed based on the first page data. The first node and the second node are any two of the multiple nodes, and the second time period is the period following the first time period. This two-stage transaction scheduling mechanism, processing only intra-node transactions in the first time period and cross-node transactions in the second time period, significantly reduces the frequency of page ownership transfers, lowers system overhead, and improves overall system performance and resource utilization efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of a shared storage provided in an embodiment of this application;

[0020] Figure 2 This is one of the schematic diagrams of a database transaction scheduling method provided in the embodiments of this application;

[0021] Figure 3 This is a flowchart of a database transaction scheduling method provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of a shared storage architecture provided in an embodiment of this application;

[0023] Figure 5 This is a second schematic diagram of a database transaction scheduling method provided in the embodiments of this application;

[0024] Figure 6 This is a schematic diagram of the structure of a database transaction scheduling device provided in an embodiment of this application;

[0025] Figure 7This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] To facilitate understanding of the embodiments of this application, the following description is provided.

[0028] like Figure 1 As shown, the shared storage architecture connects all master nodes to a unified shared storage system, enabling cross-node data sharing and multi-point writing without moving the physical location of the data. In the shared storage architecture, data is stored in the storage layer, and master nodes can process any transaction without restriction. When master node N processes a read (R(x)) or write (W(x)) operation on data item x in transaction T, the process includes the following steps:

[0029] (1) O_T(xp): If N does not already own page xp, then ownership of it is transferred to N. This includes notifying other nodes that N now manages xp and ensuring that the page is synchronized. Since page xp is not bound to any particular master node, remote operation O_T(xp) is usually called frequently.

[0030] (2) L_T(xp): Acquires the lock of page xp;

[0031] (3) R (xp): Reads page xp;

[0032] (4) L_K(x): Acquire a shared (read) or exclusive (write) lock on x;

[0033] (5) R(x) / W(x): Perform a read (R(x)) or write (W(x)) on x;

[0034] (6) U_L(xp): Releases the lock on page xp.

[0035] In a shared storage architecture, all master nodes can directly access and modify pages on the same physical storage without explicit data migration or sharding. The storage layer typically employs highly available distributed block storage systems, which ensure data persistence and reliability through underlying multi-replica redundancy and consistency mechanisms. Compute nodes are interconnected with shared storage via high-speed networks, forming a layered architecture that separates storage and computation, enabling the system to have elastic scalability and independent resource scheduling capabilities.

[0036] However, while multi-write databases based on shared storage architecture solve the problems of high distributed transaction costs and complex data partitioning in shared-nothing architectures, their core bottleneck is also quite prominent: frequent page ownership transfers. In shared storage architectures, the cost of page ownership transfers is high, mainly due to two factors. First, the architecture relies on a component called the Global Page Lock Manager (GPLM), which can be centralized or distributed, to track the ownership of each page across various master nodes. When master node N requests ownership of page xp through operation O_T(xp), the GPLM instructs the current owner to release the page and grant ownership to N. This process, along with page data synchronization, involves four round-trip times (RTTs). Second, transactions typically access multiple pages, leading to a large number of ownership transfers. The problem is even more pronounced for hot pages, as multiple nodes compete for ownership, further increasing the overhead.

[0037] Therefore, although shared storage architecture has advantages in resource scheduling, deployment flexibility and high availability, it still faces significant challenges in transaction scheduling and consistency control. There is an urgent need for a more intelligent and efficient transaction scheduling mechanism to avoid the system performance degradation caused by frequent ownership transfers.

[0038] First-Come First-Served (FCFS), as a scheduling pattern, provides a simple and clear basis for the execution order of database transactions due to its intuitive logic. The advantage of this scheduling strategy lies in its clear implementation logic, which effectively avoids the additional computational overhead caused by complex scheduling algorithms, while also ensuring the fairness of transaction processing to a certain extent.

[0039] However, when focusing on the specific scenario of cloud-native databases based on shared storage architecture, the impact of transaction scheduling order is further amplified. In a shared storage architecture, multiple transactions may simultaneously access and operate on the same data resources, and the data ownership transfer operations involved in each transaction consume system resources. Because the FCFS scheduling model only processes transactions based on their arrival order, ignoring the access patterns and dependencies of different data resources, different transaction scheduling orders can lead to drastically different numbers of ownership transfers. Excessive ownership transfers not only increase system communication overhead and resource contention but may also trigger frequent lock waits and deadlock risks, ultimately causing significant differences in the database system's performance in key performance indicators such as throughput and response time. This phenomenon highlights the necessity and urgency of optimizing the traditional FCFS scheduling model or exploring more suitable scheduling strategies in a shared storage architecture to improve the overall performance of the database system.

[0040] To reduce page ownership transfers, a data partitioning strategy is typically employed. Figure 2 For example, the pages accessed by each transaction are marked with diagonal rectangles. Data pages P1 and P2 are assigned to master node N1, while databases P3 and P4 are assigned to master node N2. Transactions are routed to different primary partitions based on the assignment. However, due to poor scheduling, the problem of frequent ownership transfers remains unresolved, resulting in significant additional system overhead.

[0041] This application provides a database transaction scheduling method, apparatus, and electronic device to address the problem of high system overhead in current scheduling methods.

[0042] See Figure 3 , Figure 3 This is a flowchart of a database transaction scheduling method provided in an embodiment of this application. The method is executed by a first node, such as... Figure 3 As shown, the method includes the following steps:

[0043] Step 301: Upon receiving a first transaction request, obtain the first page data of the database that the first transaction request needs to access, wherein the database is divided into multiple partitions, each partition corresponds to a multiple node, and each of the multiple nodes is used to manage the page data of one partition.

[0044] Step 302: When the management node of the first page data is the first node, access the first page data within the first time period and process the transaction corresponding to the first transaction request based on the first page data;

[0045] Step 303: When the management node of the first page data includes the first node and the second node, access the first page data during the second time period and process the transaction corresponding to the first transaction request based on the first page data, wherein the second time period is the time period after the first time period.

[0046] This application proposes a two-phase transaction scheduling (2PS) mechanism, which reduces the transfer of ownership of transactions within and between nodes by dividing the system into two alternating phases.

[0047] Among them, the 2PS-based shared storage architecture is as follows: Figure 4 As shown. The architecture system includes:

[0048] A distributed shared storage layer: used to store data pages and logs;

[0049] A metadata server: responsible for maintaining the global page lock manager and the global validity table;

[0050] Multiple master nodes: used to execute transactions;

[0051] A transaction router: responsible for forwarding transactions to the master node;

[0052] A coordinator node: responsible for scheduling the phases of all nodes and ensuring fault tolerance.

[0053] The transaction router is responsible for receiving transaction requests from clients and routing them to a master node in the cluster for processing. To achieve efficient transaction routing, a range partitioning strategy logically divides the database into multiple partitions, establishing an affinity relationship between the data and the master node. Each page stores only tuples belonging to the same partition. The number of partitions equals the number of master nodes. This partitioning strategy can be specified by the user or generated using a workload-aware adaptive method.

[0054] The router maintains a partition metadata table, where each partition Parᵢ is uniquely associated with a master node Nᵢ. When a transaction request is received, the router identifies the partitions accessed by the transaction. If the transaction accesses only one partition, it is classified as an "intra-node transaction" and forwarded to the master node corresponding to that partition; if it accesses multiple partitions, it is classified as an "inter-node transaction" and forwarded to one of the master nodes involved in the partitions.

[0055] The coordinator is responsible for synchronizing phase states across all master nodes and adjusting the duration of the two phases, the "partition phase" and the "global phase," as needed. Furthermore, the coordinator monitors the heartbeat status of each master node. When a master node fails, the coordinator detects the problem and notifies other surviving nodes to recover, thus ensuring system consistency. The coordinator can be deployed on one or more physical machines and ensures high availability through Paxos or Raft protocols.

[0056] Based on the above architecture, taking the first transaction as an example, upon receiving a first transaction request, the first page of data in the database that the first transaction request needs to access is obtained. Based on the first page data, the master node managing the first page data can be determined, that is, the node that owns the first page data. The first page data may include one or more page data, and the master node managing the first page data may include one or more.

[0057] When the data on the first page is managed by the first node, the data on the first page is accessed through the first node within the first time period. The first master node is a single master node that manages one or more pages of data.

[0058] When the first page data includes multiple pages of data, and these multiple pages are managed by at least two master nodes (including the first node and the second node), accessing the first page data requires acquiring ownership of the multiple pages across nodes. In this case, access to the multiple pages is handled through the first node within a second time period to minimize the transfer of page data ownership.

[0059] In the above process, the access phase of the page database is divided into two phases: the partition phase (first phase) and the global phase (second phase). In the first phase, only intra-node transactions are processed (if a transaction only accesses one partition, it is classified as an "intra-node transaction"), and only the data of a single node is accessed. In the second phase, inter-node transactions are processed (if multiple partitions are accessed, it is classified as an "inter-node transaction"), which requires obtaining data ownership across nodes.

[0060] As shown in Table 1, upon receiving a transaction request, when entering the partitioning phase, the master node initializes resources, and the phase switching thread notifies the coordinator that it is ready to enter the partitioning phase (row 4 in the table). Once all master nodes report readiness, the coordinator responds to the sysEpoch function and instructs them to begin the partitioning phase.

[0061]

[0062] When the coordinator sends a message to each master node, instructing it to enter the partitioning phase, each master node initiates local preparation work. Before this, the master node requests pages in its "affinity partition" that it does not yet own from the global page lock manager; at the same time, it also requests the set of data pages in its partition that have been marked as invalid from the global validity table (row 5 of Table 1).

[0063] After completing the above preparations, the master node sets its running status to "partitioned" (row 6 of Table 1) and begins scheduling the execution of transactions within the node. At this point, the node has full data access permissions within its affinity partition and can freely perform read and write operations on these pages without any transfer of page ownership.

[0064] Once the state transition is complete, the worker threads within the node will detect the new state and begin processing transactions (line 23). During the partitioning phase, the master node only schedules and executes transactions within its own node (i.e., transactions only access the data partitions managed by the master node), behaving similarly to handling local transactions in a shared-nothing architecture. Through this approach, the system avoids cross-node synchronization and coordination, significantly reducing the overhead of network communication and global lock operations.

[0065] As a concrete example, such as Figure 5 As shown, pages P1 and P2 belong to master node N1, and pages P3 and P4 belong to master node N2. During the partitioning phase, N1 executes transactions that only access its local pages, such as T1, T5, and T10, while transactions involving cross-partition page access, such as T3 and T8, are deferred to the global phase for processing.

[0066] Because each master node processes pre-divided and non-overlapping partitions, no page conflicts or cross-node contention occur during the partitioning phase. This "divide and conquer" approach allows all master nodes to execute transactions in parallel and efficiently without involving page ownership transfers or adding invalid page markers, minimizing the complexity of system scheduling.

[0067] The partitioning phase runs for a dynamically determined time window, denoted as tp milliseconds. Once this time is reached, the "phase switching thread" within the master node switches the node state to "switching" (row 8 of Table 1) and waits for all worker threads to complete their current transactions (row 9 of Table 1). After all worker threads have entered the waiting state, they will pause execution and await the coordinator's instruction to proceed to the next phase, the global phase.

[0068] Once the coordinator receives messages from all master nodes indicating that they are ready to enter the global phase, it will notify all master nodes to begin executing the global phase (Table 1, row 11).

[0069] In this phase, all transactions—including those initially identified as inter-node transactions by the transaction router, and those identified as cross-partition access only during the partitioning phase—will be scheduled for execution. Unlike the partitioning phase, in the global phase, each master node can access any page in all logical partitions, not just those in its affinity partition. In other words, master nodes are no longer limited to local pages and can schedule and execute any cross-partition transaction. At the beginning of the global phase, however, each master node still only owns pages in its affinity partition.

[0070] like Figure 5 As shown, during the global phase, both master nodes N1 and N2 can handle transactions accessing any page. For example, when transaction T4 wants to access page P1, and the current ownership of P1 is still held by N1 (because T3 just wrote to the page on N1), N2 first needs to check the global lock status of P1 in its local page lock manager. If it finds that it does not have ownership of P1, it must request the global page lock manager to transfer the exclusive ownership of page P1 from N1 and synchronize the latest content of the page. After the ownership transfer is completed, T4 can modify the tuples in P1 on N2. After this, T8 and T7 will also continue to perform write operations on page P1 through a similar process. Since these operations involve competition and transfer of the same page among multiple master nodes, multiple page ownership transfers will occur.

[0071] In this embodiment, a two-stage transaction scheduling mechanism is used to process only intra-node transactions in the first time period and cross-node transactions in the second time period. This can significantly reduce the frequency of page ownership transfer and improve the overall system performance and resource utilization efficiency.

[0072] Optionally, the sum of the durations of the first time period and the second time period is the total duration of the current first iteration cycle; the method further includes:

[0073] Obtain the first throughput of the first time period and the second throughput of the second time period in the second iteration cycle, wherein the second iteration cycle is the iteration cycle preceding the first iteration cycle.

[0074] Obtain the proportion of the transaction volume in the first time period of the second iteration cycle to the total transaction volume of the first iteration cycle;

[0075] The durations of the first and second time periods in the first iteration cycle are determined based on the first throughput, the second throughput, and the ratio.

[0076] The first iteration period is the current period, and the duration of the current period is the sum of the duration of the first time period and the duration of the second time period.

[0077] The second iteration period is the iteration period preceding the current period. In some implementations, the second iteration period is the previous iteration period preceding the current period, or it is the average of multiple historical iteration periods.

[0078] Each iteration cycle is divided into a first time period and a second time period, corresponding to the two stages mentioned above (i.e., the partition stage and the global stage). The transaction volume of the first time period of the second iteration cycle can be understood as the transaction volume processed in the first time period, or the transaction volume corresponding to the received transaction requests. The total transaction volume of the first iteration cycle can be understood as the transaction volume corresponding to the received transaction requests or the transaction volume processed within the total time period corresponding to the first iteration cycle.

[0079] In the 2PS algorithm, the duration tp (in milliseconds) of the partition phase and the duration tg of the global phase are dynamically determined, while the total duration of the entire iteration cycle (i.e., a complete "partition + global" phase) remains fixed. As shown in the formula: tp + tg = e.

[0080] Here, 'e' represents the total duration of a complete system iteration cycle. Its value is adjustable and can be configured by the user or through other means. The default value is consistent with the group commit interval set in the system. For example, in MySQL, it corresponds to the `binlog_group_commit_sync_delay` parameter. Group commit is an optimization strategy used by most databases. It merges multiple transactions committed within a certain period of time, thereby amortizing the cost of flushing the Write-Ahead Log (WAL) to disk. This strategy ensures that the blocking latency of most transactions is less than or close to the group commit latency, thus avoiding a significant performance degradation perceived by the user.

[0081] To dynamically adjust tp and tg, 2PS introduces the following optimization model. The coordinator periodically collects the throughput of each master node during the partitioning phase and the global phase, denoted as follows:

[0082] αp: Throughput during the partitioning phase, such as (transactions per second);

[0083] αg: Throughput in the global phase;

[0084] C: Proportion of transactions within a node

[0085] Based on this, the appropriate two-stage time formula for the coordinator is: (αp × tp) / C = (αg × tg) / (1 - C).

[0086] Since the total duration of the iteration cycle is fixed, the durations of the first and second time periods can be obtained using the above method. The formula above ensures that the amount of transactions completed within each phase matches the proportion of transactions in the actual workload. In other words, it attempts to match the transaction execution capacity of each phase with the distribution of transaction types within the current system load, thereby minimizing the risk of transactions being delayed across multiple iteration cycles. Phase switching is a strongly synchronous operation in the system, meaning all master nodes must switch states simultaneously to enter the next phase. However, since the duration of each phase is typically a few hundred milliseconds, the switching overhead has a relatively small impact on overall system performance.

[0087] By dynamically determining the duration of the partition phase (tp) and the global phase (tg) by the ratio of throughput to transaction type, the optimal matching between the processing capacity of transactions in different phases and the system load is ensured, thereby reducing cross-cycle latency.

[0088] To further reduce log write overhead, 2PS employs a group commit strategy: at the end of each partition phase and the global phase, all transactions within that phase are committed to their respective partitions. This keeps the expected commit latency for each transaction within e / 2.

[0089] Optionally, before accessing the first page data within the first time period, the method further includes:

[0090] Determine whether the first page data exists in the pre-acquired invalid page set of the first node, wherein the invalid page set is a set of page data in an invalid state;

[0091] If the first page data exists in the set of invalid pages, the second page data is obtained from the target storage area and stored. The second page data is the latest version of the first page data.

[0092] The first page data is deleted from the invalid page set, and the second page data is locked.

[0093] In this case, "page data is invalid" can be understood as the page data version not being the latest version, and a newer version already exists.

[0094] Before accessing the first page data during transaction execution, the first node first checks if the first page data exists in its invalid page set. If it does, the first page data is invalid. The first node immediately synchronizes the latest version of the page (i.e., the second page data) from the global cache or a remote node, then removes the first page data from the invalid set and locks the second page data.

[0095] During transaction execution, the Multiversion Two-Phase Locking (MV2PL) concurrency control protocol is used to lock the target tuple before reading or writing pages, ensuring isolation and consistency. The advantage of this mechanism is that it synchronizes data pages only when necessary, effectively reducing unnecessary data transfers and lock contention, further improving execution efficiency.

[0096] Optionally, before determining whether the first page data exists in the pre-acquired set of invalid pages of the first node, the method further includes:

[0097] The page data managed by the first node is determined from the pre-obtained global valid table, and the invalid page set is determined from the page data managed by the first node. The global valid table is a status table of all page data in the database, and the global valid table is used to identify the valid or invalid status of page data.

[0098] The method further includes:

[0099] Once all transactions are completed within the second time period, control each node corresponding to all transactions to release ownership of the page data in the non-affinity partition and update the global valid table, where the non-affinity partition of each node is a partition not managed by that node.

[0100] The metadata server is responsible for maintaining the global page lock manager and the global validity table, which can identify the valid or invalid status of each page's data.

[0101] Before entering the partitioning phase, the master node (taking the first node as an example) requests the set of pages marked as invalid within its partition from the global valid table (Table 1, row 5).

[0102] Once the global phase has been executed for tg milliseconds and all worker threads have completed their transactions, the master node will release ownership of all pages it holds but which are not part of its affinity partitions, and update the global valid tables if the master node operated on these pages during the period it held ownership (Table 1, row 15).

[0103] By using the above method, the globally valid table can be updated in each cycle, reducing the transfer of ownership of page data and lowering system overhead.

[0104] Optionally, the first page data includes third page data and fourth page data; accessing the first page data during the second time period includes:

[0105] If the management node of the third page data is the first node and the management node of the fourth page data is the second node, then acquire ownership of the fourth page data;

[0106] Based on ownership of the fourth page data, the third page data and the fourth page data are accessed during the second time period.

[0107] If the page data that the first transaction request needs to access includes the third page data and the fourth page data, and the ownership of the third page data and the fourth page data belongs to the first node and the second node respectively, the first node first acquires the ownership of the fourth page data, and then accesses the third page data and the fourth page data based on the ownership of the third page data already owned by the first node and the acquired ownership of the fourth page data.

[0108] By employing the above methods, the transfer of page data ownership is reduced, thereby lowering system overhead.

[0109] Optionally, the method further includes:

[0110] If the first page data includes the fifth page data managed by the third node, and the first node accesses the fifth page data within the first time period, the access operation corresponding to the first transaction request is suspended.

[0111] Access the fifth page data during the second time period.

[0112] If the transaction corresponding to the first transaction request is misjudged as an intra-node transaction when identified by the transaction router, but actually accesses the fifth page data (i.e. accesses the non-affinity partition page) during execution, and the ownership of the fifth page data belongs to the third node, the system will immediately suspend the transaction and postpone it to a subsequent global phase for rescheduled execution.

[0113] This application's embodiments can be used in a cloud-native multi-write architecture based on shared storage. In this architecture, all master nodes can access a unified shared data page through a high-speed network, and data consistency during concurrent access is ensured through a page ownership protocol. The system combines static data partitioning with transaction routing, enabling most transactions to be completed within the affinity master node, avoiding cross-node coordination, and thus circumventing the distributed scheduling overhead of the traditional two-phase commit protocol.

[0114] A two-phase transaction scheduling mechanism (2PS) based on a shared storage architecture is used to improve the transaction execution performance of multi-master database systems in high-concurrency, strong-consistency scenarios. This mechanism divides system scheduling into a partition phase and a global phase. Through transaction affinity identification and phased scheduling synchronization, the system can dynamically adapt to different load structures without knowing the transaction content in advance, thereby effectively reducing the frequency of page ownership transfer and improving the overall system throughput.

[0115] Tests conducted on SmallBank and TPC-C workloads showed that the system performance based on the two-phase transaction scheduling algorithm was improved by 97% to 168% compared to the system performance based on the first-come, first-served scheduling algorithm.

[0116] In some implementations, a global intelligent scheduling algorithm that does not require stage division, such as a machine learning-based transaction scheduler, can be used to dynamically predict the cross-node access patterns of transactions, optimize the scheduling order in advance, and reduce the frequency of page ownership transfer.

[0117] In some implementations, an asynchronous page multi-replica mechanism is introduced, allowing multiple master nodes to hold read-only copies of popular pages. Consistency issues are resolved during the commit phase through version control and conflict detection, avoiding performance bottlenecks caused by the transfer of exclusive ownership.

[0118] The 2PS transaction scheduling mechanism of this application effectively improves the throughput and response time of the database system by reducing page ownership transfer, optimizing transaction scheduling and resource utilization, and can be directly applied to commercial database products, cloud service platforms and data-intensive enterprise application systems.

[0119] See Figure 6 , Figure 6 This is a schematic diagram of the structure of a database transaction scheduling device provided in an embodiment of this application, applied to the first node, such as... Figure 6 As shown, the database transaction scheduling device 600 includes:

[0120] The first acquisition module 601 is used to acquire the first page data of the database that the first transaction request needs to access when a first transaction request is received. The database is divided into multiple partitions, and the multiple partitions correspond one-to-one with multiple nodes. Each of the multiple nodes is used to manage the page data of one partition.

[0121] The first processing module 602 is used to access the first page data within a first time period when the management node of the first page data is the first node, and to process the transaction corresponding to the first transaction request based on the first page data.

[0122] The second processing module 603 is used to access the first page data during a second time period when the management node of the first page data includes the first node and the second node, and to process the transaction corresponding to the first transaction request based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is the time period located after the first time period.

[0123] Optionally, the sum of the durations of the first time period and the second time period is the total duration of the current first iteration cycle; the device further includes:

[0124] The second acquisition module is used to acquire the first throughput of the first time period and the second throughput of the second time period of the second iteration cycle, wherein the second iteration cycle is the iteration cycle that precedes the first iteration cycle.

[0125] The third acquisition module is used to acquire the proportion of the transaction volume in the first time period of the second iteration cycle to the total transaction volume of the first iteration cycle;

[0126] The first determining module is used to determine the duration of the first time period and the second time period in the first iteration cycle based on the first throughput, the second throughput and the ratio.

[0127] Optionally, the device further includes:

[0128] The second determining module is used to determine whether the first page data exists in the pre-acquired invalid page set of the first node, wherein the invalid page set is a set of page data in an invalid state;

[0129] The fourth acquisition module is used to acquire second page data from the target storage area and store the second page data when the first page data exists in the invalid page set, wherein the second page data is the latest version of the first page data;

[0130] The third processing module is used to delete the first page data from the invalid page set and lock the second page data.

[0131] Optionally, the device further includes:

[0132] The third determining module is used to determine the page data managed by the first node from the pre-acquired global valid table, and to determine the invalid page set in the page data managed by the first node. The global valid table is a status table of all page data in the database, and the global valid table is used to identify the valid or invalid status of page data.

[0133] The device further includes:

[0134] The update module is used to control each node corresponding to all transactions to release ownership of page data in the non-affinity partitions and update the global valid table when all transaction processing is completed in the second time period. The non-affinity partitions of each node are partitions that do not belong to the management of each node.

[0135] Optionally, the first page data includes third page data and fourth page data; the second processing module is specifically used for:

[0136] If the management node of the third page data is the first node and the management node of the fourth page data is the second node, then acquire ownership of the fourth page data;

[0137] Based on ownership of the fourth page data, the third page data and the fourth page data are accessed during the second time period.

[0138] Optionally, the device further includes:

[0139] The abort module is used to abort the access operation corresponding to the first transaction request when the first page data includes the fifth page data managed by the third node, and the first node accesses the fifth page data during the first time period.

[0140] The second access module is used to access the fifth page data during the second time period.

[0141] The database transaction scheduling device can achieve Figure 1 The various processes implemented in the method embodiments can achieve the same technical effect, and will not be described again here to avoid repetition.

[0142] It should be noted that the electronic device provided in this application embodiment is a transaction scheduling device capable of executing the above-mentioned database. Therefore, all implementation methods of the transaction scheduling method embodiment of the above-mentioned database are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0143] See Figure 7 As shown in the figure, this application embodiment also provides an electronic device, including a bus 601, a transceiver 602, an antenna 603, a bus interface 604, a processor 605, and a memory 606.

[0144] Processor 605, used for:

[0145] Upon receiving a first transaction request, the system retrieves the first page data of the database that the first transaction request needs to access. The database is divided into multiple partitions, each partition corresponding to a node. Each node manages the page data of one partition.

[0146] When the management node of the first page data is the first node, the first page data is accessed within the first time period, and the transaction corresponding to the first transaction request is processed based on the first page data;

[0147] In the case where the management node of the first page data includes the first node and the second node, the first page data is accessed during the second time period, and the transaction corresponding to the first transaction request is processed based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is the time period following the first time period.

[0148] exist Figure 7 In this document, a bus architecture (represented by bus 601) is used. Bus 601 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 605 and memory represented by memory 606. Bus 601 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 604 provides an interface between bus 601 and transceiver 602. Transceiver 602 may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 605 is transmitted over a wireless medium via antenna 603, which further receives data and transmits data to processor 605.

[0149] Processor 605 manages bus 601 and general processing, and also provides various functions, including timing, peripheral interface, voltage regulation, power management, and other control functions. Memory 606 can be used to store data used by processor 605 during operation.

[0150] Alternatively, the processor 605 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD).

[0151] Optionally, the sum of the durations of the first time period and the second time period is the total duration of the current first iteration cycle; the processor 605 is also configured to execute:

[0152] Obtain the first throughput of the first time period and the second throughput of the second time period in the second iteration cycle, wherein the second iteration cycle is the iteration cycle preceding the first iteration cycle.

[0153] Obtain the proportion of the transaction volume in the first time period of the second iteration cycle to the total transaction volume of the first iteration cycle;

[0154] The durations of the first and second time periods in the first iteration cycle are determined based on the first throughput, the second throughput, and the ratio.

[0155] Optionally, the processor 605 is also used to perform:

[0156] Determine whether the first page data exists in the pre-acquired invalid page set of the first node, wherein the invalid page set is a set of page data in an invalid state;

[0157] If the first page data exists in the set of invalid pages, the second page data is obtained from the target storage area and stored. The second page data is the latest version of the first page data.

[0158] The first page data is deleted from the invalid page set, and the second page data is locked.

[0159] Optionally, the processor 605 is also used to perform:

[0160] The page data managed by the first node is determined from the pre-obtained global valid table, and the invalid page set is determined from the page data managed by the first node. The global valid table is a status table of all page data in the database, and the global valid table is used to identify the valid or invalid status of page data.

[0161] The method further includes:

[0162] Once all transactions are completed within the second time period, control each node corresponding to all transactions to release ownership of the page data in the non-affinity partition and update the global valid table, where the non-affinity partition of each node is a partition not managed by that node.

[0163] Optionally, the first page data includes third page data and fourth page data; the processor 605 is specifically used to execute:

[0164] If the management node of the third page data is the first node and the management node of the fourth page data is the second node, then acquire ownership of the fourth page data;

[0165] Based on ownership of the fourth page data, the third page data and the fourth page data are accessed during the second time period.

[0166] Optionally, the processor 605 is also used to perform:

[0167] If the first page data includes the fifth page data managed by the third node, and the first node accesses the fifth page data within the first time period, the access operation corresponding to the first transaction request is suspended.

[0168] Access the fifth page data during the second time period.

[0169] It should be noted that the electronic device provided in this application embodiment is a device capable of executing the above-described database transaction scheduling method. Therefore, all implementation methods of the above-described database transaction scheduling method embodiment are applicable to this electronic device and can achieve the same or similar beneficial effects. To avoid repetition, this embodiment will not elaborate further.

[0170] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, this computer program implements the various processes of the above-described database transaction scheduling method embodiments and achieves the same technical effects. To avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0171] This application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above-described... Figure 1 The various processes of the method embodiments shown can achieve the same technical effect, and will not be described again here to avoid repetition.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0174] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of transaction scheduling for a database, characterized by, Executed by the first node, the method includes: Upon receiving a first transaction request, the system retrieves the first page data of the database that the first transaction request needs to access. The database is divided into multiple partitions, each partition corresponding to a node. Each node manages the page data of one partition. When the management node of the first page data is the first node, the first page data is accessed within the first time period, and the transaction corresponding to the first transaction request is processed based on the first page data; In the case where the management node of the first page data includes the first node and the second node, the first page data is accessed during the second time period, and the transaction corresponding to the first transaction request is processed based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is the time period following the first time period.

2. The method of claim 1, wherein, The sum of the durations of the first time period and the second time period is the total duration of the current first iteration cycle; the method further includes: Obtain the first throughput of the first time period and the second throughput of the second time period in the second iteration cycle, wherein the second iteration cycle is the iteration cycle preceding the first iteration cycle. Obtain the proportion of the transaction volume in the first time period of the second iteration cycle to the total transaction volume of the first iteration cycle; The durations of the first and second time periods in the first iteration cycle are determined based on the first throughput, the second throughput, and the ratio.

3. The method of claim 1, wherein, Before accessing the first page data within the first time period, the method further includes: Determine whether the first page data exists in the pre-acquired invalid page set of the first node, wherein the invalid page set is a set of page data in an invalid state; If the first page data exists in the set of invalid pages, the second page data is obtained from the target storage area and stored. The second page data is the latest version of the first page data. The first page data is deleted from the invalid page set, and the second page data is locked.

4. The method of claim 3, wherein, Before determining whether the first page data exists in the pre-acquired set of invalid pages of the first node, the method further includes: The page data managed by the first node is determined from the pre-obtained global valid table, and the invalid page set is determined from the page data managed by the first node. The global valid table is a status table of all page data in the database, and the global valid table is used to identify the valid or invalid status of page data. The method further includes: Once all transactions are completed within the second time period, control each node corresponding to all transactions to release ownership of the page data in the non-affinity partition and update the global valid table, where the non-affinity partition of each node is a partition not managed by that node.

5. The method according to any one of claims 1 to 4, characterized in that, The first page data includes the third page data and the fourth page data; accessing the first page data during the second time period includes: If the management node of the third page data is the first node and the management node of the fourth page data is the second node, then acquire ownership of the fourth page data; Based on ownership of the fourth page data, the third page data and the fourth page data are accessed during the second time period.

6. The method according to claim 1, characterized in that, The method further includes: If the first page data includes the fifth page data managed by the third node, and the first node accesses the fifth page data within the first time period, the access operation corresponding to the first transaction request is suspended. Access the fifth page data during the second time period.

7. An apparatus for transaction scheduling of a database, characterized by: Applied to the first node, the device includes: The first acquisition module is used to acquire the first page data of the database that the first transaction request needs to access when a first transaction request is received. The database is divided into multiple partitions, and the multiple partitions correspond one-to-one with multiple nodes. Each of the multiple nodes is used to manage the page data of one partition. The first processing module is used to access the first page data within a first time period when the management node of the first page data is the first node, and to process the transaction corresponding to the first transaction request based on the first page data. The second processing module is used to access the first page data during a second time period when the management node of the first page data includes the first node and the second node, and to process the transaction corresponding to the first transaction request based on the first page data, wherein the first node and the second node are any two of the plurality of nodes, and the second time period is the time period located after the first time period.

8. An electronic device, comprising: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the transaction scheduling method for the database as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the database transaction scheduling method as described in any one of claims 1 to 6.

10. A computer program product, characterised in that, It includes computer instructions that, when executed by a processor, implement the steps of the transaction scheduling method for the database as described in any one of claims 1 to 6.

Images