A parallel execution method and device based on transaction merging
By merging single transactions into multiple transactions in the database, setting up commit LSN and parallel LSN, combining savepoint and rollback policies, the data conflict problems caused by transaction merging and parallel execution are solved, and efficient consistency of data synchronization is achieved.
Patent Information
- Application Number
- CN202111503811.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-09
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-12-09
AI Technical Summary
During the real-time database replication process, when using transaction merging and parallel execution strategies, data conflicts are easily caused and the consistency of data synchronization is affected.
By combining single transactions into multiple transactions, setting up multi-transaction commit LSN and parallel LSN, registering them to the execution thread link list, and using the judgments of parallel LSN, separator LSN and operation merge execution to handle single transactions, ensuring that it is avoided during parallel execution Data conflicts, use savepoint and rollback policies to handle operations that do not meet the criteria.
It effectively avoids data conflicts caused by parallel execution, ensures consistency of data synchronization, and improves the performance of data synchronization.
Smart Images

Figure CN114296887B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to the technical field of database data processing, and in particular to a parallel execution method and device based on transaction merging. Background Art
[0002] At present, heterogeneous database replication technology based on log parsing is widely used. This technology captures the database log at the source end, parses the INSERT, UPDATE, and DELETE operations in the log, and then sends it to the target end. The target end reversely generates the log information and restores it to SQL statement mode, and then uses the database general interface to apply it to the target database to achieve data replication. Therefore, in the process of real-time database replication, the execution efficiency of the target end is an important factor affecting the performance of data synchronization. When executing synchronization transactions on the target database, it is necessary to execute transaction operations based on the transaction submission order, otherwise errors will occur when the related operations are executed, thereby affecting the correctness of synchronization. In order to pursue the performance of data synchronization, the transaction merging strategy is often adopted when executing synchronization transactions, merging small transactions into large transactions. This can not only reduce the number of executions by merging the same operations of different transactions, but also reduce the number of transaction submissions, thereby achieving the purpose of improving synchronization performance. In addition, by increasing the number of threads executing transactions and adopting a multi-threaded parallel strategy to simultaneously store multiple merged transactions, the synchronization performance can be further improved. However, when transaction merging and parallel execution strategies are adopted at the same time, data conflicts will occur during parallel execution, which will in turn affect the consistency of data synchronization.
[0003] In view of this, overcoming the defects of the prior art is an urgent problem to be solved in the field of this technology. Summary of the invention
[0004] One of the technical problems to be solved by the present invention is to solve the problem of data conflicts caused by adopting conventional transaction merging and multi-threaded parallel execution strategies, thereby affecting the consistency of data synchronization.
[0005] The present invention adopts the following technical solution:
[0006] In a first aspect, the present invention provides a parallel execution method based on transaction merging, the method comprising:
[0007] Merge a single transaction into multiple transactions and set the commit LSN and parallel LSN of the multiple transactions, and register the multiple transactions to the execution thread list;
[0008] Single transaction processing step: The execution thread extracts the next single transaction from the multiple transactions and starts executing it, taking the commit LSN of the single transaction as the current separator LSN;
[0009] Enter the corresponding step by judging the parallel LSN size, the separator LSN size, the number of rows affected by the current operation, and the judgment of the operation merge execution;
[0010] Thread submission judgment step: judge whether the current single transaction is the last single transaction, if so, execute the thread submission operation; if not, re-enter the single transaction processing step.
[0011] Preferably, the deployment preparation step is also included:
[0012] Deploy data synchronization services on both the source database and the target database;
[0013] After the target-side data synchronization service is started, initialize the pending execution list, execution thread, wake-up list, and execution thread list;
[0014] After receiving the source operation, the target data synchronization service classifies and manages it according to the transaction ID of the operation. When a transaction is committed, it is stored in the pending execution linked list in the order of the LSN size of the committed operation.
[0015] Preferably, the single transaction processing step further includes: determining the wake-up linked list of the current execution thread, and waking up the execution thread whose wake-up type is execution waiting and whose execution LSN is less than the dividing line LSN.
[0016] Preferably, the corresponding steps are entered by judging the parallel LSN size, the separator LSN size, the number of rows affected by the current operation, and the judgment of the operation merge execution:
[0017] Parallel LSN determination step: selecting to enter the operation type determination step or the separator line LSN determination step according to the LSN of the current operation and the size of the parallel LSN of the execution thread before the current execution thread;
[0018] Separator LSN judgment step: choose whether to generate a save point or wait for wakeup based on the LSN of the current operation and the size of the separator LSN of the execution thread before the current execution thread;
[0019] Determination of operation merging and execution: Extract operations to be executed according to the operation sequence in the current single transaction, and choose whether to merge and execute based on whether the current operation meets the conditions for merging and execution;
[0020] Determine the number of rows affected by the current operation: Compare the number of rows affected by the current operation with the expected number of rows affected to choose whether to continue executing the operation in the current single transaction or perform a partial rollback operation.
[0021] Preferably, the parallel LSN determination step specifically includes: determining whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads arranged before the current execution thread, if so, executing the current operation, if not, entering the separation line LSN determination step.
[0022] Preferably, the dividing line LSN judgment step specifically includes: judging whether the LSN of the current operation is smaller than the dividing line LSNs of all execution threads arranged before the current execution thread, and if so, entering the conflict detection step; if not, searching from back to front for the first execution thread whose dividing line LSN is smaller than the LSN of the current operation, and adding the current execution thread to the wake-up linked list of the found execution thread, and marking the wake-up type as execution wait.
[0023] Preferably, the judgment step of the operation merge execution specifically includes: judging whether the ready execution LSN extracted according to the current single transaction operation sequence is smaller than the parallel LSN, if so, performing merge execution, otherwise, generating a save point SAVEPOINT and then executing, and then entering the judgment step of the number of rows affected by the current operation.
[0024] Preferably, the step of judging the number of rows affected by the current operation specifically includes: judging whether the number of rows affected by the operation is consistent with expectations; if so, entering the step of judging the operation merge execution; if not, performing a rollback operation, rolling back to the previous save point SAVEPOINT, and then adding the current execution thread to the wake-up list of the previous execution thread in the execution thread list, setting the wake-up type to submit wait, and then suspending and waiting to be awakened.
[0025] Preferably, the thread submission judgment step specifically includes: judging whether the current single transaction is the last single transaction, if so, executing the submission and waking up all the waiting execution threads in the wake-up list, and then removing the execution thread from the execution thread list; otherwise, re-entering the single transaction processing step.
[0026] In a second aspect, the present invention further provides a parallel execution device based on transaction merging, which is used to implement the parallel execution method based on transaction merging described in the first aspect, and the device includes:
[0027] At least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor, they are used to complete the parallel execution method based on transaction merging described in the first aspect.
[0028] The present invention aims at the above-mentioned problems existing in the prior art, and formulates a feasible solution for the efficient parallel execution of multiple transactions formed after the single transaction is merged. The LSN submitted by the single transaction is divided into execution intervals, and the conditions for parallel execution are formulated. The operations that meet the parallel execution conditions in the execution interval are merged and executed, and the operations that do not meet the conditions are rolled back and wait for the previous multiple transactions to be submitted before execution. The rollback strategy can effectively avoid data conflicts caused by parallel execution, thereby avoiding data loss or operation suspension, and ensuring the consistency of data synchronization during the operation execution process. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0030] Figure 1 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0031] Figure 2 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0032] Figure 3 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0033] Figure 4 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0034] Figure 5 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0035] Figure 6 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0036] Figure 7 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0037] Figure 8 It is a flowchart of a method for parallel execution based on transaction merging provided by an embodiment of the present invention;
[0038] Fig. 9 It is a structural diagram of a parallel execution device based on transaction merging provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be arbitrarily combined with each other to form a feasible technical solution, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.
[0040] In the description of the present invention, the terms "inside", "outside", "longitudinal", "lateral", "upper", "lower", "top", "bottom" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and do not require that the present invention must be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on the present invention.
[0041] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0042] Embodiment 1:
[0043] For the convenience of description, the transaction in the source log of the present invention is called a "single transaction", and the transaction formed by merging multiple single transactions is called a "multi-transaction".
[0044] The embodiment of the present invention provides a parallel execution method based on transaction merging, such as Figure 1 As shown, the specific steps are as follows:
[0045] Step 101: merge a single transaction into multiple transactions, set commit LSNs and parallel LSNs of the multiple transactions, and register the multiple transactions into an execution thread linked list.
[0046] The process of merging a single transaction into multiple transactions in step 101 requires preliminary deployment work. In order to highlight the method of the present invention, the deployment process before the transaction merger is described later. The execution thread extracts N single transactions from the to-be-executed transaction linked list in the order of transaction commit LSNs and merges them into one multiple transaction, and sets the commit LSN of the last single transaction as the commit LSN of the multiple transaction, and registers in the execution thread linked list according to the commit LSN of the multiple transaction. At this time, the execution thread sets the commit LSN of the first single transaction of the multiple transaction as the parallel LSN. Since the extraction process in the multiple transaction is extracted in order from small to large according to the commit LSN of the single transaction, the multiple transaction contains the smallest commit LSN in the single transaction, which is the set parallel LSN. At this time, the commit LSN of the last single transaction in the multiple transaction is the maximum value of the commit LSN of the single transaction in the multiple transaction.
[0047] Step 102: The execution thread extracts the next single transaction from the multiple transactions and starts to execute it, takes the commit LSN of the single transaction as the current dividing line LSN, determines the wakeup linked list of the current execution thread, and wakes up the execution thread whose wakeup type is execution wait and whose execution LSN is less than the dividing line LSN.
[0048] Step 102 represents a single transaction processing step. In this step, since the execution thread extracts multiple transactions merged from multiple single transactions, when the next single transaction extracted by the execution thread starts to execute, the single transaction needs to be executed in the order of the multiple transactions merged. At this time, the setting of the dividing line is to distinguish the single transaction currently executed from the single transaction that has been executed before. By judging the current execution thread, the execution thread with the wake-up type of execution waiting and the execution time less than the dividing line LSN is selected for wake-up.
[0049] Step 103: Select to enter the operation type determination step or the separation line LSN determination step according to the LSN of the current operation and the size of the parallel LSN of the execution thread before the current execution thread.
[0050] Step 104: whether to generate a save point or wait for wakeup is selected according to the LSN of the current operation and the size of the separator LSN of the execution thread before the current execution thread.
[0051] Step 105: extract operations to be executed according to the operation sequence in the current single transaction, and select whether to merge the operations according to whether the current operations meet the conditions for merge execution.
[0052] Step 106: Compare the number of rows affected by the currently executed operation with the expected number of rows affected to select whether to continue to execute the operation in the current single transaction or to perform a partial rollback operation.
[0053] Steps 103-106 represent the judgment steps executed by the current operation in different situations, wherein step 103 represents the parallel LSN judgment step, which determines whether to enter the operation type judgment step or the separator line LSN judgment step by comparing the extracted LSN of the current operation with the parallel LSN of the execution thread before the current execution thread; step 104 represents the separator line LSN judgment step, which enters the corresponding step operation according to the size of the LSN extracted from the current operation and the separator line LSN before this execution thread; steps 105 and 106 respectively represent the judgment of the operation merge execution and the judgment of the number of rows affected by the current operation. The corresponding judgment rules have been stated in the steps and will not be repeated here.
[0054] Step 107: Determine whether the current single transaction is the last single transaction. If so, execute the thread commit operation; if not, re-enter the single transaction processing step.
[0055] Step 107 represents the thread commit step, in which the LSN of the current single transaction is extracted. When the LSN is the largest and the current operation is a commit operation, the thread commit operation is executed, otherwise the single transaction execution step is re-entered.
[0056] In addition, in the preferred solution of this embodiment, before merging transactions, it is necessary to deploy synchronization services on the source end and the target end, and it is also necessary to initialize the corresponding linked lists. Figure 2 As shown in the figure, the specific deployment preparation steps include:
[0057] Step 201: Deploy data synchronization services on both the source database and the target database.
[0058] Step 202: After the target-side data synchronization service is started, the to-be-executed linked list, the execution thread, the wake-up linked list, and the execution thread linked list are initialized.
[0059] Step 203: After receiving the source operation, the target-side data synchronization service classifies and manages the operation according to the transaction ID. When a transaction is committed, the transaction is stored in the to-be-executed linked list in the order of the LSN size of the committed operation.
[0060] Steps 201-203 provide a feasible premise for deployment preparation steps, merging of subsequent transactions, and implementation of corresponding parallel execution methods.
[0061] Step 103 is a step for determining the parallel LSN. In this preferred embodiment, step 103 can be further refined, such as Figure 3 As shown, the specific steps include:
[0062] Step 1031: Determine whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads before the current execution thread. In this step, the LSN of the current operation is first extracted, and the LSN is compared with the parallel LSN size of the execution threads before the current execution thread to enter the following determination step.
[0063] Step 1032: If yes, execute the current operation; if no, enter the separation line LSN determination step.
[0064] Step 104 is a separation line LSN determination step. In this preferred embodiment, step 104 can be further refined, such as Figure 4 As shown, the specific steps include:
[0065] Step 1041: Determine whether the LSN of the current operation is smaller than the LSN of the separators of all execution threads that are arranged before the current execution thread. In this step, the LSN of the current operation is first extracted, and the LSN is compared with the LSN of the separators of all execution threads that are arranged before the current execution thread to enter the following determination step.
[0066] Step 1042: If yes, enter the conflict detection step; if not, find the first execution thread whose separator line LSN is smaller than the LSN of the current operation from back to front, add the current execution thread to the wake-up list of the found execution thread, and mark the wake-up type as execution wait.
[0067] Step 105 is a step for determining whether to merge and execute operations. In this preferred embodiment, step 105 can be further refined, such as Figure 5 As shown, the specific steps include:
[0068] Step 1051: Determine whether the prepared execution LSN extracted according to the current single transaction operation sequence is smaller than the parallel LSN. In this step, the current operation LSN is first extracted and compared with the parallel LSN to enter the following determination step.
[0069] Step 1052: If yes, perform a merge execution; otherwise, generate a save point SAVEPOINT and execute again, and then enter the step of determining the number of rows affected by the current operation.
[0070] Step 106 is a step for determining the number of rows affected by the current operation. In this preferred embodiment, step 106 can be further refined, such as Figure 6 As shown, the specific steps include:
[0071] Step 1061: Determine whether the number of rows affected by the operation is consistent with the expected number. In this step, the LSN of the current operation is first extracted, and the number of rows affected is compared with the expected number of rows affected, and then the following determination step is entered. It is worth noting that when the merge is executed, the number of merge operations is the expected number of rows affected. If the number of rows affected obtained after execution is inconsistent with the data of the merge operation, it means that one or more operations in the merged batch of operations have not been successfully executed.
[0072] Step 1062: If yes, continue to extract the next operation; otherwise, roll back to the previous save point SAVEPOINT, and then add the current execution thread to the wake-up list of the previous execution thread in the execution thread list, set the wake-up type to submit wait, and then suspend waiting to be awakened.
[0073] Step 107 is a thread submission step judgment step. In this preferred embodiment, step 107 can be further refined, such as Figure 7 As shown, the specific steps include:
[0074] Step 1071 determines whether the current single transaction is the last single transaction. In this step, the LSN of the current single transaction is first extracted, and the LSN of the current transaction is used to determine whether it is the last single transaction, and then the following determination step is entered. It is worth noting that the merged single transactions have been sorted according to the size of the LSN submitted by the transaction in the deployment step. When the transactions are merged, the last single transaction actually corresponds to the single transaction with the largest LSN submitted in this batch.
[0075] If yes, step 1072 executes the commit and wakes up all waiting execution threads in the wake-up linked list, and then removes the execution thread from the execution thread linked list; otherwise, re-enters the single transaction processing step.
[0076] like Figure 8 As shown, it represents the whole process flow chart of the parallel execution method based on transaction merging. Before merging transactions, certain preparation steps are required, and then the parallel execution rules are formulated. The parallel execution method of transaction merging is implemented in the corresponding preparation steps. First, by deploying synchronization services on the source and target ends, after the target end data synchronization service is started, the initial to-be-executed linked list, execution thread, wake-up linked list and execution thread linked list are generated. At this time, the to-be-executed linked list consists of submitted single transactions and is stored in the to-be-executed linked list in the order of the submitted LSN size; after the preparation steps are completed, the transaction merging rules and the parallel execution method are formulated. By adopting the strategy of partially rolling back to the last save point and then waiting for the execution thread in front to submit and re-execute, the parallelism of the conflict-free data synchronization operation can be optimized, thereby improving the performance of data synchronization.
[0077] In the embodiment of the method, first, when two execution threads execute multiple transactions in parallel, the commit LSN of a single transaction is used to divide different parallel intervals. Since the resources accessed by these operations in the first parallel interval do not conflict, the operation LSNs in the two execution threads can be unconditionally parallel in the first parallel interval; secondly, starting from the second parallel interval, the execution thread with the commit LSN in the back may access the data modified by the execution thread with the commit LSN in the front in the first parallel interval in this interval, but the modification of these data has not been committed at this time, so the subsequent execution thread may not be able to access the data it wants. For this situation, the subsequent execution thread creates a save point before executing in parallel, and then identifies this possibility by checking the number of rows affected by the execution operation. When this situation occurs, a partial rollback to the previous save point is adopted and then the execution thread in the front is waited for submission and re-execution. Since there are multiple single transactions in the multiple transactions executed by the execution thread, different parallel execution intervals are divided by using the commit LSN of each single transaction. When multiple execution threads are executed, the operations falling in the same parallel execution interval can be executed in parallel, thereby improving the performance of data synchronization.
[0078] Embodiment 2:
[0079] The embodiment of the present invention is based on the method described in Example 1, and uses a specific example scenario to show the implementation process of the method. The details are as follows:
[0080] The source database table T1 (ID INT, DATA INT) contains 3 rows of data. Its data is distributed in the database as follows:
[0081]
[0082]
[0083] The source side has the following log operations:
[0084] LSN Transaction ID operate 1 1 UPDATE T1 SET DATA=11 WHERE ID=1 AND DATA=1; 2 3 UPDATE T1 SET DATA=33 WHERE ID=3 AND DATA=3; 3 1 COMMIT; 4 2 UPDATE T1 SET DATA=22 WHERE ID=2 AND DATA=2; 5 3 UPDATE T1 SET DATA=333 WHERE ID=3 AND DATA=33; 6 4 UPDATE T1 SET DATA=111 WHERE ID=1 AND DATA=11; 7 2 COMMIT; 8 3 COMMIT; 9 4 COMMIT;
[0085] 1. Create two execution threads A and B
[0086] 2. Execution thread A extracts transactions with single transaction IDs 1 and 2 and merges them into a multi-transaction. It uses the commit LSN 7 of single transaction 2 as the commit LSN of the multi-transaction and registers it in the execution thread linked list.
[0087] 3. Execution thread B extracts transactions with single transaction IDs 3 and 4 and merges them into a multi-transaction. It uses the commit LSN 9 of single transaction 4 as the commit LSN of the multi-transaction and registers it in the execution thread linked list. At this time, execution thread A already exists in the execution thread linked list. Since the commit LSN of B is larger than that of A, A is ranked ahead of B in the execution thread linked list.
[0088] 4. Execution thread A divides the parallel intervals into [1-3] and [3-7] according to the commit LSN of the single transaction, and sets the commit LSN 3 of the first single transaction as the parallel LSN.
[0089] 5. Execution thread B divides the parallel intervals into [7-8] and [8-9] according to the commit LSN of the single transaction, and sets the commit LSN 8 of the first single transaction as the parallel LSN.
[0090] 6. Execution threads A and B start executing in parallel.
[0091] The execution flow diagram is as follows:
[0092]
[0093]
[0094]
[0095]
[0096] In the specific example scenario of the present invention, when a single transaction is merged into multiple transactions, execution thread B needs to wait until execution thread A commits before it can start execution, because execution thread B cannot determine whether the data it accesses conflicts with execution thread A. Serial execution can ensure data synchronization consistency but sacrifices synchronization performance.
[0097] It can be seen from the above parallel execution process of the present invention that when the execution thread B executes the operation with LSN of 2, since it is smaller than the parallel LSN of the execution thread A, it can be unconditionally parallel, which is a point to improve the synchronization performance. When executing the operation with LSN of 5, it needs to wait for the execution thread A to complete all operations of single transaction 1, and then execute by setting a save point, and the execution result is consistent with the expectation, which is another point to improve the synchronization performance. When executing the operation with LSN of 6, it is executed by setting a save point, but the check fails when affecting the number of rows, so it rolls back to the previous save point and adds itself to the wake-up list of the execution thread A, and executes again after the execution thread A submits, thereby ensuring the consistency of data synchronization when parallel execution fails.
[0098] Comparing the present invention with the existing solutions, the present invention uses the commit LSN of a single transaction in multiple transactions to divide the parallel intervals, and then combines the method of setting save points to perform trial parallel execution to improve the synchronization performance. If an execution error occurs, the consistency of the synchronized data is ensured by rolling back to the previous save point.
[0099] Embodiment 3:
[0100] like Fig. 9 FIG. 1 is a schematic diagram of a parallel execution device based on transaction merging according to an embodiment of the present invention. The parallel execution device based on transaction merging according to this embodiment includes one or more processors 21 and a memory 22. Fig. 9 A processor 21 is taken as an example.
[0101] The processor 21 and the memory 22 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.
[0102] The memory 22 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs and non-volatile computer executable programs, such as the parallel execution method based on transaction merging in Example 1. The processor 21 executes the parallel execution method based on transaction merging by running the non-volatile software programs and instructions stored in the memory 22.
[0103] The memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the processor 21 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0104] The program instructions / modules are stored in the memory 22, and when executed by the one or more processors 21, the parallel execution method based on transaction merging in the above embodiment 1 is executed, for example, the above described Figure 1-8 The steps shown.
[0105] It is worth noting that the information interaction, execution process, etc. between the modules and units within the above-mentioned devices and systems are based on the same concept as the processing method embodiment of the present invention. The specific contents can be found in the description of the method embodiment of the present invention and will not be repeated here.
[0106] A person skilled in the art may understand that all or part of the steps in the various methods of the embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.
[0107] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A parallel execution method based on transaction merging, characterized in that: include: Merge a single transaction into multiple transactions and set the commit LSN and parallel LSN of the multiple transactions, and register the multiple transactions to the execution thread list; Single transaction processing step: The execution thread extracts the next single transaction from the multiple transactions and starts executing it, taking the commit LSN of the single transaction as the current separator LSN; Entering corresponding steps by judging the parallel LSN size, the separator LSN size, the number of rows affected by the current operation, and the judgment of the operation merge execution; specifically including: parallel LSN judgment step: selecting to enter the operation type judgment step or the separator LSN judgment step according to the LSN of the current operation and the size of the parallel LSN of the execution thread before the current execution thread; Separator LSN judgment step: choose whether to generate a save point or wait for wakeup based on the LSN of the current operation and the size of the separator LSN of the execution thread before the current execution thread; Determination of operation merging and execution: Extract operations to be executed according to the operation sequence in the current single transaction, and choose whether to merge and execute based on whether the current operation meets the conditions for merging and execution; Determine the number of rows affected by the current operation: Compare the number of rows affected by the current operation with the expected number of rows affected to choose whether to continue to execute the operation in the current single transaction or perform a partial rollback operation; Thread submission judgment step: judge whether the current single transaction is the last single transaction, if so, execute the thread submission operation; if not, re-enter the single transaction processing step; It also includes deployment preparation steps: deploying data synchronization services on both the source database and the target database; after the target data synchronization service is started, initializing the pending execution list, execution thread, wake-up list and execution thread list; after the target data synchronization service receives the source operation, it classifies and manages it according to the transaction ID of the operation. When a transaction is committed, the transaction is stored in the pending execution list in the order of the LSN size of the committed operation.
2. The parallel execution method based on transaction merging according to claim 1, characterized in that: The single transaction processing step also includes: determining the wake-up linked list of the current execution thread, and waking up the execution thread whose wake-up type is execution waiting and whose execution LSN is less than the dividing line LSN.
3. The parallel execution method based on transaction merging according to claim 1, characterized in that: The parallel LSN determination step specifically includes: determining whether the LSN of the current operation is smaller than the parallel LSNs of all execution threads arranged before the current execution thread, if so, executing the current operation, if not, entering the separation line LSN determination step.
4. The parallel execution method based on transaction merging according to claim 3, characterized in that: The dividing line LSN judgment step specifically includes: judging whether the LSN of the current operation is smaller than the dividing line LSNs of all execution threads arranged before the current execution thread, and if so, entering the conflict detection step; if not, searching from back to front for the first execution thread whose dividing line LSN is smaller than the LSN of the current operation, and adding the current execution thread to the wake-up linked list of the found execution thread, and marking the wake-up type as execution wait.
5. The parallel execution method based on transaction merging according to claim 4, characterized in that: The judgment step of the operation merge execution specifically includes: judging whether the ready execution LSN extracted according to the current single transaction operation sequence is smaller than the parallel LSN, if so, performing merge execution, otherwise, generating a save point SAVEPOINT and then executing, and then entering the judgment step of the number of rows affected by the current operation.
6. The parallel execution method based on transaction merging according to claim 5, characterized in that: The step of judging the number of rows affected by the current operation specifically includes: judging whether the number of rows affected by the operation is consistent with expectations; if so, entering the judgment step of merging and executing the operations; if not, performing a rollback operation to the last save point SAVEPOINT, and then adding the current execution thread to the wake-up list of the last execution thread in the execution thread list, setting the wake-up type to submit wait, and then suspending and waiting to be awakened.
7. The parallel execution method based on transaction merging according to any one of claims 1 to 6, characterized in that: The thread submission judgment step specifically includes: judging whether the current single transaction is the last single transaction, if so, executing submission and waking up all waiting execution threads in the wake-up linked list, and then removing the execution thread from the execution thread linked list; otherwise, re-entering the single transaction processing step.
8. A parallel execution device based on transaction merging, characterized in that: The method comprises at least one processor and a memory, wherein the at least one processor and the memory are connected via a data bus, and the memory stores instructions that can be executed by the at least one processor, and after being executed by the processor, the instructions are used to complete the parallel execution method based on transaction merging as described in any one of claims 1 to 7.
