Database snapshot data processing method and apparatus, electronic device, and storage medium

Abstract

The present disclosure relates to a database snapshot data processing method and apparatus, an electronic device, and a storage medium. The method comprises: obtaining snapshot parameters, wherein the snapshot parameters comprise current global transaction commit sequence numbers corresponding to a snapshot time, the current global transaction commit sequence numbers are transaction commit sequence numbers recorded in a database at the snapshot time and used for obtaining a snapshot, and transaction commit sequence numbers are in one-to-one correspondence with transaction write-ahead log address sequence numbers of corresponding transactions; on the basis of the current global transaction commit sequence numbers, determining the visibility of transaction data conforming to the snapshot parameters; and filtering out transaction data invisible under the snapshot to obtain data conforming to snapshot visibility. The problems in the related art of poor performance, low snapshot efficiency and a low speed caused when determination of snapshot visibility based on transaction commit sequence numbers needs to be performed after transactions are completed are solved.

Description

Database snapshot data processing method, device, electronic device, and storage medium Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, electronic device, and storage medium for processing database snapshot data. Background Art

[0002] In databases, Multi-Version Concurrency Control (MVCC) has been widely used as a concurrency control technology to ensure transaction consistency. The emergence of Commit Sequence Number (CSN) technology has led to its development by addressing the performance bottleneck of traditional MVCC technology, which requires traversing all transaction states.

[0003] In the current CSN technology, when determining visibility, the CSN of a transaction in the database can only be obtained after the transaction is completed and submitted to the clog, while the LSN (clog sequence number) of a transaction can only be obtained after the transaction enters the commit date. When a snapshot determines transaction visibility based solely on the CSN, the update of the CSN depends on the growth of the transaction's LSN. Visibility determination can only be performed after the transaction corresponding to the previous CSN is completed and submitted. Only after obtaining the previous CSN and accumulating CSNs can the submission process and visibility determination of the transaction corresponding to the next LSN be performed.

[0004] That is, when determining the visibility of a snapshot transaction, it is necessary to wait for the completion of the transaction in progress. This will cause multiple transactions submitted in parallel to be submitted one by one in a fixed order before the visible rows can be determined. This will result in performance losses in parallel operation and cause low snapshot efficiency and slow speed.

[0005] Regarding the problem in related technologies where snapshot visibility is determined based on transaction commit sequence numbers, it is necessary to wait for the transaction to end, resulting in low performance, low snapshot efficiency, and slow speed. No effective solution has yet been proposed.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a method, device, electronic device, and storage medium for processing database snapshots, which at least address the problem in related technologies that when determining snapshot visibility based on transaction commit sequence numbers, it is necessary to wait for the transaction to end, resulting in low performance, low snapshot efficiency, and slow speed.

[0008] An embodiment of the present disclosure provides a data processing method for a database snapshot, comprising: obtaining snapshot parameters, wherein the snapshot parameters include a current global transaction commit sequence number corresponding to a snapshot time, the current global transaction commit sequence number being a transaction commit sequence number used to obtain the snapshot, recorded in the database at the snapshot time, and having a one-to-one correspondence between the transaction commit sequence number and a transaction write-ahead log address sequence number of the corresponding transaction; determining the visibility of transaction data that meets the snapshot parameters based on the current global transaction commit sequence number; and filtering out transaction data that is not visible under the snapshot to obtain data that meets the snapshot visibility.

[0009] The beneficial effects of the embodiments of the present disclosure are as follows: by obtaining the current global transaction commit sequence number corresponding to the snapshot time and the transaction pre-write log address sequence number of the corresponding transaction, as a snapshot parameter and as a basis for visibility judgment, after the parallel transaction enters the transaction completion process, the transaction pre-write log address sequence number is obtained, and the transaction commit sequence number can be accumulated. There is no need to rely on the completion and submission of the transaction corresponding to the previous transaction pre-write log address sequence number, and then accumulate the transaction commit sequence number to update the current global transaction commit sequence number, so that the transaction commit sequence number can have the characteristics of the transaction pre-write log address sequence number, that is, the transaction commit sequence number before the current global transaction commit sequence number can be considered to have ended the transaction completion process, and the transaction visibility can be judged based on this, without waiting for the completion of the transaction corresponding to the previous transaction pre-write log address sequence number, thereby improving the efficiency of visibility judgment and the efficiency of snapshots, and reducing the loss of parallel performance.

[0010] As an optional embodiment, the snapshot parameters also include a minimum active transaction identifier and a latest active transaction identifier. The visibility of transaction data that meets the snapshot parameters is determined based on the current global transaction commit sequence number, including: determining the visibility of a first transaction in the database whose transaction identifier is greater than or equal to the latest active transaction identifier as invisible, wherein the transactions in the database are assigned transaction identifiers from small to large in the order of creation; for a second transaction in the database whose transaction identifier is less than the minimum active transaction identifier, determining the visibility of the second transaction based on the transaction completion status corresponding to the second transaction; for a third transaction in the database whose transaction identifier is greater than or equal to the minimum active transaction identifier and less than the latest active transaction identifier, determining the visibility of the third transaction based on the current global transaction commit sequence number.

[0011] Since transaction identifiers are assigned in the order in which transactions are created, when determining visibility, we first compare the transaction identifier with the minimum active transaction identifier and the latest active transaction identifier to determine the first and second transactions whose visibility can be clearly determined. For the third transaction whose transaction identifier is between the minimum active transaction identifier and the latest active transaction identifier, we use the current global transaction commit sequence number to perform visibility determination to ensure the accuracy of visibility determination of transactions in the database.

[0012] As an optional embodiment, determining the visibility of the second transaction according to the transaction status corresponding to the second transaction includes: querying the transaction status of the second transaction according to the transaction completion log of the second transaction, wherein the transaction completion log is a log in the database that records the transaction execution and transaction completion process; when the transaction status indicates that the transaction is completed, determining the visibility of the second transaction as visible; when the transaction status indicates that the transaction is not completed, determining the visibility of the second transaction as invisible.

[0013] For the second transaction whose transaction ID is less than the minimum active transaction ID, it means that the transaction has completed the transaction completion process and may have been committed or rolled back and is still running. Therefore, the visibility of the second transaction depends on whether its transaction status indicates the end of the transaction. The second transaction that indicates the end of the transaction is determined to be visible, and the second transaction that indicates that the transaction is not ended is determined to be invisible.

[0014] As an optional embodiment, determining the visibility of the third transaction according to the current global transaction commit sequence number includes: querying the transaction commit sequence number and transaction status of the third transaction according to the mapping log of the third transaction, wherein the mapping log is generated according to the transaction commit sequence number, the transaction write-ahead log address sequence number, the transaction identifier, and the transaction status during the execution of the transaction completion process of the transaction in the database, and the transaction commit sequence number and the transaction write-ahead log address sequence number have a one-to-one correspondence; if the transaction commit sequence number of the third transaction is less than the current global transaction commit sequence number and the transaction status of the third transaction indicates that the transaction is completed, determining the visibility of the third transaction to be visible; if the transaction status of the third transaction indicates that the transaction is not completed, or does not have the transaction commit sequence number, or the transaction commit sequence number is greater than the current global transaction commit sequence number, determining the visibility of the third transaction to be invisible.

[0015] For the third transaction, which is also a transaction in the process of transaction completion, it may or may not be completed. First, we can determine whether the transaction commit sequence number of the third transaction is less than the current global transaction commit sequence number, that is, the third transaction was committed before the current global transaction commit sequence number in the snapshot parameters. Then, based on whether the transaction status of the third transaction indicates transaction completion, we determine whether it is invisible. If the transaction is not completed, it is invisible. If the transaction is completed, the visibility of the third transaction is determined to be visible. If the third transaction does not have a transaction commit sequence number, or the transaction commit sequence number is greater than the current global transaction commit sequence number, it means that it was committed after the snapshot time and is not visible to the snapshot.

[0016] As an optional embodiment, the method further includes: executing a transaction completion process in parallel with the transaction in the database to update the snapshot parameters, wherein the transaction completion process includes a transaction commit process and a transaction rollback process; obtaining the snapshot parameters includes: obtaining the snapshot parameters corresponding to the snapshot time according to the snapshot instruction.

[0017] Snapshot parameters are used to obtain transaction data of the database at the snapshot time according to the snapshot instruction. Snapshot parameters include the minimum active transaction identifier, the latest active transaction identifier, and the current global transaction commit sequence number. These parameters are updated during the transaction completion process, so that the current global transaction commit sequence number can reflect the visibility of the transaction in the database, so that the visibility of the snapshot transaction can be judged based on the current global transaction commit sequence number.

[0018] As an optional embodiment, the transactions in the database execute a transaction completion process in parallel and update the snapshot parameters, including: in the transaction completion process, after the data of the transaction in the database is determined, generating a transaction completion log; after generating the transaction completion log, updating the latest active transaction identifier; updating the current global transaction commit sequence number according to the transaction commit sequence number of the current transaction executing the transaction completion process; and updating the minimum active transaction identifier after resetting the transaction identifier of the data structure in the database.

[0019] During the transaction completion process of a database transaction, after the transaction completion log is generated, it can be determined that the data of the transaction will not change. The three snapshot parameters required for snapshot visibility judgment can be updated, including the minimum active transaction identifier, the latest active transaction identifier, and the current global transaction commit sequence number. This allows the snapshot parameters to effectively reflect the visibility of the transaction in the snapshot, so that the visibility of the snapshot transaction can be judged subsequently based on the current global transaction commit sequence number.

[0020] As an optional embodiment, before generating the transaction completion log, it also includes: after the transaction in the database enters the transaction completion process, generating a pre-write log, and obtaining the log address sequence number of the pre-write log as the transaction pre-write log address sequence number; and obtaining the current global latest transaction commit sequence number, adding one to accumulate, and obtaining the transaction commit sequence number; writing the transaction commit sequence number, the transaction pre-write log address sequence number, and the transaction identifier and transaction status into a mapping array to correspond the transaction commit sequence number and the transaction pre-write log address sequence number one to one, wherein the mapping array is used to generate a mapping log.

[0021] Before generating a transaction completion log, you need to generate a write-ahead log, write the transaction commit sequence number and the transaction write-ahead log address sequence number into the mapping number for binding, and write the transaction identifier and transaction status into the mapping array to facilitate subsequent updates to the current global transaction commit sequence number and query and obtain relevant parameters during visibility judgment.

[0022] As an optional embodiment, updating the current global transaction commit sequence number according to the transaction commit sequence number of the current transaction executing the transaction completion process includes: obtaining the transaction pre-write log address sequence number of the transaction corresponding to the current global transaction commit sequence number as the current global transaction write-ahead log address sequence number; determining whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number according to the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number; and if the current transaction needs to update the current global transaction commit sequence number, updating the current global transaction commit sequence number to the transaction commit sequence number.

[0023] When updating the current global transaction commit sequence number, it is necessary to rely on the current global transaction write-ahead log address sequence number. Based on the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number, if the current transaction needs to update the current global transaction commit sequence number, the current global transaction commit sequence number is updated to the transaction commit sequence number to complete the update of the current global transaction commit sequence number.

[0024] As an optional embodiment, determining whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number based on the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number includes: when the current global transaction write-ahead log address sequence number is greater than the transaction write-ahead log address sequence number, determining that the current transaction is a non-target transaction, wherein the non-target transaction is a transaction that does not need to update the current global transaction commit sequence number; when the current global transaction write-ahead log address sequence number is less than or equal to the transaction write-ahead log address sequence number, locking the current transaction, and after acquiring the lock, determining again whether the current global transaction Whether the write-ahead log address sequence number is greater than the locked transaction write-ahead log address sequence number; if the current global transaction write-ahead log address sequence number is greater than the locked transaction write-ahead log address sequence number, determine that the current transaction is a non-target transaction and release the lock; if the current global transaction write-ahead log address sequence number is less than or equal to the locked transaction write-ahead log address sequence number, obtain the mapping array, traverse the mapping transactions in the mapping array from the current global transaction commit sequence number to the current global latest transaction commit sequence number, select the latest mapping transaction whose transaction write-ahead log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number as the target transaction, and release the lock.

[0025] Specifically, the current global transaction write-ahead log address sequence number is compared with the transaction write-ahead log address sequence number of the current transaction to determine whether the current transaction is the target transaction for which the current global transaction commit sequence number needs to be updated. If the current transaction is not the target transaction, the target transaction is found by combining the mapping array, and the current global transaction commit sequence number is updated based on the mapping array of the target transaction. The current global transaction write-ahead log address sequence number is thus used to avoid confusion, frequent updates, and erroneous updates of the current global transaction commit sequence number due to the different speeds of transaction completion processes among multiple parallel transactions. This ensures the accuracy of the current global transaction commit sequence number and, in turn, the accuracy of snapshot visibility judgment.

[0026] As an optional embodiment, traversing the mapping transactions in the mapping array from the current global transaction commit sequence number to the current global latest transaction commit sequence number, selecting the latest mapping transaction whose transaction pre-write log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number as the target transaction, includes: judging the difference between the transaction pre-write log address sequence number of the mapping transaction corresponding to the traversed transaction commit sequence number and the current global transaction write-ahead log address sequence number; when the transaction pre-write log address sequence number of the mapping transaction corresponding to the traversed transaction commit sequence number is less than or equal to the current global transaction write-ahead log address sequence number, and when the transaction pre-write log address sequence number of the mapping transaction corresponding to the next traversed transaction commit sequence number is greater than the current global transaction write-ahead log address sequence number, determining that the mapping transaction corresponding to the traversed transaction commit sequence number is the target transaction.

[0027] When the current transaction is not a target transaction, the mapping array is traversed according to the current global transaction write-ahead log address sequence number, and the transaction write-ahead log address sequence number of the latest corresponding mapped transaction that meets the transaction commit sequence number is selected. The transaction that is less than or equal to the current global transaction write-ahead log address sequence number is selected as the target transaction to update the current global transaction commit sequence number to ensure the accuracy of the current global transaction commit sequence number.

[0028] As an optional embodiment, after traversing the mapping transactions in the mapping array from the global current transaction commit sequence number to the global latest transaction commit sequence number, and selecting the latest mapping transaction whose transaction pre-write log address sequence number is less than or equal to the transaction pre-write log address sequence number as the target transaction, it also includes: according to the transaction pre-write log address sequence number of the target transaction, obtaining the corresponding transaction identifier, transaction status and transaction commit sequence number from the mapping array, and generating a mapping log; updating the current global transaction write-ahead log address sequence number to the transaction write-ahead log address sequence number of the target transaction.

[0029] Determine the target transaction to update the current global transaction commit sequence number. Generate a mapping log for query during visibility judgment. Update the current global transaction write-ahead log address sequence number to facilitate other transactions to judge the target transaction and the next update judgment of the current global transaction commit sequence number.

[0030] As an optional embodiment, the method further includes: when read-only hot standby of the database is enabled, generating a playback snapshot parameter by replaying the write-ahead log of the transaction, wherein the current global transaction commit sequence number of the snapshot parameter corresponds one-to-one to the transaction write-ahead log address sequence number of the corresponding transaction; and obtaining corresponding playback snapshot data according to the snapshot parameter, wherein the playback snapshot data is synchronized with the content of the snapshot data when read-only hot standby is not enabled.

[0031] Taking snapshots in read-only hot standby scenarios ensures snapshot data synchronization. This means that any transaction visibility corresponding to the current global transaction commit sequence number in a read-only hot standby snapshot can be found in the same version in a snapshot taken without read-only hot standby enabled. This avoids the issue of snapshot data being out of sync due to replay in read-only hot standby scenarios. Similarly, this also solves the problem of waiting for transaction completion to determine snapshot visibility in read-only hot standby scenarios, improving snapshot acquisition efficiency in read-only hot standby scenarios.

[0032] An embodiment of the present disclosure provides a data processing device for a database snapshot, comprising: a snapshot module configured to obtain snapshot parameters, wherein the snapshot parameters include a current global transaction commit sequence number corresponding to a snapshot time, the current global transaction commit sequence number being a transaction commit sequence number recorded in the database at the snapshot time for obtaining the snapshot, and the transaction commit sequence number corresponding one-to-one to a transaction write-ahead log address sequence number of the corresponding transaction; a judgment module configured to judge the visibility of transaction data that meets the snapshot parameters based on the current global transaction commit sequence number; and a determination module configured to filter out transaction data that is not visible under the snapshot to obtain data that meets the snapshot visibility.

[0033] An electronic device provided by an embodiment of the present disclosure includes: a processor, and a memory storing a program, wherein the program includes instructions, and when the instructions are executed by the processor, the processor executes any one of the methods described above.

[0034] An embodiment of the present disclosure provides a non-transitory machine-readable medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the above methods.

[0035] The details of one or more embodiments of the present disclosure are set forth in the accompanying drawings and the description below to make other features, objects, and advantages of the present disclosure more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] To more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other embodiments based on these drawings without inventive effort.

[0037] FIG1 is a flowchart of visibility determination of a database snapshot according to an embodiment of the present disclosure;

[0038] FIG2 is a flowchart of a method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0039] FIG3 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0040] FIG4 is a flowchart of another visibility determination of a database snapshot according to an embodiment of the present disclosure.

[0041] FIG5 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0042] FIG6 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0043] FIG7 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0044] FIG8 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure.

[0045] FIG9 is a flowchart of a data processing device for a database snapshot according to an embodiment of the present disclosure.

[0046] FIG10 is a schematic structural diagram of an electronic device according to this embodiment. DETAILED DESCRIPTION

[0047] The following describes embodiments of the present invention in more detail with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0048] First, the professional terms that may be involved in this embodiment are explained and described as follows:

[0049] CSN: Commit sequence number, a unique identifier used to identify the order in which transactions are committed in a database. Each transaction is assigned a unique CSN number when it is committed.

[0050] CLOG: commit log, commit log

[0051] A transaction snapshot is a mirror image of the consistent state of a database at a specific point in time. It captures the data state of all committed transactions up to that point in time and provides a readable, static view that is unaffected by subsequent transactions. After obtaining a snapshot, transaction visibility must be determined.

[0052] xid: transaction identifier

[0053] The Log Sequence Number (LSN) is the sequence number of the storage address of the transaction write-ahead log (WAL). The WAL log is used to identify each data change operation in the database.

[0054] RO Hot Standby: A standby node that supports read-only services. (Read-only Hot Standby) refers to a database system in which a backup server is set to read-only mode, allowing users to perform read-only queries on the backup server. RO Hot Standby provides data redundancy and fault tolerance without affecting the performance of the primary server.

[0055] RW nodes: Read-write master nodes. RW snapshots are a database snapshot technology that allows read and write operations to be performed within a snapshot. Unlike traditional read-only snapshots, RW snapshots create a writable copy at a specific point in time, allowing modifications to be made without affecting the original data. This allows users to use snapshots for experiments, testing, or rollbacks without affecting the original data.

[0056] Traditional CSN technology advances and generates a transaction commit sequence number (CSN) for each non-read-only transaction when it is committed. Other transactions read the latest committed CSN as a transaction snapshot to determine data visibility. It strictly adheres to the rule that if the transaction CSN is less than the current latest CSN (that is, the current global transaction commit sequence number in the snapshot parameter, global_snapshot_current_csn), the transaction commit results are visible, while the remaining transaction commit results have not been committed at the time of obtaining the snapshot and are therefore invisible.

[0057] FIG1 is a flow chart of visibility determination of a database snapshot according to an embodiment of the present disclosure. In the above scheme, when determining visibility, it is necessary to wait for the committing transaction to complete the commit before proceeding. The specific process is shown in FIG1 .

[0058] When taking a snapshot, record the minimum active transaction ID (xid) as the minimum active transaction ID (snapshot.min) in the snapshot parameters. The ID of the most recently committed transaction (xid+1) is used as the latest active transaction ID (snapshot.max) in the snapshot parameters. The commit sequence number (csn+1) of the most recently committed transaction is used as the snapshot CSN (snapshot.csn). It should be noted that the snapshot CSN serves the same purpose as the current global transaction commit sequence number and, to some extent, has the same meaning. However, different descriptions are required in different scenarios.

[0059] When making visibility judgments, if the transaction sequence number xid of a transaction is greater than or equal to the latest active transaction identifier snapshot.max, the transaction is not visible.

[0060] If the transaction sequence number xid is less than the minimum active transaction identifier snapshot.min, it means that the transaction corresponding to xid has ended before the current transaction started. Query the transaction commit status through the transaction commit log clog. If the transaction is committed, it is visible; if the transaction has been rolled back, it is not visible.

[0061] When the transaction sequence number xid is between the minimum active transaction identifier snapshot.min and the latest active transaction identifier snapshot.max, you need to query the mapping log csnlog. If the transaction commit sequence number csn has a value and is smaller than snapshot.csn, the transaction is visible; otherwise, it is invisible. If the queried transaction status is committing, you need to wait until the corresponding transaction is committed to make a judgment.

[0062] When determining transaction visibility based on the obtained snapshot, if the corresponding transaction is in the committing state, you need to wait for the transaction to be committed. According to the transaction commit process, the transition from committing to committed requires waiting for the WAL write-ahead log to be persisted. The persistence process is a time-consuming disk I / O operation, which directly prolongs the transaction visibility determination time and affects transaction execution efficiency.

[0063] To more clearly illustrate the update of parameters when a transaction is committed, the transaction commit process is taken as an example. The transaction rollback process is similar.

[0064] The transaction submission process is as follows:

[0065] Update the latest completed transaction identifier (latest_completed_xid), which can be used to determine the latest active transaction identifier (snapshot.max) in the snapshot parameter above. Set the commit status corresponding to the transaction identifier (xid) to committing. Cumulatively update the transaction commit sequence number (csn) value (NextCSN++). Construct the transaction WAL write-ahead log and commit it. Flush the WAL write-ahead log and wait for the WAL write-ahead log to be persisted. Update the commit status to committed and write to the mapping log (csnlog). Write to the transaction commit log (clog). Update the process transaction status.

[0066] When a database uses read-only hot standby (RO) read-only standby, a new log type must be added to synchronize the committing state. However, when the hot standby obtains a snapshot and uses the current snapshot, if the corresponding transaction is in the committing state, visibility cannot be determined until the log replay completes the transaction. This increases read latency and reduces read efficiency for the RO read-only hot standby. Furthermore, inconsistencies between the snapshot obtained by the hot standby and the read-write (RW) primary node can lead to RO read inconsistencies.

[0067] In order to solve the problem in the related art that when determining snapshot visibility based on transaction commit sequence numbers, it is necessary to wait for the transaction to end, resulting in low performance, low snapshot efficiency and slow speed, the embodiment of the present disclosure provides a data processing method for database snapshots.

[0068] FIG2 is a flow chart of a method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG2 , the method for processing data of a database snapshot includes the following steps:

[0069] Step S201: Obtain snapshot parameters, where the snapshot parameters include the current global transaction commit sequence number corresponding to the snapshot time. The current global transaction commit sequence number is the transaction commit sequence number used to obtain the snapshot recorded in the database at the snapshot time. The transaction commit sequence number corresponds one-to-one with the transaction write-ahead log address sequence number of the corresponding transaction.

[0070] Step S202: Determine the visibility of transaction data that meets the snapshot parameters based on the current global transaction commit sequence number.

[0071] Step S203 : Filter out transaction data that is not visible under the snapshot to obtain data that meets the snapshot visibility.

[0072] The data processing method for the database snapshot provided by the embodiment of the present disclosure obtains the current global transaction commit sequence number corresponding to the snapshot time and the transaction write-ahead log address sequence number of the corresponding transaction as a snapshot parameter and as a basis for visibility judgment. Therefore, after the parallel transaction enters the transaction completion process, the transaction write-ahead log address sequence number is obtained, and the transaction commit sequence number can be accumulated. There is no need to rely on the completion and submission of the transaction corresponding to the previous transaction write-ahead log address sequence number to accumulate the transaction commit sequence number and update the current global transaction commit sequence number. As a result, the transaction commit sequence number can have the characteristics of the transaction write-ahead log address sequence number. That is, the transaction commit sequence number before the current global transaction commit sequence number can be considered to have completed the transaction completion process. This can be used to judge transaction visibility without waiting for the transaction corresponding to the previous transaction write-ahead log address sequence number to complete the submission. This improves the efficiency of visibility judgment and snapshot efficiency and reduces the loss of parallel performance.

[0073] The above snapshot parameters may include three parameters: the first parameter is the snapshot minimum transaction identifier snapshot.min, referred to as min, which is the smallest transaction identifier among the active transactions in the database corresponding to the snapshot time. Since transaction identifiers are assigned in ascending order when transactions are created, it can be understood that transactions with transaction identifiers xid before the snapshot minimum transaction identifier snapshot.min are all inactive transactions in the completed transaction process. These inactive transactions in the completed transaction process can be judged as visible to the snapshot based on whether they have ended their operation.

[0074] The second parameter is the snapshot's latest transaction identifier, snapshot.max (max for short), which is the latest transaction identifier among active transactions in the database at the snapshot time. Transactions with transaction identifiers xid after snapshot.max are transactions that have not yet started the transaction completion process and are therefore invisible to the snapshot.

[0075] Transactions whose transaction identifier xid is between the minimum transaction identifier snapshot.min and the latest transaction identifier snapshot.max can be understood as active transactions in the database corresponding to the snapshot time. They may have been committed or may be in the process of transaction completion. At this time, the third parameter of the snapshot parameter is needed, the current global transaction commit sequence number global_snapshot_current_csn. The current global transaction commit sequence number global_snapshot_current_csn is the latest transaction commit sequence number in the database at the snapshot time.

[0076] In the related art, database transactions are in the transaction completion process, wherein the transaction completion process includes the transaction commit process and the transaction rollback process. After entering the transaction completion process, the global CSN of the current database is accumulated to generate the corresponding transaction commit sequence number CSN. After the write-ahead log WAL is generated and persisted, the status of the transaction completion process is updated to a completed transaction. The transaction write-ahead log address sequence number LSN is also generated during the persistence of the write-ahead log WAL. In this way, the generation of the transaction commit sequence number CSN depends on the global CSN of the current database, and the transaction commit sequence numbers CSN before the global CSN of the current database all represent transactions that have completed the transaction completion process. That is, the accumulation of CSN by a transaction in the transaction completion process depends on the transaction corresponding to the previous transaction commit sequence number CSN to end the transaction submission process.

[0077] In this way, if the current global transaction commit sequence number global_snapshot_current_csn required for the snapshot has not been updated by the transaction, it is necessary to wait for the transaction in the database and continue the transaction completion process until the current database global CSN is accumulated to the current global transaction commit sequence number global_snapshot_current_csn of the snapshot parameter. Only then can visibility judgment be performed, that is, a snapshot can be generated.

[0078] In this embodiment, the transaction completion process is improved, and the transaction submission sequence number CSN of the transaction is mapped one-to-one with the transaction pre-write log address sequence number LSN of the corresponding transaction. After entering the transaction completion process, the address sequence number LSN of the transaction pre-write log WAL is first obtained, and at the same time, the global CSN of the current database is accumulated to generate the corresponding transaction submission sequence number CSN, and the transaction submission sequence number CSN is bound to the transaction pre-write log address sequence number LSN.

[0079] The transaction commit sequence number (CSN) is assigned after the transaction completes its process. It only indicates the start of the transaction completion process. The transaction write-ahead log (WAL) records the valid data of the transaction. After the WAL is committed and persistently stored, the transaction data layer does not change. Subsequent processes are all interaction- or process-related steps and do not affect the transaction data changes.

[0080] In this embodiment, the transaction completion process, upon entering the transaction completion process, first obtains the address sequence number (LSN) of the transaction's write-ahead log (WAL). This is equivalent to pre-acquiring the WAL address sequence number, reserving a persistent storage location for the WAL in the transaction completion process, and then accumulating the transaction commit sequence number. Therefore, after the transaction enters the transaction completion process, the transaction commit sequence number (CSN) is updated and compared with the current global transaction commit sequence number (global_snapshot_current_csn) as the basis for determining visibility.

[0081] Because the WAL log address sequence number is reserved in advance and bound to the transaction commit sequence number (CSN), the earlier transaction commit sequence number (CSN) will continue the transaction completion process until the WAL is persisted and the subsequent transaction completion process is complete. Once the earlier transaction commit sequence number (CSN) is determined, the later transaction commit sequence number (CSN) can be updated and the transaction completion process continues.

[0082] Therefore, when obtaining a snapshot, there is no need to wait for the transaction completion process to end in sequence before accumulating the subsequent transaction submission sequence numbers CSN, nor is it necessary for the transaction completion processes of subsequent transactions to end one by one in the order of the transaction submission sequence numbers CSN.

[0083] This greatly ensures the parallelism of the transaction completion process, improves the parallel efficiency of the transaction completion process, and ensures the timeliness and accuracy of the database's global transaction submission sequence number (CSN). After obtaining the snapshot parameters, the transaction visibility of the snapshot can be determined as quickly as possible, which improves the efficiency and speed of snapshot acquisition and reduces the loss of parallel performance.

[0084] The actions of the transaction completion process described above are automatically triggered and executed in the database based on the running status of parallel transactions. When obtaining a snapshot, you only need to obtain the snapshot parameters. The snapshot parameters contain the current global transaction commit sequence number global_snapshot_current_csn, which is updated instantly by each parallel transaction in the database. When obtaining a snapshot, the current global transaction commit sequence number global_snapshot_current_csn corresponding to the snapshot time is used as the criterion for determining transaction visibility.

[0085] Based on the current global transaction commit sequence number global_snapshot_current_csn in the snapshot parameter, the visibility of the transaction data that meets the snapshot parameter is determined, the invisible transaction data is deleted, and the snapshot data is determined.

[0086] FIG3 is a flow chart of another method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG3 , the process of determining transaction visibility includes:

[0087] When obtaining a snapshot, the currently active minimum transaction identifier (xid) is recorded as snapshot.min, and the latest submitted transaction is represented by xid+1 as snapshot.max. It should be noted that the latest submitted transaction represented by xid+1 here is for subsequent judgment. If it is greater than or equal to snapshot.max, it is considered invisible. It is also acceptable if it is not +1, but the judgment method needs to be changed accordingly. That is, if it is greater than snapshot.max, it is considered invisible.

[0088] The current global transaction commit sequence number global_snapshot_current_csn+1 is used as snapshot.csn. It should be noted that the current global transaction commit sequence number global_snapshot_current_csn+1 here has a similar purpose to the latest committed transaction xid+1 mentioned above, and it does not need to be +1.

[0089] When the transaction identifier xid of a transaction is greater than or equal to snapshot.max, the transaction is not visible.

[0090] If the transaction identifier xid is less than snapshot.min, the transaction has already ended before the start of the minimum active transaction at the time of the snapshot. Since the transaction completion process includes commit and rollback, the transaction ends when the commit process completes, but the transaction may still be running after the rollback process. Therefore, the transaction completion status can be queried using the transaction completion log (clog).

[0091] The transaction completion log (clog) is generated for both transaction commits and transaction rollbacks. A commit generates a commit log (submit-clog) corresponding to the commit status, while a rollback generates an abort-clog corresponding to the abort status. If the transaction completion log (clog) indicates that the transaction has been committed, it is visible; if it indicates that the transaction has been rolled back, it is invisible.

[0092] When the transaction identifier xid corresponding to the transaction is between snapshot.min and snapshot.max, it is necessary to query from the xid-csn mapping log, namely csnlog. If the transaction has no transaction commit sequence number csn (indicating that the transaction completion process has not started) or has been rolled back abort, it is not visible. If it has been committed, the transaction commit sequence number csn is determined to have a value and is smaller than snapshot.csn. In this case, the transaction is visible; otherwise, it is not visible.

[0093] Compared to Figure 1, when the transaction ID xid of a transaction is between snapshot.min and snapshot.max, querying the mapping log csnlog shows that if the queried transaction status is committing, the judgment must wait until the corresponding transaction commits. The process in Figure 3 does not require waiting for the corresponding transaction to commit, improving the efficiency of visibility judgment.

[0094] FIG4 is a flowchart of another database snapshot visibility determination process according to an embodiment of the present disclosure. As shown in FIG4 , as an optional embodiment, the snapshot parameters further include the minimum active transaction identifier and the latest active transaction identifier. The visibility of transaction data that meets the snapshot parameters is determined based on the current global transaction commit sequence number, including:

[0095] Step S401: Determine the visibility of the first transaction in the database whose transaction ID is greater than or equal to the latest active transaction ID as invisible, wherein the transactions in the database are assigned transaction IDs in ascending order according to the order in which they were created;

[0096] Step S402: for a second transaction whose transaction ID is smaller than the minimum active transaction ID in the database, determine the visibility of the second transaction according to the transaction completion status corresponding to the second transaction;

[0097] Step S403 : for a third transaction in the database whose transaction ID is greater than or equal to the minimum active transaction ID and less than the latest active transaction ID, determine the visibility of the third transaction according to the current global transaction commit sequence number.

[0098] Since transaction identifiers are assigned in the order in which transactions are created, when determining visibility, we first compare the transaction identifier with the minimum active transaction identifier and the latest active transaction identifier to determine the first and second transactions whose visibility can be clearly determined. For the third transaction whose transaction identifier is between the minimum active transaction identifier and the latest active transaction identifier, we use the current global transaction commit sequence number to perform visibility determination to ensure the accuracy of visibility determination of transactions in the database.

[0099] Because transaction IDs in the database are assigned in ascending order based on the order in which transactions are created, after obtaining the snapshot parameters, you can obtain the database's minimum active transaction ID, snapshot.min, and the latest active transaction ID, snapshot.max, at the snapshot time.

[0100] Therefore, the database transactions at the snapshot time are divided into the first transaction whose transaction identifier xid is greater than or equal to the latest active transaction identifier snapshot.max, the second transaction whose transaction identifier xid is less than the minimum active transaction identifier snapshot.min, and the third transaction whose transaction identifier xid is greater than or equal to the minimum active transaction identifier snapshot.min and less than the latest active transaction identifier snapshot.max.

[0101] For the first transaction, it can be understood that transactions with transaction identifiers xid after the latest transaction identifier snapshot.max of the snapshot are all transactions that have not started the transaction completion process. These transactions that have not started the transaction completion process are invisible to the snapshot.

[0102] The second transaction can be understood as transactions with transaction identifiers xid before the minimum transaction identifier snapshot.min of the snapshot, which are all inactive transactions in the completed transaction process.

[0103] Since the transaction completion process includes commit and rollback, the transaction needs to continue running after the rollback process. For inactive transactions that have completed the transaction completion process, if they have completed running, for example, they are in the committed state, then they are visible to the snapshot. If they have not completed running, for example, they are in the rolled back state, then they are invisible to the snapshot.

[0104] The third transaction can be understood as an active transaction in the database corresponding to the snapshot time. It may have been committed or may be in the process of transaction completion. At this time, it is necessary to use the third parameter of the snapshot parameter, the current global transaction commit sequence number global_snapshot_current_csn. The current global transaction commit sequence number global_snapshot_current_csn is the latest transaction commit sequence number in the database at the snapshot time.

[0105] Since the global transaction commit sequence numbers of the database are also gradually accumulated, in theory, the current global transaction commit sequence number global_snapshot_current_csn is used to determine transaction visibility. In practice, this requires that the transaction commit sequence number of the transaction be less than the current global transaction commit sequence number global_snapshot_current_csn at the snapshot time.

[0106] Furthermore, since the transaction commit sequence number (CSN) in this embodiment is bound to the transaction write-ahead log address sequence number (LSN) and is obtained when the transaction completion process begins, and the transaction completion process may include subsequent locking and unlocking steps, or some types of transaction completion processes may continue to run after completion, such as rollback. Therefore, transactions that have not completed the transaction completion process or have completed the rollback process but are still running are not visible to the snapshot.

[0107] As an optional embodiment, determining the visibility of the second transaction based on the transaction status corresponding to the second transaction includes: querying the transaction status of the second transaction based on the transaction completion log of the second transaction, wherein the transaction completion log is a log in a database that records the transaction execution and transaction completion process; when the transaction status indicates that the transaction is completed, determining the visibility of the second transaction as visible; when the transaction status indicates that the transaction is not completed, determining the visibility of the second transaction as invisible.

[0108] For the second transaction whose transaction ID is smaller than the minimum active transaction ID, it indicates that the transaction has completed the transaction completion process and may have been committed or rolled back and is still running.

[0109] Therefore, the visibility of the second transaction needs to depend on whether its transaction status indicates that the transaction is completed. The second transaction that indicates that the transaction is completed is determined to be visible, and the second transaction that indicates that the transaction is not completed is determined to be invisible.

[0110] The second transaction can be understood as transactions with transaction identifiers xid before the minimum transaction identifier snapshot.min of the snapshot, which are all inactive transactions in the completed transaction process.

[0111] Since the transaction completion process includes commit and rollback, the transaction needs to continue running after the rollback process. For inactive transactions that have completed the transaction completion process, if they have completed running, for example, they are in the committed state, then they are visible to the snapshot. If they have not completed running, for example, they are in the rolled back state, then they are invisible to the snapshot.

[0112] As an optional embodiment, the visibility of the third transaction is determined according to the current global transaction commit sequence number, including: querying the transaction commit sequence number and transaction status of the third transaction according to the mapping log of the third transaction, wherein the mapping log is generated according to the transaction commit sequence number, the transaction pre-write log address sequence number, the transaction identifier and the transaction status during the execution of the transaction completion process in the database, and the transaction commit sequence number and the transaction pre-write log address sequence number have a one-to-one correspondence; when the transaction commit sequence number of the third transaction is less than the current global transaction commit sequence number and the transaction status of the third transaction indicates that the transaction is completed, the visibility of the third transaction is determined to be visible; when the transaction status of the third transaction indicates that the transaction is not completed, or there is no transaction commit sequence number, or the transaction commit sequence number is greater than the current global transaction commit sequence number, the visibility of the third transaction is determined to be invisible.

[0113] For the third transaction, which is also a transaction in the process of transaction completion, it may or may not be completed. First, we can determine whether the transaction commit sequence number of the third transaction is less than the current global transaction commit sequence number, that is, the third transaction was committed before the current global transaction commit sequence number in the snapshot parameters. Then, based on whether the transaction status of the third transaction indicates transaction completion, we determine whether it is invisible. If the transaction is not completed, it is invisible. If the transaction is completed, the visibility of the third transaction is determined to be visible. If the third transaction does not have a transaction commit sequence number, or the transaction commit sequence number is greater than the current global transaction commit sequence number, it means that it was committed after the snapshot time and is not visible to the snapshot.

[0114] For the third transaction, you need to query the xid-csn mapping log, namely csnlog. If the transaction has no transaction commit sequence number csn (indicating that the transaction completion process has not started) or has been rolled back abort, it is not visible. If it has been committed, and the transaction commit sequence number csn has a value and is smaller than snapshot.csn, the transaction is visible; otherwise, it is not visible.

[0115] The mapping log csnlog is generated for transactions in the database during the transaction completion process based on the transaction commit sequence number, transaction write-ahead log address sequence number, transaction identifier, and transaction status. The transaction commit sequence number and the transaction write-ahead log address sequence number correspond one-to-one.

[0116] If the transaction status of the third transaction indicates that the transaction has ended, for example, it has been committed, and the transaction commit sequence number CSN is less than or equal to the current global transaction commit sequence number global_snapshot_current_csn, the visibility of the third transaction is determined to be visible. If the transaction status of the third transaction indicates that the transaction has not ended, for example, it has been rolled back, or it has no transaction commit sequence number, indicating that the transaction has not entered the transaction completion process, or the transaction commit sequence number is greater than the current global transaction commit sequence number global_snapshot_current_csn, indicating that the transaction entered the transaction commit process after the snapshot time, in all three cases, the visibility of the third transaction can be determined to be invisible.

[0117] FIG5 is a flow chart of another method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG5 , as an optional embodiment, the method further includes:

[0118] Step S501: The transactions in the database execute a transaction completion process in parallel to update snapshot parameters, wherein the transaction completion process includes a transaction commit process and a transaction rollback process;

[0119] Step S201 of obtaining snapshot parameters includes: Step S502 of obtaining snapshot parameters corresponding to the snapshot time according to the snapshot instruction.

[0120] Snapshot parameters are used to obtain transaction data of the database at the snapshot time according to the snapshot instruction. Snapshot parameters include the minimum active transaction identifier, the latest active transaction identifier, and the current global transaction commit sequence number. These parameters are updated during the transaction completion process, so that the current global transaction commit sequence number can reflect the visibility of the transaction in the database, so that the visibility of the snapshot transaction can be judged based on the current global transaction commit sequence number.

[0121] The update of snapshot parameters is an immediate update of each transaction in the database. When taking a snapshot, the snapshot parameters corresponding to the snapshot time are obtained according to the snapshot instruction, which serves as the basis for judging the visibility of subsequent transactions.

[0122] As an optional embodiment, transactions in the database execute a transaction completion process in parallel and update snapshot parameters, including: in the transaction completion process, after the transaction data in the database is determined, generating a transaction completion log; after generating the transaction completion log, updating the latest active transaction identifier; updating the current global transaction commit sequence number according to the transaction commit sequence number of the current transaction executing the transaction completion process; and updating the minimum active transaction identifier after resetting the transaction identifier of the data structure in the database.

[0123] During the transaction completion process of a database transaction, after the transaction completion log is generated, it can be determined that the data of the transaction will not change. The three snapshot parameters required for snapshot visibility judgment can be updated, including the minimum active transaction identifier, the latest active transaction identifier, and the current global transaction commit sequence number. This allows the snapshot parameters to effectively reflect the visibility of the transaction in the snapshot, so that the visibility of the snapshot transaction can be judged subsequently based on the current global transaction commit sequence number.

[0124] In the transaction completion process of the above database transactions, including the commit process and the rollback process, a transaction completion log (clog) is generated after the transaction data is finalized. The commit completion log is generated during the commit process, and the rollback completion log is generated during the rollback process.

[0125] Therefore, after the transaction completion log clog is generated, that is, the transaction is completed, the latest completed transaction identifier latest_completed_xid is updated, that is, the latest active transaction identifier is ended. The next transaction identifier after it can be the latest active transaction identifier snapshot.max. The general update method is to accumulate based on the current value.

[0126] After generating the transaction completion log clog, snapshot parameters need to be updated and unlocked. Therefore, the minimum active transaction identifier is updated after the last step of the transaction completion process, for example, after resetting the transaction identifier of the data structure in the database.

[0127] The current transaction's commit sequence number (CSN) is bound to the transaction's write-ahead log address sequence number (LSN). Furthermore, the CSN and LSN are determined at the beginning of the transaction's completion process. Therefore, the subsequent commit sequence is not necessarily strictly based on the CSN. This means that for concurrently active transactions in the completion process, the commit order is determined solely by their own execution speed and has no explicit correlation with the CSN increment.

[0128] This results in multiple concurrently active transactions entering the transaction completion process based on their transaction commit sequence number (CSN) potentially updating the current global transaction commit sequence number (global_snapshot_current_csn). The current global transaction commit sequence number (global_snapshot_current_csn) must match the CSN of the latest active transaction entering the transaction completion process; otherwise, visibility will be affected.

[0129] Multiple parallel transactions arranged by transaction commit sequence number (CSN) may complete one after another. This can cause some transactions with earlier CSNs to commit later, while others with later CSNs to commit earlier. Naturally, transactions submitted earlier have their CSNs updated earlier, while transactions submitted later have their CSNs updated later. This can cause confusion in the current global transaction commit sequence number (global_snapshot_current_csn), causing it to not accumulate gradually and even regress. This is an update error that directly affects the correctness of snapshot visibility.

[0130] This requires each transaction to confirm whether it needs to update the current transaction's CSN before updating the current transaction's CSN. To address this, this embodiment proposes updating the current global transaction's CSN based on the current transaction's CSN during the transaction completion process. The details are explained below.

[0131] FIG6 is a flow chart of another method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG6 , as an optional embodiment, before generating a transaction completion log, the method further includes:

[0132] Step S601: After a transaction in a database enters a transaction completion process, a write-ahead log is generated, and a log address sequence number of the write-ahead log is obtained as the transaction write-ahead log address sequence number;

[0133] Step S602, obtain the current global latest transaction submission sequence number, add one to the cumulative number, and obtain the transaction submission sequence number;

[0134] Step S603 , write the transaction commit sequence number, the transaction write-ahead log address sequence number, the transaction identifier, and the transaction status into a mapping array to establish a one-to-one correspondence between the transaction commit sequence number and the transaction write-ahead log address sequence number, wherein the mapping array is used to generate a mapping log.

[0135] Before generating a transaction completion log, you need to generate a write-ahead log, write the transaction commit sequence number and the transaction write-ahead log address sequence number into the mapping number for binding, and write the transaction identifier and transaction status into the mapping array to facilitate subsequent updates to the current global transaction commit sequence number and query and obtain relevant parameters during visibility judgment.

[0136] In related technologies, after generating the write-ahead log (WAL), the corresponding address sequence number (LSN) is generally obtained before persistent writing is performed. This approach requires that the transaction completion process complete the WAL before the WAL is committed and persisted in order to obtain the WAL address sequence number (LSN). Since the WAL is a necessary step in the transaction completion process, obtaining its LSN is also necessary for the transaction completion process. However, it can only be obtained after the WAL is committed, which still requires waiting for the transaction to commit.

[0137] Therefore, in this embodiment, after a transaction in the database enters the transaction completion process, a write-ahead log WAL is generated and the log address sequence number of the write-ahead log WAL is obtained as the transaction write-ahead log address sequence number LSN. This is to reserve storage space for the write-ahead log WAL. The write-ahead log WAL is also stored in sequence.

[0138] After obtaining the transaction write-ahead log address sequence number (LSN), the current global latest transaction commit sequence number (global_snapshot_current_csn) is obtained and incremented by one to obtain the transaction commit sequence number (CSN). This global latest transaction commit sequence number (global_snapshot_current_csn) is also the current global latest transaction commit sequence number (CSN) for the database executing the transaction completion process.

[0139] The transaction commit sequence number (CSN), the transaction write-ahead log address sequence number (LSN), the transaction identifier (xid), and the transaction status (status) are then written into a mapping array to establish a one-to-one correspondence between the transaction commit sequence number and the transaction write-ahead log address sequence number. The mapping array is used to generate a mapping log, thus associating the transaction commit sequence number (CSN) with the transaction write-ahead log address sequence number (LSN).

[0140] This allows the transaction commit sequence number CSN to have the characteristics of the transaction write-ahead log address sequence number LSN. That is, the transactions corresponding to the transaction commit sequence number CSN before the current global transaction commit sequence number global_snapshot_current_csn can be considered to have completed the transaction completion process. This can be used to determine transaction visibility without waiting for the transaction corresponding to the previous transaction write-ahead log address sequence number LSN to complete submission. This improves the efficiency of visibility judgment and snapshot efficiency, and reduces the loss of parallel performance.

[0141] As an optional embodiment, the current global transaction commit sequence number is updated according to the transaction commit sequence number of the current transaction that executes the transaction completion process, including: obtaining the transaction pre-write log address sequence number of the transaction corresponding to the current global transaction commit sequence number as the current global transaction write-ahead log address sequence number; determining whether the current transaction that executes the transaction completion process needs to update the current global transaction commit sequence number according to the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number; if the current transaction needs to update the current global transaction commit sequence number, updating the current global transaction commit sequence number to the transaction commit sequence number.

[0142] When updating the current global transaction commit sequence number, it is necessary to rely on the current global transaction write-ahead log address sequence number. Based on the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number, if the current transaction needs to update the current global transaction commit sequence number, the current global transaction commit sequence number is updated to the transaction commit sequence number to complete the update of the current global transaction commit sequence number.

[0143] Get the transaction write-ahead log address sequence number of the transaction corresponding to the current global transaction commit sequence number global_snapshot_current_csn, which is the current global transaction write-ahead log address sequence number global_snapshot_current_lsn. This current global transaction write-ahead log address sequence number global_snapshot_current_lsn is the transaction that updated the previous current global transaction commit sequence number global_snapshot_current_csn. When the transaction that updated the current global transaction commit sequence number global_snapshot_current_csn was updated, the current global transaction write-ahead log address sequence number global_snapshot_current_lsn was updated. This corresponds to the current global transaction commit sequence number global_snapshot_current_csn in the database at the current moment.

[0144] The above-mentioned current global transaction write-ahead log address sequence number global_snapshot_current_lsn can be understood as the last updated current global transaction commit sequence number global_snapshot_current_csn as the transaction commit sequence number CSN corresponding to the associated transaction write-ahead log address sequence number LSN in the mapping array.

[0145] Since the transaction write-ahead log address sequence number LSN is associated with the transaction commit sequence number CSN, both the transaction commit sequence number CSN and the transaction write-ahead log address sequence number LSN are obtained when entering the transaction completion process. The transaction commit sequence number CSN cannot represent the order in which transactions are completed, while the transaction write-ahead log address sequence number LSN can represent it to a certain extent. Therefore, when determining whether the current transaction needs to update the current global transaction commit sequence number global_snapshot_current_csn, the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is used to determine whether the current transaction needs to update the current global transaction commit sequence number global_snapshot_current_csn.

[0146] Based on the difference between the current global transaction write-ahead log address sequence number (global_snapshot_current_lsn) and the transaction write-ahead log address sequence number (LSN) (submit_lsn for commit and abort_lsn for rollback), determine whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number. If the current transaction needs to update the current global transaction commit sequence number, update the current global transaction commit sequence number (lobal_snapshot_current_csn) to the transaction commit sequence number (CSN), and update the current global transaction write-ahead log address sequence number (global_snapshot_current_lsn) to the transaction write-ahead log address sequence number (LSN).

[0147] FIG7 is a flowchart of another method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG7 , as an optional embodiment, determining whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number based on the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number includes:

[0148] Step S701: If the current global transaction write-ahead log address sequence number is greater than the transaction write-ahead log address sequence number, determine that the current transaction is a non-target transaction, wherein the non-target transaction is a transaction that does not need to update the current global transaction commit sequence number;

[0149] Step S702: If the current global transaction write-ahead log address sequence number is less than or equal to the transaction write-ahead log address sequence number, lock the current transaction. After acquiring the lock, determine again whether the current global transaction write-ahead log address sequence number is greater than the locked transaction write-ahead log address sequence number.

[0150] Step S703: If the address sequence number of the current global transaction write-ahead log is greater than the address sequence number of the locked transaction write-ahead log, the current transaction is determined to be a non-target transaction, and the lock is released.

[0151] Step S704: When the current global transaction write-ahead log address sequence number is less than or equal to the locked transaction write-ahead log address sequence number, obtain the mapping array, traverse the mapping transactions in the mapping array from the current global transaction commit sequence number to the current global latest transaction commit sequence number, select the latest mapping transaction whose transaction write-ahead log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number as the target transaction, and release the lock.

[0152] Specifically, the current global transaction write-ahead log address sequence number is compared with the transaction write-ahead log address sequence number of the current transaction to determine whether the current transaction is the target transaction for which the current global transaction commit sequence number needs to be updated. If the current transaction is not the target transaction, the target transaction is found by combining the mapping array, and the current global transaction commit sequence number is updated based on the mapping array of the target transaction. The current global transaction write-ahead log address sequence number is thus used to avoid confusion, frequent updates, and erroneous updates of the current global transaction commit sequence number due to the different speeds of transaction completion processes among multiple parallel transactions. This ensures the accuracy of the current global transaction commit sequence number and, in turn, the accuracy of snapshot visibility judgment.

[0153] If the current global transaction write-ahead log address sequence number, global_snapshot_current_lsn, is greater than the transaction write-ahead log address sequence number, LSN, this indicates that the current transaction completed earlier than the transaction corresponding to the current global transaction write-ahead log address sequence number, global_snapshot_current_lsn. The current transaction's transaction commit sequence number, CSN, already exists and is earlier than the transaction commit sequence number, CSN, corresponding to the current global transaction write-ahead log address sequence number, global_snapshot_current_lsn. Therefore, there is no need to update the current global transaction commit sequence number, global_snapshot_current_csn. This means that the current transaction is a non-target transaction.

[0154] If the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is less than or equal to the transaction write-ahead log address sequence number LSN, it means that the transaction completion process of the current transaction is later than the transaction corresponding to the current global transaction write-ahead log address sequence number global_snapshot_current_lsn, and the transaction commit sequence number CSN corresponding to the current transaction is greater than or equal to the current global transaction commit sequence number global_snapshot_current_csn. In theory, the transaction commit sequence number CSN of the current transaction needs to update the current global transaction commit sequence number global_snapshot_current_csn.

[0155] However, the transaction commit sequence number (CSN) only indicates the order in which transactions are committed, and does not indicate whether the transaction has been completed. For transactions that have not yet completed the transaction completion process, the current global transaction commit sequence number (global_snapshot_current_csn) does not need to be updated. In this case, visibility needs to be determined based on the current transaction status.

[0156] It should be noted that because the current transaction may be running, it is necessary to lock the current transaction. During the locking process, the current transaction continues to run and may change its state. Therefore, after acquiring the lock, it is necessary to check again whether the current global transaction write-ahead log address sequence number (global_snapshot_current_lsn) is greater than the write-ahead log address sequence number (LSN) of the locked transaction.

[0157] If the current global transaction write-ahead log address sequence number (global_snapshot_current_lsn) is greater than the locked transaction write-ahead log address sequence number (LSN), then the current transaction's transaction commit sequence number (CSN) already exists and is earlier than the transaction commit sequence number (CSN) corresponding to the current global transaction write-ahead log address sequence number (global_snapshot_current_lsn). Therefore, there is no need to update the current global transaction commit sequence number (global_snapshot_current_csn). This means that the current transaction is not the target transaction, and the lock is released to allow subsequent processes to proceed.

[0158] If the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is still less than or equal to the locked transaction write-ahead log address sequence number LSN, it means that the transaction commit sequence number CSN of the current transaction is later than the transaction commit sequence number CSN corresponding to the current global transaction write-ahead log address sequence number global_snapshot_current_lsn. In theory, the transaction commit sequence number CSN of the current transaction needs to update the current global transaction commit sequence number global_snapshot_current_csn.

[0159] However, similar to the above, the transaction commit sequence number CSN only indicates the order in which transactions are committed, and does not indicate whether the transaction has been completed. For transactions that have not yet completed the transaction completion process, there is no need to update the current global transaction commit sequence number global_snapshot_current_csn. In this case, visibility needs to be determined based on the current transaction status.

[0160] Specifically, obtain the mapping array, traverse the mapping transactions in the mapping array from the global current transaction commit sequence number global_snapshot_current_csn to the current global latest transaction commit sequence number global_latest_current_csn, select the latest mapping transaction whose transaction pre-write log address sequence number LSN is less than or equal to the current global transaction pre-write log address sequence number global_snapshot_current_lsn as the target transaction, and release the lock.

[0161] The mapping from the current global transaction commit sequence number (global_snapshot_current_csn) to the global latest transaction commit sequence number (global_latest_current_csn) indicates that the global latest transaction commit sequence number (global_latest_current_csn), also known as the current transaction commit sequence number (CSN), is greater than the global current transaction commit sequence number (global_snapshot_current_csn) in the snapshot parameter. This means that the global current transaction commit sequence number (global_snapshot_current_csn) is outdated and needs to be updated.

[0162] The latest mapping transaction whose transaction write-ahead log address sequence number LSN is less than or equal to the current global transaction write-ahead log address sequence number global_latest_current_lsn is selected from the mapping array as the target transaction in order to update it in one go. The mapping array is also maintained and updated globally in real time by the database, recording the mapping relationship between each parallel transaction at the current time. The mapping array can be used to find the target transaction with the largest current transaction commit sequence number CSN and the largest transaction write-ahead log address sequence number LSN, and update the global current transaction commit sequence number global_snapshot_current_csn.

[0163] As an optional embodiment, the mapping transactions between the current global transaction commit sequence number and the current global latest transaction commit sequence number are traversed in the mapping array, and the latest transaction whose transaction pre-write log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number is selected as the target transaction, including: judging the difference between the transaction pre-write log address sequence number of the mapping transaction corresponding to the traversed transaction commit sequence number and the current global transaction write-ahead log address sequence number; if the transaction pre-write log address sequence number of the mapping transaction corresponding to the traversed transaction commit sequence number is less than or equal to the current global transaction write-ahead log address sequence number, and if the transaction pre-write log address sequence number of the mapping transaction corresponding to the next traversed transaction commit sequence number is greater than the current global transaction write-ahead log address sequence number, determining that the mapping transaction corresponding to the traversed transaction commit sequence number is the target transaction.

[0164] When the current transaction is not a target transaction, the mapping array is traversed according to the current global transaction write-ahead log address sequence number, and the transaction write-ahead log address sequence number of the latest corresponding mapped transaction that meets the transaction commit sequence number is selected. The transaction that is less than or equal to the current global transaction write-ahead log address sequence number is selected as the target transaction to update the current global transaction commit sequence number to ensure the accuracy of the current global transaction commit sequence number.

[0165] Determine the size of the transaction write-ahead log address sequence number LSN of the corresponding mapping transaction of the traversed transaction commit sequence number CSN and the current global transaction write-ahead log address sequence number global_latest_current_lsn, and determine the latest transaction whose transaction write-ahead log address sequence number LSN is less than or equal to the global transaction write-ahead log address sequence number global_latest_current_lsn as the target transaction.

[0166] The transaction write-ahead log address sequence number LSN of the corresponding mapping transaction of the transaction commit sequence number CSN is required to be less than or equal to the current global transaction write-ahead log address sequence number global_latest_current_lsn, and the transaction write-ahead log address sequence number LSN of the corresponding mapping transaction of the next traversed transaction commit sequence number CSN is greater than the current global transaction write-ahead log address sequence number global_latest_current_lsn. The corresponding mapping transaction of the traversed transaction commit sequence number is determined to be the target transaction.

[0167] As an optional embodiment, after traversing the mapping transactions in the mapping array from the global current transaction commit sequence number to the global latest transaction commit sequence number, and selecting the latest mapping transaction whose transaction pre-write log address sequence number is less than or equal to the transaction pre-write log address sequence number as the target transaction, it also includes: obtaining the corresponding transaction identifier, transaction status and transaction commit sequence number from the mapping array according to the transaction pre-write log address sequence number of the target transaction, and generating a mapping log; updating the current global transaction write-ahead log address sequence number to the transaction write-ahead log address sequence number of the target transaction.

[0168] Determine the target transaction to update the current global transaction commit sequence number. Generate a mapping log for query during visibility judgment. Update the current global transaction write-ahead log address sequence number to facilitate other transactions to judge the target transaction and the next update judgment of the current global transaction commit sequence number.

[0169] FIG8 is a flow chart of another method for processing data of a database snapshot according to an embodiment of the present disclosure. As shown in FIG8 , as an optional embodiment, the method further includes:

[0170] Step S801: When read-only hot standby of the database is enabled, a replay snapshot parameter is generated by replaying the write-ahead log of the transaction, wherein the current global transaction commit sequence number of the snapshot parameter corresponds one-to-one with the transaction write-ahead log address sequence number of the corresponding transaction;

[0171] Step S802 : acquiring corresponding playback snapshot data according to the snapshot parameters, wherein the playback snapshot data is synchronized with the content of the snapshot data for which read-only hot standby is not enabled.

[0172] Taking snapshots in read-only hot standby scenarios ensures snapshot data synchronization. This means that any transaction visibility corresponding to the current global transaction commit sequence number in a read-only hot standby snapshot can be found in the same version in a snapshot taken without read-only hot standby enabled. This avoids the issue of snapshot data being out of sync due to replay in read-only hot standby scenarios. Similarly, this also solves the problem of waiting for transaction completion to determine snapshot visibility in read-only hot standby scenarios, improving snapshot acquisition efficiency in read-only hot standby scenarios.

[0173] In the related art, in read-only hot standby scenarios, it is necessary to replay the WAL log to achieve a snapshot, that is, to obtain the above-mentioned replay snapshot. During replay, the transaction commit sequence number (CSN) and the transaction write-ahead log address sequence number (LSN) are also generated. The transaction commit sequence number only indicates the order in which transactions were submitted, and the transaction write-ahead log address sequence number (LSN) only indicates the modified version of the transaction. During replay, due to the different order of the WAL log content, the transaction commit sequence number (CSN) and transaction write-ahead log address sequence number (LSN) generated are completely different from those when the WAL was written.

[0174] Different sequence numbers will also cause the playback snapshot data corresponding to different transaction commit sequence numbers CSN to be unique. The snapshots corresponding to all transaction commit sequence numbers CSN when the WAL log is written may be different, which leads to the insynchronization of the snapshot content.

[0175] After binding the transaction commit sequence number (CSN) and the transaction write-ahead log address sequence number (LSN), the generated transaction commit sequence number (CSN) and transaction write-ahead log address sequence number (LSN) may differ during replay. However, the replay snapshot data corresponding to a transaction commit sequence number (CSN) will always find the same snapshot data when the WAL log is written. Although the transaction commit sequence number (CSN) and transaction write-ahead log address sequence number (LSN) corresponding to the snapshot data and the replay snapshot data may differ, their data content, including transaction visibility, remains consistent. This achieves snapshot content synchronization and snapshot data consistency.

[0176] It should be noted that this embodiment also provides an optional implementation method, which is described in detail below.

[0177] In related technologies, transaction submission requires generating a commit write-ahead log and obtaining the corresponding write-ahead log address sequence number, which is then persisted to ensure that all commits prior to the corresponding write-ahead log address sequence number have been completed. To avoid issues with traditional CSN solutions, the transaction commit sequence number is synchronized with the transaction write-ahead log address sequence number. This ensures that the transaction commit sequence number meets the aforementioned conditions. This means that currently committing transactions with a sequence number less than the database's latest global transaction commit sequence number are eliminated during the synchronization process. This ensures that all transactions with a sequence number less than the database's latest global transaction commit sequence number are completed, resolving the issue of waiting for commits and improving performance.

[0178] This embodiment unifies the time node for obtaining the transaction commit sequence number with the node for obtaining the transaction commit write-ahead log address sequence number, and binds the order of the transaction commit sequence number and the transaction write-ahead log address sequence number. Since transaction commit requires the generation and persistence of a transaction write-ahead log, as long as the write-ahead log is written, the transaction and the transactions before this write-ahead log address sequence number have been committed in fact. This is used as the basis for judgment to determine that all transactions corresponding to the write-ahead log address sequence numbers before this transaction write-ahead log address sequence number have been completed, and the transaction commit sequence numbers of these transactions have also been generated and updated. At the same time, the current global transaction commit sequence number in the snapshot parameter obtained when obtaining the snapshot is updated to obtain the maximum transaction commit sequence number corresponding to the current latest transaction write-ahead log commit sequence number plus 1 as the snapshot. This satisfies the snapshot visibility judgment logic and avoids transactions waiting to be committed.

[0179] The relevant data structure is as follows:

[0180] The current global transaction commit sequence number global_snapshot_current_csn is used to determine the snapshot parameters. In this embodiment, the current global transaction commit sequence number of the snapshot parameters is global_snapshot_current_csn+1.

[0181] The current global latest transaction commit sequence number global_latest_current_csn is the latest global transaction commit sequence number in the database. It is updated by each transaction during the commit process. After the update, the updated current global latest transaction commit sequence number can be used as the transaction commit sequence number of the transaction. The next transaction commit sequence number is global_latest_current_csn+1.

[0182] The current global transaction write-ahead log address sequence number global_snapshot_current_lsn and the current global transaction commit sequence number global_snapshot_current_csn are updated by each transaction during the commit process.

[0183] The global_csn_xid_mapping[max_backends] mapping array stores the mapping between the transaction commit sequence number (csn) and the transaction identifier (xid), as well as the mapping between the transaction commit sequence number (csn) and the transaction write-ahead log address (lsn). The array length is equal to the maximum number of processes currently allowed, or max_backends. Array id = csn % max_backends.

[0184] The storage structure of the csnlog transaction commit log is that each transaction commit sequence number csn occupies 8 bytes. A transaction commit log csnlog writes 1024 transaction commit sequence numbers csn to a page.

[0185] Transaction status includes INPROGRESS / ABORTED / COMMITTING / FROZEN, etc.

[0186] The process of obtaining a csn snapshot is as follows:

[0187] Read the current value of oldest_active_xid, the transaction identifier of the smallest currently active transaction in the world, as min, which is snapshot.min.

[0188] Read the current global transaction commit sequence number global_snapshot_current_csn as snapshot.csn; the current global transaction commit sequence number global_snapshot_current_csn is the transaction commit sequence number csn of the transaction corresponding to the maximum LSN smaller than the current latest flush LSN (LSN data stream).

[0189] Read the current value of the transaction identifier latest_completed_xid of the latest currently active transaction in the world as max, which is snapshot.max.

[0190] The current global transaction commit sequence number global_snapshot_current_csn and the current global transaction write-ahead log address sequence number global_snapshot_current_lsn are generated during the transaction commit process or rollback process.

[0191] The transaction submission process is as follows:

[0192] 1. Get the address sequence number lsn of the transaction commit write-ahead log wal as submit_lsn, and the current global latest transaction commit sequence number global_latest_current_csn++, and save them as the transaction commit sequence number csn of the transaction;

[0193] 2. According to the transaction commit sequence number csn value, record the transaction commit sequence number csn, transaction identifier xid, submit_lsn, and commit status commit status in the mapping array global_csn_xid_mapping[global_latest_current_csn];

[0194] 3. Generate and submit the data flow log flush xlog with the commit sequence number of the pre-written log. If it is an asynchronous commit, there is no need to wait for flush xlog. The other steps are the same.

[0195] 4. Write transaction commit log clog;

[0196] 5. Update the transaction identifier latest_completed_xid of the latest currently active transaction globally;

[0197] 6. If the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is greater than submit_lsn, jump to the next step 7. Otherwise, lock it. After acquiring the lock, determine again whether the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is greater than submit_lsn. If so, release the lock and jump to the next step 7. Otherwise, judge in order from the mapping array global_csn_xid_mapping [current global transaction submission sequence number global_snapshot_current_csn% max_backends] to global_csn_xid_mapping [current global latest transaction submission sequence number global_latest_current_csn% max_backends]. If the transaction write-ahead log address sequence number lsn corresponding to a transaction submission sequence number is less than or equal to submit_lsn, then set the transaction identifier xid, transaction submission sequence number csn, and transaction status status corresponding to the transaction identifier xid in the mapping array to the mapping log csnlog value, and mark the transaction status. Finally, update the current global transaction commit sequence number global_snapshot_current_csn to the last valid transaction commit sequence number csn that meets the judgment condition, the current global transaction write-ahead log address sequence number global_snapshot_current_lsn to submit_lsn, and release the lock;

[0198] 7. Add the shared lock share ProcArrayLock, reset the transaction identifier xid and other variables in the PGXACT, PGPROC and other data structures, and release the shared lock share ProcArrayLock;

[0199] 8. Advance the transaction identifier oldest_active_xid of the smallest currently active transaction in the world.

[0200] The transaction rollback process is as follows:

[0201] 1. Get the address sequence number lsn of the write-ahead log wal of the transaction rollback abort as abort_lsn, and the current global latest transaction commit sequence number global_latest_current_csn++, and save it as the transaction commit sequence number csn value of the transaction;

[0202] 2. According to the transaction commit sequence number csn value, the transaction commit sequence number csn, transaction identifier xid, abort_lsn, and rollback status abort status are mapped in the global_csn_xid_mapping[global_latest_current_csn] array;

[0203] 3. Generate and submit the data flow log flush xlog with the commit sequence number of the pre-written log. If it is an asynchronous commit, there is no need to wait for flush xlog. The other steps are the same.

[0204] 4. Write transaction commit log clog;

[0205] 5. Update the transaction identifier latest_completed_xid of the latest currently active transaction globally;

[0206] 6. If the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is greater than abort_lsn, jump to the next step 7. Otherwise, lock it. After acquiring the lock, determine again whether the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is greater than abort_lsn. If so, release the lock and jump to the next step 7. Otherwise, write the address sequence number from the mapping array global_csn_xid_mapping [current global transaction commit sequence number global_snapshot_current_csn% max_backends] to the mapping array global_csn_xid_mapping [current global latest transaction commit sequence number global_l atest_current_csn%max_backends] is judged in order. If the transaction write-ahead log address sequence number lsn corresponding to a transaction commit sequence number is less than or equal to abort_lsn, the transaction identifier xid and transaction commit sequence number csn corresponding to the transaction commit sequence number in the mapping array are set, and the transaction status status sets the mapping log csnlog value corresponding to the transaction identifier xid, and marks the transaction status. Finally, the current global transaction commit sequence number global_snapshot_current_csn is updated to the last valid transaction commit sequence number csn that meets the judgment condition, the current global transaction write-ahead log address sequence number global_snapshot_current_lsn is abort_lsn, and the lock is released;

[0207] 7. Add the shared lock share ProcArrayLock, reset the transaction identifier xid and other variables in the PGXACT, PGPROC and other data structures, and release the shared lock share ProcArrayLock;

[0208] 8. Advance the transaction identifier oldest_active_xid of the smallest transaction that is currently active globally.

[0209] As shown in Figure 3, the transaction visibility judgment process is as follows:

[0210] i. When obtaining a snapshot, record the smallest transaction identifier oldest_active_xid that is currently active as snapshot.min, the transaction latest_completed_xid + 1 that is currently the latest committed as snaoshot.max, and the global_snapshot_current_csn + 1 as snapshot.csn;

[0211] ii. When the corresponding xid is greater than or equal to snapshot.max, the transaction xid is not visible;

[0212] iii. When xid < snapshot.min, it means that the transaction corresponding to xid has ended before the current transaction started. Query the commit status of the transaction through clog. If it is committed, it is visible; if the transaction has been rolled back, it is not visible;

[0213] iv. When xid is between snapshot.min and snapshot.max, it is necessary to query from the xid-csn mapping log, that is, csnlog. If the transaction has no csn or is aborted, it is not visible. If it is committed, judge that if csn has a value and is smaller than snapshot.csn, the transaction is visible; otherwise, it is not visible.

[0214] Transaction visibility judgment on the RO hot standby. The RO hot standby obtains the transaction snapshot through wal log replay; for the redo transaction commit, wal csn++ and lsn are used as the global global_snapshot_current_csn and global_snapshot_current_lsn, and csnlog is generated; for the redo transaction abort, wal csn++ and lsn are used as the global global_snapshot_current_csn and global_snapshot_current_lsn, and csnlog is generated; global min and global max are generated for the redo.

[0215] The RO snapshot acquisition only needs to read global min; read global_snapshot_current_csn as snapshot.csn; read global max.

[0216] The transaction visibility determination process is identical to the RW master node logic. Because the RO hot standby utilizes the redo WAL log and the CSN is synchronized with the LSN, the hot standby redo WAL advances the LSN while also obtaining a CSN state consistent with the RW. This ensures snapshot synchronization between the RO and RW nodes, resolves the snapshot asynchrony issue between RW and RO nodes in traditional CSNs, and achieves read consistency for RO hot standby.

[0217] FIG9 is a flowchart of a data processing device for a database snapshot according to an embodiment of the present disclosure. As shown in FIG9 , based on the above-mentioned data processing method for a database snapshot provided by an embodiment of the present disclosure, an embodiment of the present disclosure further provides a data processing device for a database snapshot, which includes: a snapshot module 901, a judgment module 902, and a determination module 903. The device is described in detail below.

[0218] The snapshot module 901 is configured to obtain snapshot parameters, where the snapshot parameters include the current global transaction commit sequence number corresponding to the snapshot time. The current global transaction commit sequence number is the transaction commit sequence number recorded in the database at the snapshot time for obtaining the snapshot, and the transaction commit sequence number corresponds one-to-one with the transaction write-ahead log address sequence number of the corresponding transaction; the judgment module 902 is connected to the above-mentioned snapshot module 901 and is configured to judge the visibility of transaction data that meets the snapshot parameters based on the current global transaction commit sequence number; the determination module 903 is connected to the above-mentioned judgment module 902 and is configured to filter out transaction data that is not visible under the snapshot to obtain data that meets the snapshot visibility.

[0219] The data processing device for the above-mentioned database snapshot provided by the embodiment of the present disclosure obtains the current global transaction commit sequence number that corresponds one-to-one to the transaction write-ahead log address sequence number of the corresponding transaction as a snapshot parameter and as a basis for visibility judgment. Therefore, after the parallel transaction enters the transaction completion process, the transaction write-ahead log address sequence number is obtained, and the accumulation of transaction commit sequence numbers can be achieved. There is no need to rely on the completion and submission of the transaction corresponding to the previous transaction write-ahead log address sequence number to accumulate the transaction commit sequence number and update the current global transaction commit sequence number. Therefore, after the transaction starts to commit, the transaction commit sequence number required for snapshot visibility judgment can be accumulated to judge the transaction visibility. There is no need to wait for the completion of the submission of the transaction corresponding to the previous transaction write-ahead log address sequence number, thereby improving the efficiency of visibility judgment and snapshot efficiency, and reducing the loss of parallel performance.

[0220] The present disclosure also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, wherein the computer program, when executed by the at least one processor, causes the electronic device to perform the method of the present disclosure.

[0221] The embodiments of the present disclosure further provide a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method of the embodiments of the present disclosure.

[0222] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method of the embodiments of the present disclosure.

[0223] With reference to Figure 10, a block diagram of an electronic device that can serve as a server or client of an embodiment of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0224] As shown in Figure 10, the electronic device includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the electronic device can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0225] Multiple components within the electronic device are connected to the I / O interface 1005, including an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 can be any type of device capable of inputting information into the electronic device. The input unit 1006 can receive input numeric or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1007 can be any type of device capable of presenting information and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 may include, but is not limited to, a magnetic disk or an optical disk. The communication unit 1009 allows the electronic device to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0226] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units for running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present disclosure may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 1002 and / or communication unit 1009. In some embodiments, the computing unit 1001 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0227] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0228] In the context of the embodiments of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0229] It should be noted that the term "including" and its variations used in the embodiments of the present disclosure are open inclusions, that is, "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present disclosure are illustrative and not restrictive. Those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

[0230] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0231] The various steps described in the method implementations provided in the embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method implementations may include additional steps and / or omit the steps shown. The scope of protection of the present disclosure is not limited in this respect.

[0232] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present disclosure. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same or similar parts between the various embodiments are referenced to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0233] The above-described embodiments merely represent several implementation methods of the present disclosure. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present disclosure, all of which fall within the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be determined by the appended claims.

Claims

1. A method for processing data of a database snapshot, comprising: Obtaining snapshot parameters, wherein the snapshot parameters include a current global transaction commit sequence number corresponding to the snapshot time, the current global transaction commit sequence number being a transaction commit sequence number for obtaining a snapshot recorded in the database at the snapshot time, and the transaction commit sequence number having a one-to-one correspondence with a transaction write-ahead log address sequence number of the corresponding transaction; Determining visibility of transaction data that meets the snapshot parameters according to the current global transaction commit sequence number; Filter out transaction data that is not visible under the snapshot to obtain data that meets the snapshot visibility.

2. The method according to claim 1, wherein: The snapshot parameters also include the minimum active transaction identifier and the latest active transaction identifier. According to the current global transaction commit sequence number, the visibility of transaction data that meets the snapshot parameters is determined, including: Determine the visibility of the first transaction in the database whose transaction identifier is greater than or equal to the latest active transaction identifier as invisible, wherein the transactions in the database are assigned transaction identifiers in ascending order according to the order of creation; For a second transaction in the database whose transaction identifier is smaller than the minimum active transaction identifier, determining visibility of the second transaction according to a transaction completion status corresponding to the second transaction; For a third transaction in the database whose transaction identifier is greater than or equal to the minimum active transaction identifier and less than the latest active transaction identifier, visibility of the third transaction is determined according to the current global transaction commit sequence number.

3. The method according to claim 2, wherein: Determining visibility of the second transaction according to a transaction state corresponding to the second transaction includes: querying the transaction status of the second transaction according to the transaction completion log of the second transaction, wherein the transaction completion log is a log in the database that records the transaction execution and transaction completion process; In a case where the transaction state indicates that the transaction is finished, determining visibility of the second transaction as visible; When the transaction status indicates that the transaction is not completed, the visibility of the second transaction is determined to be invisible.

4. The method according to claim 3, wherein: Determining visibility of the third transaction according to the current global transaction commit sequence number includes: querying a transaction commit sequence number and a transaction status of the third transaction according to a mapping log of the third transaction, wherein the mapping log is generated for the transaction in the database in the process of executing a transaction completion process according to a transaction commit sequence number, a transaction write-ahead log address sequence number, a transaction identifier, and a transaction status, and the transaction commit sequence number corresponds to the transaction write-ahead log address sequence number in one-to-one correspondence; When the transaction commit sequence number of the third transaction is less than the current global transaction commit sequence number and the transaction state of the third transaction indicates that the transaction is finished, determine that the visibility of the third transaction is visible; When the transaction state of the third transaction indicates that the transaction is not completed, or there is no transaction commit sequence number, or the transaction commit sequence number is greater than the current global transaction commit sequence number, the visibility of the third transaction is determined to be invisible.

5. The method according to claim 1, wherein: The method further comprises: The transactions in the database execute a transaction completion process in parallel to update the snapshot parameters, wherein the transaction completion process includes a transaction commit process and a transaction rollback process; The snapshot parameters include: According to the snapshot instruction, obtain the snapshot parameters corresponding to the snapshot time.

6. The method according to claim 5, wherein: The transactions in the database execute the transaction completion process in parallel and update the snapshot parameters, including: In the transaction completion process, after the data of the transaction is determined, a transaction completion log is generated for the transaction in the database; After generating the transaction completion log, update the latest active transaction identifier; Update the current global transaction commit sequence number according to the transaction commit sequence number of the current transaction executing the transaction completion process; After resetting the transaction identifier of the data structure in the database, the minimum active transaction identifier is updated.

7. The method according to claim 6, wherein: Before generating the transaction completion log, it also includes: After the transaction in the database enters the transaction completion process, a write-ahead log is generated, and a log address sequence number of the write-ahead log is obtained as the transaction write-ahead log address sequence number; And obtain the current global latest transaction submission sequence number, add one to the cumulative number, and get the transaction submission sequence number; The transaction commit sequence number, the transaction write-ahead log address sequence number, the transaction identifier and the transaction status are written into a mapping array to correspond the transaction commit sequence number and the transaction write-ahead log address sequence number one to one, wherein the mapping array is used to generate a mapping log.

8. The method according to claim 7, wherein: Updating the current global transaction commit sequence number according to the transaction commit sequence number of the current transaction executing the transaction completion process includes: Get the transaction write-ahead log address sequence number of the transaction corresponding to the current global transaction commit sequence number as the current global transaction write-ahead log address sequence number; Determining whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number according to the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number; In the case where the current transaction needs to update the current global transaction commit sequence number, the current global transaction commit sequence number is updated to the transaction commit sequence number.

9. The method according to claim 8, wherein: Determining whether the current transaction executing the transaction completion process needs to update the current global transaction commit sequence number according to the size of the current global transaction write-ahead log address sequence number and the transaction write-ahead log address sequence number includes: When the current global transaction write-ahead log address sequence number is greater than the transaction write-ahead log address sequence number, determining that the current transaction is a non-target transaction, wherein the non-target transaction is a transaction that does not need to update the current global transaction commit sequence number; If the current global transaction write-ahead log address sequence number is less than or equal to the transaction write-ahead log address sequence number, lock the current transaction, and after acquiring the lock, determine again whether the current global transaction write-ahead log address sequence number is greater than the locked transaction write-ahead log address sequence number; When the current global transaction write-ahead log address sequence number is greater than the locked transaction write-ahead log address sequence number, determining that the current transaction is a non-target transaction and releasing the lock; When the current global transaction write-ahead log address sequence number is less than or equal to the locked transaction write-ahead log address sequence number, obtain the mapping array, traverse the mapping array, from the current global transaction commit sequence number to the locked transaction write-ahead log address sequence number, The mapping transaction between the current global latest transaction commit sequence number, select the latest mapping transaction whose transaction write-ahead log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number as the target transaction, and release the lock.

10. The method according to claim 9, wherein: Traversing the mapping transactions in the mapping array from the current global transaction commit sequence number to the current global latest transaction commit sequence number, selecting the latest transaction whose transaction write-ahead log address sequence number is less than or equal to the current global transaction write-ahead log address sequence number as the target transaction, including: Determine the size of the transaction write-ahead log address sequence number of the corresponding mapping transaction of the traversed transaction commit sequence number and the current global transaction write-ahead log address sequence number; When the transaction write-ahead log address sequence number of the corresponding mapping transaction of the traversed transaction commit sequence number is less than or equal to the current global transaction write-ahead log address sequence number, and when the transaction write-ahead log address sequence number of the corresponding mapping transaction of the next traversed transaction commit sequence number is greater than the current global transaction write-ahead log address sequence number, the corresponding mapping transaction of the traversed transaction commit sequence number is determined to be the target transaction.

11. The method according to claim 10, wherein: After traversing the mapping transactions in the mapping array from the global current transaction commit sequence number to the global latest transaction commit sequence number, and selecting the latest mapping transaction whose transaction write-ahead log address sequence number is less than or equal to the transaction write-ahead log address sequence number as the target transaction, the method further includes: According to the transaction write-ahead log address sequence number of the target transaction, the corresponding transaction identifier, transaction status and transaction commit sequence number are obtained from the mapping array to generate a mapping log; The current global transaction write-ahead log address sequence number is updated to the transaction write-ahead log address sequence number of the target transaction.

12. The method according to any one of claims 1 to 11, wherein: The method further comprises: When the read-only hot standby of the database is turned on, a replay snapshot parameter is generated by replaying the write-ahead log of the transaction, wherein the current global transaction commit sequence number of the snapshot parameter corresponds one-to-one to the transaction write-ahead log address sequence number of the corresponding transaction; The corresponding playback snapshot data is acquired according to the snapshot parameters, wherein the playback snapshot data is synchronized with the content of the snapshot data for which read-only hot standby is not enabled.

13. A data processing device for a database snapshot, comprising: A snapshot module is configured to obtain snapshot parameters, wherein the snapshot parameters include a current global transaction commit sequence number corresponding to the snapshot time, the current global transaction commit sequence number is a transaction commit sequence number for obtaining the snapshot recorded in the database at the snapshot time, and the transaction commit sequence number has a one-to-one correspondence with a transaction write-ahead log address sequence number of the corresponding transaction; A judgment module is configured to judge the visibility of transaction data that meets the snapshot parameters according to the current global transaction submission sequence number; The determination module is configured to filter out transaction data that is not visible under the snapshot and obtain data that meets the snapshot visibility.

14. An electronic device comprising: A processor, and a memory storing a program, the program comprising instructions which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 12.

15. A non-transitory machine-readable medium storing computer instructions for causing the computer to execute the method according to any one of claims 1 to 12.

Images