Query method and device based on global consistency snapshot, equipment and medium

Abstract

The invention relates to the technical field of distributed databases, and discloses a query method, device and equipment based on a global consistency snapshot and a medium, and the method comprises the steps that a transaction management node generates an active transaction list carrying snapshot version information, writing a transaction log at the transaction management node and each data fragment node, and asynchronously transmitting the transaction log to a destination end database; and when a destination end database receives a data query request, analyzing the transaction log, determining common active transaction list version information of each data fragment node as a global consistency snapshot, completing data visibility judgment based on the global consistency snapshot, and returning a query result. According to the method, the global consistency snapshot is determined on the basis of the active transaction list version information of the transaction management node and each data fragment node on the destination end database side, so that uniform data visibility judgment can be completed under the condition of transaction information asynchronization and synchronization in the query stage, and thus consistent return of query results is realized.

Description

Technical Field

[0001] This invention relates to the field of distributed database technology, and in particular to a query method, apparatus, device, and medium based on globally consistent snapshots. Background Technology

[0002] In distributed database systems, data is typically divided and stored across multiple data shard nodes. Transactions on different shards interact during concurrent execution, necessitating a consistent data visibility mechanism to handle cross-shard access and data replication scenarios. To achieve atomicity and isolation of transactions, existing technologies often introduce a global transaction management node to centrally maintain transaction identifier allocation and transaction status.

[0003] In existing technologies, distributed databases uniformly allocate transaction identifiers through a global transaction management node and maintain a set of outstanding transaction identifiers in the current system, thereby constructing a data visibility snapshot for querying. The query operation compares the range of transaction identifiers recorded in this snapshot with the corresponding transaction identifiers of the data records to determine whether the data is in a visible state. In a master database environment, this approach can meet basic transaction consistency and query requirements.

[0004] However, existing technologies have significant shortcomings in data replication or disaster recovery synchronization scenarios for distributed databases. Transaction logs between the transaction management node and each data shard node are typically transmitted through independent log streams. The arrival times of transaction-related information on different nodes are not consistent, making it difficult for the standby server to obtain a set of corresponding transaction status information at the same time. In this situation, transaction snapshots built based on a single node or localized information cannot accurately reflect the true transaction status across multiple data shards. Summary of the Invention

[0005] The main objective of this invention is to provide a query method, apparatus, device, and storage medium based on globally consistent snapshots. This invention aims to solve the technical problem in the prior art where the asynchronous synchronization of transaction states between the transaction management node and each data shard node makes it difficult for the destination database to complete a consistent data visibility judgment based on the received transaction information during the query phase, thus affecting the consistency of query results.

[0006] To achieve the above objectives, the present invention provides a query method based on globally consistent snapshots, comprising: In the transaction management node of the source database of the distributed database, an active transaction list is generated, which includes committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated, and corresponding snapshot version information is assigned to the active transaction list. The list of active transactions carrying the snapshot version information is distributed from the transaction management node to each data shard node of the source database; The list of active transactions is written into the transaction log of the transaction management node and the transaction log of each data shard node at predetermined intervals based on the snapshot version information corresponding to the same moment. The transaction logs of the transaction management node and the transaction logs of each data shard node are asynchronously transmitted to the destination database through independent log streams; When the destination database receives a data query request, it parses the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. Based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the common active transaction list version information of each data shard node is determined as a globally consistent snapshot. Based on the list of active transactions corresponding to the globally consistent snapshot, the visibility of the data to be accessed by the data query request is determined and the query result is returned.

[0007] Furthermore, to achieve the above objectives, the present invention provides a query apparatus based on a globally consistent snapshot, comprising: The active transaction list generation module is used to generate an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers, and transaction ranges to be allocated in the transaction management node of the source database of the distributed database, and to allocate corresponding snapshot version information to the active transaction list; An active transaction list distribution module is used to distribute the active transaction list carrying the snapshot version information from the transaction management node to each data shard node of the source database; The transaction log writing module is used to write the list of active transactions into the transaction log of the transaction management node and the transaction log of each data shard node according to a predetermined period based on the snapshot version information corresponding to the same moment. An independent log stream transmission module is used to asynchronously transmit the transaction logs of the transaction management node and the transaction logs of each data shard node to the destination database via an independent log stream. The version information extraction module is used to parse the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database when the destination database receives a data query request, and extract the version information of the active transaction list of the transaction management node and the version information of the active transaction list of each data shard node. The global consistency snapshot determination module is used to determine the common active transaction list version information of each data shard node as a global consistency snapshot based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. The visibility determination query module is used to determine the visibility of the data to be accessed by the data query request based on the list of active transactions corresponding to the global consistency snapshot and return the query results.

[0008] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and a query program based on a globally consistent snapshot stored in the memory and executable on the processor, wherein when the query program based on a globally consistent snapshot is executed by the processor, it implements the steps of the query method based on a globally consistent snapshot as described above.

[0009] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a query program based on a globally consistent snapshot, wherein the query program based on a globally consistent snapshot, when executed by a processor, implements the steps of the query method based on a globally consistent snapshot as described above.

[0010] Beneficial Effects: This invention relates to the field of distributed database technology and discloses a query method, apparatus, device, and medium based on a globally consistent snapshot. The method includes: a transaction management node generating an active transaction list carrying snapshot version information, writing transaction logs at the transaction management node and each data shard node, and asynchronously transmitting the list to the destination database; upon receiving a data query request, the destination database parses the transaction logs, determines the version information of the common active transaction list of each data shard node as a globally consistent snapshot, and performs data visibility judgment based on this globally consistent snapshot, returning the query results. This invention, by determining a globally consistent snapshot on the destination database side based on the version information of the active transaction list of the transaction management node and each data shard node, enables the query phase to complete a unified data visibility judgment while transaction information is asynchronously synchronized, thereby achieving a consistent return of query results. Attached Figure Description

[0011] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a schematic diagram of an application environment for a query method based on globally consistent snapshots according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating an embodiment of the query method based on globally consistent snapshots according to the present invention. Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the query device based on globally consistent snapshots of the present invention. Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention; Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0012] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0013] The query method based on globally consistent snapshots provided in this invention can be applied to applications such as... Figure 1 In this application environment, the client communicates with the server via a network. The server can generate an active transaction list carrying snapshot version information at the transaction management node through the client, and asynchronously transmit it to the destination database after writing transaction logs at the transaction management node and each data shard node. When the destination database receives a data query request, it parses the transaction logs, determines the common active transaction list version information of each data shard node as a globally consistent snapshot, and completes data visibility judgment based on this globally consistent snapshot and returns the query results. This invention determines a globally consistent snapshot on the destination database side based on the active transaction list version information of the transaction management node and each data shard node, enabling the query phase to complete a unified data visibility judgment while transaction information is asynchronously synchronized, thereby achieving consistent return of query results. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will be described in detail below through specific embodiments.

[0014] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the query method based on globally consistent snapshots provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0015] like Figure 2 As shown, the query method based on globally consistent snapshots proposed in this invention includes the following steps: S10, generate an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated in the transaction management node of the source database of the distributed database, and allocate corresponding snapshot version information to the active transaction list; In this embodiment, in a distributed database, to ensure the deterministic nature of subsequent transaction state analysis, a unified representation of the current transaction running state is required at the source end. The transaction management node continuously maintains a set of information related to the transaction lifecycle during system operation. The committed transaction boundary information delineates the range of transactions that have been committed and entered a stable state before the current time point. This information originates from transaction commit confirmation events and is dynamically updated as transactions complete. The uncommitted transaction identifier identifies the set of transactions that have been assigned transaction identifiers but have not yet entered the committed completion state. It originates from the execution interval between the transaction creation event and the transaction end event, reflecting transactions that are currently in an uncertain state. The pending transaction range describes the range of transaction identifiers that have not yet been occupied by any transaction. This range is continuously maintained by the transaction management node according to the transaction identifier generation rules to ensure the continuity and uniqueness of transaction identifier allocation.

[0016] The transaction management node combines the above information at the same logical point in time to form an active transaction list. The active transaction list comprehensively represents the overall operational status of transactions in the system at the current moment. It includes a range of stable transactions, a set of unstable transactions, and an allocable transaction interval, thus avoiding the problem that single transaction status information cannot cover global transaction relationships. To ensure that this active transaction list has clear distinguishability at different points in time, corresponding snapshot version information needs to be assigned to it. Snapshot version information identifies the logical time position corresponding to the active transaction list. Its generation is based on an increasing sequence or logical time marker maintained internally by the transaction management node, establishing a comparable order relationship between different active transaction lists. The snapshot version information is generated and bound together with the active transaction list, ensuring that the list accurately corresponds to the transaction state at the time of its generation when referenced subsequently.

[0017] During implementation, the transaction management node generates an active transaction list by freezing the current transaction state structure in memory and writing snapshot version information during generation, thereby avoiding impact on the transactions being executed and ensuring the integrity and consistency of the transaction state expression.

[0018] This embodiment centrally organizes the transaction running status at the source transaction management node and generates a list of active transactions carrying snapshot version information, so that the transaction status can be expressed in a stable and distinguishable form, providing a unified version benchmark for subsequent parsing of the transaction status and reducing the uncertain impact of changes in transaction status on consistency judgment.

[0019] S20, the list of active transactions carrying the snapshot version information is distributed from the transaction management node to each data shard node of the source database; In this embodiment, in a distributed database, transaction states need to form a consistent understanding across multiple data shards. Therefore, after the transaction management node generates an active transaction list carrying snapshot version information, this active transaction list needs to be synchronized to each data shard node in the source database. The active transaction list expresses the overall running state of transactions in the system at the time of generation, and the snapshot version information identifies the logical time position corresponding to the transaction state expression. Both are transmitted as an inseparable whole during the distribution process to avoid shard nodes receiving incomplete or inconsistent transaction state information.

[0020] During the distribution process, the transaction management node reads the list of generated and frozen active transactions, and encapsulates the transaction boundary information, uncommitted transaction identifiers, transaction ranges to be allocated, and corresponding snapshot version information. This encapsulates the information, enabling shard nodes to directly parse and build a locally usable transaction state view upon receipt. The distribution operation is executed on all data shard nodes in the source database, ensuring that each shard node perceives the transaction state based on the same snapshot version information, thereby eliminating cognitive biases between shards caused by different transaction state sources.

[0021] In practice, the distribution behavior does not depend on the transaction execution path, but is executed as a state synchronization behavior independent of the specific transaction processing flow, so that a stable transaction state propagation channel is formed between the transaction management node and the data sharding node.

[0022] This embodiment distributes a list of active transactions carrying snapshot version information from the source database to each data shard node. This enables each shard node to perceive the transaction operation status in the system based on the same transaction status, reducing the risk of inconsistent understanding of transaction status between shards and providing a unified foundation for subsequent information processing based on transaction status.

[0023] S30, the list of active transactions is written into the transaction log of the transaction management node and the transaction log of each data shard node according to a predetermined period based on the snapshot version information corresponding to the same moment; In this embodiment, in a distributed database environment, transaction states need to be traceable and time-consistent. Therefore, after the active transaction list is generated and distributed, it needs to be permanently recorded in the form of a log. Write operations are triggered according to a predetermined period, ensuring that transaction states are persisted at a stable pace and preventing missing or discontinuous state records due to transient events. The predetermined period is used to limit the write frequency, and its source can be a system scheduling strategy, a time wheel triggering mechanism, or the transaction state change density assessment result.

[0024] During the write process, the snapshot version information corresponding to the same moment is used as a common time alignment basis, ensuring that the active transaction lists recorded in the logs of the transaction management node and each data shard node point to the same logical time position. This same moment does not require the physical clock to be completely consistent, but is aligned through logical time identifiers or a global time base, thereby ensuring the comparability of transaction states recorded in the logs of different nodes.

[0025] The active transaction list is used in the transaction log of the transaction management node to record the global transaction allocation and commit status, and in the transaction log of each data shard node to record the transaction status perceived by the shard side. Writing to these separate nodes ensures that the log records align with the node's responsibilities while maintaining consistent references to snapshot version information, avoiding time drift between logs on different nodes.

[0026] This embodiment improves the traceability and cross-node alignment capabilities of transaction states by writing active transaction lists at predetermined intervals and based on snapshot version information corresponding to the same moment at the transaction management node and each data shard node. This ensures that transaction states form a time-consistent recording basis in the logs of different nodes.

[0027] S40, the transaction logs of the transaction management node and the transaction logs of each data shard node are asynchronously transmitted to the destination database through an independent log stream; In this embodiment, after the source database completes the transaction log writing, to avoid mutual interference between transmissions of different nodes, the transaction logs of the transaction management node and the transaction logs of each data shard node are transmitted through independent log streams. Independent log streams are used to isolate transaction status data from different sources, ensuring that delays, congestion, or failures during log transmission do not have a cascading effect between nodes.

[0028] Asynchronous transmission removes the synchronization constraint between log writing and log transmission, allowing transaction management nodes and data shard nodes to continue processing subsequent transaction state changes after completing local log writing without waiting for transmission to complete. Asynchronous mode also allows the log stream to be scheduled according to its own sending rhythm, thereby adapting to network bandwidth fluctuations and the receiving capacity of the destination database.

[0029] Transaction logs maintain their original chronological order and snapshot version information during transmission, enabling the destination database to accurately identify the transaction status of different nodes at different times based on the received log content. Transaction management node logs and data shard node logs are transmitted separately, allowing the destination database to distinguish the source of global transaction status information from that of sharded transaction status information.

[0030] This embodiment transmits the transaction logs of the transaction management node and the transaction logs of each data shard node asynchronously as independent log streams, thereby isolating transaction status data from different sources during transmission, reducing the impact of transmission latency differences on the overall transaction status synchronization, and improving the stability of log transmission and the overall continuous processing capability of the system.

[0031] S50, when the destination database receives a data query request, it parses the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. In this embodiment, after the destination database receives a data query request, it first performs parsing processing on the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received and stored locally. Transaction log parsing is used to identify data units related to the transaction status from continuous log records, so that the transaction version information implicit in the logs can be extracted and used in a structured manner.

[0032] The transaction logs of the transaction management node contain records reflecting the global transaction allocation and commit status. By parsing these logs, version identifier information corresponding to the active transaction list can be extracted, which describes the range of transaction status perceived by the transaction management node at a specific time point. The transaction logs of each data shard node record the execution and commit status of local transactions on the shard. By parsing these logs, the version information of the active transaction list for the corresponding shard can be extracted, which describes the transaction status reflected by different shards under the log synchronization progress.

[0033] The parsing process only targets the log content already received by the destination database, without relying on the real-time status of the source database, enabling query processing to be performed independently based on currently available transaction information. The extracted version information of the active transaction list from the transaction management node is kept distinct from the version information of the active transaction list from each data shard node, providing a basis for subsequent comparisons of transaction status across different nodes.

[0034] This embodiment parses the received transaction logs of the transaction management node and the transaction logs of each data shard node on the destination database side and extracts the corresponding active transaction list version information. This enables the query processing to make judgments based on the currently synchronized transaction status, avoiding dependence on the source database and thus improving the certainty of the transaction status recognition during the query phase.

[0035] S60, based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, determine the common active transaction list version information of each data shard node as a globally consistent snapshot. In this embodiment, assuming the destination database has already obtained the active transaction list version information from the transaction management node and the active transaction list version information from each data shard node, it is necessary to perform a unified analysis of this version information to determine the active transaction list version information that can be supported by all data shard nodes. This determination process is used to resolve the version inconsistency problem caused by asynchronous transmission of transaction logs from different nodes, enabling subsequent processing to be based on a unified transaction view.

[0036] The version information of the active transaction list on the transaction management node reflects the time boundaries of global transaction allocation and commit status, while the version information of the active transaction list on each data shard node reflects the actual progress of local transaction execution and log synchronization on the shard. By introducing both types of version information, a constrained version comparison space can be built on the destination database side, avoiding misjudgments of transaction status caused by relying solely on information from a single node.

[0037] The process of determining the common active transaction list version information for all data shard nodes involves finding a version position among multiple version information that is covered by all data shard nodes, and the active transaction list corresponding to this version position can be consistently identified by all data shard nodes. This common version information logically represents the unified transaction state that can be safely used on the current destination database side, and is defined as a globally consistent snapshot, used to characterize the transaction boundaries visible during the query phase.

[0038] This embodiment determines the common active transaction list version information by integrating the active transaction list version information of the transaction management node and each data shard node on the destination database side. This enables query processing to be based on a transaction view that is supported by all data shard nodes, thereby reducing the risk of transaction state inconsistency caused by asynchronous log transmission.

[0039] S70, based on the list of active transactions corresponding to the global consistency snapshot, perform visibility judgment on the data to be accessed by the data query request and return the query result.

[0040] In this embodiment, given that a globally consistent snapshot has been determined, the destination database needs to perform a visibility assessment on the data to be accessed by the data query request based on the active transaction list corresponding to the globally consistent snapshot. The active transaction list contains transaction boundary information, incomplete transaction identifiers, and transaction allocation ranges, which can fully describe which transaction states are considered completed and which are still incomplete or invisible from the snapshot's perspective. By introducing the active transaction list corresponding to the globally consistent snapshot, data visibility assessment no longer depends on the local transaction state of a single data shard node, but is uniformly controlled by the consistent transaction view already confirmed on the destination database side.

[0041] During visibility assessment, the destination database maps data query requests to specific data access objects and compares the transaction identifier associated with each access object with the transaction boundary information in the active transaction list. If the transaction identifier is within the committed transaction boundary and not included in the uncommitted transaction identifier set, the corresponding data state is determined to be visible; if the transaction identifier is within the uncommitted transaction identifier set or exceeds the transaction range allowed by the snapshot, the corresponding data state is determined to be invisible. In this way, visibility assessment is strictly limited to the transaction range defined by the globally consistent snapshot, avoiding inconsistencies in data visibility caused by differences in synchronization progress among different data shard nodes.

[0042] After determining the visibility of the data involved in the query, the destination database constructs the query results based only on the data that is determined to be visible, and returns the query results, thereby ensuring that the query results are consistent with the global consistency snapshot at the transaction level.

[0043] This embodiment performs visibility judgment on the queried data based on the list of active transactions corresponding to the globally consistent snapshot, enabling the destination database to form a unified data visibility standard in an environment of asynchronous and synchronous transaction states, thereby ensuring that the query results remain consistent across multiple data shard replicas.

[0044] In one embodiment, step S10 above includes: S101, a maintenance mechanism for an active transaction list is established through the transaction management node of the source database of the distributed database. The maintenance mechanism for the active transaction list is used to manage the boundary information of committed transactions, the identifier of uncommitted transactions, and the scope of transactions to be allocated. S102, monitor the transaction start event through the transaction management node, update the range of transactions to be allocated, and add the newly allocated transaction identifier to the uncommitted transaction identifier; S103, respond to the transaction commit event through the transaction management node, and update the committed transaction boundary information and the uncommitted transaction identifier; S104, when the transaction management node arrives at the predetermined period, it obtains the current committed transaction boundary information, uncommitted transaction identifiers and pending transaction ranges to form an active transaction list; S105, the transaction management node generates snapshot version information based on the current system state and a predefined version generation strategy, writes the snapshot version information as metadata into the header area of ​​the active transaction list, and records the correspondence between the snapshot version information and the active transaction list in the distributed metadata service.

[0045] In this embodiment, when the transaction management node generates an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges and allocates snapshot version information, it first establishes a maintenance mechanism for the active transaction list to continuously maintain the committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges. This maintenance mechanism can be implemented as a maintenance process within the transaction management node that coordinates a set of in-memory data structures with persistent constraints. The committed transaction boundary information represents the boundary of transaction numbers that have been committed, derived from the incrementing watermark of the transaction commit confirmation sequence or commit confirmation log. The uncommitted transaction identifiers represent the set of allocated but not yet committed transaction numbers, derived from the transaction number allocation results generated by the transaction start event and the recovery results of the transaction end event. The pending transaction range represents a continuous interval of subsequently allocable transaction numbers, derived from the current cursor of the transaction number generator and the progress of the allocation strategy. The three types of information form a constraint relationship in the maintenance mechanism. The committed transaction boundary information provides a boundary reference for completed commits, the uncommitted transaction identifier limits the set of transaction numbers that have been allocated but cannot be regarded as committed, and the pending transaction range limits the starting position and available range of the subsequently allocated transaction numbers. This enables the active transaction list to express the segmented structure of the committed, uncommitted, and pending transaction number space.

[0046] When the transaction management node monitors transaction start events, these events can originate from transaction session establishment, explicit transaction start commands, or implicit transaction trigger requests. Based on this event, the transaction management node allocates new transaction identifiers from the scope of transactions to be allocated. The allocation process can advance the starting point of the scope of transactions to be allocated using sequential allocation or batch pre-allocation. The allocated transaction identifier is deducted from the scope of transactions to be allocated and written into the uncommitted transaction identifier to maintain consistency constraints between the scope of transactions to be allocated and the uncommitted transaction identifier. To reduce allocation conflicts under high concurrency, the scope of transactions to be allocated can be maintained by atomic cursors, and the uncommitted transaction identifier can be maintained by bitmaps, skip lists, range sets, or hash sets. After completing the allocation, the transaction management node establishes an association record between the transaction identifier and the transaction session identifier, enabling subsequent transaction commit events to accurately locate the corresponding uncommitted transaction identifier entry.

[0047] When the transaction management node responds to a transaction commit event, the event can originate from a commit request confirmation, a two-phase commit commit decision notification, or a commit receipt. Based on this event, the transaction management node updates the committed transaction boundary information and the uncommitted transaction identifiers. The update action includes at least removing the corresponding transaction identifier from the uncommitted transaction identifiers and performing a progress check on the committed transaction boundary information. This progress check needs to avoid directly including transaction numbers with gaps in the committed transaction boundary information. Therefore, a continuous commit watermark progression rule can be used. After removing a transaction identifier, it checks whether there are still blocked numbers in the uncommitted transaction identifiers after the committed transaction boundary information. If not, the committed transaction boundary information is advanced to the maximum position that maintains a continuous committed interval. This progression rule ensures that the committed transaction boundary information has a monotonically increasing property, while ensuring that the uncommitted transaction identifiers accurately represent the set of transaction numbers that have not yet been committed, avoiding the distorted representation of the active transaction list caused by incorrectly folding uncommitted transactions into the committed transaction boundary information.

[0048] When the predetermined period arrives, the transaction management node acquires the current committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges to form an active transaction list. The predetermined period can be implemented as a time period, an event-triggered period, or a hybrid triggering period. The time period can be driven by a timer, the event-triggered period can be driven by a transaction start event count threshold or a transaction commit event count threshold, and the hybrid triggering period can trigger a generation action when either the time or event is satisfied. Acquiring this action requires ensuring the consistency of the three types of information at the same sampling moment. Therefore, atomic reads can be achieved when forming the active transaction list using read-write locks, version stamps, or snapshot replication. The committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges are combined into a structured list entry. This structured list entry can contain a commit boundary field, an uncommitted set field, and a pending range field. The uncommitted set field can be expressed in interval compression to reduce size, and the pending range field can be expressed using a start number and an upper bound number to reduce redundancy. This ensures that the active transaction list not only expresses the transaction status but also facilitates subsequent propagation and writing.

[0049] When a transaction management node generates snapshot version information based on the current system state and a predefined version generation strategy, the current system state may include the transaction management node's local timestamp, the current value of committed transaction boundary information, a summary of uncommitted transaction identifiers, the cursor position of the transaction range to be allocated, and the node's identity identifier. The predefined version generation strategy is used to map the system state into comparable, sortable, and traceable snapshot version information. The version generation strategy can employ a monotonically increasing sequence number strategy, a timestamp plus node sequence strategy, or a log position-based watermark strategy. The monotonically increasing sequence number strategy relies on an atomic counter to generate a globally increasing version number. The timestamp plus node sequence strategy uses intra-node sequences to fill in the gaps within the same time slice to avoid duplication. The log position-based watermark strategy uses the write or commit position in the transaction log of the transaction management node as the version source. After generation, the snapshot version information is written as metadata to the header area of ​​the active transaction list. The header area may include a version field, a generation time field, a generation node field, and a verification field, enabling the active transaction list to directly identify version boundaries when read or transmitted. Subsequently, the correspondence between snapshot version information and the active transaction list is recorded in the distributed metadata service. The distributed metadata service can be implemented using key-value stores, metadata tables, or consistent metadata logs. The snapshot version information is used as the key, and the storage location, digest value, or content hash of the active transaction list is used as the value to establish the mapping. A lifecycle status field can also be attached to distinguish between valid, expired, or cleaned-up states, thereby providing a locatable basis for subsequent indexing of the active transaction list by version.

[0050] This embodiment, through the above steps, enables the active transaction list to have comparable version boundaries and indexable location criteria, reducing the probability of ambiguity and mismatch during the generation, retrieval, and consistency verification of the active transaction list.

[0051] In one embodiment, step S20 above includes: S201, the transaction management node serializes the list of active transactions carrying snapshot version information into a distributed message format to generate a message of the list of active transactions carrying snapshot version information. S202, the transaction management node sends the active transaction list message carrying snapshot version information to each data shard node of the source database based on the message broadcasting mechanism; S203, receive the active transaction list message carrying snapshot version information through each data shard node, and parse the active transaction list carrying snapshot version information to obtain the active transaction list. S204, each data sharding node sends a message reception confirmation signal to the transaction management node; S205, the integrity of the distribution operation is verified by the transaction management node based on the message reception confirmation signals of all data shard nodes.

[0052] In this embodiment, when the transaction management node distributes the active transaction list carrying snapshot version information to each data shard node of the source database, it first serializes the active transaction list carrying snapshot version information into a distributed message format to generate an active transaction list message carrying snapshot version information. Serialization is used to convert the active transaction list from the data structure within the node into a transmittable byte sequence or field sequence. The distributed message format is used to limit the field order, field type, length boundaries, and verification rules, so that different data shard nodes can restore the active transaction list in a consistent parsing method in a heterogeneous operating environment. The active transaction list message carrying snapshot version information needs to retain the binding relationship between the snapshot version information and the active transaction list content in the message payload. The snapshot version information can appear as a message header field or a fixed field in the payload, and the active transaction list can appear as a structured set of fields, so that the data shard nodes can locate the snapshot version information first and then locate the field content of the active transaction list during parsing, thereby avoiding mismatch between the snapshot version information and the active transaction list content. To reduce message size and improve transmission efficiency, during serialization, range compression or bitmap compression can be applied to set-type fields in the active transaction list, dictionary encoding can be used for duplicate fields, and presence bits can be used for optional fields. This allows the active transaction list message carrying snapshot version information to maintain parsability while reducing network overhead. To improve parsing reliability, the active transaction list message carrying snapshot version information can include a checksum field. The checksum field can be calculated from the snapshot version information and the active transaction list content, and is used at the receiving end to verify that the message has not been truncated or tampered with during transmission.

[0053] When the transaction management node sends an active transaction list message carrying snapshot version information to each data shard node of the source database using a message broadcast mechanism, the message broadcast mechanism distributes the same active transaction list message carrying snapshot version information to multiple receiving targets, avoiding the differences in sending paths and timing caused by unicast. The message broadcast mechanism can be implemented as a publish-subscribe channel, a broadcast topic, or a multicast group. When sending, the transaction management node writes the active transaction list message carrying snapshot version information into the corresponding broadcast channel, enabling each data shard node to pull or receive the message as a subscriber. To achieve traceability and suppression of duplicate reception, the active transaction list message carrying snapshot version information can carry a message identifier. The message identifier can be derived from the snapshot version information or generated by the transaction management node and associated with the snapshot version information, allowing each data shard node to determine whether it has already processed the same active transaction list message carrying snapshot version information based on the message identifier. To maintain the predictability of the receiving order under the broadcast mechanism, the transaction management node can also add a monotonically increasing sequence field between the continuously sent active transaction list messages carrying snapshot version information, enabling the data sharding node to detect out-of-order data and process it in order or trigger compensation processing.

[0054] When each data shard node receives and parses an active transaction list message carrying snapshot version information, the receiving action retrieves the message content from the channel corresponding to the message broadcast mechanism. The parsing action restores the message content into a structured active transaction list according to the field definitions of the distributed message format and reads the snapshot version information within it. During parsing, the verification fields of the active transaction list message carrying snapshot version information can be validated first to confirm the integrity of the message content. Then, the snapshot version information is read to establish a version context. Subsequently, the content of each field of the active transaction list is read, and storage entries for the active transaction list are created locally. To ensure that the active transaction list carrying snapshot version information can be referenced by subsequent processing on the data shard node side, the data shard node can bind the snapshot version information and the active transaction list and write them to the local memory cache or local persistent area. The snapshot version information can be used as an index key or participate in the index key generation, allowing the same data shard node to locate the corresponding active transaction list by snapshot version information. If the received active transaction list message carrying snapshot version information is delivered repeatedly, the data shard node can perform idempotency determination based on the message identifier or snapshot version information, and perform overwrite, skip, or merge processing on the corresponding entries of the existing snapshot version information. Overwrite is used to refresh the same version entries with the latest received content, skip is used to keep the previously existing entries unchanged, and merge is used to perform field-level merging when there are incremental fields in the shard node, so as to avoid state drift caused by repeated processing.

[0055] When each data shard node sends a message reception acknowledgment signal to the transaction management node, the acknowledgment signal indicates that the active transaction list message carrying snapshot version information has been received and parsed at the corresponding data shard node. The acknowledgment signal should include relevant information to locate the corresponding message and avoid ambiguity. This relevant information can include snapshot version information, a message identifier, or a combination of both, enabling the transaction management node to establish a one-to-one correspondence between the acknowledgment signal and a specific active transaction list message carrying snapshot version information. To distinguish between successful reception and successful reception and parsing, the acknowledgment signal can also include an acknowledgment status field. This field identifies the completion stage, which can cover received, verified, parsed, or written to a local entry, allowing the transaction management node to make more granular judgments about the reliability of the distribution chain. To prevent the transaction management node from misjudging distribution failure due to lost acknowledgment signals, each data shard node can employ a retry mechanism when sending the acknowledgment signal, maintaining consistency in the acknowledgment content corresponding to the same snapshot version information during the retry process, allowing the transaction management node to perform idempotent processing.

[0056] When the transaction management node verifies the integrity of the distribution operation based on message reception confirmation signals from all data shard nodes, this verification confirms that the active transaction list message carrying snapshot version information has covered all data shard nodes in the source database. This prevents version inconsistencies caused by only some data shard nodes updating the active transaction list. The transaction management node can maintain a list of data shard node sets, match the message reception confirmation signal of each data shard node with the node identifier in the list, and use the snapshot version information as the aggregation dimension to calculate the confirmation coverage, thereby obtaining the distribution completion status for that snapshot version information. If message reception confirmation signals from all data shard nodes are not received within a preset waiting window, the transaction management node can mark the distribution completion status as incomplete and record the set of missing node identifiers for subsequent resending or exception handling. If message reception confirmation signals from all data shard nodes are received, the transaction management node can mark the distribution completion status corresponding to that snapshot version information as complete and establish an association record between the completion status and the snapshot version information. This allows subsequent steps to determine whether the active transaction list corresponding to that snapshot version information has been consistently updated across all data shard nodes in the source database based on the distribution completion status. To improve traceability, the transaction management node can also save the confirmation timestamp and confirmation status field of each data shard node when recording the completion status of distribution, forming an auditable record of the distribution operation, which can be used to locate the abnormal position of the distribution link when a version gap or node failure occurs.

[0057] This embodiment, through the above steps, enables the active transaction list and snapshot version information to form a resolvable, traceable, and verifiable distribution link across multiple shards, reducing the probability of inconsistencies in shard states caused by incomplete distribution coverage and version mismatches.

[0058] In one embodiment, step S30 above includes: S301, when the predetermined period arrives, the same time reference is obtained based on the global time synchronization mechanism; S302, Based on the snapshot version information corresponding to the same time reference, determine the list of active transactions associated with the snapshot version information; S303, Write the list of active transactions and the snapshot version information corresponding to the same time reference into the transaction log of the transaction management node; S304, based on the snapshot version information corresponding to the same time reference, control each data shard node to write the list of active transactions into the transaction log of the corresponding data shard node.

[0059] In this embodiment, when the active transaction list is written to the transaction logs of the transaction management node and each data shard node at a predetermined period, the predetermined period is used to limit the write trigger frequency and write granularity. It originates from the incremental change requirements of the transaction status on the time axis and can slice the status of the active transaction list at fixed intervals, which is convenient for subsequent time-based location and playback. The predetermined period can be expressed using period parameters at the millisecond, second, or minute level, or it can be expressed as an equivalent period of "triggering a write once every time a preset number of transaction start events or transaction commit events are accumulated," so that the write frequency matches the system throughput. In this context, the active transaction list plays the role of a carrier of the transaction status set. Its content comes from the continuous maintenance results of transaction start events and transaction commit events by the transaction management node. The write action requires that the active transaction list has a definite boundary at the moment of writing. Therefore, the write trigger point needs to be bound to the same time base to freeze the active transaction list view at that time, so as to avoid inconsistency between the written content and the snapshot version information caused by concurrent updates during the write process.

[0060] When the predetermined period arrives, a common time reference is obtained. The global time synchronization mechanism ensures that the transaction management node and each data shard node have an aligned time reference for understanding the same moment, preventing write time drift caused by different nodes using their own local clocks. The common time reference describes the time anchor commonly used within a write trigger window. It can be a unified timestamp, a logical time scale, or a comparable time point identifier within the synchronization system. The global time synchronization mechanism can be implemented by synchronizing time on the node side and providing a unified time read interface based on a time synchronization service, or by generating a time scale on the transaction management node side based on a logical clock and broadcasting it to each data shard node, so that each data shard node uses the same scale as the write reference upon receiving the scale. This common time reference needs to be referenced by the write record; therefore, it can be stored as a field in the transaction log entry so that the unified time corresponding to the write can be restored when parsing the transaction log later.

[0061] When determining the list of active transactions associated with snapshot version information based on snapshot version information corresponding to the same time reference, the snapshot version information is used to express the version identifier of the active transaction list at a certain time anchor point. It originates from the process of the transaction management node allocating versions to the active transaction list, and can bind the active transaction list written once with the version identifier written once. Snapshot version information corresponding to the same time reference means that the generation or selection of snapshot version information has a definite mapping relationship with the same time reference. This mapping relationship can be formed by "reading the current snapshot version information at the trigger time of the same time reference," or by "calculating the version number or timestamp field of the snapshot version information according to the same time reference." When determining the list of active transactions associated with snapshot version information, it is necessary to use the snapshot version information as an index key to locate the corresponding entry in the active transaction list cache maintained by the transaction management node, or to use the snapshot version information as a filter condition to filter the active transaction list generation pipeline to select the active transaction list with matching versions, thereby ensuring that the active transaction list being written is consistent with the snapshot version information used for this write. For example, when the reference point is a unified timestamp, the snapshot version information may include the unified timestamp or its derived sequence number. The active transaction list writes the unified timestamp into the header area when it is generated, so the relationship can be determined by matching the header field.

[0062] When writing the active transaction list and the snapshot version information corresponding to the same time reference to the transaction management node's transaction log, the transaction log is used to carry the transaction management node's time-series record of transaction status and version. The write operation requires that the active transaction list and snapshot version information be written to disk in the same log record or the same group of related log records, so that the active transaction list content and snapshot version information can be restored simultaneously when the transaction management node's transaction log is parsed subsequently. The combination relationship between the active transaction list and snapshot version information needs to be maintained atomically during the write to avoid gaps in parsing caused by writing only one of them. Therefore, a single transaction log record containing both fields can be used, or two records can be used with the same snapshot version information as the association key. The transaction management node's transaction log also needs to record the write order. Therefore, an incrementing sequence or offset field can be added to the log record so that the active transaction list at the same time reference can be restored in the order of writes when replaying the transaction log.

[0063] When controlling each data shard node to write its active transaction list to its corresponding transaction log based on the snapshot version information corresponding to the same time reference, the control action triggers each data shard node to perform writes under the same time reference, thereby forming parallel disk writes under the same snapshot version information within the shard range. The control signal can carry the same time reference and snapshot version information, enabling each data shard node to use the snapshot version information as a write identifier and write the active transaction list and snapshot version information to its transaction log during the write process. The corresponding data shard node's transaction log records the write trajectory of the active transaction list for that shard. The write content needs to be consistent with that of the transaction management node, and at least must include snapshot version information and the active transaction list content in the log record so that the destination database can extract the active transaction list version information of each data shard node and align it with the active transaction list version information of the transaction management node when parsing the transaction logs of each data shard node. To avoid discrepancies in the write times of different data shard nodes, the same time reference can be used as part of the write trigger condition. After receiving the control signal, each data shard node executes the write based on the same time reference, or waits locally for the trigger point corresponding to the same time reference before writing, so that the transaction log records of different shards can be compared in terms of time semantics.

[0064] This embodiment enables the transaction management node and each data shard node to form aligned transaction log records under the same time semantics and version identifier through the above steps, reducing the difficulty of version alignment caused by inconsistent write times or inconsistent version bindings in a multi-shard environment.

[0065] In one embodiment, step S50 above includes: S501, Receive data query request through the destination database, and parse the consistency level requirements and query scope information in the data query request; S502, scan the portion of the transaction log of the received transaction management node that is related to the query range information through the destination database, and parse the active transaction list version information of the transaction management node from the scanned portion; S503, based on the consistency level requirements, determine the parallel scanning strategy, scan the part of the transaction log of each data shard node that has been received according to the parallel scanning strategy through the destination database, and parse the active transaction list version information of each data shard node from the scanned part; S504, Based on the consistency level requirements, a verification standard is determined, and the timestamp consistency and integrity of the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node are verified through the destination database according to the verification standard. S505 caches the version information of the active transaction list of the transaction management node that has passed the timestamp consistency and integrity verification, as well as the version information of the active transaction list of each data shard node, in the query context environment through the destination database.

[0066] In this embodiment, when the destination database receives a data query request, it performs parsing and extraction. The data query request, as the triggering input, originates from a read request initiated by the client, application service, or intermediate query gateway to the destination database. The data query request serves as a carrier of two types of constraints in this process: one is the consistency level requirement used to limit read consistency semantics, and the other is the query range information used to limit the scope of transaction information to be retrieved. The consistency level requirement describes the degree of visibility consistency that the destination database needs to achieve. It can be expressed as a level that mandates the use of a globally consistent snapshot, or a level that allows alignment within a smaller range, thus determining the stringency of subsequent parallel scanning strategies and verification standards. The query range information constrains the target interval of the transaction log scan. Its source can be the target shard set, target table, or key range carried in the data query request, or it can be the shard coverage range deduced by the destination database through the routing table after parsing the data query request, causing the transaction log scan to converge from a full traversal to the part related to the query range information. In terms of receiving and parsing, the destination database can perform syntax-level parsing on the data query request to extract the consistency level requirement field and query scope information field, or it can perform protocol-level unpacking on the data query request and locate the consistency level requirement and query scope information in the request header or extended fields. The parsing result needs to be written into the query context environment so that subsequent scanning, verification and caching actions can share the same set of constraint inputs.

[0067] When scanning the transaction logs of the received transaction management nodes for portions related to the query range information and parsing the active transaction list version information of the transaction management nodes, the transaction logs of the transaction management nodes are represented on the destination database side as a set of received log fragments or a set of log files. The scanning action is used to locate the log record intervals that match the query range information within these sets. The portions related to the query range information can be determined through various mapping methods, such as mapping the time boundaries in the query range information to the timestamp field range of the transaction logs, mapping the snapshot version information candidate range in the query range information to the version field range of the transaction logs, and mapping the shard coverage range in the query range information to the filtering conditions of the corresponding shard identifier field in the transaction management node logs. The scanning process needs to be able to output the intermediate result of the scanned portion, which can be the set of hit log records, the hit log offset interval, or the set of hit index entries, to facilitate decoupling of scanning and parsing. When parsing the version information of the active transaction list of the transaction management node, it is necessary to extract a combination of fields that can characterize the version evolution of the active transaction list on the transaction management node side from the scanned portion. These fields could include snapshot version information, timestamp fields corresponding to the same time reference, log sequence number fields, or offset fields. The parsing result forms the version information of the active transaction list of the transaction management node and serves as input for subsequent verification. To improve the determinism of the parsing, the destination database can prioritize selecting the latest available set of records that meet the consistency level requirements during the scan phase, or it can select a set of records covering the time window corresponding to the query range information, to avoid incomplete version information due to only hitting a portion of the records.

[0068] When determining a parallel scanning strategy based on consistency level requirements and scanning the transaction logs of each received data shard node according to the parallel scanning strategy, the parallel scanning strategy describes the concurrency, scheduling, and resource boundaries for scanning the transaction logs of multiple data shard nodes. The input to the parallel scanning strategy comes from the consistency level requirements. The stricter the consistency level requirements, the more likely it is that the scans of each shard need to be completed within a closer time window to reduce the probability of verification failures caused by cross-shard version misalignment. Therefore, the parallel scanning strategy can set higher concurrency, tighter scan time windows, and more explicit shard priorities. The implementation of the parallel scanning strategy can be done by assigning scan tasks to each data shard node at the granularity of shards, or by dividing the transaction log of each data shard node into several scan segments at the granularity of log segments and executing them concurrently, while maintaining the task status in the query context environment for aggregation. When scanning the parts of the received transaction logs of each data shard node that are related to the query range information, it is necessary to use the query range information to limit the scan boundaries of each shard to avoid scanning irrelevant shards or irrelevant log intervals. When parsing the version information of the active transaction list for each data shard node during the scanning process, the parsing logic needs to be comparable to the version information of the active transaction list for the transaction management node. It must at least include version and timestamp fields related to the snapshot version information to enable subsequent timestamp consistency and integrity checks. The version information of the active transaction list for each data shard node needs to be aggregated by shard identifier to form a version information set indexed by shard, facilitating shard-by-shard alignment or set-level alignment checks during the verification phase.

[0069] When determining the verification standard based on consistency level requirements and verifying the consistency and integrity of the timestamps of the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node according to the verification standard, the verification standard is used to define the judgment conditions, tolerance range, and failure handling boundaries of the verification. Its rigor is driven by the consistency level requirements. When the consistency level requirement is high, the verification standard may require that the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node be completely consistent in the timestamp field or that a common timestamp set exists and is not empty. When the consistency level requirement is low, the verification standard may allow the timestamps to have a preset deviation window, but still requires that an alignable timestamp within the deviation window can be found. Timestamp consistency verification is used to determine whether there is an alignable relationship between the timestamp sequence on the transaction management node side and the timestamp sequences on the data shard node side. The verification method can be to find the intersection, find the maximum common timestamp, or find a common timestamp that meets the conditions within the timestamp reference range. Integrity verification is used to determine whether the extracted version information covers the necessary intervals corresponding to the query range information. This determination can include whether at least one version information has been extracted from each shard, whether key fields are missing, and whether there are log breaks causing sequence discontinuities. The verification output needs to distinguish between pass and fail states, forming a filtered result that can be used for caching. The filtered result can be represented as the set of active transaction list version information for the transaction management node that passed the verification, and the set of active transaction list version information for each data shard node that passed the verification, or as a subset of version information marked as usable within each shard.

[0070] When caching the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node that have passed timestamp consistency and integrity checks in the query context environment, the query context environment serves to carry the shared state in a data query request processing chain. Its content originates from the consistency level requirements and query range information obtained during the receiving and parsing phase, as well as the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node obtained during the scanning and parsing phase, and the passed set output by the verification phase. The goal of caching is to fix the version information that subsequent steps need to reuse within the lifecycle of a single query, avoiding rescanning of the transaction log when determining a global consistency snapshot or performing visibility checks. The cache structure can be organized in a two-level structure: one level stores the active transaction list version information of the transaction management node using the transaction management node as the key, and the other level stores the active transaction list version information of each data shard node using the shard identifier as the key, with a verification pass flag and timestamp reference range appended to the query context environment, facilitating subsequent direct comparison and selection based on the cached content. Cache writes need to be bound to data query requests to avoid version confusion caused by reuse between different data query requests. Therefore, a request identifier can be assigned to the query context environment and cache entries can be isolated by the request identifier. At the same time, the cached content in the query context environment can be cleaned up after the query ends or times out to release resources.

[0071] This embodiment, through the above steps, enables a single data query request to form a usable and aligned set of version information within the scope of transaction information already received by the destination database. This reduces the impact of irrelevant log scanning and cross-shard version misalignment on parsing and extraction, and provides stable input for subsequent determination of globally consistent snapshots based on version information.

[0072] In one embodiment, step S60 above includes: S601, collect the version information of the active transaction list of the transaction management node and the version information of the active transaction list of each data shard node to form a complete set of version information; S602, determine the timestamp reference range based on the active transaction list version information of the transaction management node in the complete version information set; S603, compare the active transaction list version information of each data shard node in the complete version information set with the timestamp sequence within the timestamp reference range, and identify the active transaction list version information corresponding to the largest common timestamp contained in each data shard node; S604, verify the existence and consistency of the active transaction list version information corresponding to the maximum common timestamp in each data shard node, and use the active transaction list version information that has passed the existence and consistency verification as the common active transaction list version information of each data shard node. S605, mark the version information of the active transaction list common to each data shard node as a global consistency snapshot, and generate a unique identifier for the global consistency snapshot; S606, record the mapping relationship between the global consistency snapshot and the version information of the common active transaction list of each data shard node, and store the generation timestamp and snapshot metadata of the global consistency snapshot.

[0073] In this embodiment, when determining the common active transaction list version information of each data shard node as a globally consistent snapshot based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the active transaction list version information of the transaction management node plays the role of input for global time reference and version boundary constraints, and the active transaction list version information of each data shard node plays the role of input for shard-side reachability status. The two types of version information are aligned using timestamps as a comparable field. When collecting version information from the active transaction lists of the transaction management node and the active transaction lists of each data shard node to form a complete version information set, the collection process needs to cover all relevant version information records received by the destination database. The version information from the active transaction lists of the transaction management node can come from the parsed or cached output of the transaction logs of the transaction management node, and the version information from the active transaction lists of each data shard node can come from the parsed or cached output of the transaction logs of their respective data shard nodes. When forming a complete version information set, it is necessary to maintain source differentiation and an index structure. The index structure can include a version information list with node identifier as the key, an inverted index mapping with timestamp as the key, and an aggregate mapping with snapshot version information as the key, thereby supporting subsequent comparisons by limiting the comparison range based on timestamp reference ranges and reducing comparison costs. During the construction of the complete version information set, duplicate version information needs to be deduplicated. Deduplication can be based on a combination of timestamp and version sequence number fields, or a combination of snapshot version information fields and log offset fields. After deduplication, the source node identifier of each version information record is still retained, facilitating the location of the corresponding data shard node and the execution of existence and consistency checks during the subsequent verification phase.

[0074] When determining the timestamp reference range based on the active transaction list version information of the transaction management nodes in the complete version information set, the timestamp reference range is used to constrain the selectable interval of candidate timestamps, avoiding invalid comparisons of excessively old or new timestamps in the active transaction list version information of each data shard node. The timestamps carried in the active transaction list version information of the transaction management nodes can be considered as reference points for global time series. When determining the timestamp reference range, a closed interval can be formed by extracting the minimum and maximum reference timestamps from the active transaction list version information of the transaction management nodes. Alternatively, a target timestamp can be extracted and combined with a preset window width to generate a sliding interval. Multiple intervals can also be formed based on the active transaction list version information of multiple consecutive transaction management nodes, and the comparison can be attempted sequentially according to these intervals. The generation of the timestamp reference range needs to match the actual coverage of the complete version information set. The generation process can begin by statistically analyzing the timestamp distribution of the active transaction list version information of the transaction management nodes, and then mapping the statistical results to the timestamp reference range. Mapping rules can include removing abnormal timestamps, removing records with missing fields, and removing isolated points with excessive jumps, preventing abnormal records from expanding the timestamp reference range and causing the candidate set to swell. Once the timestamp reference range is generated, it needs to be written into the comparison context of the complete version information set so that subsequent comparison operations can directly filter the active transaction list version information of each data shard node using the timestamp reference range.

[0075] When comparing the active transaction list version information of each data shard node within a complete version information set and identifying the version information of the active transaction list corresponding to the largest common timestamp contained in all data shard nodes, the comparison object is limited to the active transaction list version information of each data shard node, the filtering condition is limited to the timestamp reference range, and the comparison goal is to identify the largest common timestamp. A timestamp sequence can be understood as the set of version information timestamps visible to each data shard node on the destination database side. During comparison, the timestamp sequence of each data shard node can first be filtered and sorted according to a range, based on whether the timestamps are increasing or decreasing. Then, the intersection of multiple timestamp sequences is obtained to obtain a common timestamp set, and the largest common timestamp is selected from this set. The selection of the largest common timestamp is intended to select version information that is closer to the current synchronization progress, given that all shards possess it, thus reducing read lag caused by selecting an older timestamp. To improve identification efficiency, the timestamp sequences of each data shard node can be constructed as a bitmap or ordered set structure. A multi-way merge algorithm can then be used to locate the common timestamp. Alternatively, the algorithm can iterate backwards from the upper bound of the timestamp reference range, checking in each iteration whether the timestamp exists simultaneously in the timestamp sequences of all data shard nodes. The first matching timestamp is the maximum common timestamp. After identifying the maximum common timestamp, it needs to be mapped back to the active transaction list version information in the complete version information set. The mapping rule uses the timestamp field as the primary key and combines it with the shard identifier field to select the corresponding record, thus obtaining the active transaction list version information corresponding to the maximum common timestamp. This information is then used as input for the subsequent verification stage.

[0076] When verifying the existence and consistency of the active transaction list version information corresponding to the maximum common timestamp across all data shard nodes, and using the active transaction list version information that passes existence and consistency verification as the common active transaction list version information for all data shard nodes, existence verification confirms that each data shard node does indeed have the active transaction list version information record corresponding to the maximum common timestamp on the destination database side. Consistency verification confirms that the version information corresponding to the maximum common timestamp across all data shard nodes can be determined to be the same version in key fields. The existence verification can be implemented by retrieving the complete version information set by shard identifier, checking whether each shard has a version information entry with a timestamp equal to the maximum common timestamp, and further checking whether the log sequence number and log offset of the version information entry fall within the received range to avoid false existence due to missing log fragments. Consistency verification requires determining the consistency judgment field set, which can include a snapshot version information field, a timestamp field, a version sequence number field, and an active transaction list summary field. The active transaction list summary field can consist of the header metadata of the active transaction list, a checksum, or a hash digest, used to compare consistency without expanding all content. After successful consistency verification, when confirming the active transaction list version information as the common active transaction list version information for all data shard nodes, a common version information object needs to be created. This object should record the maximum common timestamp, the set of data shard nodes participating in the verification, and the values ​​of the consistency judgment field set. This ensures that subsequent marking and recording actions reuse the same common version information object. If existence and consistency verification fail, a failure reason flag needs to be generated in the verification context. This flag can indicate missing shards, inconsistent fields, or verification failure, providing feedback input for subsequent re-identification of the maximum common timestamp.

[0077] When marking the common active transaction list version information of each data shard node as a globally consistent snapshot and generating a unique identifier for the globally consistent snapshot, the marking action promotes the common version information object to a referenceable globally consistent snapshot entity. The unique identifier of the globally consistent snapshot provides a stable reference key to support query paths, audit paths, and backtracking paths. The unique identifier of the globally consistent snapshot can be generated by combining the maximum common timestamp and the participating shard set identifier, or by combining the snapshot version information field and the maximum common timestamp field, or by combining the difference between the upper bound of the timestamp reference range and the maximum common timestamp and the version sequence number. After generation, the unique identifier must satisfy the requirements of uniqueness and repeatability within the destination database. The marking action can set a snapshot flag bit on the common version information object in the memory structure, or it can write a globally consistent snapshot record to the persistent metadata table and point the reference field of the common version information object to this record, so that subsequent processing can locate the globally consistent snapshot through the unique identifier.

[0078] When recording the mapping relationship between the globally consistent snapshot and the version information of the common active transaction list of each data shard node, and storing the generation timestamp and snapshot metadata of the globally consistent snapshot, the mapping relationship is used to establish a traceable link between the globally consistent snapshot entity and the common version information object. The generation timestamp is used to characterize the moment when the globally consistent snapshot is confirmed on the destination database side, and the snapshot metadata is used to carry additional information required for subsequent queries and maintenance. The mapping relationship can include a mapping between the unique identifier of the globally consistent snapshot and the maximum common timestamp field, a mapping between the unique identifier of the globally consistent snapshot and the version sequence number field of the version information of the common active transaction list of each data shard node, or a mapping between the unique identifier of the globally consistent snapshot and the set of data shard nodes participating in the verification. The generation timestamp can be taken from the local time source of the destination database or from the unified time base provided by the global time synchronization mechanism. After taking the value, it is written together with the unique identifier of the globally consistent snapshot into the snapshot metadata storage area. Snapshot metadata can include timestamp reference range, a set of fields for existence and consistency verification, checksum, offset range of version information source log, number of shards, etc., to support the subsequent visibility judgment process based on globally consistent snapshots to quickly locate relevant version information and reduce redundant calculations.

[0079] This embodiment enables the destination database to determine the common active transaction list version information of each data shard node and stably mark it as a globally consistent snapshot by the asynchronous arrival of shard transaction logs, through the above steps. At the same time, by recording the mapping relationship and generating timestamps and snapshot metadata, it provides a snapshot basis that can be referenced repeatedly for subsequent query paths, thereby reducing the impact of cross-shard version misalignment on the determination of consistent snapshots and improving the traceability and stability of snapshot references.

[0080] In one embodiment, step S70 above includes: S701, parse the data query request, and determine the set of data object identifiers to be accessed by the data query request and the corresponding data access conditions; S702, based on the list of active transactions corresponding to the global consistency snapshot, obtain multiple historical version data of each data object in the data object identifier set from the destination database; S703, for multiple historical versions of data for each data object, determine the visibility of each data version to the current query based on the transaction visibility strategy defined in the active transaction list corresponding to the global consistency snapshot; S704, Based on the visibility judgment result, filter out the data version that is visible to the data query request from multiple historical version data of each data object; S705, Assemble the visible data versions of all data objects to form a complete query result dataset corresponding to the data query request; S706, the complete query result dataset is returned to the initiator of the data query request.

[0081] In this embodiment, when parsing a data query request and determining the data object identifier set and data access conditions, the data query request can carry query statements, parameterized predicates, target tables, or target key ranges. The parsing action requires breaking down the data query request into executable access units. The data object identifier set is used to precisely point to the data object to be read. The data object can correspond to row keys, primary keys, document identifiers, graph node identifiers, or composite keys. The identifier set can be expressed as a list, range expression, or Bloom filter. The data access conditions are used to constrain the reading range and filtering logic. Conditions can include equality conditions, range conditions, time conditions, column projection conditions, sorting and pagination conditions, etc. In implementation, the data query request can be lexically and syntactically parsed to obtain an abstract syntax tree, and then the abstract syntax tree can be mapped to the data object identifier set and data access conditions. Alternatively, fields can be directly extracted from a structured data query request to generate the data object identifier set and data access conditions. To ensure the feasibility of retrieving multiple historical versions of data, access plan fragments can be generated synchronously during the parsing phase. For example, the data object identifier set can be divided into shard subsets according to sharding routing rules, and each shard subset can be bound to the same query context environment with the data access conditions. The query context environment is used to run through the subsequent retrieval, judgment, filtering, assembly and return.

[0082] When retrieving multiple historical versions of each data object from the data object identifier set in the destination database based on the active transaction list corresponding to the globally consistent snapshot, the destination database serves as the read-side carrier after data replication and persistence. The retrieval action is directed towards the persistent data version structure within the destination database. Multiple historical versions represent the version sequence formed by the same data object at different transaction write times. This version sequence can be implemented using version chains, timestamp indexes, commit sequence number indexes, or multi-version storage tables. The active transaction list corresponding to the globally consistent snapshot does not require pruning to a unique version during the retrieval phase, but it can be used to limit the range of candidate versions to be retrieved. For example, only versions with transaction numbers no greater than the upper bound of the committed transaction boundary information can be retrieved, or backtracking can be stopped when a version earlier than the committed transaction boundary information is encountered during version chain backtracking. In implementation, the version chain can be read sequentially according to the data object identifier set, or a range scan can be performed based on data access conditions, and the version chain can be pulled for each scanned record. Alternatively, a batch interface can be used to pull the version header pointers of multiple data objects at once and backtrack the version chain in parallel. To reduce the random read pressure on the target database, a layered caching and pre-read strategy can be adopted to obtain multiple historical versions of data. For example, the version header information of the most recently accessed data object can be cached in the query context, or the version metadata can be read in blocks for consecutive key ranges and then the version content can be read as needed.

[0083] When determining the visibility of each data version to the current query based on the transaction visibility strategy defined in the active transaction list for multiple historical versions of each data object, the transaction visibility strategy is used to convert the transaction status information in the active transaction list into executable judgment rules. The transaction visibility strategy can include rules for committed transaction boundary information, rules for uncommitted transaction identifiers, rules for the scope of transactions to be allocated, and rules related to deletion markers, rollback markers, and read-write operations within the same transaction. The judgment action requires extracting version metadata for each data version and matching it with the active transaction list. Version metadata can include write transaction number, commit timestamp, deletion marker, version sequence number, etc. In implementation, the active transaction list can be preprocessed into a fast judgment structure; for example, uncommitted transaction identifiers can be constructed as hash sets, committed transaction boundary information can be converted into upper and lower bound comparators, and the scope of transactions to be allocated can be converted into threshold comparators, thus completing the inclusion judgment in constant time when judging each data version. Visibility determination results can be expressed as Boolean flags, tri-state flags, or priority flags. Tri-state flags are used to distinguish between visible, invisible, and states that require backtracking to the next version. Priority flags are used to select the most appropriate version when multiple visible versions exist.

[0084] When filtering data versions visible to a data query request from multiple historical versions based on visibility judgment results, the filtering action needs to bind the visibility judgment results generated in the judgment phase with the version sequence to form a set of visible data versions. The filtering strategy can aim to "select one visible data version for each data object" or "retain all visible data versions for each data object," depending on the semantics of the data query request. For snapshot-consistent reads, it tends to select one visible data version that satisfies the snapshot semantics for each data object. In implementation, it can traverse the version sequence from newest to oldest, terminating the traversal when the first visible version is encountered. Alternatively, it can first filter out all visible versions and then perform secondary filtering based on data access conditions, such as column projection, predicate conditions, or sorting rules for further pruning. To ensure the filtering results are compatible with subsequent assembly actions, the visible data version corresponding to each data object can be encapsulated into a row structure or key-value structure during the filtering phase, and the correspondence between the data object identifier and the visible data version can be recorded in the query context, facilitating the subsequent merging of data returned from different shards.

[0085] When assembling the visible data versions of all data objects to form the complete query result dataset corresponding to the data query request, the assembly process needs to unify the scattered visible data versions into a returnable structured result. The complete query result dataset can contain three parts: a set of data rows, a set of columns, and a set of metadata. The metadata set can include information such as the number of returned entries, pagination cursor, sorting status, and scan statistics. The assembly process needs to follow the projection, filtering, sorting, and pagination requirements carried in the data access conditions. Projection is used to determine the set of returned fields, filtering is used to remove records that do not meet the conditions, sorting is used to determine the return order, and pagination is used to truncate the return window. In implementation, it can be assembled first according to the original order of the data object identifier set, and then merge sort is performed according to the sorting conditions. Alternatively, it can be partially sorted by shards during the filtering stage and multi-way merge is performed during the assembly stage. To reduce memory pressure during the assembly stage, the complete query result dataset can adopt a streaming output structure, that is, assembling and outputting simultaneously, while maintaining the output progress and pagination cursor in the query context. The pagination cursor can be formed by combining the sort key of the last record and the data object identifier.

[0086] When returning the complete query result dataset to the initiator of the data query request, the initiator can be an application service, gateway component, or interactive client. The return action needs to encode the complete query result dataset into a protocol format acceptable to the initiator. In implementation, the complete query result dataset can be serialized into a binary protocol, a matrix block, a JSON structure, or a custom result frame, and the return process should include the request identifier, validation information, and pagination cursor corresponding to the data query request. To maintain consistency with preceding processing, the return phase can return the unique identifier of the globally consistent snapshot or the timestamp of the globally consistent snapshot as part of the result metadata, enabling the initiator to associate the globally consistent snapshot used in this read for subsequent tracing or retry control.

[0087] This embodiment enables the destination database to make consistent visibility decisions on the data versions of different data objects based on the same active transaction list, even in an environment where the states of multiple sharded transactions arrive asynchronously. This reduces the impact of cross-shard version misalignment on the consistency of query results and reduces the additional processing overhead caused by invisible data entering the result assembly stage.

[0088] In one embodiment, a query apparatus based on a globally consistent snapshot is provided, which corresponds one-to-one with the query method based on a globally consistent snapshot in the above embodiments. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the query device based on a globally consistent snapshot of the present invention. The modules include: an active transaction list generation module 10, an active transaction list distribution module 20, a transaction log writing module 30, an independent log stream transmission module 40, a version information extraction module 50, a globally consistent snapshot determination module 60, and a visibility judgment query module 70. Detailed descriptions of each functional module are as follows: The active transaction list generation module 10 is used to generate an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated in the transaction management node of the source database of the distributed database, and to allocate corresponding snapshot version information to the active transaction list; The active transaction list distribution module 20 is used to distribute the active transaction list carrying the snapshot version information from the transaction management node to each data shard node of the source database; The transaction log writing module 30 is used to write the list of active transactions into the transaction log of the transaction management node and the transaction log of each data shard node according to a predetermined period based on the snapshot version information corresponding to the same moment. The independent log stream transmission module 40 is used to asynchronously transmit the transaction logs of the transaction management node and the transaction logs of each data shard node to the destination database through an independent log stream. The version information extraction module 50 is used to parse the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database when the destination database receives a data query request, and extract the version information of the active transaction list of the transaction management node and the version information of the active transaction list of each data shard node. The global consistency snapshot determination module 60 is used to determine the common active transaction list version information of each data shard node as a global consistency snapshot based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. The visibility determination query module 70 is used to determine the visibility of the data to be accessed by the data query request based on the list of active transactions corresponding to the global consistency snapshot and return the query results.

[0089] For specific limitations regarding the query apparatus based on globally consistent snapshots, please refer to the aforementioned limitations on the query method based on globally consistent snapshots, which will not be repeated here. Each module in the aforementioned query apparatus based on globally consistent snapshots can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0090] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements server-side functions or steps of a query method based on globally consistent snapshots.

[0091] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of a query method based on globally consistent snapshots.

[0092] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: In the transaction management node of the source database of the distributed database, an active transaction list is generated, which includes committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated, and corresponding snapshot version information is assigned to the active transaction list. The list of active transactions carrying the snapshot version information is distributed from the transaction management node to each data shard node of the source database; The list of active transactions is written into the transaction log of the transaction management node and the transaction log of each data shard node at predetermined intervals based on the snapshot version information corresponding to the same moment. The transaction logs of the transaction management node and the transaction logs of each data shard node are asynchronously transmitted to the destination database through independent log streams; When the destination database receives a data query request, it parses the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. Based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the common active transaction list version information of each data shard node is determined as a globally consistent snapshot. Based on the list of active transactions corresponding to the globally consistent snapshot, the visibility of the data to be accessed by the data query request is determined and the query result is returned.

[0093] In one embodiment, a computer-readable storage medium is provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, performs the following steps: In the transaction management node of the source database of the distributed database, an active transaction list is generated, which includes committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated, and corresponding snapshot version information is assigned to the active transaction list. The list of active transactions carrying the snapshot version information is distributed from the transaction management node to each data shard node of the source database; The list of active transactions is written into the transaction log of the transaction management node and the transaction log of each data shard node at predetermined intervals based on the snapshot version information corresponding to the same moment. The transaction logs of the transaction management node and the transaction logs of each data shard node are asynchronously transmitted to the destination database through independent log streams; When the destination database receives a data query request, it parses the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. Based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the common active transaction list version information of each data shard node is determined as a globally consistent snapshot. Based on the list of active transactions corresponding to the globally consistent snapshot, the visibility of the data to be accessed by the data query request is determined and the query result is returned.

[0094] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0095] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A query method based on globally consistent snapshots, characterized in that, Includes the following steps: In the transaction management node of the source database of the distributed database, an active transaction list is generated, which includes committed transaction boundary information, uncommitted transaction identifiers and transaction ranges to be allocated, and corresponding snapshot version information is assigned to the active transaction list. The list of active transactions carrying the snapshot version information is distributed from the transaction management node to each data shard node of the source database; The list of active transactions is written into the transaction log of the transaction management node and the transaction log of each data shard node at predetermined intervals based on the snapshot version information corresponding to the same moment. The transaction logs of the transaction management node and the transaction logs of each data shard node are asynchronously transmitted to the destination database through independent log streams; When the destination database receives a data query request, it parses the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. Based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the common active transaction list version information of each data shard node is determined as a globally consistent snapshot. Based on the list of active transactions corresponding to the globally consistent snapshot, the visibility of the data to be accessed by the data query request is determined and the query result is returned.

2. The query method based on globally consistent snapshots as described in claim 1, characterized in that, The transaction management node of the source database in the distributed database generates an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges, and assigns corresponding snapshot version information to the active transaction list, including: A mechanism for maintaining an active transaction list is established through the transaction management node of the source database in the distributed database. This mechanism is used to manage the boundary information of committed transactions, the identifiers of uncommitted transactions, and the scope of transactions to be allocated. The transaction management node monitors the transaction start event, updates the range of transactions to be allocated, and adds the newly allocated transaction identifier to the uncommitted transaction identifier. The transaction management node responds to transaction commit events, updating the boundary information of committed transactions and the identifiers of uncommitted transactions; When the predetermined period arrives, the transaction management node obtains the current committed transaction boundary information, uncommitted transaction identifiers, and pending transaction ranges to form an active transaction list. The transaction management node generates snapshot version information based on the current system state and a predefined version generation strategy. The snapshot version information is written as metadata into the header area of ​​the active transaction list, and the correspondence between the snapshot version information and the active transaction list is recorded in the distributed metadata service.

3. The query method based on globally consistent snapshots as described in claim 1, characterized in that, Distributing the list of active transactions carrying the snapshot version information from the transaction management node to each data shard node of the source database includes: The transaction management node serializes the list of active transactions carrying snapshot version information into a distributed message format to generate a message containing the list of active transactions carrying snapshot version information. The transaction management node sends the list of active transactions carrying snapshot version information to each data shard node of the source database through a message broadcasting mechanism. Each data shard node receives the active transaction list message carrying snapshot version information and parses it to obtain the active transaction list carrying snapshot version information. Each data sharding node sends a message reception confirmation signal to the transaction management node; The integrity of the distribution operation is verified by the transaction management node based on the message reception confirmation signals of all data shard nodes.

4. The query method based on globally consistent snapshots as described in claim 1, characterized in that, The list of active transactions is written to the transaction log of the transaction management node and the transaction log of each data shard node at predetermined intervals based on the snapshot version information corresponding to the same moment, including: When the predetermined period arrives, the same time reference is obtained based on the global time synchronization mechanism; Based on the snapshot version information corresponding to the same time reference, determine the list of active transactions associated with the snapshot version information; Write the list of active transactions and the snapshot version information corresponding to the same time reference into the transaction log of the transaction management node; Based on the snapshot version information corresponding to the same time reference, control each data shard node to write the list of active transactions into the transaction log of the corresponding data shard node.

5. The query method based on globally consistent snapshots as described in claim 1, characterized in that, When the destination database receives a data query request, it parses the transaction logs of the transaction management node and each data shard node that have been received in the destination database, and extracts the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, including: The system receives data query requests from the destination database and parses the consistency level requirements and query scope information in the data query requests. The system scans the transaction logs of the received transaction management node in the destination database to obtain the portion related to the query scope information, and then parses the active transaction list version information of the transaction management node from the scanned portion. Based on the consistency level requirements, a parallel scanning strategy is determined. The target database scans the part of the transaction log of each data shard node that has been received that is related to the query range information according to the parallel scanning strategy, and the active transaction list version information of each data shard node is parsed from the scanned part. Based on the consistency level requirements, a verification standard is determined. The timestamp consistency and integrity of the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node are verified through the destination database according to the verification standard. The active transaction list version information of the transaction management node that has passed the timestamp consistency and integrity verification, as well as the active transaction list version information of each data shard node, are cached in the query context environment through the destination database.

6. The query method based on globally consistent snapshots as described in claim 1, characterized in that, Based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node, the common active transaction list version information of each data shard node is determined as a globally consistent snapshot, including: Collect the version information of the active transaction list of the transaction management node and the version information of the active transaction list of each data shard node to form a complete set of version information; The timestamp reference range is determined based on the version information of the active transaction list of the transaction management node in the complete version information set; By comparing the active transaction list version information of each data shard node in the complete version information set with the timestamp sequence within the timestamp reference range, the active transaction list version information corresponding to the largest common timestamp contained in each data shard node is identified. Verify the existence and consistency of the active transaction list version information corresponding to the maximum common timestamp in each data shard node, and use the active transaction list version information that passes the existence and consistency verification as the common active transaction list version information of each data shard node. The version information of the active transaction list shared by each data shard node is marked as a globally consistent snapshot, and a unique identifier for the globally consistent snapshot is generated. Record the mapping relationship between the global consistency snapshot and the version information of the common active transaction list of each data shard node, and store the generation timestamp and snapshot metadata of the global consistency snapshot.

7. The query method based on globally consistent snapshots as described in claim 1, characterized in that, Based on the list of active transactions corresponding to the globally consistent snapshot, the visibility of the data to be accessed by the data query request is determined and the query results are returned, including: Parse the data query request to determine the set of data object identifiers to be accessed by the data query request and the corresponding data access conditions; Based on the list of active transactions corresponding to the global consistency snapshot, obtain multiple historical version data of each data object in the data object identifier set from the destination database; For multiple historical versions of data for each data object, the visibility of each data version to the current query is determined according to the transaction visibility strategy defined in the active transaction list corresponding to the global consistency snapshot; Based on the visibility determination results, the data versions visible to the data query request are selected from multiple historical versions of each data object. Assemble the visible data versions of all data objects to form a complete query result dataset corresponding to the data query request; The complete query result dataset is returned to the originating end of the data query request.

8. A query device based on globally consistent snapshots, characterized in that, The query device based on globally consistent snapshots includes: The active transaction list generation module is used to generate an active transaction list containing committed transaction boundary information, uncommitted transaction identifiers, and transaction ranges to be allocated in the transaction management node of the source database of the distributed database, and to allocate corresponding snapshot version information to the active transaction list; An active transaction list distribution module is used to distribute the active transaction list carrying the snapshot version information from the transaction management node to each data shard node of the source database; The transaction log writing module is used to write the list of active transactions into the transaction log of the transaction management node and the transaction log of each data shard node according to a predetermined period based on the snapshot version information corresponding to the same moment. An independent log stream transmission module is used to asynchronously transmit the transaction logs of the transaction management node and the transaction logs of each data shard node to the destination database via an independent log stream. The version information extraction module is used to parse the transaction logs of the transaction management node and the transaction logs of each data shard node that have been received in the destination database when the destination database receives a data query request, and extract the version information of the active transaction list of the transaction management node and the version information of the active transaction list of each data shard node. The global consistency snapshot determination module is used to determine the common active transaction list version information of each data shard node as a global consistency snapshot based on the active transaction list version information of the transaction management node and the active transaction list version information of each data shard node. The visibility determination query module is used to determine the visibility of the data to be accessed by the data query request based on the list of active transactions corresponding to the global consistency snapshot and return the query results.

9. A computer device, characterized in that, The computer device includes a memory, a processor, and a query program based on a globally consistent snapshot stored in the memory and executable on the processor, wherein the query program based on a globally consistent snapshot, when executed by the processor, implements the steps of the query method based on a globally consistent snapshot as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a query program based on a globally consistent snapshot, which, when executed by a processor, implements the steps of the query method based on a globally consistent snapshot as described in any one of claims 1-7.

Images