Concurrency control method for cross-model database transaction processing

Abstract

The invention provides a concurrency control method for cross-model database transaction processing, which relates to the technical field of database transaction processing and comprises the following steps: analyzing a multi-model transaction currently sent by a user, and distributing each database operation statement in the multi-model transaction to a database of a corresponding data model; generating a standardized library-level logic object access set based on the global logic object mapping table; summarizing the standardized library-level logic object access sets from different databases to generate a transaction-level logic object access set of the multi-model transaction; and performing comparison and conflict judgment on the transaction-level logic object access set of the multi-model transaction currently sent by the user and the transaction-level logic object access sets of all concurrent multi-model transactions, making a decision for the final state of the multi-model transaction, and executing corresponding data persistence operation.

Description

Technical Field

[0001] This invention belongs to the field of database transaction processing technology, and specifically relates to a concurrency control method for cross-model database transaction processing. Background Technology

[0002] As business system data models become increasingly diverse, modern application architectures are shifting from single-data storage to multi-model databases or a hybrid approach of multiple single-model databases. This aims to fully leverage the unique advantages of different data models in handling specific business scenarios. While this "multi-model" or "hybrid persistence" architecture enhances system design flexibility and functional adaptability, it also significantly increases the complexity of ensuring data consistency. This is particularly evident in ensuring that multiple operations across different data models (such as relational, key-value, and graph databases) within a single business transaction unit still strictly meet the atomicity, consistency, and durability requirements of the ACID properties of database transactions.

[0003] In such heterogeneous environments, concurrency control becomes a core challenge. When multiple transactions execute concurrently and attempt to access the same data, effectively detecting and handling cross-model read / write conflicts is crucial for maintaining data consistency. In a monolithic, homogeneous database system, mature concurrency control mechanisms (such as locking, multi-version concurrency control, or timestamp sorting) can operate effectively based on a unified data model and operational semantics. However, due to fundamental differences in data organization, access interfaces, and transaction support capabilities among different data models, the lack of a unified coordination layer to globally perceive and adjudicate conflicts between these cross-model operations makes traditional single-model solutions difficult to apply directly.

[0004] Existing technical solutions attempt to address this problem from different paths, but all have significant limitations. While distributed transactions based on the XA protocol can achieve strong consistency among databases supporting the protocol through two-phase commit, its synchronous blocking and long-term resource locking mechanisms severely limit system throughput and are unsuitable for most NoSQL databases that do not natively support XA. Solutions such as Saga, which employ an eventual consistency model, avoid distributed locks through post-event compensation mechanisms, but their programming models are complex, requiring developers to explicitly define and maintain the inverse operations of all business operations, increasing development burden and operational risks. Furthermore, their inherent transient inconsistencies cannot meet the stringent requirements of core business scenarios demanding strong consistency. In addition, while using a single transactional database that internally supports multiple models can simplify transaction management, this approach easily leads to vendor lock-in risks, and its performance on different data models may not be comparable to dedicated "best practice" databases.

[0005] In summary, existing technologies lack a universal solution that can effectively shield the heterogeneity of underlying data models, perform cross-model conflict detection and coordination in a unified manner, and provide strong consistency guarantees without significantly sacrificing system performance. This technological gap exposes developers to significant design complexity and reliability risks at the data consistency level when building modern hybrid persistent applications, necessitating a novel concurrency control method to overcome this bottleneck. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a concurrency control method for cross-model database transaction processing. This invention introduces a global logical object layer independent of the underlying physical storage. By establishing a mapping relationship between business logic objects and multi-model physical data, and performing concurrency control based on logical object versions, it achieves atomicity and consistency of cross-model transactions at the business level.

[0007] In a first aspect, the present invention provides a concurrency control method for cross-model database transaction processing, comprising the following steps:

[0008] Initialize the global logical object mapping table based on the multi-model transactions executed during the deployment phase;

[0009] A unified query agent is used to receive and parse the multi-model transaction currently sent by the user. The multi-model transaction includes several database operation statements. According to the data model type of the database operation statements, each database operation statement in the multi-model transaction is distributed to the database of the corresponding data model. Databases of different model types receive the corresponding database operation statements and generate a standardized library-level logical object access set based on the global logical object mapping table.

[0010] The transaction coordinator is used to aggregate standardized library-level logical object access sets from different databases to generate transaction-level logical object access sets for multi-model transactions.

[0011] The transaction coordinator compares the transaction-level logical object access set of the user's current multi-model transaction with the transaction-level logical object access set of all concurrent multi-model transactions, and performs conflict resolution based on different concurrency control strategies to obtain conflict detection results.

[0012] Based on the conflict detection results, the transaction coordinator makes a decision on the final state of the multi-model transaction currently sent by the user and performs the corresponding data persistence operations.

[0013] Furthermore, the multi-model transactions executed during the deployment phase are a series of multi-model transactions executed during the deployment and business initialization phases, used to create various business entities in the business;

[0014] The business entity refers to a conceptual object with clear semantics and lifecycle in the business. The global logical object mapping table is used to record the mapping relationship between physical data items and logical objects, and to maintain the version information of logical objects.

[0015] The logical object refers to a unified entity obtained by abstracting one or more physical data items at the business semantic level. The logical object has a unique logical object identifier, which is located in a unified namespace and is globally unique.

[0016] The mapping relationships include one-to-one mapping, one-to-many mapping, and many-to-one mapping;

[0017] The one-to-one mapping refers to one physical data item corresponding to one logical object;

[0018] The one-to-many mapping refers to the storage of multiple attributes of the same logical object in different models or different physical data items;

[0019] The many-to-one mapping refers to the complete state of a logical object formed by multiple physical data items that may cross models.

[0020] The physical data item refers to the smallest storage unit that is directly read and written in the database storage engine, including at least: keys and their corresponding key-value records in the key-value database, data rows and column fields within rows in the relational database, and physical records such as vertices, edges, and attributes of vertices or edges in the graph database.

[0021] When executing multi-model transactions during the deployment phase, the global logical object mapping table is initialized. When the multi-model transactions executed during the deployment phase are committed, logical object identifiers are assigned to the business entities created by the multi-model transactions executed during the deployment phase, and the logical object identifiers and the physical data items associated with the business entities are written into the global logical object mapping table.

[0022] Furthermore, the specific method for generating a standardized library-level logical object access set based on the global logical object mapping table is as follows:

[0023] Different model types of databases use built-in query parsers to perform lexical analysis, syntax analysis, and semantic analysis on the received database operation statements to generate physical execution plans. The physical execution plans include the physical data items that need to be accessed to execute the database operation statements.

[0024] Databases of different model types execute their respective physical execution plans in parallel before the current multi-model transaction is committed by the user. Each database includes a lightweight data access collector, which captures physical data item access records, including native identifiers, operation types, and data values, during the execution of the physical execution plan.

[0025] The specific method for capturing physical data item access records is as follows:

[0026] For key-value databases, capture Get, Put, and Delete operations, and record the key, value, and operation type of the captured operation;

[0027] For relational databases, capture read and write operations on data rows. Read operations include index scans and full table scans, while write operations include insert, update, and delete operations.

[0028] For graph databases, capture operations such as traversal of vertices or edges, attribute reading, and modification.

[0029] The data access collector converts captured physical data item access records into logical object access records based on the global logical object mapping table. Multiple logical object access records are then combined to form a standardized library-level logical object access set. The specific method is as follows:

[0030] The global logical object mapping table is queried based on the physical data item access record to determine the logical object to which the physical data item access record belongs and the current version information of the logical object. The current version information is one of version number, version vector or hybrid logical timestamp.

[0031] Based on the mapping rules, the operation type and data value in the physical data item access record are mapped to logical operation type and logical operation value, respectively; the mapping rules are: operations that do not change the state of the logical object are mapped to logical read operation READ, and operations that change the state of the logical object are mapped to logical write operation WRITE.

[0032] Generate standardized logical object access records in the following format: <Logical Object Identifier, Logical Operation Type, Logical Operation Value, Current Version Information>. The logical object identifier is unique within the global namespace, and its naming convention is: domain: / / <business domain> / <entity type> / <entity ID>. The logical operation type includes READ and WRITE operations. READ records a snapshot of the data read; WRITE records the data to be written or incremental modifications; and the current version information records the globally consistent version of the logical object at the time the logical operation is executed.

[0033] Furthermore, the specific method for conflict resolution is as follows:

[0034] The transaction coordinator, for each logical object X involved in the transaction-level logical object access set, makes a decision based on at least one or more of the following conflict criteria, which include:

[0035] If the version information of the logical object X read by the user's current multi-model transaction T1 is inconsistent with the latest version information of the logical object X, it is determined that a version verification conflict has been detected.

[0036] If the logical object X that the user intends to write in the current multi-model transaction T1 has already been written by another uncommitted concurrent multi-model transaction T2, then it is determined that a logical write-write conflict has been detected.

[0037] If the logical object X that the user intends to write in the current multi-model transaction T1 has been read by another uncommitted concurrent multi-model transaction T2, it is determined that a logical write-read conflict has been detected.

[0038] If a logical object X that has been read by the user's current multi-model transaction T1 has been modified by another uncommitted concurrent transaction request T2, then a logical read-write conflict is detected.

[0039] Furthermore, conflict resolution is performed based on different concurrency control strategies, including:

[0040] When adopting a pessimistic control strategy based on logical object locks, a logical object-level read lock or write lock is requested when accessing a logical object. During the lock holding period, incompatible accesses to the same logical object by other concurrent transactions are prevented, thus preventing conflicts from occurring.

[0041] When adopting an optimistic concurrency control strategy based on version verification, version consistency verification is achieved by comparing the version information of the logical objects accessed in the transaction-level logical object access center of the multi-model transaction currently sent by the user with the latest committed version information of the logical objects in the transaction coordinator.

[0042] When adopting a multi-version concurrency control strategy, equivalent conflict avoidance is achieved by maintaining multiple version information for logical objects and comparing the version information corresponding to the transaction-level logical object access set.

[0043] When using a timestamp sorting strategy, a globally unique timestamp is assigned to the multi-model transaction at the start of the transaction, and the serializability of the scheduling result is guaranteed by verifying the timestamp order of each logical read operation (READ) and logical write operation (WRITE) in the transaction-level logical object access set.

[0044] Furthermore, based on the conflict detection results, the transaction coordinator makes a decision on the final state of the multi-model transaction. The specific method is as follows:

[0045] If no conflict is detected, the transaction coordinator decides to commit the multi-model transaction currently sent by the user, and the transaction coordinator performs the persistence operation.

[0046] If a conflict is detected, the transaction coordinator decides to abort the multi-model transaction currently being sent by the user. The transaction coordinator returns a failure message to the user and sends an abort notification to all databases.

[0047] The transaction coordinator generates and returns retryable information to the user, instructing the user to retry the current multi-model transaction. When the user initiates a retry, each database generates a retry standardized library-level logical object access set based on the cached library-level logical object access set during the retry process, and sends the retry standardized library-level logical object access set to the transaction coordinator. The transaction coordinator summarizes the retry standardized library-level logical object access sets from different databases, generates a retry transaction-level logical object access set, and re-executes the conflict resolution: if no conflict with concurrent transactions is detected, the transaction coordinator submits the retry of the multi-model transaction and generates a redo log, which is sent to each database for persistence; if a concurrent conflict is detected, the transaction coordinator aborts the current retry attempt and returns the retryable information again, or prompts the user to retry later.

[0048] Upon receiving an abort notification, each database cleans up the temporary write results and temporary states generated during the execution of multi-model transactions, while retaining retry information; when it is detected that a user has given up retrying or the preset retry validity period has expired, the retry information is cleaned up.

[0049] Furthermore, the transaction coordinator performs persistence operations including:

[0050] The transaction coordinator generates new version information for all logical objects to be updated in the multi-model transaction currently sent by the user. The generation of version information follows a strict monotonically increasing rule.

[0051] The transaction coordinator expands the logical object access records of the current multi-model transaction's transaction-level logical object access set, where the logical operation type is logical write (WRITE), into a set of write operations for physical data items, generating a set of redo logs. The redo logs include the target physical data item identifier, the physical write operation type, the data value to be written, and the new version information. Sequence numbers are assigned to redo logs sent to different database instances, and these sequence numbers are unique and monotonically increasing within the corresponding database instance.

[0052] The transaction coordinator distributes the redo logs to the corresponding databases. After receiving the redo logs, the databases complete the write-to-disk process and return an acknowledgment result.

[0053] After all databases confirm successful persistence, the transaction coordinator notifies each database to clean up all write results and temporary states temporarily stored during the execution of this multi-model transaction; writes the new version information to and updates the current version information of the corresponding logical object in the global logical object mapping table; and returns a successful transaction commit response to the user.

[0054] Secondly, this application proposes an electronic device comprising: one or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the concurrency control method for cross-model database transaction processing.

[0055] Thirdly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the concurrency control method for cross-model database transaction processing.

[0056] Fourthly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned concurrency control method for cross-model database transaction processing.

[0057] The beneficial effects of adopting the above technical solution are as follows: The concurrency control method for cross-model database transaction processing provided by this invention establishes and maintains a global logical object mapping table independent of the underlying physical storage, and abstracts access across heterogeneous data models such as relational, key-value, and graph data into a standardized library-level logical object access set. The transaction coordinator then aggregates the library-level logical object access sets generated by each database to form a transaction-level logical object access set. Subsequently, the transaction coordinator performs centralized conflict resolution around concurrency control metadata such as logical object version / lock / timestamp, thereby providing configurable isolation semantics and consistency guarantees for cross-model transactions without relying on a single vendor's multi-model database. At the same time, the global logical object mapping table makes conflict detection and commit protocols reusable and scalable across different data models, significantly reducing the consistency design complexity and engineering implementation cost of hybrid persistent applications, and improving the reliability and maintainability of the system while taking performance into account. Attached Figure Description

[0058] Figure 1 A flowchart of the concurrency control method for cross-model database transaction processing provided in Embodiment 1 of the present invention;

[0059] Figure 2 A schematic diagram of the logical object access set generation process provided in Embodiment 1 of the present invention;

[0060] Figure 3 A schematic diagram illustrating the principle of conflict detection and successful execution of multi-model transactions provided in Embodiment 1 of the present invention;

[0061] Figure 4 The schematic diagram of the principle of multi-model transactions performing conflict detection and aborting due to conflict provided in Embodiment 1 of the present invention. Detailed Implementation

[0062] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0063] Example 1:

[0064] This embodiment, using a specific cross-model transaction example, details the implementation process of a concurrency control method for cross-model database transaction processing provided in this embodiment. This embodiment is based on an e-commerce purchase scenario, where multi-model transactions involve updating user balances (key-value model), deducting product inventory (relational model), and updating user purchase counts (graph model).

[0065] In light of the e-commerce purchase scenario described above, this embodiment presents a concurrency control method for cross-model database transaction processing, such as... Figure 1 As shown, it includes the following steps:

[0066] Step 1: Initialize the global logical object mapping table based on the multi-model transactions executed during the deployment phase;

[0067] The multi-model transactions executed during the deployment phase are a series of multi-model transactions executed during the deployment and business initialization phases, used to create various business entities in the business;

[0068] The business entity refers to a conceptual object with clear semantics and lifecycle in the business. In this embodiment, e-commerce business is involved, and the business entities include users, products and orders.

[0069] The global logical object mapping table is used to record the mapping relationship between physical data items and logical objects, and to maintain the version information of logical objects;

[0070] The logical object refers to a unified entity obtained by abstracting one or more physical data items at the business semantic level. The logical object has a unique logical object identifier, which is used to represent the complete state information associated with the business entity. The logical object identifier is in a unified namespace and is globally unique, and is used to consistently refer to the same business entity involved in different data model databases.

[0071] The core function of the global logical object mapping table is to map the underlying heterogeneous physical data storage upwards into a unified data view oriented towards business logic. The mapping relationship includes one-to-one mapping, one-to-many mapping, and many-to-one mapping.

[0072] The one-to-one mapping refers to one physical data item corresponding to one logical object;

[0073] The one-to-many mapping refers to the storage of multiple attributes of the same logical object in different models or different physical data items;

[0074] The many-to-one mapping refers to the complete state of a logical object formed by multiple physical data items that may cross models.

[0075] The physical data item refers to the smallest storage unit that is directly read and written in the database storage engine, including at least: keys and their corresponding key-value records in the key-value database, data rows and column fields within rows in the relational database, and physical records such as vertices, edges, and attributes of vertices or edges in the graph database.

[0076] When executing multi-model transactions during the deployment phase, the global logical object mapping table is initialized. Upon commit of the multi-model transactions executed during the deployment phase, logical object identifiers are assigned to the business entities created by the multi-model transactions, and the logical object identifiers and the physical data items associated with the business entities are written to the global logical object mapping table. In this embodiment, an e-commerce purchase scenario is created, and the globally logical object mapping table constructed and maintained is as follows: Figure 2 As shown in the figure, the logical data items are standardized intermediate representations introduced to achieve a unified mapping from logical objects to physical data items of different models.

[0077] During subsequent business operations, when users send multi-model transactions to insert, update, or delete business records, the global logical object mapping table is incrementally updated. When multi-model transactions create new business entities during runtime, logical object identifiers are also assigned to them and the global logical object mapping table is updated, thereby ensuring that the global logical object mapping table always reflects the latest business data distribution status.

[0078] Step 2: Receive and parse the multi-model transaction currently sent by the user using a unified query agent. The multi-model transaction includes several database operation statements. Based on the data model type of the database operation statements, distribute each database operation statement in the multi-model transaction to the database of the corresponding data model.

[0079] In this embodiment, the unified query agent receives the multi-model transaction currently sent by the user and assigns a globally unique transaction ID to the multi-model transaction: TxID-20240530-001. The multi-model transaction includes three database operation statements, namely:

[0080] Database operation statement 1: PUT user_balance_789 150.50 / / , means updating the balance of user ID 789 to 150.50; the data model type of database operation statement 1 is a key-value model operation, and database operation statement 1 is dispatched to the LevelDB database:

[0081] Database operation statement 2: UPDATE inventory SET stock = stock-1 WHERE item_id = 'item_abc' / / indicates that the inventory of product 'item_abc' will be reduced by 1; the data model type of database operation statement 2 is relational model operation, and database operation statement 2 is dispatched to the openGauss database:

[0082] Database operation statement 3: UPDATE VERTEX 'user_789' SET user.purchase_count += 1 / / This means incrementing the purchase count attribute of user vertex 'user_789' by 1; the data model type of database operation statement 3 is graph model operation, and database operation statement 3 is dispatched to the Nebula Graph database:

[0083] Step 3: Databases of different model types receive corresponding database operation statements and generate a standardized library-level logical object access set based on the global logical object mapping table;

[0084] Step 3.1: Databases of different model types use their built-in query parser to perform lexical analysis, syntax analysis, and semantic analysis on the received database operation statements to generate a physical execution plan. The physical execution plan includes the physical data items that need to be accessed to execute the database operation statements.

[0085] In this embodiment, databases of different model types include key-value databases, relational databases, and graph databases, such as... Figure 1 As shown, LevelDB database receives database operation statement 1, openGauss database receives database operation statement 2, and Nebula Graph database receives database operation statement 3.

[0086] For LevelDB databases, database operation statements (such as Get / Put / Delete) can be directly mapped to fixed data access paths without the need for complex parsing and optimization processes, thus eliminating the need to generate physical execution plans.

[0087] Step 3.2: Before the user's current multi-model transaction is committed, databases of different model types execute their respective physical execution plans in parallel. Each database includes a lightweight data access collector. During the execution of the physical execution plan, the data access collector captures physical data item access records, including native identifiers, operation types, and data values.

[0088] The data access collector is deployed between the database's execution engine and storage engine. When the database's execution engine initiates data read / write operations according to the physical execution plan, the data access collector accurately extracts physical data item access records from the input parameters and output results of the data read / write operations, such as... Figure 2The operation of the data access collector is shown below;

[0089] The specific method for capturing physical data item access records is as follows:

[0090] For key-value databases, capture Get, Put, and Delete operations, and record the key, value, and operation type of the captured operation;

[0091] For relational databases, capture read and write operations on data rows. Read operations include index scans and full table scans, while write operations include insert, update, and delete operations.

[0092] For graph databases, capture operations such as traversal of vertices or edges, attribute reading, and modification.

[0093] Step 3.3: The data access collector converts the captured physical data item access records into logical object access records based on the global logical object mapping table, and combines multiple logical object access records to form a standardized library-level logical object access set. The specific method is as follows:

[0094] The global logical object mapping table is queried based on the physical data item access record to determine the logical object to which the physical data item access record belongs and the current version information of the logical object. The current version information is one of version number, version vector or hybrid logical timestamp.

[0095] Based on the mapping rules, the operation type and data value in the physical data item access record are mapped to logical operation type and logical operation value, respectively; the mapping rules are: operations that do not change the state of the logical object are mapped to logical read operation READ, and operations that change the state of the logical object are mapped to logical write operation WRITE.

[0096] Generate standardized logical object access records in the following format: <Logical Object Identifier, Logical Operation Type, Logical Operation Value, Current Version Information>. The logical object identifier is unique within the global namespace, and its naming convention is: domain: / / <business domain> / <entity type> / <entity ID>. The logical operation type includes READ and WRITE operations. READ records a snapshot of the data read; WRITE records the data to be written or incremental modifications; and the current version information records the globally consistent version of the logical object at the time the logical operation is executed.

[0097] In this embodiment, the LevelDB database executes database operation statement 1: PUT user_balance_789150.50. The LevelDB database parses database operation statement 1 as a write operation (WRITE), locates the key user_balance_789, and prepares to write the data 150.50. The data access collector integrated into the LevelDB database intercepts the PUT operation call. The captured physical data item access record is: native identifier user_balance_789, logical operation type logical write operation (WRITE), operation value 150.50. The data access collector queries the global logical object mapping table, confirms that the key user_balance_789 is mapped to the logical object biz: / / ecommerce / User / 789, obtains the current version information (V5) from the logical object biz: / / ecommerce / User / 789, and generates a logical object access record. <biz: / / ecommerce / User / 789, WRITE, {"balance": 150.50}, V5>, and added to the standardized library-level logical object access set.

[0098] Executing database operation statement 2 in OpenGauss database: UPDATE inventory SET stock = stock - 1 When `WHERE item_id = 'item_abc'`, the openGauss database query parser performs lexical and syntactic analysis, and the optimizer generates an execution plan. For example, it locates the row of data with `item_id = 'item_abc'` through an index scan. The data access collector intercepts this read operation, specifically: the native identifier is `inventory.table::row[item_abc]`, the logical operation type is a logical read operation `READ`, and the operation value is `{stock: 50, ...}`. The execution engine calculates the new value and prepares to write it to `stock = 49`. The data access collector intercepted this write operation again. Specifically, the native identifier is inventory.table::row[item_abc], the logical operation type is logical write operation WRITE, and the operation value is {stock: 49, ...}. The data access collector queries the global logical object mapping table to determine that the physical data item inventory.table::row[item_abc] is mapped to the logical object biz: / / ecommerce / Inventory / item_abc. The data access collector obtains the current version information of this logical object as V12 and generates two logical object access records, respectively.<biz: / / ecomm-erce / Inventory / item_abc, READ, {stock:50},V12> and <biz: / / ecommerce / Inventory / item_abc, WR-ITE, {stock: 49}, V12>, and added to the standardized library-level logical object access set.

[0099] Executing database operation statement 3 in Nebula Graph database: UPDATE VERTEX 'user_789' SET user.purchase_count += 1: The Nebula Graph database query parser recognizes database operation statement 3 as a vertex update operation. The execution plan is as follows: locate vertex user_789, read its purchase_count attribute, calculate the new value, and write it back. The execution engine reads the current attribute `purchase_count: 10` from vertex `user_789`. The data access collector intercepts and records this: the native identifier is `vertex[user_789]`, the logical operation type is a logical read operation (READ), and the operation value is `{purchase_count: 10, ...}`. The execution engine then writes a new attribute value `purchase_count: 11`. The data access collector intercepts and records this: the native identifier is `vertex[user_789]`, the logical operation type is a logical write operation (WRITE), and the operation value is `{purchase_count: 11, ...}`. The data access collector queries the global logical object mapping table to determine that the physical data item `vertex[user_789]` is mapped to `biz: / / ecommerce / User / 789`. The data access collector obtains the current version information of this logical object as V5 and generates two logical object access records, respectively... <biz: / / ecommerce / User / 789, READ, {purchase_count: 10}, V5> and<biz: / / ecommerce / User / 789, WRITE,{purchase_count: 11}, V5> It is also added to the standardized library-level logical object access set.

[0100] Step 4: Use the transaction coordinator to aggregate the standardized library-level logical object access sets from different databases, and generate the transaction-level logical object access set of multi-model transactions, which fully represents the data access behavior of multi-model transaction requests at the business level.

[0101] When a user submits a multi-model transaction TxID-20240530-001, databases of different model types send all of their generated standardized library-level logical object access sets to a centralized transaction coordinator. It is important to note that this embodiment replaces the local transaction processing and commit mechanisms of each database with a centralized transaction coordinator. During the transaction execution phase, databases of different model types are only responsible for executing local operations; their internal data access collectors are responsible for generating the standardized library-level logical object access sets, while the database's own transaction manager remains in a waiting state.

[0102] In this embodiment, the transaction coordinator collects all standardized library-level logical object access sets from three databases and summarizes them to form the transaction-level logical object access set of multi-model transaction TxID-20240530-001. The behavior of multi-model transaction TxID-20240530-001 at the business level is shown in Table 1. It reads the purchase count of user 789 and the inventory of product abc, and intends to update the balance and purchase count of user 789, as well as the inventory of product abc.

[0103] Table 1. Business-level behavior of model transaction TxID-20240530-001

[0104]

[0105] Step 5: Use the transaction coordinator to compare the transaction-level logical object access set of the user's current multi-model transaction with the transaction-level logical object access set of all concurrent multi-model transactions, and make conflict resolutions according to different concurrency control strategies to obtain conflict detection results.

[0106] The specific method for conflict resolution is as follows:

[0107] The transaction coordinator, for each logical object X involved in the transaction-level logical object access set, makes a decision based on at least one or more of the following conflict criteria, which include:

[0108] If the version information of logical object X read by the user's current multi-model transaction T1 is inconsistent with the latest version information of logical object X, it is determined that a version verification conflict has been detected. This indicates that logical object X has been modified by other multi-model transactions during the execution of the current multi-model transaction T1. Version verification conflict helps to prevent non-repeatable reads and ensure data consistency.

[0109] If the logical object X that the user's current multi-model transaction T1 intends to write has already been written by another uncommitted concurrent multi-model transaction T2, it is determined that a logical write-write conflict has been detected. Logical write-write conflicts are the root cause of lost update problems. Prohibiting logical write-write conflicts helps to ensure atomicity and achieve Read Committed and higher isolation levels.

[0110] If the logical object X that the user's current multi-model transaction T1 intends to write has been read by another uncommitted concurrent multi-model transaction T2, it is determined that a logical write-read conflict has been detected. Detecting and prohibiting logical write-read conflicts can prevent non-repeatable reads and help achieve the repeatable read isolation level.

[0111] If a logical object X that has been read by the user's current multi-model transaction T1 has been modified by another uncommitted concurrent transaction request T2, it is determined that a logical read-write conflict has been detected. Detecting and prohibiting logical read-write conflicts can prevent dirty reads and help achieve the Read Committed isolation level.

[0112] Conflict resolution is performed based on different concurrency control strategies, unifying the concurrency control problem of multi-model transactions into: centralized conflict resolution on the transaction coordinator side, focusing on the concurrency control metadata of each logical object in the transaction-level logical object access set, including but not limited to version information, lock flags, timestamps, etc., including:

[0113] When adopting a pessimistic control strategy based on logical object locks, by requesting a logical object-level read lock or write lock when accessing a logical object, other concurrent transactions are prevented from making incompatible accesses to the same logical object during the lock holding period, thereby preventing conflicts from occurring at the source.

[0114] When adopting an optimistic concurrency control strategy based on version verification, version consistency verification is achieved by comparing the version information of the logical objects in the transaction-level logical object access center of the multi-model transaction currently sent by the user with the latest committed version information of the logical objects in the transaction coordinator.

[0115] When adopting a multi-version concurrency control strategy, equivalent conflict avoidance is achieved by maintaining multiple version information for logical objects and comparing the version information corresponding to the transaction-level logical object access set.

[0116] When using a timestamp sorting strategy, a globally unique timestamp is assigned to the multi-model transaction at the start of the transaction, and the serializability of the scheduling result is guaranteed by verifying the timestamp order of each logical read operation (READ) and logical write operation (WRITE) in the transaction-level logical object access set.

[0117] The core of the concurrency control mechanism is to use a unified logical object as an anchor point to standardize and aggregate the concurrency control information required for cross-model access to the transaction coordinator, thereby making consistent decisions on conflicts between multi-model transactions and providing consistent and reliable transaction isolation semantic guarantees in a multi-model hybrid environment.

[0118] In this embodiment, when the multi-model transaction TxID-20240530-001 is committed, there is another concurrently committed multi-model transaction TxID-20240530-000. The concurrently committed multi-model transaction TxID-20240530-000 successfully updated the version information of the logical object biz: / / ecommerce / User / 789 to V6.

[0119] The transaction coordinator begins conflict detection for multi-model transaction TxID-20240530-001. The coordinator examines each logical object in its transaction-level logical object access set. For example, it checks logical object biz: / / ecommerce / User / 789. The baseline version information for multi-model transaction TxID-20240530-001 is V5, but the latest committed version information for logical object biz: / / ecommerce / User / 789 is V6, indicating a version verification conflict. Figure 4 As shown.

[0120] According to the preset repeatable read isolation level rules, version verification conflicts directly cause the multi-model transaction TxID-20240530-001 to fail to commit, and the transaction coordinator does not need to check other conflict types.

[0121] Step 6: Based on the conflict detection results, use the transaction coordinator to make a decision on the final state of the multi-model transaction and execute the corresponding data persistence operations;

[0122] If no conflict is detected, the transaction coordinator decides to commit the multi-model transaction currently sent by the user, and performs the following persistence operations:

[0123] The transaction coordinator generates new version information for all logical objects to be updated in the multi-model transaction currently sent by the user. The version information generation follows a strict monotonically increasing rule and is implemented based on a global sequence number, a hybrid logical clock, or other distributed unique ID scheme to ensure the sortability of version information within the same logical object.

[0124] The transaction coordinator expands the logical object access records of the transaction-level logical object access set of the multi-model transaction currently sent by the user, which are logical write operations of type WRITE, into a set of write operations for physical data items, and generates a set of redo logs. Each redo log includes the target physical data item identifier, physical write operation type, data value to be written, and new version information. The redo logs sent to different databases are assigned a sequence number, which is unique and monotonically increasing within the corresponding database instance.

[0125] The transaction coordinator distributes the redo logs to the corresponding databases. After receiving the redo logs, the databases complete the write-to-disk process and return an acknowledgment result.

[0126] After all databases confirm successful persistence, the transaction coordinator notifies each database to clean up all write results and temporary states temporarily stored during the execution of this multi-model transaction, writes the new version information to and updates the current version information of the corresponding logical object in the global logical object mapping table, and returns a success response to the user.

[0127] If a conflict is detected, the transaction coordinator decides to abort the multi-model transaction currently being sent by the user. The transaction coordinator returns a failure message to the user and sends an abort notification to all databases.

[0128] The transaction coordinator generates and returns retryable information to the user, which is used to retry the current multi-model transaction. When the user initiates a retry, each database generates a retry standardized library-level logical object access set based on the cached library-level logical object access set, and sends the retry standardized library-level logical object access set to the transaction coordinator. The transaction coordinator summarizes the retry standardized library-level logical object access sets from different databases, generates a retry transaction-level logical object access set, and re-executes the conflict resolution: if no conflict with concurrent transactions is detected, the transaction coordinator submits the retry of the multi-model transaction and generates a redo log record, which is sent to each database for persistence; if a concurrent conflict is detected, the transaction coordinator aborts the current retry attempt and returns the retryable information again, or prompts the user to retry later.

[0129] Upon receiving an abort notification, each database cleans up temporary write results and temporary states generated during the execution of multi-model transactions, while retaining retry information. When it is detected that a user has abandoned retrying or the preset retry validity period has expired, the retry information is cleaned up.

[0130] In this embodiment, due to a conflict detected in step 5, the transaction coordinator resolves to abort the transaction request TxID-20240530-001. The coordinator returns the failure message "Transaction aborted due to concurrentmodification on logical object: biz: / / ecommerce / User / 789" to the client. Simultaneously, the transaction coordinator sends an abort notification to all database instances participating in the multi-model transaction execution. Subsequently, it generates and returns retryable information to the user. When the user initiates a retry, each database executor performs fast access to the relevant data based on the cached library-level logical object access set during the retry process, forming a new library-level logical object access set. This new set is similar to the one in step 3, except that the operation value and version information of the logical object biz: / / ecommerce / User / 789 are updated to the latest V6 version, and the new library-level logical object access set is sent to the transaction coordinator. After the transaction coordinator aggregates and generates the transaction-level logical object access set for retry transactions, it executes a conflict resolution process consistent with that of ordinary transactions. Since the logical object biz: / / ecommerce / User / 789 that was initially committed in conflict was updated during the retry process and there are no other concurrent transactions in this embodiment, the retry commit is successful, and then the persistence operation consistent with that of ordinary transactions is performed.

[0131] If the multi-model transaction is successfully committed, meaning there are no concurrent multi-model transactions TxID-20240530-000, the transaction coordinator will perform the persistence operation after confirming that multi-model transaction TxID-20240530-001 has no conflicts. Figure 3 As shown. Logical object access records with the logical operation type WRITE in the transaction-level logical object access set are expanded into write operations targeting physical data items; new version numbers are generated, upgrading the version of logical object biz: / / ecommerce / User / 789 from V5 to V6, and the version of logical object biz: / / ecommerce / Inventory / item_abc from V12 to V13; sequence numbers are assigned to redo logs sent to different databases, and these sequence numbers are unique and monotonically increasing within the corresponding database instance. Finally, three redo logs are obtained as follows:

[0132] Redo log sent to LevelDB: L-LSN-20240530001 | PUT | key="user_balance_789", new_value="150.50", new_version="V6".

[0133] Redo log sent to openGauss: O-LSN-20240530001 | UPDATE | table="inventory", pk_value="item_abc", column="stock", new_value=49, new_version="V13".

[0134] Redo log sent to Nebula Graph: N-LSN-20240530001 | VERTEX_UPDATE | vid= "user_789", tag="user", property="purchase_count", new_value=11, new_version="V6".

[0135] Subsequently, the transaction coordinator distributes the redo log containing the new value and version information to all corresponding databases. Upon receiving the redo log, each database completes the disk persistence process and returns a confirmation result. After all databases confirm successful persistence, the transaction coordinator instructs each database to clean up all write results and temporary states temporarily stored during the execution of this multi-model transaction; writes the new version information to and updates the current version information of the corresponding logical object in the global logical object mapping table, thereby making the logical object version in the global logical object mapping table the authoritative basis for globally consistent versions; and returns a transaction commit success response to the user. Further, in some implementations, each database may write the new_version carried in the redo log to the version field, attribute, or bypass metadata of the corresponding physical data item during disk persistence to support idempotency verification, fault recovery, or consistency reconciliation for persistent replay; however, this is not used as the authoritative source of the logical object version.

[0136] Example 2:

[0137] This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to execute the concurrency control method for cross-model database transaction processing.

[0138] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the concurrency control method for cross-model database transaction processing as described in the embodiments. It is understood that the electronic device may also include input / output (I / O) interfaces and communication components.

[0139] The processor is used to execute all or part of the steps in the concurrency control method for cross-model database transaction processing as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0140] The processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the concurrent control method for cross-model database transaction processing described in the above embodiments.

[0141] Example 3:

[0142] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0143] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the concurrency control method for cross-model database transaction processing described in the various embodiments of this application.

[0144] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) application stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned concurrency control method for cross-model database transaction processing.

[0145] Example 4:

[0146] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the concurrency control method for cross-model database transaction processing.

[0147] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0148] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0149] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A concurrency control method for cross-model database transaction processing, characterized in that, Includes the following steps: Initialize the global logical object mapping table based on the multi-model transactions executed during the deployment phase; The unified query agent receives and parses the multi-model transaction currently sent by the user. The multi-model transaction includes several database operation statements. According to the data model type of the database operation statements, each database operation statement in the multi-model transaction is distributed to the database of the corresponding data model. Different model types of databases receive corresponding database operation statements and generate a standardized library-level logical object access set based on the global logical object mapping table; The transaction coordinator is used to aggregate standardized library-level logical object access sets from different databases to generate transaction-level logical object access sets for multi-model transactions. The transaction coordinator compares the transaction-level logical object access set of the user's currently sent multi-model transaction with the transaction-level logical object access sets of all concurrent multi-model transactions that are being committed, and makes conflict resolution based on different concurrency control strategies to obtain conflict detection results. Based on the conflict detection results, the transaction coordinator makes a decision on the final state of the multi-model transaction currently sent by the user and performs the corresponding data persistence operations.

2. The concurrency control method for cross-model database transaction processing according to claim 1, characterized in that, The multi-model transactions executed during the deployment phase are a series of multi-model transactions executed during the deployment and business initialization phases, used to create various business entities in the business; The business entity refers to a conceptual object with clear semantics and lifecycle in the business. The global logical object mapping table is used to record the mapping relationship between physical data items and logical objects, and to maintain the version information of logical objects. The logical object refers to a unified entity obtained by abstracting one or more physical data items at the business semantic level. The logical object has a unique logical object identifier, which is located in a unified namespace and is globally unique. The mapping relationships include one-to-one mapping, one-to-many mapping, and many-to-one mapping; The one-to-one mapping refers to one physical data item corresponding to one logical object; The one-to-many mapping refers to the storage of multiple attributes of the same logical object in different models or different physical data items; The many-to-one mapping refers to the complete state of a logical object formed by multiple physical data items that may cross models. The physical data item refers to the smallest storage unit that is directly read and written in the database storage engine, including at least: keys and their corresponding key-value records in the key-value database, data rows and column fields within rows in the relational database, and physical records such as vertices, edges, and attributes of vertices or edges in the graph database. When executing multi-model transactions during the deployment phase, the global logical object mapping table is initialized. When the multi-model transactions executed during the deployment phase are committed, a global logical object identifier is assigned to the business entity created by the multi-model transactions executed during the deployment phase, and the global logical object identifier and the physical data item associated with the business entity are written into the global logical object mapping table.

3. The concurrency control method for cross-model database transaction processing according to claim 2, characterized in that, The specific method for generating a standardized library-level logical object access set based on the global logical object mapping table is as follows: Different model types of databases use built-in query parsers to perform lexical analysis, syntax analysis, and semantic analysis on the received database operation statements to generate physical execution plans. The physical execution plans include the physical data items that need to be accessed to execute the database operation statements. Databases of different model types execute their respective physical execution plans in parallel before the current multi-model transaction is committed by the user. Each database includes a lightweight data access collector. During the execution of the physical execution plan, the data access collector captures physical data item access records, including native identifiers, operation types, and data values. The specific method for capturing physical data item access records is as follows: For key-value databases, capture Get, Put, and Delete operations, and record the key, value, and operation type of the captured operation; For relational databases, capture read and write operations on data rows. Read operations include index scans and full table scans, while write operations include insert, update, and delete operations. For graph databases, capture operations such as traversal of vertices or edges, attribute reading, and modification. The data access collector converts captured physical data item access records into logical object access records based on the global logical object mapping table. Multiple logical object access records are then combined to form a standardized library-level logical object access set. The specific method is as follows: The global logical object mapping table is queried based on the physical data item access record to determine the logical object to which the physical data item access record belongs and the current version information of the logical object. The current version information is one of version number, version vector or hybrid logical timestamp. The current version information of a logical object is obtained through the transaction coordinator. The current version information is one of the version number, version vector, or hybrid logical timestamp. Based on the mapping rules, the operation type and data value in the physical data item access record are mapped to logical operation type and logical operation value, respectively; the mapping rules are: operations that do not change the state of the logical object are mapped to logical read operation READ, and operations that change the state of the logical object are mapped to logical write operation WRITE. Generate standardized logical object access records in the following format: <Logical Object Identifier, Operation Type, Operation Value, Current Version Information>. The logical object identifier is unique within the global namespace, and its naming convention is: domain: / / <Business Domain> / <Entity Type> / <Entity ID>. The operation type includes READ and WRITE logical operations. The READ operation records a snapshot of the data read; the WRITE operation records the data to be written or incremental modifications; and the current version information records the globally consistent version of the logical object at the time the logical operation is executed.

4. The concurrency control method for cross-model database transaction processing according to claim 3, characterized in that, The specific method for conflict resolution is as follows: The transaction coordinator, for each logical object X involved in the transaction-level logical object access set, makes a decision based on at least one or more of the following conflict criteria, which include: If the version information of the logical object X read by the user's current multi-model transaction T1 is inconsistent with the latest version information of the logical object X, it is determined that a version verification conflict has been detected. If the logical object X that the user intends to write in the current multi-model transaction T1 has already been written by another uncommitted concurrent multi-model transaction T2, then it is determined that a logical write-write conflict has been detected. If the logical object X that the user intends to write in the current multi-model transaction T1 has been read by another uncommitted concurrent multi-model transaction T2, it is determined that a logical write-read conflict has been detected. If a logical object X that has been read by the user's current multi-model transaction T1 has been modified by another uncommitted concurrent transaction request T2, then a logical read-write conflict is detected.

5. A concurrency control method for cross-model database transaction processing according to claim 4, characterized in that, The conflict resolution based on different concurrency control strategies includes: When adopting a pessimistic control strategy based on logical object locks, a logical object-level read lock or write lock is requested when accessing a logical object. During the lock holding period, incompatible accesses to the same logical object by other concurrent transactions are prevented, thus preventing conflicts from occurring. When adopting an optimistic concurrency control strategy based on version verification, version consistency verification is achieved by comparing the version information of the logical objects accessed in the transaction-level logical object access center of the multi-model transaction currently sent by the user with the latest committed version information of the logical objects in the transaction coordinator. When adopting a multi-version concurrency control strategy, equivalent conflict avoidance is achieved by maintaining multiple version information for logical objects and comparing the version information corresponding to the transaction-level logical object access set. When using a timestamp sorting strategy, a globally unique timestamp is assigned to the multi-model transaction at the start of the transaction, and the serializability of the scheduling result is guaranteed by verifying the timestamp order of each logical read operation (READ) and logical write operation (WRITE) in the transaction-level logical object access set.

6. The concurrency control method for cross-model database transaction processing according to claim 5, characterized in that, Based on the conflict detection results, the transaction coordinator makes a decision on the final state of the multi-model transaction. The specific method is as follows: If no conflict is detected, the transaction coordinator decides to commit the multi-model transaction currently sent by the user, and the transaction coordinator performs the persistence operation. If a conflict is detected, the transaction coordinator decides to abort the multi-model transaction currently being sent by the user. The transaction coordinator returns a failure message to the user and sends an abort notification to all databases. The transaction coordinator generates and returns retryable information to the user, which instructs the user to retry the current multi-model transaction. When the user initiates a retry, each database generates a retry standardized library-level logical object access set based on the cached library-level logical object access set during the retry process, and sends the retry standardized library-level logical object access set to the transaction coordinator. The transaction coordinator aggregates the retry standardized library-level logical object access sets from different databases, generates a retry transaction-level logical object access set, and re-executes conflict resolution: if no conflict with concurrent transactions is detected, the transaction coordinator submits the retry of the multi-model transaction and generates a redo log, which is sent to each database for persistence. If a concurrent conflict is detected, the transaction coordinator will abort the retry attempt and return a retryable message again, or prompt the user to retry later. Upon receiving an abort notification, each database cleans up the temporary write results and temporary states generated during the execution of multi-model transactions, while retaining retry information; when it is detected that a user has given up retrying or the preset retry validity period has expired, the retry information is cleaned up.

7. A concurrency control method for cross-model database transaction processing according to claim 6, characterized in that, The transaction coordinator performs persistence operations including: The transaction coordinator generates new version information for all logical objects to be updated in the multi-model transaction currently sent by the user. The generation of version information follows a strict monotonically increasing rule. The transaction coordinator expands the logical object access records of the current multi-model transaction's transaction-level logical object access set, where the logical operation type is logical write (WRITE), into a set of write operations for physical data items, generating a set of redo logs. The redo logs include the target physical data item identifier, the physical write operation type, the data value to be written, and the new version information. Sequence numbers are assigned to redo logs sent to different database instances, and these sequence numbers are unique and monotonically increasing within the corresponding database instance. The transaction coordinator distributes the redo logs to the corresponding databases. After receiving the redo logs, the databases complete the write-to-disk process and return an acknowledgment result. After all databases confirm successful persistence, the transaction coordinator notifies each database to clean up all write results and temporary states temporarily stored during the execution of this multi-model transaction; writes the new version information to and updates the current version information of the corresponding logical object in the global logical object mapping table; and returns a successful transaction commit response to the user.

Images