Database access methods, devices, electronic devices and readable storage media

Abstract

The present disclosure relates to the field of database technology, and in particular to a database access method, device, electronic device, and readable storage medium. The database access method comprises: opening a transaction group according to a single database access statement for a specified data table, wherein the transaction group comprises multiple transactions; determining a data set visible to the transaction in the specified data table, wherein the data set is the same for all transactions in the transaction group; and performing access operations on the data set through the multiple transactions. The present disclosure can solve the problem of low efficiency when executing a single complex database access statement.

Description

Technical Field

[0001] The present disclosure relates to the field of database technology, and in particular to a database access method, device, electronic device and readable storage medium. Background Art

[0002] Distributed databases include CN (Compute Node, responsible for parsing, optimizing, and generating execution plans for database access statements) clusters and DN (Data Node, data node) clusters. In order to improve the performance of distributed databases, when processing complex and time-consuming single database access statements, if the target data tables are distributed on the same target DN, the distributed database will split the database access statement into more fine-grained subtasks and execute the transactions corresponding to the subtasks in parallel through multiple database connections to the target DN. By default, transactions on different database connections are independent of each other, and logical global consistency reading cannot be guaranteed, which will lead to serious execution result errors. Summary of the invention

[0003] In order to solve the problems in the related art, the embodiments of the present disclosure provide a database access method, device, electronic device and readable storage medium.

[0004] In a first aspect, an embodiment of the present disclosure provides a database access method, characterized by comprising:

[0005] Opening a transaction group according to a single database access statement for a specified data table, wherein the transaction group includes multiple transactions;

[0006] Determine a data set in the specified data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group;

[0007] Access operations are performed on the data set through the multiple transactions.

[0008] According to an embodiment of the present disclosure, wherein:

[0009] The specified data table includes one or more data tables on the specified data node;

[0010] The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

[0011] According to an embodiment of the present disclosure, different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

[0012] According to an embodiment of the present disclosure, wherein:

[0013] The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions;

[0014] The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated node when the main transaction is started.

[0015] According to an embodiment of the present disclosure, wherein:

[0016] The master transaction has read and write permissions for the specified data table, and the slave transaction only has read permissions for the specified data table.

[0017] According to an embodiment of the present disclosure, the method further includes:

[0018] When opening the transaction group, assigning a transaction group identifier and a transaction identifier to the transaction;

[0019] Searching for a transaction among active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group;

[0020] If there is no transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as the main transaction, the identifier of the specified transaction is determined as the corresponding main transaction identifier of the specified transaction, and when the specified transaction is opened, the active transaction set on the specified data node is determined as the corresponding active transaction set of the specified transaction.

[0021] According to an embodiment of the present disclosure, wherein:

[0022] The transaction group identifier includes a specified field in the transaction name of the transaction;

[0023] The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

[0024] According to an embodiment of the present disclosure, the method further includes:

[0025] If there is a transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a slave transaction, the corresponding master transaction identifier of the transaction having a transaction group identifier matching the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and the corresponding active transaction set of the transaction having a transaction group identifier matching the specified transaction is used as the corresponding active transaction set of the specified transaction.

[0026] According to an embodiment of the present disclosure, the performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions:

[0027] Determine a transaction that creates the specified data to be accessed by any of the transactions;

[0028] If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

[0029] In a second aspect, an embodiment of the present disclosure provides a database access device, characterized in that it includes:

[0030] An opening module, configured to open a transaction group according to a single database access statement for a specified data table, wherein the transaction group includes a plurality of transactions;

[0031] A first determination module is configured to determine a data set in the specified data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group;

[0032] The execution module is configured to execute access operations on the data set through the multiple transactions.

[0033] According to an embodiment of the present disclosure, wherein:

[0034] The specified data table includes one or more data tables on the specified data node;

[0035] The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

[0036] According to an embodiment of the present disclosure, different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

[0037] According to an embodiment of the present disclosure, wherein:

[0038] The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions;

[0039] The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated node when the main transaction is started.

[0040] According to an embodiment of the present disclosure, wherein:

[0041] The master transaction has read and write permissions for the specified data table, and the slave transaction only has read permissions for the specified data table.

[0042] According to an embodiment of the present disclosure, the device further includes:

[0043] an allocation module, configured to allocate a transaction group identifier and a transaction identifier to the transaction when the transaction group is opened;

[0044] A search module configured to search for a transaction in active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group;

[0045] The second determination module is configured to determine the designated transaction as a main transaction if there is no transaction with a transaction group identifier matching the designated transaction among the active transactions opened on the designated data node, determine the identifier of the designated transaction as the corresponding main transaction identifier of the designated transaction, and determine the active transaction set on the designated data node as the corresponding active transaction set of the designated transaction when the designated transaction is opened.

[0046] According to an embodiment of the present disclosure, wherein:

[0047] The transaction group identifier includes a specified field in the transaction name of the transaction;

[0048] The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

[0049] According to an embodiment of the present disclosure, the device further includes:

[0050] The third determination module is configured to determine the designated transaction as a slave transaction if there is a transaction having a transaction group identifier matching the designated transaction among the active transactions opened on the designated data node, determine the corresponding master transaction identifier of the transaction having the transaction group identifier matching the designated transaction as the corresponding master transaction identifier of the designated transaction, and use the corresponding active transaction set of the transaction having the transaction group identifier matching the designated transaction as the corresponding active transaction set of the designated transaction.

[0051] According to an embodiment of the present disclosure, the performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions:

[0052] Determine a transaction that creates the specified data to be accessed by any of the transactions;

[0053] If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

[0054] In a third aspect, an embodiment of the present disclosure provides a distributed database, including a computing node cluster and a data node cluster, wherein:

[0055] The computing node obtains a single database access statement for a specified data table;

[0056] The designated data node storing the designated data table in the data node cluster starts a transaction group according to the single database access statement, the transaction group includes multiple transactions, determines a data set in the designated data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group, and performs access operations on the data set through the multiple transactions.

[0057] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory is used to store one or more computer instructions, and wherein the one or more computer instructions are executed by the processor to implement a method as described in any one of the first aspects.

[0058] In a fifth aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the method described in the first aspect is implemented.

[0059] In a sixth aspect, a computer program product is provided in an embodiment of the present disclosure, comprising computer instructions, which, when executed by a processor, implement the method as described in any one of the first aspects.

[0060] According to the technical solution provided by the embodiment of the present disclosure, a group of transactions corresponding to subtasks obtained by splitting a single database access statement are logically divided into a transaction group, and it is stipulated that the data accessible to all transactions in a transaction group is the same, ensuring that transactions in a transaction group read logically globally consistent data, thereby achieving the same result as a single transaction execution task.

[0061] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0063] Figure 1 A schematic diagram of the structure of a distributed database according to an embodiment of the present disclosure is shown.

[0064] Figure 2 A schematic diagram showing how to split the execution of a single database access statement into multiple subtasks according to an embodiment of the present disclosure.

[0065] Figure 3 A schematic diagram of a globally incremented transaction ID according to an embodiment of the present disclosure is shown.

[0066] Figure 4 A flowchart of a database access method according to an embodiment of the present disclosure is shown.

[0067] Figure 5 A structural block diagram of a database access device according to an embodiment of the present disclosure is shown.

[0068] Figure 6 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0069] Figure 7 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0070] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0071] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0072] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0073] Figure 1 A schematic diagram of the structure of a distributed database according to an embodiment of the present disclosure is shown.

[0074] like Figure 1 As shown, the distributed database includes a DN cluster, a CN cluster and a GMS (Global Metadata Service) cluster.

[0075] DN is a storage engine used by distributed databases. Each DN can be a cluster consisting of a leader and two followers. DN uses the MVCC (Multi-Version Concurrency Control) mechanism for concurrency control, that is, inserting, modifying, deleting, and other operations on data will create a new version of the data. When other transactions read the data later, they will determine which versions are visible based on the time when the current transaction was started and the status of all currently committed and active transactions. This process is called "visibility judgment."

[0076] CN is a distributed database access engine used by distributed databases. CN has two processing modes for database access statements sent by clients. The first is to directly parse, optimize, and generate an execution plan for the database access statement, and send the generated execution plan to DN for execution. The second is to forward the database access statement to DN, and DN parses, optimizes, and generates an execution plan for the database access statement, and executes it locally. The database access statement can be, for example, a SQL (Structured Query Language) statement. When executing a database access statement, CN will create and maintain a database session to the DN where the target data is located. The target DN executes the execution plan corresponding to the database access statement, returns the execution result to CN through the database session for further processing, and finally sends the processing result to the client.

[0077] When DN executes the execution plan corresponding to the database access statement, a transaction is started. Transactions can be divided into general transactions and distributed transactions. When the accessed data is all on a single DN, a general transaction is used. When the accessed data is on different DNs in a distributed system, a distributed transaction is used.

[0078] In DN, a transaction is bound to a database session, which means that a transaction can only be executed in one session. If a transaction contains a single database access statement that is complex and takes a long time to run (for example, a query on a large table), the database access statement can only be executed serially, with poor performance. Typical scenarios include union, insert, and select.

[0079] In order to improve the performance of the distributed database system, when CN processes such a single database access statement, if it finds that multiple data tables used by it are distributed on the same DN, it will split the execution of the database access statement into multiple subtasks and use multiple database sessions to the target DN to execute the transactions corresponding to the subtasks respectively, so as to achieve the effect of parallel execution of the database access statement, such as Figure 2 shown.

[0080] Figure 2 A schematic diagram showing how to split the execution of a single database access statement into multiple subtasks according to an embodiment of the present disclosure.

[0081] like Figure 2 As shown, when CN finds that multiple data tables used by a single database access statement are distributed on the same DN, it splits the execution of the database access statement into multiple subtasks, and uses multiple database sessions to the target DN to execute the transactions corresponding to the subtasks respectively, thereby achieving the effect of executing the database access statement in parallel. Figure 2 The case where the database access uses two data tables T1 and T2 and is split into two subtasks (corresponding to transactions TRX1 and TRX2 respectively) is shown, but it can be understood that the database access can also use more data tables and be split into more subtasks accordingly.

[0082] DN needs to open a transaction when executing a transaction through each database session, and determine which data is visible to the current transaction through visibility judgment. If you simply replace a single database session with multiple database sessions, since the transactions opened on different database sessions are independent of each other, some modifications made by other concurrent transactions may only be visible to transactions opened on some database sessions, but not to transactions opened on other database sessions. As a result, logically globally consistent data cannot be read, and ultimately an incorrect execution result is obtained.

[0083] DN maintains a globally incremented transaction ID, which is incremented globally within a single DN. A newly opened transaction uses the currently available global transaction ID as its transaction ID. After the current global transaction ID is used, the global transaction ID increments by 1. The smaller the transaction ID, the earlier the transaction was opened. At the same time, DN also maintains a global active transaction linked list for quickly obtaining information about all active transactions. When a transaction is committed or rolled back, the transaction is removed from the linked list.

[0084] When a new transaction is started, DN traverses the global active transaction list and obtains the ID set of all currently active transactions. The minimum transaction ID in the set is marked as trx_id_min, and the currently available global transaction ID is marked as trx_id_max. The structure composed of this information is called a read view.

[0085] Figure 3 A schematic diagram of a globally incremented transaction ID according to an embodiment of the present disclosure is shown.

[0086] like Figure 3 As shown in the figure, assuming that when the current transaction is newly opened, the currently available global transaction ID is 1005, then DN assigns the global transaction ID 1005 to the newly opened current transaction, and traverses the global active transaction list to determine that the currently active transactions are trx1 (ID 1001), trx3 (ID 1003), and trx4 (ID 1004). The transactions before trx1 (ID 1001) and trx2 (ID 1002) are committed transactions. Therefore, the read view of the newly opened current transaction contains the following information:

[0087] 1. Active transaction ID set: (1001, 1003, 1004)

[0088] 2. trx_id_min is the minimum value in the set of active transaction IDs. In this example, the value is 1001

[0089] 3. trx_id_max is the value of the current transaction, in this case the value is 1005

[0090] When the newly opened current transaction reads a data row, it obtains the ID of the transaction that created the data (including inserting or modifying the data) from the data row, marked as trx_id, and combines the read view of the current transaction to make visibility judgment according to the following rules:

[0091] 1. If trx_id is less than the minimum value trx_id_min in the active transaction ID set (for example, all transactions with ID less than 1001), it means that the transaction has been committed when the current transaction is opened, and the data it created is visible to the current transaction.

[0092] 2. If trx_id is greater than trx_id_max (for example, all transactions with ID greater than 1005), it means that the transaction was started later than the current transaction and the data it created is not visible to the current transaction.

[0093] 3. If trx_id is included in the active transaction ID set (for example, ID is 1003), it means that the transaction has been opened and is active when the current transaction is opened, and the data created by it is not visible to the current transaction.

[0094] 4. If trx_id is not included in the active transaction ID set (for example, ID is 1002), it means that the transaction has been committed when the current transaction is opened, and the data created by it is visible to the current transaction.

[0095] 5. If the active transaction ID set is empty, it means that there are no other active transactions when the current transaction is opened. In other words, the transaction that created the data has been committed at that time, and the data it created is visible to the current transaction.

[0096] The status of globally active transactions changes dynamically over time (for example, new transactions are started, transactions are committed, rolled back, etc.). If different transactions are started at different points in time using different database sessions, these transactions will have different transaction IDs and different sets of active transaction IDs. As a result, data created or data modified by some concurrent transactions may be visible only to transactions started on some database sessions, but not to transactions started on other database sessions.

[0097] In this way, if you simply use different database sessions to initiate transactions to execute the subtasks obtained by splitting a single database access statement, these subtasks will be executed as independent transactions that have no connection with each other, and the effect of global consistent read cannot be achieved, thus causing logical errors. In order to avoid logical errors, logical read consistency must be guaranteed when executing these subtasks, that is, for a certain data row, it is either visible to all transactions where all subtasks are located, or it is not visible to all transactions where all subtasks are located.

[0098] In order to solve this problem, the embodiment of the present disclosure logically divides a group of transactions corresponding to subtasks obtained by splitting a single database access statement into a transaction group, and stipulates that the data that can be accessed by all transactions in a transaction group is the same, thereby ensuring that transactions in a transaction group read logically globally consistent data, thereby achieving the same result as a single transaction execution task.

[0099] Figure 4 FIG. 1 is a flowchart showing a database access method according to an embodiment of the present disclosure. Figure 4 As shown, the database access method includes the following steps S401-S403:

[0100] In step S401, a transaction group is opened according to a single database access statement for a specified data table, wherein the transaction group includes multiple transactions;

[0101] In step S402, a data set visible to the transaction in the specified data table is determined, wherein the data set is the same for all transactions in the transaction group;

[0102] In step S403, access operations are performed on the data set through the multiple transactions.

[0103] According to an embodiment of the present disclosure, the designated data table includes one or more data tables on a designated data node. According to an embodiment of the present disclosure, different transactions in a transaction group can access the same designated data table or different designated data tables. Since the data visible to the transactions in the transaction group is the same, logical read consistency can be guaranteed even if multiple transactions access the same data table.

[0104] According to an embodiment of the present disclosure, opening a transaction group based on a single database access statement for a specified data table may, for example, include dividing multiple transactions corresponding to multiple subtasks obtained by splitting the single database access statement for accessing the specified data table into one transaction group.

[0105] According to an embodiment of the present disclosure, different transactions in the transaction group correspond to different database sessions from a designated computing node to the designated data node. The designated computing node may be a computing node that receives the single database access statement from the client, or any other computing node used to feed back data access results to the client.

[0106] According to an embodiment of the present disclosure, performing access operations on the data set through the multiple transactions includes performing access operations on the data set in parallel through at least two transactions in the multiple transactions. Performing access operations on the data set in parallel through at least two transactions in the multiple transactions includes that the time when the at least two transactions perform access operations on the data set at least partially overlaps.

[0107] For example, for a single complex SQL statement "SELECT*FROM t1 UNION ALL select*FROM t2", in serial processing mode, a transaction is started using the same database session to DN, data tables t1 and t2 are scanned serially, and the scan results are sent to CN for aggregation. In parallel mode, CN splits the statement into two subtasks, "SELECT*FROM t1" and "select*FROM t2", and initiates transaction execution through two database sessions to DN. The two transactions scan t1 and t2 in parallel, and send the scan results to CN for aggregation.

[0108] According to an embodiment of the present disclosure, the single SQL statement includes any of the following types of SQL statements: mergesort join, union all, insert, select...

[0109] According to an embodiment of the present disclosure, the transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transaction; the data set visible to the transaction in the designated data table includes data created by the master transaction, and data created by a transaction that is started earlier than the master transaction and does not belong to the corresponding active transaction set of the transaction, and the corresponding active transaction set of the transaction is the active transaction set on the designated node when the master transaction is started.

[0110] According to an embodiment of the present disclosure, the master transaction has read and write permissions on the specified data table, and the slave transaction only has read permissions on the specified data table.

[0111] For example, the first transaction opened in a transaction group is called the master transaction. Only the master transaction can perform write operations (including insert, delete, update, etc.), and there can only be one master transaction in a transaction group. Other transactions are slave transactions and can only perform read operations. The data modifications of the master transaction are visible not only to itself, but also to all other transactions in the same transaction group.

[0112] In this way, when executing CN split subtasks through different database sessions, the transactions of these subtasks can be configured into the same transaction group. They will use the same corresponding active transaction set and main transaction ID, logically ensuring global consistent reads.

[0113] The following is an explanation using a simple example. Assuming that transactions trx 1 and trx 2 are enabled and they form a transaction group, according to the embodiment of the present disclosure:

[0114] 1. If trx 1 is started first, it will become the master transaction and can perform read and write operations. The transaction ID of trx 1 is the master transaction ID of the transaction group;

[0115] 2. trx 2 is a slave transaction and can only perform read operations. trx 2 will copy the corresponding active transaction set and master transaction ID of trx 1;

[0116] 3. If trx 1 modifies data, these modifications are also visible to trx 2.

[0117] Take the SQL statement "insert into t1 values(...); ...; insert into t2 select*fromt1" as an example. CN first inserts data into t1, then obtains all the data in t1 and inserts it into t2. In order to achieve parallel processing, CN splits the SQL statement into two transactions and executes them through different database sessions. Among them, the main transaction is used to insert data into t1 and insert the data obtained from CN into t2, and the slave transaction is used to read data from t1 and send it to CN. Since the main transaction is not committed, by default, the other slave transactions cannot read the data inserted into t1 by the main transaction. In transaction group mode, since each transaction in the transaction group saves the main transaction ID, the changes of the main transaction are visible to all transactions in the transaction group, so each transaction in the transaction group can read the data inserted into t1 by the main transaction.

[0118] According to an embodiment of the present disclosure, a new system variable is added in the kernel layer of DN to explicitly turn on or off the transaction group mode. The transaction group mode is turned off by default. When the transaction group mode needs to be turned on or off, it needs to be set in advance in the database connection.

[0119] If the transaction group mode is turned on in the current database session, the transaction group mode is only effective for the current database session. If the transaction group mode is turned on globally, the transaction group mode is effective for all database sessions. After this variable is modified, it is only effective for new transactions opened subsequently, and the currently executing transaction is not affected.

[0120] According to an embodiment of the present disclosure, in the transaction group mode, transactions opened within a period of time need to be logically divided into one or more different transaction groups. However, opening a transaction is a relatively independent process, so it is necessary to determine which transactions belong to the same transaction group as the transaction to be opened.

[0121] There are many ways to determine which transactions belong to the same transaction group as the transaction to be opened. For example, the most direct method is to specify the transaction group ID separately when opening a transaction. However, this solution requires modifying the SQL syntax, and when making visibility judgments, it is necessary to traverse the global active transaction list multiple times to find transactions with the same transaction group ID, which has a direct impact on performance and is costly.

[0122] Alternatively, according to an embodiment of the present disclosure, the transaction group identifier includes a specified field in the transaction name of the transaction, and the assignment of the transaction group identifier to the transaction includes assigning a transaction group identifier to the transaction by assigning a value to the specified field when starting the transaction. For example, CN can use an SQL statement such as "XA START xid" to start a transaction, where xid is the transaction name and the format is gtrid[,bqual[,formatID]], wherein gtrid and bqual are both strings that can be arbitrarily specified, so their assignments can be agreed upon to achieve transaction group matching. According to an embodiment of the present disclosure, xids that meet the following preset rules belong to the same transaction group:

[0123] 1. gtrid is the same; and

[0124] 2. The format of bqual is "value@xxxx". The "value" part is the same, and the "xxxx" part is 4 digits and cannot be repeated.

[0125] Specifically, when the xid and gtrid of two transactions are the same, the value of value@xxxx is the same, but xxxx is different, the two transactions belong to the same transaction group.

[0126] According to an embodiment of the present disclosure, when a transaction is opened, the specified fields gtrid and bqual in xid can be assigned values ​​according to the above-mentioned preset rules to divide the transaction into transaction groups. According to an embodiment of the present disclosure, the gtrid and value@xxxx fields in xid can be regarded as transaction group identifiers. When two transaction group identifiers meet the preset matching rules, the two transaction group identifiers are matching transaction group identifiers. For example, when the gtrid of two transaction group identifiers is the same, the value of value@xxxx is the same, and xxxx is different, the two transaction group identifiers are matching transaction group identifiers.

[0127] According to an embodiment of the present disclosure, when the transaction group is opened, a transaction group identifier and a transaction identifier are assigned to the transaction; a transaction having a transaction group identifier matching the specified transaction in the transaction group is searched for among the active transactions opened on the specified data node; if there is no transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a master transaction, the identifier of the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and when the specified transaction is opened, the active transaction set on the specified node is determined as the corresponding active transaction set of the specified transaction. If there is a transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a slave transaction, the corresponding master transaction identifier of the transaction having a transaction group identifier matching the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and the corresponding active transaction set of the transaction having a transaction group identifier matching the specified transaction is used as the corresponding active transaction set of the specified transaction.

[0128] After the transaction group mode is turned on, the information related to the transaction group needs to be recorded. A member variable trx_group_master_trx_id is added to the object in memory for each transaction to save the master transaction ID of the transaction group to which the current transaction belongs.

[0129] If the transaction group mode is off, the process of creating a read view is the same as a normal transaction. In addition, you need to set the trx_group_master_trx_id of the current transaction to the ID of the current transaction.

[0130] If the transaction group mode is turned on, when a current transaction is opened, the currently available global transaction ID is assigned to the current transaction as its transaction ID, and the current transaction is matched with the active transaction opened on the corresponding DN according to the above xid assignment rules. If there is no active transaction belonging to the same transaction group as the current transaction, it means that the current transaction is the first transaction opened in this transaction group, and it may also be the only transaction in this transaction group. Then the trx_group_master_trx_id of the current transaction is set to the transaction ID of the current transaction, and the read view of the current transaction is generated. The readview includes the corresponding active transaction set of the current transaction (represented by the active transaction ID set on the DN when the current transaction is opened), the minimum transaction ID in the active transaction ID set, and the transaction ID of the current transaction.

[0131] If it is found according to the above xid assignment rules that the current transaction group already has a transaction opened and in an active state, that is, a matching transaction for the current transaction group is found, the trx_group_master_trx_id of the current transaction is set to the trx_group_master_trx_id of the matching transaction, and the read view of the matching transaction is copied.

[0132] In this way, when all transactions belonging to the same transaction group are started, they will have a common read view and master transaction ID (ie, trx_group_master_trx_id).

[0133] According to an embodiment of the present disclosure, performing access operations on the data set through the multiple transactions includes, for any transaction among the multiple transactions: determining a transaction that creates the specified data to be accessed by the any transaction; if the transaction that creates the specified data is opened before the corresponding main transaction of the any transaction and does not belong to the corresponding active transaction set of the any transaction, or the transaction that creates the specified data is the corresponding main transaction of the any transaction, then the specified data is visible to the any transaction, otherwise the specified data is not visible to the any transaction.

[0134] According to an embodiment of the present disclosure, the read views of all transactions in a transaction group are the same, and their corresponding main transaction IDs are the same. Therefore, for any transaction in the transaction group, the data visibility determined based on the read view and the main transaction ID is the same, thereby ensuring the global consistency read of transactions in the transaction group.

[0135] According to the embodiments of the present disclosure, the above database access method can be implemented on DN based on the storage engine layer of DN, and cooperate with CN to split and parallelize complex and time-consuming SQL statements, and test SQL statements for the orders table (1.5 million rows of data) and the customer table (150,000 rows of data): SQL: SELECT o_orderkey, o_custkey, c_custkey, c_name FROM orders JOIN customer on o_custkey=c_custkey ORDER BY o_custkeylimit 5. The test results show that the execution time of enabling the transaction group function is less than 1 second, which saves about 2 seconds of execution time compared to not enabling the transaction group function, and the time is greatly shortened. It can be seen that the database access method provided by the present disclosure can effectively improve the processing speed of complex SQL.

[0136] Figure 5The structural block diagram of the database access device according to the embodiment of the present disclosure is shown. The device can be implemented as part or all of the DN through software, hardware or a combination of both.

[0137] like Figure 5 As shown, the database access device 500 includes an opening module 510 , a first determining module 520 , and an executing module 530 .

[0138] The opening module 510 is configured to open a transaction group according to a single database access statement to a specified data table, wherein the transaction group includes multiple transactions;

[0139] The first determination module 520 is configured to determine a data set in the specified data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group;

[0140] The execution module 530 is configured to execute access operations on the data set through the multiple transactions.

[0141] According to an embodiment of the present disclosure, wherein:

[0142] The specified data table includes one or more data tables on the specified data node;

[0143] The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

[0144] According to an embodiment of the present disclosure, different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

[0145] According to an embodiment of the present disclosure, wherein:

[0146] The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions;

[0147] The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated node when the main transaction is started.

[0148] According to an embodiment of the present disclosure, wherein:

[0149] The master transaction has read and write permissions for the specified data table, and the slave transaction only has read permissions for the specified data table.

[0150] According to an embodiment of the present disclosure, the device 500 further includes:

[0151] an allocation module 540, configured to allocate a transaction group identifier and a transaction identifier to the transaction when the transaction group is opened;

[0152] A search module 550 is configured to search for a transaction in active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group;

[0153] The second determination module 560 is configured to determine the designated transaction as a primary transaction if there is no transaction having a transaction group identifier matching the designated transaction among the active transactions opened on the designated data node, determine the identifier of the designated transaction as the corresponding primary transaction identifier of the designated transaction, and determine the active transaction set on the designated data node as the corresponding active transaction set of the designated transaction when the designated transaction is opened.

[0154] According to an embodiment of the present disclosure, wherein:

[0155] The transaction group identifier includes a specified field in the transaction name of the transaction;

[0156] The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

[0157] According to an embodiment of the present disclosure, the device 500 further includes:

[0158] The third determination module 570 is configured to determine the designated transaction as a slave transaction if there is a transaction having a transaction group identifier matching the designated transaction among the active transactions opened on the designated data node, determine the corresponding master transaction identifier of the transaction having the transaction group identifier matching the designated transaction as the corresponding master transaction identifier of the designated transaction, and use the corresponding active transaction set of the transaction having the transaction group identifier matching the designated transaction as the corresponding active transaction set of the designated transaction.

[0159] According to an embodiment of the present disclosure, the performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions:

[0160] Determine a transaction that creates the specified data to be accessed by any of the transactions;

[0161] If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

[0162] The present disclosure also discloses an electronic device, Figure 6 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0163] like Figure 6 As shown, the electronic device includes a memory and a processor, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method according to an embodiment of the present disclosure.

[0164] The present disclosure provides a database access method, which includes:

[0165] Opening a transaction group according to a single database access statement for a specified data table, wherein the transaction group includes multiple transactions;

[0166] Determine a data set in the specified data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group;

[0167] Access operations are performed on the data set through the multiple transactions.

[0168] According to an embodiment of the present disclosure, wherein:

[0169] The specified data table includes one or more data tables on the specified data node;

[0170] The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

[0171] According to an embodiment of the present disclosure, different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

[0172] According to an embodiment of the present disclosure, wherein:

[0173] The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions;

[0174] The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated node when the main transaction is started.

[0175] According to an embodiment of the present disclosure, wherein:

[0176] The master transaction has read and write permissions for the specified data table, and the slave transaction only has read permissions for the specified data table.

[0177] According to an embodiment of the present disclosure, the method further includes:

[0178] When opening the transaction group, assigning a transaction group identifier and a transaction identifier to the transaction;

[0179] Searching for a transaction among active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group;

[0180] If there is no transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as the main transaction, the identifier of the specified transaction is determined as the corresponding main transaction identifier of the specified transaction, and when the specified transaction is opened, the active transaction set on the specified data node is determined as the corresponding active transaction set of the specified transaction.

[0181] According to an embodiment of the present disclosure, wherein:

[0182] The transaction group identifier includes a specified field in the transaction name of the transaction;

[0183] The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

[0184] According to an embodiment of the present disclosure, the method further includes:

[0185] If there is a transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a slave transaction, the corresponding master transaction identifier of the transaction having a transaction group identifier matching the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and the corresponding active transaction set of the transaction having a transaction group identifier matching the specified transaction is used as the corresponding active transaction set of the specified transaction.

[0186] According to an embodiment of the present disclosure, the performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions:

[0187] Determine a transaction that creates the specified data to be accessed by any of the transactions;

[0188] If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

[0189] The present disclosure also provides a distributed database, including a computing node cluster and a data node cluster, wherein:

[0190] The computing node obtains a single database access statement for a specified data table;

[0191] The designated data node storing the designated data table in the data node cluster starts a transaction group according to the single database access statement, the transaction group includes multiple transactions, determines a data set in the designated data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group, and performs access operations on the data set through the multiple transactions.

[0192] According to an embodiment of the present disclosure, a distributed database may have the following Figure 1 The structure shown.

[0193] According to an embodiment of the present disclosure, wherein:

[0194] The specified data table includes one or more data tables on the specified data node;

[0195] The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

[0196] According to an embodiment of the present disclosure, wherein:

[0197] Different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

[0198] According to an embodiment of the present disclosure, wherein:

[0199] The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions;

[0200] The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated node when the main transaction is started.

[0201] According to an embodiment of the present disclosure, wherein:

[0202] The master transaction has read and write permissions for the specified data table, and the slave transaction only has read permissions for the specified data table.

[0203] According to an embodiment of the present disclosure, the designated data node is further configured as:

[0204] When opening the transaction group, assigning a transaction group identifier and a transaction identifier to the transaction;

[0205] Searching for a transaction among active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group;

[0206] If there is no transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as the main transaction, the identifier of the specified transaction is determined as the corresponding main transaction identifier of the specified transaction, and when the specified transaction is opened, the active transaction set on the specified data node is determined as the corresponding active transaction set of the specified transaction.

[0207] According to an embodiment of the present disclosure, wherein:

[0208] The transaction group identifier includes a specified field in the transaction name of the transaction;

[0209] The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

[0210] According to an embodiment of the present disclosure, the designated data node is further configured as:

[0211] If there is a transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a slave transaction, the corresponding master transaction identifier of the transaction having a transaction group identifier matching the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and the corresponding active transaction set of the transaction having a transaction group identifier matching the specified transaction is used as the corresponding active transaction set of the specified transaction.

[0212] According to an embodiment of the present disclosure, the performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions:

[0213] Determine a transaction that creates the specified data to be accessed by any of the transactions;

[0214] If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

[0215] Figure 7 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0216] like Figure 7 As shown, the computer system includes a processing unit, which can perform the various methods in the above-mentioned embodiments according to the program stored in the read-only memory (ROM) or the program loaded from the storage part into the random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The processing unit, ROM and RAM are connected to each other through a bus. The input / output (I / O) interface is also connected to the bus.

[0217] The following components are connected to the I / O interface: an input part including a keyboard, a mouse, etc.; an output part including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage part including a hard disk, etc.; and a communication part including a network interface card such as a LAN card, a modem, etc. The communication part performs a communication process via a network such as the Internet. The drive is also connected to the I / O interface as needed. Removable media, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on the drive as needed so that the computer program read therefrom is installed into the storage part as needed. Among them, the processing unit can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0218] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above method. In such an embodiment, the computer program can be downloaded and installed from a network through a communication part, and / or installed from a removable medium.

[0219] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0220] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0221] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0222] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A database access method, characterized in that: include: Opening a transaction group according to a single database access statement for a specified data table, wherein the transaction group includes multiple transactions; Determine a data set in the designated data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group; the designated data table includes one or more data tables on a designated data node; performing access operations on the data set through the multiple transactions; The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions; The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated data node when the main transaction is started.

2. The method according to claim 1, wherein: The opening of the transaction group according to the single database access statement for the designated data table includes dividing the multiple transactions corresponding to the multiple subtasks obtained by splitting the single database access statement into one transaction group.

3. The method according to claim 1, wherein: Different transactions in the transaction group correspond to different database sessions from a specified computing node to the specified data node.

4. The method according to claim 1, wherein: The master transaction has read and write permissions on the specified data table, and the slave transaction only has read permissions on the specified data table.

5. The method according to claim 1, further comprising: When opening the transaction group, assigning a transaction group identifier and a transaction identifier to the transaction; Searching for a transaction among active transactions opened on the designated data node that has a transaction group identifier that matches a designated transaction in the transaction group; If there is no transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as the main transaction, the identifier of the specified transaction is determined as the corresponding main transaction identifier of the specified transaction, and when the specified transaction is opened, the active transaction set on the specified data node is determined as the corresponding active transaction set of the specified transaction.

6. The method according to claim 5, wherein: The transaction group identifier includes a specified field in the transaction name of the transaction; The allocating a transaction group identifier to the transaction includes allocating a transaction group identifier to the transaction by assigning a value to the designated field when starting the transaction.

7. The method according to claim 5, further comprising: If there is a transaction having a transaction group identifier matching the specified transaction among the active transactions opened on the specified data node, the specified transaction is determined as a slave transaction, the corresponding master transaction identifier of the transaction having a transaction group identifier matching the specified transaction is determined as the corresponding master transaction identifier of the specified transaction, and the corresponding active transaction set of the transaction having a transaction group identifier matching the specified transaction is used as the corresponding active transaction set of the specified transaction.

8. The method according to claim 7, wherein: The performing access operations on the data set through the multiple transactions includes, for any transaction in the multiple transactions: Determine a transaction that creates the specified data to be accessed by any of the transactions; If the transaction that creates the specified data is opened before the corresponding main transaction of any of the transactions and does not belong to the corresponding active transaction set of any of the transactions, or the transaction that creates the specified data is the corresponding main transaction of any of the transactions, then the specified data is visible to any of the transactions; otherwise, the specified data is not visible to any of the transactions.

9. A database access device, characterized in that: include: An opening module, configured to open a transaction group according to a single database access statement for a specified data table, wherein the transaction group includes a plurality of transactions; A first determination module is configured to determine a data set in the specified data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group; the specified data table includes one or more data tables on a specified data node; an execution module, configured to perform access operations on the data set through the multiple transactions; The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions; The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated data node when the main transaction is started.

10. A distributed database, comprising a computing node cluster and a data node cluster, wherein: The computing node obtains a single database access statement for a specified data table; The designated data node storing the designated data table in the data node cluster starts a transaction group according to the single database access statement, the transaction group includes multiple transactions, determines a data set in the designated data table that is visible to the transaction, wherein the data set is the same for all transactions in the transaction group, the designated data table includes one or more data tables on the designated data node, and access operations are performed on the data set through the multiple transactions; The transaction group includes a master transaction and multiple slave transactions, and the master transaction is started before the slave transactions; The data set visible to the transaction in the designated data table includes data created by the main transaction and data created by a transaction that is started earlier than the main transaction and does not belong to the corresponding active transaction set of the transaction, where the corresponding active transaction set of the transaction is the active transaction set on the designated data node when the main transaction is started.

11. An electronic device, characterized in that: The method comprises a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the method steps described in any one of claims 1 to 8.

12. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

13. A computer program product, comprising computer instructions, which, when executed by a processor, implement the method steps of any one of claims 1 to 8.

Images