A transaction execution method for a distributed database, an electronic device and a storage medium

Abstract

The invention discloses a transaction execution method, electronic device and storage medium of a distributed database. The method comprises: receiving a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement through an auxiliary transaction; based on the execution sequence number and a streaming reading threshold, streamingly executing the first statement to be executed through the auxiliary transaction, batch-reading data rows and returning to the distributed database entry until all data rows are read, the batch-read data rows are processed by the distributed database entry, and data to be updated is generated; receiving the data to be updated and a second statement to be executed fed back by the distributed database entry in sequence through a main transaction, executing the second statement to be executed in parallel through the main transaction, and writing the data to be updated, thereby solving the problem of insufficient memory resources and poor performance caused by the single-machine database in the distributed database being able to execute some statements only serially, saving memory resources and reducing execution time.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a transaction execution method, electronic equipment and storage medium of a distributed database. Background Art

[0002] A distributed database is a database that decomposes the business's Structured Query Language (SQL) requests to multiple stand-alone databases for execution. The main architecture of a distributed database includes: the distributed database entrance Gate and multiple stand-alone databases. The Gate can directly forward the SQL sent by the user to each stand-alone database. When executing some special SQL, the Gate cannot directly send the SQL sent by the user to each stand-alone database.

[0003] In order to solve the problem that Gate cannot directly send the SQL sent by the user to the stand-alone database, the existing main method is as follows: Gate parses the SQL to obtain the single-machine database set single_db_list where all the data rows to be updated are stored; for each stand-alone database in single_db_list, a respective read_sql_with_lock for reading data with write lock and a delete_sql for deleting data are generated; Gate opens a stand-alone transaction on each stand-alone database in single_db_list and executes the respective read_sql_with_lock to query all the data rows to be updated ori_datas and save them in Gate memory; in Gate memory, the values ​​of the shard key columns of all the queried data rows are updated to obtain updated_datas; according to updated_datas, the single-machine database set updated_single_db_list where all the updated data rows are stored is obtained; for each stand-alone database in updated_single_db_list, a respective insert_sql for inserting data is generated; Gate executes the transaction in single_db_l Each single-machine database in ist executes its own delete_sql to delete the data row to be updated from the single-machine database. Gate executes its own insert_sql on each single-machine database in updated_single_db_list to insert the updated data row into the single-machine database.

[0004] The above method requires the Gate process to have a large enough memory space. When the number of data rows updated by the user sql is large, it is necessary to lock all the rows from each stand-alone database and query them to the Gate memory, which may cause the Gate process to crash due to insufficient memory resources. In addition, the above method has performance problems. When the number of data rows updated by the user sql is large, it is necessary to lock all the rows from each stand-alone database and query them to the Gate, then delete all the data from each stand-alone database, and then insert all the updated data into each stand-alone database. The whole process is serial. In real business scenarios, the time for full lock reading may be longer due to the need to wait for the row locks of some data rows, resulting in a longer time to execute this user sql. Summary of the invention

[0005] The present invention provides a transaction execution method, electronic device and storage medium for a distributed database, so as to solve the problem of insufficient memory resources and poor performance caused by the fact that a single-machine database in a distributed database can only execute some statements serially.

[0006] According to one aspect of the present invention, a transaction execution method for a distributed database is provided, which is applied to a stand-alone database in the distributed database, comprising:

[0007] Receiving, through an auxiliary transaction in an auxiliary transaction model, a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement, wherein the auxiliary transaction model includes a main transaction and an auxiliary transaction in one-to-one correspondence;

[0008] Based on the execution sequence number and a predetermined streaming reading threshold, the first statement to be executed is stream-executed through the auxiliary transaction, and data rows are read in batches and returned to the distributed database entry until all data rows are read, wherein the batch-read data rows are processed by the distributed database entry and data to be updated is generated;

[0009] The data to be updated and the second statement to be executed fed back by the distributed database entry are received in sequence through the main transaction in the auxiliary transaction model, and the second statement to be executed is executed in parallel through the main transaction to write the data to be updated.

[0010] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0011] at least one processor, and a memory communicatively coupled to the at least one processor;

[0012] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the transaction execution method of the distributed database described in any embodiment of the present invention.

[0013] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transaction execution method of a distributed database described in any embodiment of the present invention when executed.

[0014] According to the technical solution of the embodiment of the present invention, a stand-alone database receives a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement through an auxiliary transaction in an auxiliary transaction model, wherein the auxiliary transaction model includes a one-to-one corresponding main transaction and auxiliary transaction; based on the execution sequence number and a predetermined streaming reading threshold, the first statement to be executed is streamed through the auxiliary transaction, and data rows are read in batches and returned to the distributed database entry until all data rows are read, wherein the batch-read data rows are processed by the distributed database entry and data to be updated is generated; the data to be updated and the second statement to be executed fed back by the distributed database entry are received in sequence through the main transaction in the auxiliary transaction model, and the second statement to be executed is executed in parallel through the main transaction to write the data to be updated, thereby solving the problem of a stand-alone database in a distributed database. In order to solve the problem of insufficient memory resources and poor performance caused by the fact that some statements can only be executed serially, the present application provides a new transaction model, namely, an auxiliary transaction model, in which statements are executed in parallel by the main transaction and the auxiliary transaction in the auxiliary transaction model, and the auxiliary transaction executes the first statement to be executed in a streaming manner to read data rows and feeds them back to the distributed database entry. The distributed database entry processes the data to generate data to be updated and feeds them back to the stand-alone database. The stand-alone database receives the data to be updated and the second statement to be executed in sequence. After each receipt of the data to be updated and the second statement to be executed, the main transaction executes the second statement to be executed to write the data to be updated. At this time, the first statement to be executed can read data rows in a streaming manner, and the parallel execution of the second statement to be executed and the first statement to be executed is realized, that is, the reading and writing of data are executed in parallel, which saves the memory resources of the distributed database entry, reduces the execution time, and improves the database performance.

[0015] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 is a flowchart of a transaction execution method of a distributed database provided according to Embodiment 1 of the present invention;

[0018] Figure 2 is an architecture diagram of a distributed database provided according to Embodiment 1 of the present invention;

[0019] Figure 3 is a flowchart of a transaction execution method of a distributed database provided according to Embodiment 2 of the present invention;

[0020] Figure 4 This is an example diagram of a process of applying for a row lock of a data row to be read through an auxiliary transaction according to Embodiment 2 of the present invention;

[0021] Figure 5 This is an example diagram of a process of waking up a transaction waiting for a lock in a session according to Embodiment 2 of the present invention;

[0022] Figure 6 is a schematic diagram of the structure of a distributed database provided according to Embodiment 3 of the present invention;

[0023] Figure 7 It is a structural schematic diagram of an electronic device for implementing a transaction execution method of a distributed database according to an embodiment of the present invention; DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] Embodiment 1

[0027] Figure 1 A flowchart of a transaction execution method for a distributed database provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of executing transactions in a distributed database in parallel. The method can be executed by a stand-alone database of the distributed database, and the stand-alone database in the distributed database can be deployed in an electronic device.

[0028] Figure 2 The invention provides an architecture diagram of a distributed database. The distributed database implements operations such as data query and update by decomposing the structured query language (SQL) request of the business to multiple stand-alone databases for execution. The distributed database includes a client 11, a distributed database portal 12 and at least two stand-alone databases 13. The distributed database portal 12 is a Gate. The distributed database portal 11 is an independent process, which is mainly responsible for processing the request of the client 11 and distributing the request to each stand-alone database 13, and is also responsible for controlling the execution process of distributed transactions. Distributed transactions will start stand-alone transactions on multiple stand-alone databases 13, that is, distributed transactions contain multiple stand-alone transactions, and distributed transactions guarantee strict ACID properties. The stand-alone database 13 stores data in the distributed database and is an independent process. It is responsible for executing the request issued by the distributed database portal 12 and returning the result to the distributed database portal 12, and is also responsible for controlling the execution process of stand-alone transactions. The stand-alone database 13 uses enhanced strict two-phase locking as the concurrency control strategy of the stand-alone transaction to ensure the ACID property of the stand-alone transaction. The lock granularity supports the data row level, and the lock mode supports write lock, read lock, etc. The distributed database entry 12 receives the SQL sent by the client 11, parses the SQL into SQL1-n and sends it to the corresponding stand-alone database 13, and at the same time receives n results fed back by the stand-alone database 13, summarizes the results returned by each stand-alone database 13 and returns them to the client 11.

[0029] like Figure 1 As shown, the method includes:

[0030] S101. Receive a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement through an auxiliary transaction in an auxiliary transaction model. The auxiliary transaction model includes a one-to-one corresponding main transaction and auxiliary transaction.

[0031] In this embodiment, the auxiliary transaction model can be specifically understood as a model that includes a main transaction and an auxiliary transaction, providing a streaming asynchronous execution method. The main transaction and the auxiliary transaction are one-to-one corresponding, that is, a main transaction can and can only have one auxiliary transaction, and an auxiliary transaction can and can only have one main transaction. The main transaction is started before the auxiliary transaction. The main transaction can execute any SQL that reads and writes data, and the auxiliary transaction only supports the execution of SQL that reads and writes data with a write lock. Each SQL that reads and writes data executed in the main transaction or auxiliary transaction needs to contain a serial number, and the serial numbers of multiple SQLs received in the same transaction remain monotonically the same or increasing. The auxiliary transaction model can be started in advance and used directly when it is necessary to execute two transactions in parallel through the main transaction and the auxiliary transaction, or it can be started when needed, which can be determined according to the specific usage scenario.

[0032] The operation statement can be understood as the operation statement of the database sent by the user through the client, for example, the SQL to update the value of the shard key column. The first statement to be executed can be understood as the statement for reading data sent by the distributed data entry to the stand-alone database for execution. The first statement to be executed is determined according to the operation statement. The execution sequence number can be understood as a sequence number used to identify the execution order of the operation statement. The execution sequence number is generated by the distributed database entry.

[0033] After receiving the operation statement input by the user, the distributed database portal parses the operation statement to determine the specific meaning of the operation statement, the specific operation to be performed and the order of the operation, etc. When the operation statement is a type of statement that needs to be locked and read into the memory, the data to be written is calculated in the memory and then written (for example, SQL to update the value of the sharding key column), in order to save execution time and resources, it is executed in parallel through the auxiliary transaction model. The distributed database portal parses the operation statement and generates the corresponding first statement to be executed. The distributed database portal maintains the execution sequence number corresponding to different operation statements and generates an execution sequence number for the currently received operation statement. The distributed database portal determines the stand-alone database communicating with it by parsing the operation statement, and determines whether the auxiliary transaction model has been started on the stand-alone database in the current session. If it has been started, the first statement to be executed and the execution sequence number are directly sent to the stand-alone database through the auxiliary transaction in the auxiliary transaction model. The stand-alone database receives the first statement to be executed and the execution sequence number through the auxiliary transaction. If the auxiliary transaction model is not started on the stand-alone database, the stand-alone database is instructed to start the auxiliary transaction model, that is, to start the main transaction and the auxiliary transaction, and the first statement to be executed and the execution sequence number are sent to the stand-alone database through the auxiliary transaction in the auxiliary transaction model. The stand-alone database receives the first statement to be executed and the execution sequence number through the auxiliary transaction.

[0034] S102. Based on the execution sequence number and a predetermined streaming reading threshold, the first statement to be executed is stream-executed through an auxiliary transaction, and data rows are read in batches and returned to the distributed database entry until all data rows are read, wherein the batch-read data rows are processed by the distributed database entry and data to be updated is generated.

[0035] In this embodiment, the stream reading threshold can be specifically understood as the maximum value of the data rows read each time when streaming data is read. The data to be updated can be specifically understood as the data that needs to be updated and written into the stand-alone database.

[0036] The streaming read threshold is set in advance. The streaming read threshold can be set to a fixed value or according to the type of operation statement, the business type corresponding to the data, and other data. The streaming read threshold can be determined by the distributed database entry and sent to the stand-alone database when sending the first statement to be executed, or it can be set by the stand-alone database itself. Before reading the data, the streaming read threshold is obtained, and the data rows are read in batches by executing the first statement to be executed through the auxiliary transaction. In the process of reading the data rows, it is determined whether each row of data rows can be read based on the execution sequence number. The judgment process is as follows: the execution sequence number is used to determine whether the last modification of the data row is executed by the main transaction corresponding to the auxiliary transaction when the corresponding data modification is executed according to the current operation statement. If so, it can be determined that this data row does not need to be read, otherwise, it is determined that this data row can be read. The data rows are read in batches according to the streaming read threshold until all the data rows are read. The stand-alone database feeds back the read data rows to the distributed database entry, and the distributed database entry performs corresponding data processing to generate data to be updated. The distributed database sends the generated data to be updated to the stand-alone database through the main transaction of the stand-alone database for subsequent data writing process.

[0037] S103: receiving the data to be updated and the second statement to be executed fed back by the distributed database entry in sequence through the main transaction in the auxiliary transaction model, executing the second statement to be executed in parallel through the main transaction, and writing the data to be updated.

[0038] In this embodiment, the second statement to be executed can be understood as a statement that performs a write operation on data executed by a stand-alone database. The second statement to be executed is determined by a distributed database entry. For example, the second statement to be executed is determined based on the operation statement and the data to be updated. Each time the distributed database entry receives a data row read by an auxiliary transaction, it processes it as data to be updated, determines the second statement to be executed, and sends the second statement to be executed and the data to be updated to the corresponding main transaction in the stand-alone database. The stand-alone database receives the data to be updated and the second statement to be executed fed back by the distributed database entry through the main transaction, and executes the second statement to be executed to write the data to be updated into the stand-alone database.

[0039] Taking the streaming read threshold as N as an example, when the stand-alone database reads data rows in batches, after each N rows of data rows are read, the N rows of data rows are fed back to the distributed database portal. The distributed database portal processes the N rows of data rows and sends them to the main transaction of the stand-alone database. At the same time, the distributed database portal sends the second statement to be executed to the stand-alone database through the main transaction of the stand-alone database. The stand-alone database executes the second statement to be executed through the main transaction and writes the data to be updated to the corresponding position in the database. After receiving the data of each N row of data rows, the distributed database processes it as a row of data to be updated and sends it to the main transaction of the stand-alone database to execute data writing, that is, the stand-alone database executes the first statement to be executed through the auxiliary transaction to read the data, and executes the second statement to be executed through the main transaction to write the data. The first execution statement returns data in batches until all data rows are read. After reading all data rows, the stand-alone database notifies the distributed database portal that all data has been read. For each batch of data, the distributed database portal generates the first second statement to be executed, the second second statement to be executed, and the nth second statement to be executed. The first statement to be executed is executed in parallel with each second statement to be executed.

[0040] Since the first statement to be executed and the second statement to be executed are executed in parallel, that is, while the first statement to be executed is reading data, the second statement to be executed will write to the data and change the data in the data row. If the first statement to be executed directly reads the data row, the data written this time will be read. In fact, the data written this time should not be read. By judging whether the data row is read by executing the sequence number, the data written this time can be avoided from being read, data confusion can be avoided, and the correctness and reliability of parallel execution can be maintained.

[0041] A transaction execution method for a distributed database provided by an embodiment of the present invention solves the problem of insufficient memory resources and poor performance caused by the fact that a stand-alone database in a distributed database can only execute some statements serially. The present application provides a new transaction model, namely an auxiliary transaction model, in which statements are executed in parallel by a main transaction and an auxiliary transaction in the auxiliary transaction model. The auxiliary transaction executes a first statement to be executed in a streaming manner to read data rows and feeds back the statement to the distributed database entry. The distributed database entry processes the data to generate data to be updated and feeds back the data to the stand-alone database. The stand-alone database receives the data to be updated and the second statement to be executed in sequence. After each receipt of the data to be updated and the second statement to be executed, the main transaction executes the second statement to be executed to write the data to be updated. At this time, the first statement to be executed can read data rows in a streaming manner, thereby realizing the parallel execution of the second statement to be executed and the first statement to be executed, namely, the reading and writing of data are executed in parallel, thereby saving memory resources of the distributed database entry, reducing execution time, and improving database performance.

[0042] Embodiment 2

[0043] Figure 3 This is a flow chart of a transaction execution method of a distributed database provided in Embodiment 2 of the present invention. This embodiment is refined on the basis of the above embodiment. Figure 3 As shown, the method includes:

[0044] S201. Receive a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement through an auxiliary transaction in an auxiliary transaction model. The auxiliary transaction model includes a one-to-one corresponding main transaction and auxiliary transaction.

[0045] As an optional embodiment, this optional embodiment is further optimized to include: if there is no first stand-alone transaction between the stand-alone database and the distributed database entrance and the auxiliary transaction model start-up condition is met, the auxiliary transaction model is started; if there is a first stand-alone transaction between the stand-alone database and the distributed database entrance and the auxiliary transaction model start-up condition is met, the auxiliary transaction model is started based on the first stand-alone transaction.

[0046] In this embodiment, the first stand-alone transaction can be understood as a stand-alone transaction, which is a transaction for data processing between a stand-alone database and a distributed database entry. The auxiliary transaction model startup condition can be understood as a condition for determining whether to start the auxiliary transaction model. The auxiliary transaction model startup condition can be set according to the type of operation statement. For example, when the operation statement is a statement that needs to be locked and read into the memory, the data to be written is calculated in the memory and then written, it is determined that the auxiliary transaction model startup condition is met.

[0047] Before starting the auxiliary transaction model, the stand-alone database may not start any transaction, that is, it may not interact with data, or it may interact with the distributed database portal through the first stand-alone transaction and execute simple database statements. After receiving the operation statement, the distributed database portal first determines whether the auxiliary transaction model startup conditions are met according to the operation statement. If the auxiliary transaction model startup conditions are met, it determines whether there is a first stand-alone transaction between the stand-alone database and the distributed database portal. If so, the auxiliary transaction model is started based on the first stand-alone transaction; if not, the auxiliary transaction model is started directly.

[0048] As an optional embodiment, this optional embodiment further optimizes the startup auxiliary transaction model, including steps A1-A2:

[0049] A1. After receiving a transaction start command and a first global transaction identifier through a first session, a first stand-alone transaction is started, and a global transaction identifier attribute of the first stand-alone transaction is set to the first global transaction identifier, and a primary transaction attribute and an auxiliary transaction attribute of the first stand-alone transaction are set to be empty.

[0050] In this embodiment, the first session can be specifically understood as a session between a stand-alone database and a distributed database portal, which is used to send data to the stand-alone database. The transaction start command can be specifically understood as a command instructing to start a stand-alone transaction. The first global transaction identifier is a global transaction identifier, which can uniquely identify a transaction and is used to distinguish all transactions created in a distributed database. The global transaction identifier is maintained by the distributed database portal. The global transaction identifier attribute is used to store the global transaction identifier. The main transaction attribute is used to store the main transaction to which the stand-alone transaction belongs, and the auxiliary transaction attribute is used to store the auxiliary transaction to which the stand-alone transaction belongs.

[0051] The embodiment of the present application adds a main transaction attribute and an auxiliary transaction attribute to a stand-alone transaction. When both are empty, it means that the transaction is a normal transaction; when the main transaction attribute is not empty, it means that the transaction is an auxiliary transaction, and the main transaction attribute stores the identifier of the main transaction to which the transaction belongs; when the auxiliary transaction attribute is not empty, it means that the transaction is a main transaction, and the auxiliary transaction attribute stores the identifier of the auxiliary transaction corresponding to the main transaction. Both cannot be empty at the same time.

[0052] When the first stand-alone transaction does not exist between the stand-alone database and the distributed database portal, the distributed database portal generates a transaction start command and a first global transaction identifier, and sends the transaction start command and the first global transaction identifier to the corresponding stand-alone database through the first session. After receiving the transaction start command and the first global transaction identifier through the first session, the stand-alone database starts the first stand-alone transaction, sets the global transaction identifier attribute of the first stand-alone transaction to the first global transaction identifier, and sets the primary transaction attribute and the auxiliary transaction attribute of the first stand-alone transaction to be empty.

[0053] A2. Start the auxiliary transaction and use the auxiliary transaction as the auxiliary transaction of the first stand-alone transaction. Use the first stand-alone transaction as the main transaction and form an auxiliary transaction model based on the main transaction and the auxiliary transaction.

[0054] The stand-alone database starts a transaction again as an auxiliary transaction. The stand-alone database can start the auxiliary transaction automatically or by receiving a command. The auxiliary transaction is used as an auxiliary transaction of the first stand-alone transaction, and the first stand-alone transaction is used as the main transaction. An auxiliary transaction model is formed based on the main transaction and the auxiliary transaction.

[0055] As an optional embodiment, this optional embodiment further optimizes the auxiliary transaction model started based on the first stand-alone transaction into steps B1-B3:

[0056] B1. If the main transaction attribute and the auxiliary transaction attribute of the first stand-alone transaction are both empty, start the auxiliary transaction and use the auxiliary transaction as the auxiliary transaction of the first stand-alone transaction. Use the first stand-alone transaction as the main transaction and form an auxiliary transaction model based on the main transaction and the auxiliary transaction.

[0057] When there is a first stand-alone transaction between the stand-alone database and the distributed database entry, the first stand-alone transaction can be a common transaction or a transaction in the auxiliary transaction model. The type of the first stand-alone transaction is determined according to the main transaction attribute and the auxiliary transaction attribute of the first stand-alone transaction.

[0058] If the main transaction attribute and auxiliary transaction attribute of the first stand-alone transaction are both empty, the first stand-alone transaction is an ordinary transaction. The stand-alone database can automatically start the auxiliary transaction, or receive a command from the distributed database entry to start the auxiliary transaction. After starting the auxiliary transaction, the stand-alone database uses the auxiliary transaction as the auxiliary transaction of the first stand-alone transaction, and uses the first stand-alone transaction as the main transaction, and forms an auxiliary transaction model based on the main transaction and the auxiliary transaction.

[0059] B2. If the main transaction attribute of the first stand-alone transaction is empty and the auxiliary transaction attribute is not empty, determine the first stand-alone transaction as the main transaction, determine the auxiliary transaction corresponding to the auxiliary transaction attribute, and form an auxiliary transaction model based on the main transaction and the auxiliary transaction.

[0060] If the main transaction attribute of the first stand-alone transaction is empty and the auxiliary transaction attribute is not empty, the first stand-alone transaction can be determined to be the main transaction, the transaction identifier is read from the auxiliary transaction attribute, and the transaction corresponding to the identifier is used as the auxiliary transaction of the main transaction, and an auxiliary transaction model is formed according to the main transaction and the auxiliary transaction.

[0061] B3. If the main transaction attribute of the first stand-alone transaction is not empty and the auxiliary transaction attribute is empty, determine that the first stand-alone transaction is an auxiliary transaction, determine the main transaction corresponding to the main transaction attribute, and form an auxiliary transaction model based on the main transaction and the auxiliary transaction.

[0062] If the main transaction attribute of the first stand-alone transaction is not empty and the auxiliary transaction attribute is empty, the first stand-alone transaction is determined to be an auxiliary transaction, the transaction identifier is read from the main transaction attribute, the transaction corresponding to the identifier is used as the main transaction, and an auxiliary transaction model is formed according to the main transaction and the auxiliary transaction.

[0063] As an optional embodiment, this optional embodiment further starts an auxiliary transaction and uses the auxiliary transaction as an auxiliary transaction of the first stand-alone transaction, and optimizes the steps C1-C2:

[0064] C1. Receive an auxiliary transaction start command and a second global transaction identifier through a second session, start an auxiliary transaction, and set the main transaction attribute and the auxiliary transaction attribute of the auxiliary transaction to empty.

[0065] In this embodiment, the second session can be specifically understood as a session between a stand-alone database and a distributed database entry, which is used to send data to the stand-alone database, and the first session and the second session are two different sessions. The auxiliary transaction start command can be specifically understood as a command to instruct the start of an auxiliary transaction. The second global transaction identifier is a global transaction identifier, and the second global transaction identifier is used to indicate the identifier of the main transaction of the auxiliary transaction, that is, when the distributed database entry instructs the stand-alone database to start an auxiliary transaction, it needs to send the second global transaction identifier to the stand-alone database, and the second global transaction identifier is the global transaction identifier of the main transaction corresponding to this auxiliary transaction. The stand-alone database receives the auxiliary transaction start command and the second global transaction identifier sent by the distributed database entry through the second session, starts the auxiliary transaction according to the instruction of the auxiliary transaction start command, and sets the main transaction attribute and the auxiliary transaction attribute of the auxiliary transaction to empty.

[0066] C2. Match the second global transaction identifier with the global transaction identifier attributes of all started stand-alone transactions. If the matched stand-alone transaction is the first stand-alone transaction, use the auxiliary transaction as the auxiliary transaction of the first stand-alone transaction, and set the main transaction attribute of the auxiliary transaction to the identifier of the first stand-alone transaction.

[0067] Determine the global transaction identifier in the global transaction identifier attribute of all currently started stand-alone transactions, match the second global transaction identifier with all the obtained global transaction identifiers, that is, determine whether there is an identifier that is the same as the second global transaction identifier in the global transaction identifier. If so, determine that the match is successful, otherwise, the match fails. If the match is successful, determine whether the single-machine transaction that successfully matches is the first stand-alone transaction. If not, the auxiliary transaction of the first stand-alone transaction is not found at this time, and the auxiliary transaction is restarted; if so, the auxiliary transaction is used as the auxiliary transaction of the first stand-alone transaction, and the main transaction attribute of the auxiliary transaction is set to the identifier of the first stand-alone transaction, and the relationship between the auxiliary transaction and the main transaction is established. In the subsequent data interaction process, the auxiliary transaction model can be directly used. In the actual application process, when starting the auxiliary transaction of the first stand-alone transaction, the identifier of the first stand-alone transaction is sent to the stand-alone database as the second global transaction identifier, that is, the first global transaction identifier and the second global transaction identifier are the same, and the auxiliary transaction model can be started.

[0068] For a stand-alone database, when starting an auxiliary transaction of any transaction, it can be implemented by the following process: receiving the auxiliary transaction start command sent by the distributed database entry and the global transaction identifier of the corresponding main transaction (i.e., the second global transaction identifier) ​​through the session, starting the auxiliary transaction, and setting the main transaction and auxiliary transaction attributes to empty; matching the received global transaction identifier of the main transaction with the global transaction identifier attribute of the opened transaction, if a transaction with the same global transaction identifier is matched and its auxiliary transaction attribute is empty, setting the auxiliary transaction of the matched transaction to the auxiliary transaction started this time, and setting the main transaction attribute of the auxiliary transaction started this time to the identifier of the matched transaction.

[0069] In the embodiment of the present application, since the main transaction attribute and the auxiliary transaction attribute of the first stand-alone transaction are both empty, when determining that the successfully matched stand-alone transaction is the first stand-alone transaction, it is not necessary to determine whether the auxiliary transaction attribute of the first stand-alone transaction is empty. In actual application, it is also possible to determine whether the auxiliary transaction attribute of the first stand-alone transaction is empty again.

[0070] After the stand-alone transaction is started, multiple SQL statements for reading and writing data sent by the distributed database entry are sequentially received in its session. The session refers to a fixed thread for the stand-alone database to process this stand-alone transaction. Each time a SQL statement for reading and writing data is received, the current session will process this SQL statement and return the processing result to the distributed database entry that sent this SQL statement before receiving the next SQL statement for reading and writing data. The SQL statement described here can be the first statement to be executed or the second statement to be executed. The first statement to be executed and the second statement to be executed can read or write data in the following manner.

[0071] S202: parse the first statement to be executed through the auxiliary transaction to determine at least one data row to be read.

[0072] In this embodiment, the data rows to be read can be specifically understood as the data rows that need to be read. The stand-alone database parses the first statement to be executed through the auxiliary transaction, determines the data rows that need to be read, and uses this part of the data rows as the data rows to be read. The number of data rows to be read can be one row or multiple rows, usually multiple rows.

[0073] S203: For each data row to be read, apply for a row lock of the data row to be read through an auxiliary transaction. After the row lock application is successful, determine whether to add the data row to be read to the return set through the auxiliary transaction combined with the execution sequence number.

[0074] In this embodiment, the return set can be understood as a set storing data that needs to be fed back to the distributed database entry, and the data stored in the return set is the read data rows.

[0075] For each data row to be read, determine the row lock type corresponding to the data row to be read, apply for the row lock through the auxiliary transaction, and when applying for the row lock, determine whether the data row already has a row lock of the corresponding type. If it already exists, it can be directly used as the row lock used this time, or a new row lock can be created, etc. After the row lock application is successful, read the data row to be read through the auxiliary transaction. Before reading the data row, determine whether the data row to be read is written by the second statement to be executed corresponding to the operation statement executed this time through the execution sequence number. If so, determine not to read the data row to be read, and there is no need to add the data row to be read to the return set, and directly process the next data row to be read; if not, determine to read the data row to be read, and add the data row to be read to the return set, that is, add the data in the data row to be read to the return set, and continue to process the next data row to be read.

[0076] S204. When the number of data rows to be read in the returned set is equal to the streaming read threshold, all data rows to be read in the returned set are fed back to the distributed database entry through an auxiliary transaction until all data rows are read, wherein the data rows read in batches are processed by the distributed database entry and data to be updated is generated.

[0077] Real-time monitoring of the comparison results between the number of data rows to be read in the returned set and the size of the streaming reading threshold. For example, after each row of data rows to be read is added to the returned set, the number of data rows to be read in the returned set is cumulatively added by 1, and then the number of data rows to be read is compared with the size of the streaming reading threshold. If it is less than the streaming reading threshold, continue to determine whether to add the data rows to be read to the returned set; if it is equal to the streaming reading threshold, all the data rows to be read in the returned set are fed back to the distributed database entry through an auxiliary transaction, and the distributed database entry performs data processing and generates data to be updated. Determine whether all data rows have been read. If not, continue to read data rows until all data rows have been read, thereby realizing the return of data rows to the distributed database entry in batches. After all data rows have been read, the stand-alone database can notify the distributed database entry that all data rows have been read. When returning data rows, when the last batch of data rows to be read is read, the number of data rows to be read may be less than the streaming reading threshold. If it has been detected that all data rows have been read, all data rows to be read in the return set can be directly returned to the distributed database entry. That is, when returning the read data rows to the distributed database entry, the embodiment of the present application can return when at least one of the number of data rows to be read in the return set is equal to the streaming reading threshold and all data rows have been read.

[0078] It should be noted that when applying for a row lock and reading a data row to be read, the embodiment of the present application may apply for a row lock for the data row to be read when the data row to be read is read; or row locks for all data rows to be processed may be applied first, and then read in sequence.

[0079] As an optional embodiment, this optional embodiment further optimizes the process of applying for row locks of the data rows to be read through auxiliary transactions into steps D1-D2:

[0080] D1. Determine the first target type of the row lock according to the first statement to be executed;

[0081] In this embodiment, the first target type can be understood as a type of row lock, which is determined according to the first statement to be executed. Preset the row lock type corresponding to the first statement to be executed of different types or meanings, parse the first statement to be executed, determine the specific meaning, type and other information of the first statement to be executed, and select the appropriate row lock type as the first target type. The embodiment of the present application does not limit the first target type, which can be a read lock, write lock, etc. Different types have different levels, and there are high and low levels. The specific definition rules are not limited. Generally, the level of the write lock is higher than the level of the read lock.

[0082] D2. Determine whether the auxiliary transaction has created the first row lock instance corresponding to the data row to be read and whether the type level of the first row lock instance is not lower than the level of the first target type. If so, use the first row lock instance as the row lock of the data row to be read; otherwise, obtain the main transaction attribute of the auxiliary transaction, and apply for the row lock of the data row to be read according to the main transaction attribute of the auxiliary transaction.

[0083] In this embodiment, the first row lock instance can be understood as a row lock instance. Each transaction includes a row lock instance queue attribute, which includes row lock instances of all data rows created by the transaction, and the row lock instance includes a type attribute. The row lock instance queue of the auxiliary transaction is traversed to determine whether the auxiliary transaction has created a row lock instance of the data row to be read. If it has, this row lock instance is recorded as the first row lock instance, the level of the type of the first row lock instance is determined, and the level of the type of the first row lock instance is compared with the level of the first target type. If the level of the type of the first row lock instance is not lower than the level of the first target type, the first row lock instance is used as the row lock of the data row to be read, that is, the auxiliary transaction has created the first row lock instance of the data row to be read and the level of the type of the first row lock instance is the same as or higher than the level of the first target type, and the first row lock instance is used as the row lock of the data row to be read, and the application of the row lock of the data row to be read is completed; otherwise, the main transaction attribute of the auxiliary transaction is obtained, and it is determined according to the main transaction attribute of the auxiliary transaction whether the main transaction to which the auxiliary transaction belongs applies for the row lock of the data row to be read, and the row lock of the data row to be read is applied according to the judgment result.

[0084] As an optional embodiment, this optional embodiment further applies for a row lock of the data row to be read according to the main transaction attribute of the auxiliary transaction, and optimizes it into steps E1-E3:

[0085] E1. Determine whether the main transaction attribute of the auxiliary transaction is not empty and the main transaction has created a second row lock instance corresponding to the data row to be read and the type level of the second row lock instance is not lower than the level of the first target type. If so, execute E2; otherwise, execute E3.

[0086] In this embodiment, the second row lock instance can be understood as a row lock instance. Determine whether the main transaction attribute of the auxiliary transaction is empty. If not, obtain the main transaction identifier from the main transaction attribute to determine the main transaction of the auxiliary transaction. When the main transaction of the auxiliary transaction has created a row lock instance for the data row to be read, this row lock instance is recorded as the second row lock instance. If the level of the type of the second row lock instance is not lower than the level of the first target type, execute E2 to create a row lock instance for the data row to be read; otherwise, execute E3 to create a row lock instance for the data row to be read.

[0087] E2. Create a first target row lock instance for the data row to be read, add the first target row lock instance to the row lock instance queue of the auxiliary transaction, and add the first target row lock instance to the global row lock instance list for the data row to be read.

[0088] In this embodiment, the first target row lock instance can be specifically understood as a row lock instance, and the type attribute of the first target row lock instance is the first target type. Create a row lock instance, set the type attribute of this row lock instance to the first target type, use this row lock instance as the first target row lock instance, and use the first target row lock instance as the row lock instance of the data row to be read, so as to realize the creation of the row lock instance of the data row to be read. Add the first target row lock instance to the row lock instance queue of the auxiliary transaction; each data row has an independent global row lock instance list, add the first target row lock instance to the global row lock instance list of the data row to be read, and maintain all global row lock instances of the data row to be read.

[0089] E3. Traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be read. If other transactions have created or are waiting for the third row lock instance corresponding to the data row to be read and the type of the third row lock instance conflicts with the first target type, add the auxiliary transaction to the global row lock waiting queue waiting for the data row to be read, set the waiting lock type attribute of the auxiliary transaction to the first target type, set the waiting transaction attribute of the auxiliary transaction according to the conflicting transaction, and control the session corresponding to the auxiliary transaction to sleep; if no other transaction has created or is waiting for the third row lock instance corresponding to the data row to be read or the type of the third row lock instance conflicts with the first target type, create the first target row lock instance for the data row to be read, add the first target row lock instance to the row lock instance queue of the auxiliary transaction, and add the first target row lock instance to the global row lock instance list for the data row to be read.

[0090] In step E3, other transactions refer to other transactions that are not the main transaction of the auxiliary transaction, that is, other transactions refer to non-auxiliary transactions and transactions other than the main transaction of the non-auxiliary transaction. The global row lock waiting queue can be understood as a queue in a stand-alone database waiting for row locks to be applied for data rows. The type of the waiting row lock is stored in the waiting lock type attribute, which is used to describe the type of row lock that the transaction is waiting for; the waiting transaction attribute stores the waiting transaction, for example, the transaction identifier, which is used to indicate the transaction that the current transaction is waiting for.

[0091] Traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be read, determine whether there are non-auxiliary transactions and other transactions other than the main transaction of the non-auxiliary transaction that have created or waited for the row lock instance of the data row to be read, if so, record this row lock instance as the third row lock instance, determine whether the type of the third row lock instance conflicts with the first target type, if there is a conflict, add the auxiliary transaction to the global row lock waiting queue of the data row to be read, the global row lock waiting queue records all transactions that apply for row locks and operate on the data row to be read; set the waiting lock type attribute of the auxiliary transaction to the first target type, determine a conflicting transaction or all transactions, determine to set the waiting transaction attribute of the auxiliary transaction according to the conflicting transactions, for example, set the waiting transaction attribute of the auxiliary transaction to the first transaction found that has a lock conflict with the auxiliary transaction, control the session corresponding to the auxiliary transaction to enter sleep, wait for other sessions to wake up or sleep timeout, the timeout period can be configured through system parameters.

[0092] The conflict determines the specific row lock type that different transactions can apply for the same data row at the same time. The specific logic is not limited. Generally, there is no conflict if different transactions apply for read locks, but there is a conflict if different transactions apply for write locks or one transaction applies for write locks and another transaction applies for read locks. If no other transaction has created or is waiting for the third row lock instance corresponding to the data row to be read, or if other transactions have created or are waiting for the third row lock instance corresponding to the data row to be read and the type of the third row lock instance does not conflict with the first target type, create a row lock instance, set the type attribute of this row lock instance to the first target type, use this row lock instance as the first target row lock instance, and use the first target row lock instance as the row lock instance for the data row to be read, so as to create the row lock instance for the data row to be read. Add the first target row lock instance to the row lock instance queue of the auxiliary transaction, add the first target row lock instance to the global row lock instance list of the data row to be read, and maintain all global row lock instances for the data row to be read.

[0093] As an optional embodiment, this optional embodiment further determines whether to add the to-be-read data row to the return set by combining the auxiliary transaction with the execution sequence number, and optimizes it to F1-F2:

[0094] F1. Obtain the last modification transaction of the data row to be read through the auxiliary transaction. If the last modification transaction is not the main transaction of the auxiliary transaction, add the data row to be read to the return set through the auxiliary transaction.

[0095] When each data row is modified by a transaction, the transaction that last modified the data row is recorded. The transaction may be recorded by recording the transaction identifier or other information that uniquely identifies the transaction, which is not limited in the embodiments of the present application. For example, a last modified transaction attribute is added to the data row, and the identifier of the transaction that last modified the data row is recorded through the last modified transaction attribute. Whenever a transaction modifies the data row, the information in the last modified transaction attribute of the data row is updated. The last modified transaction of the data row to be read is obtained through the auxiliary transaction, and it is determined whether the last modified transaction is the main transaction of the auxiliary transaction. If so, F2 is executed; if not, the data row to be read is added to the return set through the auxiliary transaction.

[0096] F2. If the last modification transaction is the main transaction of the auxiliary transaction, the modification sequence number of the data row to be read is obtained through the auxiliary transaction. If the modification sequence number is less than the execution sequence number, the data row to be read is added to the return set through the auxiliary transaction.

[0097] If the last modification transaction is the main transaction of the auxiliary transaction, it is necessary to determine whether the data in the data row to be read is written by the second statement to be executed in parallel with the first statement to be executed. The judgment method is as follows: obtain the modification sequence number of the data row to be read through the auxiliary transaction. If the modification sequence number is less than the execution sequence number, it is determined that the data in the data row to be read is not written by the second statement to be executed in parallel with the first statement to be executed, and the data row to be read is added to the return set through the auxiliary transaction; if the modification sequence number is not less than the execution sequence number, it can be determined that the data in the data row to be read is modified. When the auxiliary transaction executes the first data to be executed to read the data row to be read, the data should not be modified by the second statement to be executed or other database statements after the second statement to be executed. Therefore, there is no need to add this data row to be read to the return set.

[0098] S205. The data to be updated and the second statement to be executed fed back by the distributed database entry are received in sequence through the main transaction in the auxiliary transaction model, and the second statement to be executed is parsed through the main transaction to determine at least one data row to be written.

[0099] In this embodiment, the data rows to be written can be understood as the data rows that need to be written. The stand-alone database receives the data to be updated and the second statement to be executed through the main transaction. Since the auxiliary transaction reads the data rows in batches, the distributed database entry processes the data in batches accordingly, and returns the data and the second statement to be executed to the main transaction in batches. Therefore, each time the main transaction receives the data to be updated and the second statement to be executed, it needs to write the data to be updated. The stand-alone database parses the second statement to be executed through the main transaction, analyzes the specific meaning of the second statement to be executed, and determines the data rows to be written that need to be written. The number of data rows to be written can be one or more, usually more than one.

[0100] S206: For each data row to be written, apply for a row lock of the data row to be written through the main transaction, and write the data to be updated through the main transaction based on the applied row lock.

[0101] For each data row to be written, determine the row lock type corresponding to the data row to be written, apply for the row lock through the main transaction, and when applying for the row lock, determine whether the data row already has a row lock of the corresponding type. If it already exists, it can be directly used as the row lock for this time, or a new row lock can be created, etc. After the row lock application is successful, the data to be updated is written to the corresponding data row to be written through the main transaction to complete the data writing operation.

[0102] As an optional embodiment, this optional embodiment further optimizes the row lock of the data row to be written applied for by the main transaction to G1-G2:

[0103] G1. Determine a second target type of the row lock according to the second statement to be executed.

[0104] In this embodiment, the second target type can be understood as a type of row lock, which is determined according to the second statement to be executed. The row lock type corresponding to the second statement to be executed of different types or meanings is pre-set, the second statement to be executed is parsed, the specific meaning, type and other information of the second statement to be executed are determined, and the appropriate row lock type is selected as the second target type. The embodiment of the present application does not limit the second target type, which can be a read lock, a write lock, etc. Different types have different levels, and there are high and low levels. The specific definition rules are not limited. Generally, the level of the write lock is higher than the level of the read lock.

[0105] G2. Determine whether the main transaction has created the fourth row lock instance corresponding to the data row to be written and the type level of the fourth row lock instance is not lower than the level of the second target type. If so, use the fourth row lock instance as the row lock for the data row to be written; otherwise, obtain the auxiliary transaction attributes of the main transaction, and apply for the row lock for the data row to be written according to the auxiliary transaction attributes of the main transaction.

[0106] In this embodiment, the fourth row lock instance can be understood as a row lock instance. Each transaction includes a row lock instance queue attribute, which includes row lock instances of all data rows created by the transaction, and the row lock instance includes a type attribute. The row lock instance queue of the main transaction is traversed to determine whether the main transaction has created a row lock instance of the data row to be written. If it has been created, this row lock instance is recorded as the fourth row lock instance, the level of the type of the fourth row lock instance is determined, and the level of the type of the fourth row lock instance is compared with the level of the second target type. If the level of the type of the fourth row lock instance is not lower than the level of the second target type, the fourth row lock instance is used as the row lock of the data row to be written, that is, the main transaction has created the fourth row lock instance of the data row to be written and the level of the type of the fourth row lock instance is the same as or higher than the level of the second target type, and the fourth row lock instance is used as the row lock of the data row to be written, and the application of the row lock of the data row to be written is completed; otherwise, the auxiliary transaction attribute of the main transaction is obtained, and it is determined whether the auxiliary transaction of the main transaction applies for the row lock of the data row to be written according to the auxiliary transaction attribute of the main transaction, and the row lock of the data row to be written is applied according to the judgment result.

[0107] As an optional embodiment, this optional embodiment further applies for a row lock of the data row to be written according to the auxiliary transaction attribute of the main transaction, and is optimized as follows:

[0108] H1. Determine whether the auxiliary transaction attribute of the main transaction is not empty and whether the auxiliary transaction has created the fifth row lock instance corresponding to the data row to be written and the type level of the fifth row lock instance is not lower than the level of the second target type. If so, execute H2; otherwise, execute H23.

[0109] In this embodiment, the fifth row lock instance can be understood as a row lock instance. Determine whether the auxiliary transaction attribute of the main transaction is empty. If not, obtain the auxiliary transaction identifier from the auxiliary transaction attribute to determine the auxiliary transaction of the main transaction. When the auxiliary transaction of the main transaction has created a row lock instance for the data row to be written, this row lock instance is recorded as the fifth row lock instance. If the level of the type of the fifth row lock instance is not lower than the level of the second target type, execute E2 to create a row lock instance for the data row to be written; otherwise, execute E3 to create a row lock instance for the data row to be written.

[0110] H2. Create a second target row lock instance for the data row to be written, add the second target row lock instance to the row lock instance queue of the main transaction, and add the second target row lock instance to the global row lock instance list for the data row to be written.

[0111] In this embodiment, the second target row lock instance can be specifically understood as a row lock instance, and the type attribute of the second target row lock instance is the second target type. Create a row lock instance, set the type attribute of this row lock instance to the second target type, use this row lock instance as the second target row lock instance, and use the second target row lock instance as the row lock instance of the data row to be written, so as to create the row lock instance of the data row to be written. Add the second target row lock instance to the row lock instance queue of the main transaction, add the second target row lock instance to the global row lock instance list of the data row to be written, and maintain all global row lock instances of the data row to be written.

[0112] H3. Traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be written. If there is other transaction that has created or is waiting for the sixth row lock instance corresponding to the data row to be written and the type of the sixth row lock instance conflicts with the second target type, add the main transaction to the global row lock waiting queue waiting for the data row to be written, set the waiting lock type attribute of the main transaction to the second target type, set the waiting transaction attribute of the main transaction according to the conflicting transaction, and control the session corresponding to the main transaction to sleep; if there is no other transaction that has created or is waiting for the sixth row lock instance corresponding to the data row to be written or the type of the sixth row lock instance conflicts with the second target type, create the second target row lock instance for the data row to be written, add the second target row lock instance to the row lock instance queue of the main transaction, and add the second target row lock instance to the global row lock instance list for the data row to be written.

[0113] In step H3, other transactions refer to other transactions that are not auxiliary transactions of the main transaction, that is, other transactions refer to transactions other than the non-main transaction and the auxiliary transactions of the non-main transaction.

[0114] Traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be written, determine whether there are other transactions other than the non-main transaction and the auxiliary transaction of the non-main transaction that have created or are waiting for the row lock instance of the data row to be written, if so, record this row lock instance as the sixth row lock instance, determine whether the type of the sixth row lock instance conflicts with the second target type, if there is a conflict, add the main transaction to the global row lock waiting queue of the data row to be written, the global row lock waiting queue records all transactions waiting to apply for row locks and operate on the data row to be written; set the waiting lock type attribute of the main transaction to the second target type, determine a conflicting transaction or all transactions, determine to set the waiting transaction attribute of the main transaction according to the conflicting transactions, for example, set the waiting transaction attribute of the main transaction to the first transaction found that has a lock conflict with the main transaction, control the session corresponding to the main transaction to enter sleep, wait for other sessions to wake up or sleep timeout, the timeout time can be configured through system parameters.

[0115] The conflict determines the specific row lock type that different transactions can apply for the same data row at the same time. The specific logic is not limited. Generally, there is no conflict if different transactions apply for read locks, but there is a conflict if different transactions apply for write locks or one transaction applies for write locks and another transaction applies for read locks. If no other transaction has created or is waiting for the sixth row lock instance corresponding to the data row to be written, or if other transactions have created or are waiting for the sixth row lock instance corresponding to the data row to be written and the type of the sixth row lock instance does not conflict with the second target type, create a row lock instance, set the type attribute of this row lock instance to the second target type, use this row lock instance as the second target row lock instance, and use the second target row lock instance as the row lock instance for the data row to be written, so as to create the row lock instance for the data row to be written. Add the second target row lock instance to the row lock instance queue of the main transaction, add the second target row lock instance to the global row lock instance list of the data row to be written, and maintain all global row lock instances for the data row to be written.

[0116] As an optional embodiment, this optional embodiment is further optimized to include I1-I3:

[0117] I1. When the current transaction is committed, a pending data row is selected from the row lock instances of all pending data rows applied for by the current transaction as the data row to be deleted, and a row lock instance of the data row to be deleted is used as the row lock instance to be deleted.

[0118] Among them, the current transaction is a main transaction or an auxiliary transaction. When the current transaction is a main transaction, the data rows to be processed are data rows to be written. When the current transaction is an auxiliary transaction, the data rows to be processed are data rows to be read.

[0119] In this embodiment, the current transaction can be understood as the currently executed transaction, and the current transaction can be the main transaction or the auxiliary transaction in the auxiliary transaction model; the data rows to be processed can be understood as the data rows that need to be processed when the transaction is committed; the data rows to be deleted can be understood as the data rows that need to delete the row lock instance; the row lock instance to be deleted can be understood as the row lock instance that needs to be deleted. Both the main transaction and the auxiliary transaction can be committed after execution, and the submission process of the main transaction and the auxiliary transaction is the same; the embodiment of the present application can submit the main transaction as the current transaction, and the data rows to be processed are the data rows to be written. The embodiment of the present application can also submit the auxiliary transaction as the current transaction, and the data rows to be processed are the data rows to be read.

[0120] Determine the transaction that needs to be committed currently. Usually, it can be determined based on user operations, and the main transaction or auxiliary transaction that needs to be committed is used as the current transaction. Determine all row lock instances of all pending data rows applied by the current transaction, and select one data row from the pending data rows as the data row to be deleted. For example, select data rows from the front to the back in the order of each data row as the data row to be deleted. Select one row lock instance from all row lock instances of the data row to be deleted as the row lock instance to be deleted. When selecting the row lock instance to be deleted, all row lock instances of the data row to be deleted can be found from the row lock instance queue of the transaction, and each row lock instance is selected in turn as the row lock instance to be deleted.

[0121] I2. Traverse the global row lock instance list of the data row to be deleted, delete the row lock instance to be deleted from the global row lock instance list of the data row to be deleted, and delete the row lock instance to be deleted from the row lock instance queue of the current transaction.

[0122] Traverse the global row lock instance list of the data row to be deleted, determine the row lock instance in the global row lock instance list that matches the row lock instance to be deleted, and delete it from the global row lock instance list; at the same time, traverse the row lock instance queue of the current transaction, determine the row lock instance that matches the row lock instance to be deleted, and delete it from the row lock instance queue.

[0123] I3. Traverse the global row lock waiting queue of the data row to be deleted, determine the transaction waiting for the lock in the global row lock waiting queue, and wake up the session where the transaction waiting for the lock is located, until all row lock instances of all pending data rows applied by the current transaction are deleted.

[0124] Among them, the transaction that the transaction waiting for the lock is waiting for is the current transaction.

[0125] The global row lock waiting queue of the data row to be deleted is traversed. The global row lock waiting queue records the transactions waiting to apply for row locks for the data row to be deleted. The transaction that each transaction is waiting for can be determined according to the waiting transaction attribute. The waiting transaction attribute is used to determine that the waiting transaction is a transaction of the current transaction. This transaction is the transaction waiting for the lock. The session where the transaction waiting for the lock is located is awakened until all row lock instances of all data rows to be read applied by the current transaction are deleted.

[0126] When a stand-alone transaction is committed, all row lock instances of all data rows applied by the current transaction will be deleted. The main process of deleting each row lock instance is as follows: delete it from the global row lock instance list of this data row and delete it from the row lock instance queue of the transaction. After deleting a row lock instance of a data row, the global row lock waiting queue of this data row will be traversed to wake up the session of the stand-alone transaction waiting for the lock. For each transaction waiting_trx waiting for the lock, if the transaction it is waiting for is the current transaction, the session of the transaction waiting for the lock will be woken up.

[0127] As an optional embodiment, this optional embodiment further wakes up the session where the transaction waiting for the lock is located, and is optimized as follows:

[0128] J1. Obtain the waiting lock type corresponding to the transaction waiting for the lock according to the waiting lock type attribute of the transaction waiting for the lock.

[0129] In this embodiment, the waiting lock type can be understood as a row lock type. The waiting lock type attribute of the transaction waiting for the lock is read, and the row lock type stored in the waiting lock type attribute is obtained and used as the waiting lock type, that is, the row lock type that the transaction waiting for the lock is waiting for is the waiting lock type.

[0130] J2. Determine whether the main transaction attribute or the auxiliary transaction attribute of the transaction waiting for the lock is not empty and the main transaction or the auxiliary transaction of the transaction waiting for the lock has created the seventh row lock instance of the data row to be deleted and the type level of the seventh row lock instance is not lower than the level of the waiting lock type. If so, create the third target row lock instance through the transaction waiting for the lock; otherwise, if no other transaction has created the eighth row lock instance of the data row to be deleted and the type of the eighth row lock instance conflicts with the waiting lock type, create the third target row lock instance through the transaction waiting for the lock.

[0131] In step J2, other transactions refer to transactions that do not include the main transaction or auxiliary transaction of the transaction waiting for the lock. If the transaction waiting for the lock is the main transaction, other transactions do not include the auxiliary transaction of the transaction waiting for the lock; if the transaction waiting for the lock is an auxiliary transaction, other transactions do not include the main transaction of the transaction waiting for the lock. The seventh row lock instance, the third target row lock instance, and the eighth row lock instance are all a type of row lock instance.

[0132] Determine whether the main transaction attribute or auxiliary transaction attribute of the transaction waiting for the lock is empty. When the transaction waiting for the lock is the main transaction, its main transaction attribute is empty and the auxiliary transaction attribute is not empty. When the transaction waiting for the lock is the auxiliary transaction, its main transaction attribute is not empty and the auxiliary transaction attribute is empty. The following describes how to create a row lock instance when the transaction waiting for the lock is the main transaction and the auxiliary transaction:

[0133] When the transaction waiting for the lock is the main transaction, its auxiliary transaction attribute is not empty. The auxiliary transaction attribute is read to determine its auxiliary transaction, and it is determined whether the auxiliary transaction of the transaction waiting for the lock has created a row lock instance for the data row to be deleted. If it has, this row lock instance is recorded as the seventh row lock instance. If the level of the type of the seventh row lock instance is not lower than the level of the waiting lock type, the third target row lock instance is created through the transaction waiting for the lock; if it has not been created or the level of the type of the seventh row lock instance is lower than the level of the waiting lock type, it is determined whether there are other transactions that have created the row lock instance for the data row to be deleted. If so, this row lock instance is recorded as the eighth row lock instance. If the type of the eighth row lock instance conflicts with the waiting lock type, the processing flow of the transaction waiting for the lock is terminated; if no other transaction has created the eighth row lock instance for the data row to be deleted or the type of the eighth row lock instance does not conflict with the waiting lock type, the third target row lock instance is created through the transaction waiting for the lock.

[0134] When the transaction waiting for the lock is an auxiliary transaction, its main transaction attribute is not empty. The main transaction attribute is read to determine its main transaction, and it is determined whether the main transaction of the transaction waiting for the lock has created a row lock instance for the data row to be deleted. If it has, this row lock instance is recorded as the seventh row lock instance. If the level of the type of the seventh row lock instance is not lower than the level of the waiting lock type, the third target row lock instance is created through the transaction waiting for the lock; if the seventh row lock instance has not been created or the level of the type of the seventh row lock instance is lower than the level of the waiting lock type, it is determined whether there are other transactions that have created the row lock instance for the data row to be deleted. If so, this row lock instance is recorded as the eighth row lock instance. If the type of the eighth row lock instance conflicts with the waiting lock type, the processing flow of the transaction waiting for the lock is terminated; if no other transaction has created the eighth row lock instance for the data row to be deleted or the type of the eighth row lock instance does not conflict with the waiting lock type, the third target row lock instance is created through the transaction waiting for the lock.

[0135] J3. Set the type attribute of the third target row lock instance to the waiting lock type, add the third target row lock instance to the row lock instance queue of the transaction waiting for the lock, add the third target row lock instance to the global row lock instance list of the data row to be deleted, delete the transaction waiting for the lock from the global row lock waiting list, and wake up the session where the transaction waiting for the lock is located.

[0136] As an optional embodiment, this optional embodiment is further optimized to include K1-K2:

[0137] K1. If the main transaction attribute or the auxiliary transaction attribute of the transaction waiting for the lock is not empty, the main transaction or the auxiliary transaction of the transaction waiting for the lock is in the row lock waiting queue of the data row to be deleted and the waiting transaction is not the current transaction and the level of the target row lock type being waited for is not higher than the waiting lock type, create a fourth target row lock instance through the main transaction or the auxiliary transaction of the transaction waiting for the lock.

[0138] In this embodiment, the target row lock type can be understood as the type of row lock that the transaction waiting for the lock is waiting to apply for, which is also a row lock type. The fourth target row lock instance is a row lock instance.

[0139] The following describes the row lock instance creation process according to whether the transaction waiting for the lock is the main transaction or the auxiliary transaction:

[0140] When the transaction waiting for the lock is the main transaction, its auxiliary transaction attribute is not empty. The auxiliary transaction attribute is read to obtain the auxiliary transaction of the transaction waiting for the lock. If the auxiliary transaction is in the row lock waiting queue of the data row to be deleted and is not waiting for the current transaction, the waiting lock type attribute of the auxiliary transaction is read to obtain the target row lock type that the auxiliary transaction is waiting for. If the level of the target row lock type is not higher than the waiting lock type, the fourth target row lock instance is created through the auxiliary transaction of the transaction waiting for the lock.

[0141] When the transaction waiting for the lock is an auxiliary transaction, its main transaction attribute is not empty. The main transaction attribute is read to obtain the main transaction of the transaction waiting for the lock. If the main transaction is in the row lock waiting queue of the data row to be deleted and is not waiting for the current transaction, the waiting lock type attribute of the main transaction is read to obtain the target row lock type that the main transaction is waiting for. If the level of the target row lock type is not higher than the waiting lock type, the fourth target row lock instance is created through the auxiliary transaction of the transaction waiting for the lock.

[0142] K2. Set the type attribute of the fourth target row lock instance to the target row lock type, add the fourth target row lock instance to the row lock instance queue of the main transaction or auxiliary transaction of the transaction waiting for the lock, add the fourth target row lock instance to the global row lock instance list of the data row to be deleted, delete the main transaction or auxiliary transaction of the transaction waiting for the lock from the global row lock wait list, and wake up the session where the main transaction or auxiliary transaction of the transaction waiting for the lock is located.

[0143] The following describes the session wakeup process according to whether the transaction waiting for the lock is the main transaction or the auxiliary transaction:

[0144] When the transaction waiting for the lock is the main transaction, set the type attribute of the fourth target row lock instance to the target row lock type, add the fourth target row lock instance to the row lock instance queue of the auxiliary transaction of the transaction waiting for the lock, add the fourth target row lock instance to the global row lock instance list of the data row to be deleted, delete the auxiliary transaction of the transaction waiting for the lock from the global row lock wait list, and wake up the session where the auxiliary transaction of the transaction waiting for the lock is located.

[0145] When the transaction waiting for the lock is an auxiliary transaction, set the type attribute of the fourth target row lock instance to the target row lock type, add the fourth target row lock instance to the row lock instance queue of the main transaction of the transaction waiting for the lock, add the fourth target row lock instance to the global row lock instance list of the data row to be deleted, delete the main transaction of the transaction waiting for the lock from the global row lock wait list, and wake up the session where the main transaction of the transaction waiting for the lock is located.

[0146] As an optional embodiment, this optional embodiment is further optimized to include: after writing the data row, updating the modification sequence number attribute of the written data row to the execution sequence number, and updating the last modification transaction attribute of the written data row to the main transaction.

[0147] In this embodiment, the modification sequence number attribute is used to record the execution sequence number of the database statement that last modified the data row; the last modification transaction attribute is used to record the transaction that last modified the data row.

[0148] The second statement to be executed is executed in parallel through the main transaction. After the data to be updated is written to the corresponding data row, the latest operation of writing to the data row is recorded. The execution sequence number of this execution is written to the modified sequence number attribute of the data row, overwriting the sequence number originally recorded in the modified sequence number attribute, and completing the update of the sequence number. The execution sequence number can be sent to the main transaction by the distributed database entry. For example, the distributed database entry sends the execution sequence number while sending the second statement to be executed; the main transaction is written to the last modified transaction attribute of the data row, overwriting the transaction originally recorded in the last modified transaction attribute, and completing the update of the transaction. The last modified transaction attribute can store the transaction identifier.

[0149] The auxiliary transaction model solves the following problems by combining shared row locks and execution sequence numbers:

[0150] 1. Lock conflicts between auxiliary transactions and their corresponding main transactions when they are executed in parallel; the data rows that the auxiliary transaction writes and reads can be deleted, modified, or read and written by the main transaction in parallel, but other transactions cannot delete, modify, or read and write in parallel. The data rows that the main transaction writes or reads and writes can be read and locked by the auxiliary transaction in parallel, but other transactions cannot read and lock in parallel.

[0151] 2. Solved the problem of duplicate reading of auxiliary transactions; when auxiliary transactions are read with streaming write locks, the data inserted by insert_sql generated by the same operation statement can be locked successfully (it will not go to sleep due to waiting for the lock), but it cannot be read.

[0152] Take cur_sql, whose operation statement is to update the value of the shard key column, as an example to illustrate the distributed transaction execution process:

[0153] a. Get the execution sequence number cur_sql_sn of the operation statement cur_sql; the execution sequence number cur_sql is the SQL number executed in the distributed transaction; specifically, the execution sequence number is the SQL number sent by the user in the current transaction in the current session.

[0154] b. Gate parses cur_sql and obtains the single-machine database set single_db_list that stores all the data rows to be updated.

[0155] c. Gate generates a read_sql_with_lock for each stand-alone database in single_db_list, which is the first statement to be executed.

[0156] d. Gate executes the following steps in parallel on each stand-alone database in single_db_list:

[0157] a). If the auxiliary transaction model is not enabled on a stand-alone database, enable the auxiliary transaction model. The auxiliary transaction model includes one-to-one corresponding main transactions and auxiliary transactions.

[0158] b). In the auxiliary transaction, read_sql_with_lock is executed in a streaming manner, and its execution sequence number is specified as cur_sql_sn and the streaming read threshold max_num. The stand-alone database returns data rows in an incremental manner rather than in a full manner, that is, data rows are returned in batches, and the number of data rows returned in each batch does not exceed max_num.

[0159] c). Each time the distributed database portal receives a batch of data rows into memory, it executes the following steps:

[0160] 1. Generate delete_sql to delete this batch of data rows.

[0161] 2. In the single-machine master transaction, execute delete_sql and specify its execution sequence number as cur_sql_sn.

[0162] 3. The distributed database portal updates the values ​​of the shard key columns of the queried data rows in memory to obtain updated_datas, and obtains the updated_single_db_list, a collection of stand-alone databases storing all updated data rows, based on updated_datas. For each stand-alone database in updated_single_db_list, an insert_sql for inserting data is generated.

[0163] 4. The distributed database entry executes the following steps in parallel on each stand-alone database in updated_single_db_list: In the main transaction of the stand-alone database, execute insert_sql and specify its execution sequence number as the execution sequence number corresponding to the operation statement, that is, the execution sequence number corresponding to the first statement to be executed.

[0164] If the main transaction is not started on the stand-alone database, start a main transaction and specify the global transaction identifier of the main transaction as the global transaction identifier of the current distributed transaction. The embodiment of the present application has started the auxiliary transaction model, so it can be assumed that the main transaction has been started. The main transaction and auxiliary transaction in the auxiliary transaction model can also be started at different times.

[0165] 5. The distributed database portal deletes this batch of data rows in memory.

[0166] The distributed database portal generates read_sql_with_lock and write_sql based on a received SQL statement for updating the value of the shard key column. Write_sql includes delete_sql for deleting data and insert_sql for inserting data.

[0167] When a distributed transaction is committed, the opened primary transaction and auxiliary transaction will be committed on each stand-alone database.

[0168] For example, Figure 4 A flowchart showing how to apply for a row lock for a data row to be read through an auxiliary transaction is provided.

[0169] S301. Parse a first statement to be executed to determine a first target type of a row lock.

[0170] S302. Determine whether the auxiliary transaction has created a first row lock instance corresponding to the data row to be read and whether the type level of the first row lock instance is not lower than the level of the first target type. If so, execute S303; otherwise, execute S304.

[0171] S303: Use the first row lock instance as the row lock for the data row to be read.

[0172] The application for the row lock of the data row to be read can be completed through step S303.

[0173] S304: Obtain the main transaction attributes of the auxiliary transaction.

[0174] S305. Determine whether the main transaction attribute of the auxiliary transaction is not empty and the main transaction has created a second row lock instance corresponding to the data row to be read and the type level of the second row lock instance is not lower than the level of the first target type. If not, execute S306; if so, execute S307.

[0175] That is, if the main transaction attribute of the auxiliary transaction is not empty and the main transaction has created a second row lock instance corresponding to the data row to be read and the type level of the second row lock instance is not lower than the level of the first target type, execute S307; otherwise, execute S308.

[0176] S306. Determine whether other transactions of the main transaction other than the auxiliary transaction have created or are waiting for the third row lock instance corresponding to the data row to be read, and the type of the third row lock instance conflicts with the first target type; if not, execute S307; if so, execute S308.

[0177] That is, if there are other transactions of the main transaction other than the auxiliary transaction that have created or are waiting for the third row lock instance corresponding to the data row to be read and the type of the third row lock instance conflicts with the first target type, execute S308; otherwise, execute S307.

[0178] S307. Create a first target row lock instance for the data row to be read, set the type attribute of the first target row lock instance to the first target type, add the first target row lock instance to the row lock instance queue of the auxiliary transaction, and add the first target row lock instance to the global row lock instance list of the data row to be read.

[0179] The application for the row lock of the data row to be read can be completed through step S307.

[0180] S308. Add the auxiliary transaction to the global row lock waiting queue waiting for the data row to be read, set the waiting lock type attribute of the auxiliary transaction to the first target type, set the waiting transaction attribute of the auxiliary transaction according to the conflicting transaction, control the session corresponding to the auxiliary transaction to sleep, and wait for other sessions to wake up or sleep timeout.

[0181] S309: If the session corresponding to the auxiliary transaction is awakened by other sessions, the data row to be read is read; if the sleep timeout occurs, a Gate lock timeout error is returned, and the processing flow of the data row to be read is terminated.

[0182] After applying for the row lock of the data row to be read, read each data row to be read and return the set of all the read data rows to the client. When the main transaction applies for the row lock to write data, the number of written data rows is returned to the client after the data writing is completed.

[0183] Similarly, the process of applying for row locks for the data rows to be written through the main transaction is the same as Figure 4 The process principle is the same as in Figure 4 , I will not go into details here.

[0184] For example, Figure 5 A flowchart for waking up a session where a transaction waiting for a lock is located is provided, with the transaction waiting for a lock being taken as the main transaction as an example.

[0185] S401. Obtain a waiting lock type corresponding to a transaction waiting for lock according to a waiting lock type attribute of the transaction waiting for lock.

[0186] S402, determine whether the auxiliary transaction attribute of the transaction waiting for the lock is not empty and the auxiliary transaction of the transaction waiting for the lock has created the seventh row lock instance of the data row to be deleted and the type level of the seventh row lock instance is not lower than the level of the waiting lock type; if not, execute S403; if so, execute S405.

[0187] That is, if the auxiliary transaction attribute of the transaction waiting for the lock is not empty and the auxiliary transaction of the transaction waiting for the lock has created the seventh row lock instance of the data row to be deleted and the type level of the seventh row lock instance is not lower than the level of the waiting lock type, execute S405; otherwise, execute S403.

[0188] S403, determine whether other transactions have created the eighth row lock instance of the data row to be deleted and the type of the eighth row lock instance conflicts with the waiting lock type. If so, execute S404; otherwise, execute S405.

[0189] That is, if there is another transaction that has created the eighth row lock instance of the to-be-deleted data row and the type of the eighth row lock instance conflicts with the waiting lock type, execute S404 to end the processing flow; otherwise, execute S405.

[0190] S404: End the processing flow of the transaction waiting for the lock.

[0191] S405. Create a third target row lock instance through the transaction waiting for the lock, set the type attribute of the third target row lock instance to the waiting lock type, add the third target row lock instance to the row lock instance queue of the transaction waiting for the lock, add the third target row lock instance to the global row lock instance list of the data row to be deleted, delete the transaction waiting for the lock from the global row lock waiting list, and wake up the session where the transaction waiting for the lock is located.

[0192] S406. If the auxiliary transaction attribute of the transaction waiting for the lock is not empty and the auxiliary transaction of the transaction waiting for the lock is in the row lock waiting queue of the data row to be deleted and the waiting transaction is not the current transaction and the level of the target row lock type being waited for is not higher than the waiting lock type, create a fourth target row lock instance through the auxiliary transaction of the transaction waiting for the lock.

[0193] S407. Set the type attribute of the fourth target row lock instance to the target row lock type, add the fourth target row lock instance to the row lock instance queue of the auxiliary transaction of the transaction waiting for the lock, add the fourth target row lock instance to the global row lock instance list of the data row to be deleted, delete the auxiliary transaction of the transaction waiting for the lock from the global row lock wait list, and wake up the session where the auxiliary transaction of the transaction waiting for the lock is located.

[0194] Similarly, when the transaction waiting for the lock is an auxiliary transaction, the process of waking up the session where the transaction waiting for the lock is located is the same as when the transaction waiting for the lock is the main transaction. Figure 5 , which will not be elaborated here.

[0195] The transaction execution method of a distributed database provided by the embodiment of the present invention solves the problem of insufficient memory resources and poor performance caused by the fact that a single-machine database in a distributed database can only execute some statements serially, and executes the first statement to be executed and the second statement to be executed in parallel to realize parallel reading and writing of data, save memory resources, reduce execution time, and improve database performance. The auxiliary transaction model supports streaming asynchronous execution of SQL statements with write locks to read and write data, while ensuring strict transaction ACID properties.

[0196] Embodiment 3

[0197] Figure 6 This is a schematic diagram of the structure of a distributed database provided by Embodiment 3 of the present invention. Figure 6 As shown, the system includes: a distributed database entry 51 and at least two stand-alone databases 52, and the stand-alone databases 52 execute the transaction execution method of the distributed database as described in any embodiment of the present application.

[0198] The stand-alone database 52 receives the first statement to be executed and the execution sequence number determined and sent by the distributed database entry according to the operation statement through the auxiliary transaction in the auxiliary transaction model, wherein the auxiliary transaction model includes a one-to-one corresponding main transaction and auxiliary transaction;

[0199] The stand-alone database 52 executes the first statement to be executed in a streaming manner through the auxiliary transaction based on the execution sequence number and a predetermined streaming reading threshold, batch reads data rows and returns to the distributed database entry until all data rows are read;

[0200] The distributed database entry 51 processes the data rows obtained in batches, generates data to be updated and feeds it back to the stand-alone database 52;

[0201] The stand-alone database 52 receives the data to be updated and the second statement to be executed fed back from the distributed database entry in sequence through the main transaction in the auxiliary transaction model, executes the second statement to be executed in parallel through the main transaction, and writes the data to be updated.

[0202] A distributed database provided by an embodiment of the present invention solves the problem of insufficient memory resources and poor performance caused by the fact that a stand-alone database in a distributed database can only execute some statements serially. The present application provides a new transaction model, namely, an auxiliary transaction model, in which statements are executed in parallel by a main transaction and an auxiliary transaction in the auxiliary transaction model. The auxiliary transaction executes a first statement to be executed and streams data rows and feeds back to a distributed database entry. The distributed database entry processes the data and generates data to be updated and feeds back to the stand-alone database. The stand-alone database receives the data to be updated and the second statement to be executed in sequence. After each receipt of the data to be updated and the second statement to be executed, the main transaction executes the second statement to be executed to write the data to be updated. At this time, the first data to be executed can stream data rows, and the parallel execution of the second statement to be executed and the first statement to be executed is realized, that is, the reading and writing of data are executed in parallel, saving memory resources of the distributed database entry, reducing execution time, and improving database performance.

[0203] The distributed database system provided in the embodiment of the present invention includes a distributed database portal and a single-machine database that can execute the transaction execution method of the distributed database provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0204] Embodiment 4

[0205] Figure 7 A schematic diagram of the structure of an electronic device 60 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0206] like Figure 7As shown, the electronic device 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0207] A number of components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0208] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 61 executes the various methods and processes described above, such as a transaction execution method of a distributed database.

[0209] In some embodiments, the transaction execution method of the distributed database may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the transaction execution method of the distributed database described above may be executed. Alternatively, in other embodiments, the processor 61 may be configured to execute the transaction execution method of the distributed database in any other appropriate manner (e.g., by means of firmware).

[0210] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0211] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0212] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0213] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0214] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0215] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0216] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0217] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A transaction execution method for a distributed database, characterized in that: Standalone databases used in distributed databases include: Receiving, through an auxiliary transaction in an auxiliary transaction model, a first statement to be executed and an execution sequence number determined and sent by a distributed database entry according to an operation statement, wherein the auxiliary transaction model includes a main transaction and an auxiliary transaction in one-to-one correspondence; Based on the execution sequence number and a predetermined streaming reading threshold, the first statement to be executed is stream-executed through the auxiliary transaction, and data rows are read in batches and returned to the distributed database entry until all data rows are read, wherein the batch-read data rows are processed by the distributed database entry and data to be updated is generated; sequentially receiving the data to be updated and the second statement to be executed fed back by the distributed database entry through the main transaction in the auxiliary transaction model, executing the second statement to be executed in parallel through the main transaction, and writing the data to be updated; The step of streaming the first statement to be executed through the auxiliary transaction based on the execution sequence number and a predetermined streaming reading threshold, batch reading data rows and returning them to the distributed database entry includes: Parsing the first statement to be executed by the auxiliary transaction to determine at least one data row to be read; For each data row to be read, apply for a row lock of the data row to be read through the auxiliary transaction. After the row lock application is successful, determine whether to add the data row to be read to the return set through the auxiliary transaction combined with the execution sequence number; When the number of the to-be-read data rows in the returned set is equal to the streaming read threshold, all the to-be-read data rows in the returned set are fed back to the distributed database entry through the auxiliary transaction.

2. The method according to claim 1, characterized in that Also includes: If there is no first stand-alone transaction between the stand-alone database and the distributed database entry and the auxiliary transaction model startup condition is met, start the auxiliary transaction model; If a first stand-alone transaction exists between the stand-alone database and the distributed database entry and the auxiliary transaction model startup condition is met, the auxiliary transaction model is started based on the first stand-alone transaction.

3. The method according to claim 2, characterized in that Start the auxiliary transaction model, including: After receiving a transaction start command and a first global transaction identifier through a first session, a first stand-alone transaction is started, and a global transaction identifier attribute of the first stand-alone transaction is set to the first global transaction identifier, and a primary transaction attribute and an auxiliary transaction attribute of the first stand-alone transaction are set to be empty; An auxiliary transaction is started, and the auxiliary transaction is used as an auxiliary transaction of the first stand-alone transaction. The first stand-alone transaction is used as a main transaction, and an auxiliary transaction model is formed according to the main transaction and the auxiliary transaction.

4. The method according to claim 2, characterized in that: The starting of the auxiliary transaction model based on the first stand-alone transaction includes: If both the main transaction attribute and the auxiliary transaction attribute of the first stand-alone transaction are empty, start the auxiliary transaction, use the auxiliary transaction as the auxiliary transaction of the first stand-alone transaction, use the first stand-alone transaction as the main transaction, and form an auxiliary transaction model according to the main transaction and the auxiliary transaction; If the main transaction attribute of the first stand-alone transaction is empty and the auxiliary transaction attribute is not empty, determining the first stand-alone transaction as a main transaction, determining an auxiliary transaction corresponding to the auxiliary transaction attribute, and forming an auxiliary transaction model according to the main transaction and the auxiliary transaction; If the main transaction attribute of the first stand-alone transaction is not empty and the auxiliary transaction attribute is empty, determine that the first stand-alone transaction is an auxiliary transaction, determine the main transaction corresponding to the main transaction attribute, and form an auxiliary transaction model according to the main transaction and the auxiliary transaction.

5. The method according to claim 3 or 4, characterized in that: The starting of the auxiliary transaction and using the auxiliary transaction as an auxiliary transaction of the first stand-alone transaction includes: receiving an auxiliary transaction start command and a second global transaction identifier through a second session, starting an auxiliary transaction, and setting a main transaction attribute and an auxiliary transaction attribute of the auxiliary transaction to be empty; The second global transaction identifier is matched with the global transaction identifier attributes of all started stand-alone transactions; if the matched stand-alone transaction is the first stand-alone transaction, the auxiliary transaction is used as an auxiliary transaction of the first stand-alone transaction, and the main transaction attribute of the auxiliary transaction is set to the identifier of the first stand-alone transaction.

6. The method according to claim 1, characterized in that The applying for a row lock for the to-be-read data row through the auxiliary transaction includes: Determine a first target type of the row lock according to the first statement to be executed; Determine whether the auxiliary transaction has created a first row lock instance corresponding to the data row to be read and whether the type level of the first row lock instance is not lower than the level of the first target type; if so, use the first row lock instance as the row lock of the data row to be read; Otherwise, the main transaction attribute of the auxiliary transaction is obtained, and a row lock of the to-be-read data row is applied for according to the main transaction attribute of the auxiliary transaction.

7. The method according to claim 6, characterized in that The applying for a row lock for the to-be-read data row according to the main transaction attribute of the auxiliary transaction includes: Determine whether the main transaction attribute of the auxiliary transaction is not empty and the main transaction has created a second row lock instance corresponding to the data row to be read and the type level of the second row lock instance is not lower than the level of the first target type; If so, create a first target row lock instance for the data row to be read, add the first target row lock instance to the row lock instance queue of the auxiliary transaction, and add the first target row lock instance to the global row lock instance list of the data row to be read; Otherwise, traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be read; if other transactions have created or are waiting for the third row lock instance corresponding to the data row to be read and the type of the third row lock instance conflicts with the first target type, add the auxiliary transaction to the global row lock waiting queue waiting for the data row to be read, set the waiting lock type attribute of the auxiliary transaction to the first target type, set the waiting transaction attribute of the auxiliary transaction according to the conflicting transaction, and control the session corresponding to the auxiliary transaction to sleep; if no other transaction has created or is waiting for the third row lock instance corresponding to the data row to be read or the type of the third row lock instance conflicts with the first target type, create the first target row lock instance for the data row to be read, add the first target row lock instance to the row lock instance queue of the auxiliary transaction, and add the first target row lock instance to the global row lock instance list for the data row to be read.

8. The method according to claim 1, characterized in that The determining whether to add the to-be-read data row to a return set by combining the auxiliary transaction with the execution sequence number includes: Obtaining the last modification transaction of the data row to be read through the auxiliary transaction, and if the last modification transaction is not the main transaction of the auxiliary transaction, adding the data row to be read to the return set through the auxiliary transaction; If the last modified transaction is the main transaction of the auxiliary transaction, the modification sequence number of the data row to be read is obtained through the auxiliary transaction. If the modification sequence number is less than the execution sequence number, the data row to be read is added to the return set through the auxiliary transaction.

9. The method according to claim 1, characterized in that: The executing the second statement to be executed in parallel through the main transaction to write the data to be updated includes: Parsing the second statement to be executed by the main transaction to determine at least one data row to be written; For each data row to be written, a row lock of the data row to be written is applied for through the main transaction, and based on the applied row lock, the data to be updated is written through the main transaction.

10. The method according to claim 9, characterized in that The applying for a row lock for the to-be-written data row through the main transaction includes: Determine a second target type of the row lock according to the second statement to be executed; Determine whether the main transaction has created a fourth row lock instance corresponding to the data row to be written and the level of the type of the fourth row lock instance is not lower than the level of the second target type; if so, use the fourth row lock instance as the row lock of the data row to be written; Otherwise, the auxiliary transaction attribute of the main transaction is obtained, and a row lock of the to-be-written data row is applied for according to the auxiliary transaction attribute of the main transaction.

11. The method according to claim 10, characterized in that The applying for a row lock for the to-be-written data row according to the auxiliary transaction attribute of the main transaction includes: Determine whether the auxiliary transaction attribute of the main transaction is not empty and the auxiliary transaction has created a fifth row lock instance corresponding to the data row to be written and the level of the type of the fifth row lock instance is not lower than the level of the second target type; If so, create a second target row lock instance for the data row to be written, add the second target row lock instance to the row lock instance queue of the main transaction, and add the second target row lock instance to the global row lock instance list for the data row to be written; Otherwise, traverse the global row lock instance list and the global row lock waiting transaction queue of the data row to be written; if other transactions have created or are waiting for the sixth row lock instance corresponding to the data row to be written and the type of the sixth row lock instance conflicts with the second target type, add the main transaction to the global row lock waiting queue waiting for the data row to be written, set the waiting lock type attribute of the main transaction to the second target type, set the waiting transaction attribute of the main transaction according to the conflicting transaction, and control the session corresponding to the main transaction to sleep; if no other transaction has created or is waiting for the sixth row lock instance corresponding to the data row to be written or the type of the sixth row lock instance conflicts with the second target type, create the second target row lock instance for the data row to be written, add the second target row lock instance to the row lock instance queue of the main transaction, and add the second target row lock instance to the global row lock instance list for the data row to be written.

12. The method according to any one of claims 1 to 11, characterized in that: Also includes: When the current transaction is committed, a data row to be processed is selected from the row lock instances of all data rows to be processed applied by the current transaction as the data row to be deleted, and a row lock instance of the data row to be deleted is used as the row lock instance to be deleted; Traversing the global row lock instance list of the data row to be deleted, deleting the row lock instance to be deleted from the global row lock instance list of the data row to be deleted, and deleting the row lock instance to be deleted from the row lock instance queue of the current transaction; Traversing the global row lock waiting queue of the data row to be deleted, determining the transaction waiting for lock in the global row lock waiting queue and waking up the session where the transaction waiting for lock is located, until all row lock instances of all pending data rows applied by the current transaction are deleted; Among them, the transaction that the transaction waiting for the lock is waiting for is the current transaction; the current transaction is the main transaction or the auxiliary transaction, when the current transaction is the main transaction, the data rows to be processed are data rows to be written, and when the current transaction is the auxiliary transaction, the data rows to be processed are data rows to be read.

13. The method according to claim 12, characterized in that The step of waking up the session where the transaction waiting for the lock is located includes: Obtaining the waiting lock type corresponding to the transaction waiting for the lock according to the waiting lock type attribute of the transaction waiting for the lock; Determine whether the main transaction attribute or the auxiliary transaction attribute of the transaction waiting for the lock is not empty and the main transaction or the auxiliary transaction of the transaction waiting for the lock has created the seventh row lock instance of the to-be-deleted data row and the type level of the seventh row lock instance is not lower than the level of the waiting lock type; If so, create a third target row lock instance through the transaction waiting for the lock; otherwise, if no other transaction has created the eighth row lock instance of the to-be-deleted data row and the type of the eighth row lock instance conflicts with the waiting lock type, create a third target row lock instance through the transaction waiting for the lock; Set the type attribute of the third target row lock instance to the waiting lock type, add the third target row lock instance to the row lock instance queue of the transaction waiting for the lock, add the third target row lock instance to the global row lock instance list of the data row to be deleted, delete the transaction waiting for the lock from the global row lock waiting list, and wake up the session where the transaction waiting for the lock is located.

14. The method according to claim 13, characterized in that Also includes: If the main transaction attribute or the auxiliary transaction attribute of the transaction waiting for the lock is not empty, the main transaction or the auxiliary transaction of the transaction waiting for the lock is in the row lock waiting queue of the to-be-deleted data row, the transaction waiting is not the current transaction, and the level of the target row lock type being waited for is not higher than the waiting lock type, a fourth target row lock instance is created through the main transaction or the auxiliary transaction of the transaction waiting for the lock; Set the type attribute of the fourth target row lock instance to the target row lock type, add the fourth target row lock instance to the row lock instance queue of the main transaction or auxiliary transaction of the transaction waiting for the lock, add the fourth target row lock instance to the global row lock instance list of the data row to be deleted, delete the main transaction or auxiliary transaction of the transaction waiting for the lock from the global row lock wait list, and wake up the session where the main transaction or auxiliary transaction of the transaction waiting for the lock is located.

15. The method according to claim 1, characterized in that Also includes: After writing the data row, the modification sequence number attribute of the written data row is updated to the execution sequence number, and the last modification transaction attribute of the written data row is updated to the main transaction.

16. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively coupled to the at least one processor; The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the transaction execution method of a distributed database according to any one of claims 1 to 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the transaction execution method for a distributed database according to any one of claims 1 to 15 when executed.

Images