Transaction execution method and device, equipment, storage medium and program product
By analyzing the statistical information and operation types of query statements in the database, identifying and splitting large transactions, the problem of large transactions not being successfully executed is solved, and the successful execution of large transactions and the improvement of database processing capabilities is achieved.
Patent Information
- Application Number
- CN202411954697.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-01
AI Technical Summary
When the transactions processed by the database include large transactions, the large transaction fails to execute successfully, resulting in the database processing capability being affected.
By analyzing the statistical information and operation types of the table to be looked up in the database query statement, determine whether the target transaction is a large transaction. If it is a large transaction, a pre-query is performed on the database to obtain the data set to be operated, and the target transaction is split into multiple sub-transactions, and finally these sub-transactions are executed.
On the basis of ensuring that other transactions other than large transactions can be executed normally, the successful execution of large transactions is achieved and the overall ability of database processing is improved.
Smart Images
Figure CN119739470A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of financial technology or big data, and in particular to a transaction execution method, device, equipment, storage medium and program product. Background Art
[0002] In database transaction processing, transactions that occupy a large amount of resources for a long time are called large transactions. Since large transactions occupy system resources for a long time, they will block the execution of other transactions and affect the overall processing capacity of the database.
[0003] In the prior art, during the execution of database transactions, large transactions are checked and killed to ensure the execution of other transactions except large transactions. That is, when the time a transaction occupies resources exceeds a set threshold, the transaction is determined to be a large transaction and the process of the large transaction is terminated.
[0004] When the transactions processed by the database include large transactions, the large transaction is killed to ensure the execution of other transactions except the large transaction, resulting in the failure of the large transaction to be executed successfully. Summary of the invention
[0005] The present application provides a transaction execution method, apparatus, device, storage medium and program product, which are used to solve the technical problem that when the transactions processed by the database include large transactions, the large transactions are not successfully executed.
[0006] In a first aspect, the present application provides a transaction execution method, comprising:
[0007] Determine whether a target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement;
[0008] In the case where the target transaction is a large transaction, a database is pre-queried according to the database query statement to obtain a data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated;
[0009] The plurality of sub-transactions are executed.
[0010] In a second aspect, the present application provides a transaction execution device, including:
[0011] A determination module, used to determine whether a target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement;
[0012] A splitting module, used for, when the target transaction is a large transaction, pre-querying the database according to the database query statement to obtain a data set to be operated, and splitting the target transaction into multiple sub-transactions according to the data set to be operated;
[0013] An execution module is used to execute the multiple sub-transactions.
[0014] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;
[0015] The memory stores computer-executable instructions;
[0016] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect.
[0017] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.
[0018] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.
[0019] The transaction execution method, device, equipment, storage medium and program product provided by the present application determine whether the target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement. If the target transaction is a large transaction, the database is pre-queried according to the database query statement to obtain the data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated, and the multiple sub-transactions are executed. The method of the present application realizes the execution of large transactions by splitting the target transaction determined as a large transaction into multiple sub-transactions for execution, while ensuring the execution of other transactions except the large transaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0021] Figure 1 A schematic diagram of a transaction execution method provided in an embodiment of the present application Figure 1 ;
[0022] Figure 2 A schematic diagram of a transaction execution method provided in an embodiment of the present application Figure 2 ;
[0023] Figure 3 A schematic diagram of a processing flow of an execution manager provided in an embodiment of the present application;
[0024] Figure 4 A schematic diagram of the structure of a transaction execution device provided in an embodiment of the present application;
[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0026] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION
[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0029] In addition, this application involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and the use of artificial intelligence technology for automated decision-making, and a technical solution for making decisions that have a significant impact on personal rights and interests based on the results of automated decision-making. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered.
[0030] It should be noted that the transaction execution method, device, equipment, storage medium and program product provided in the present application can be used in the fields of financial technology or big data, and can also be used in any field other than the field of financial technology. The application field of the transaction execution method, device, equipment, storage medium and program product in the present application is not limited.
[0031] The specific application scenario of the present application is any scenario in which a database is operated based on a database query statement. The database query statement may be, for example, a structured query language (SQL). For example, in banking business processing, business-related data may be stored in a database, and the data stored in the database may be operated through a database query statement, for example, an update operation may be performed on any table stored in the database through a database query statement, wherein the update operation performed in the database is the process of processing a transaction.
[0032] In database transaction processing, transactions that occupy a large amount of resources for a long time during execution are called large transactions. In the prior art, when the transactions processed by the database include large transactions, the method of killing large transactions to ensure the execution of other transactions except the large transactions results in the failure of the large transaction to be executed successfully.
[0033] In view of this, the present application provides a transaction execution method, which determines whether the target transaction corresponding to the database query statement is a large transaction through a database query statement, splits the target transaction determined to be a large transaction into multiple sub-transactions, and executes the multiple sub-transactions. Among them, splitting the target transaction is to pre-query the database according to the database query statement to obtain the data set to be operated, and split the target transaction into multiple sub-transactions according to the data set to be operated. The method of the present application realizes the execution of large transactions on the basis of ensuring the execution of other transactions except large transactions.
[0034] It should be understood that the execution subject of the method of the embodiment of the present application can be a transaction execution device. It should be noted that the transaction execution device can be deployed separately on an electronic device in any environment (for example, deployed separately on an edge server in an edge environment), or all deployed in a cloud environment, or distributedly deployed in different environments.
[0035] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0036] Figure 1 A schematic diagram of a transaction execution method provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the transaction execution method of the present application specifically includes the following steps:
[0037] S101. Determine whether a target transaction corresponding to a database query statement is a large transaction according to statistical information of a table to be queried in the database query statement and an operation type of the database query statement.
[0038] A database query statement is a statement used to interact with a database and operate on the data in the database. For example, a database query statement may be a structured query language. Operations on the data in the database may be, for example, a delete operation, an update operation, or an insert operation.
[0039] The table to be queried of the database query statement is the table to be operated in the database indicated by the database query statement. Taking the structured query statement as an example, the structured query statement can indicate to delete the data in table T through "DELETE FROM T", and T is the table to be queried. The statistical information of the table to be queried may include the data volume, number of rows, primary key index, unique index, regular index, full-text index, data distribution, storage space, etc. of the table.
[0040] The operation type of a database query statement is the type of operation on the data in the database indicated in the database query statement. Taking a structured query statement as an example, the operation type of a database query statement may be, for example, an update operation, a delete operation, or an insert operation.
[0041] The target transaction corresponding to the database query statement is the operation performed on the data in the database according to the database query statement.
[0042] A large transaction is a transaction that takes up a lot of resources for a long time in database transaction processing.
[0043] One possible implementation method is to parse a database query statement based on a parsing tool in the relevant technology, obtain the table to be queried and the operation type of the database query statement, and obtain statistical information of the table to be queried, which may include the amount of data, number of rows, and storage space of the table. The statistical information of the table to be queried in the database query statement and the operation type of the database query statement are input into a pre-trained recognition model, and the recognition model can output whether the target transaction corresponding to the database query statement is a large transaction. The recognition model can be trained based on the statistical information of the table to be queried and the operation type of the database query statement in a large number of database query statements, as well as whether the database query statement is a large transaction as training data.
[0044] One possible implementation method is to decompose a database query statement into basic elements of the database query statement, such as keywords, identifiers, character strings, and comment conditions, based on a lexical analysis method, and further parse the grammatical structure of the database query statement based on a syntax analysis method to obtain a table to be queried and an operation type of the database query statement, and determine that the target transaction corresponding to the database query statement is a large transaction when the statistical information of the table to be queried exceeds a statistical information threshold corresponding to the operation type. Taking the operation type as an update type as an example, the statistical information threshold corresponding to the update type may be, for example, that the number of rows is greater than N and the storage space is greater than M.
[0045] Optionally, in this mode, incorrect characters in the database query statement can be detected based on a lexical analysis method, incorrect syntax in the database query statement can be detected based on a grammatical analysis method, and whether the database query statement conforms to the structure of the database design can be detected based on a semantic analysis method. In the event that incorrect characters or incorrect syntax are detected in the database query statement, or the database query statement does not conform to the structure of the database design, the database query statement may no longer be processed. Among them, the lexical analysis method, the grammatical analysis method, and the semantic analysis method can match the database query statement based on regular matching, or can also be implemented based on related tools of the database query statement, and this application does not limit this.
[0046] S102: When the target transaction is a large transaction, a database is pre-queried according to a database query statement to obtain a data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated.
[0047] The data set to be operated is obtained by pre-querying the database according to the database query statement.
[0048] A subtransaction is a transaction split from a target transaction. The results of executing multiple subtransactions on the database are consistent with the results of executing the target transaction on the database.
[0049] In the case where the target transaction is a large transaction, one possible implementation method is to pre-query the database according to the database query statement to obtain the data set to be operated, and split the target transaction into multiple sub-transactions according to the fields in the data set to be operated. For example, multiple screening conditions are generated according to the fields in the data set to be operated, and the screening range of each screening condition corresponds to part of the data in the data set to be operated. Based on the multiple screening conditions, on the basis of the database query statement, multiple corresponding sub-database query statements are formed, and the sub-transaction corresponding to each sub-database query statement is a transaction that operates on part of the data in the data set to be operated.
[0050] In the case where the target transaction is a large transaction, one possible implementation method is to pre-query the database according to the database query statement to obtain the data set to be operated, and split the target transaction into multiple sub-transactions according to the statistical information in the data set to be operated. Taking the value of any column in the data set to be operated as an enumeration type as an example, the multiple enumeration values included in the column are counted, and the corresponding filtering conditions are generated according to each enumeration value. Based on the multiple filtering conditions, multiple corresponding sub-database query statements are formed on the basis of the database query statement, and the sub-transaction corresponding to each sub-database query statement is a transaction that operates on part of the data in the data set to be operated.
[0051] After the target transaction is split into multiple sub-transactions according to the data set to be operated, the split sub-transactions can be confirmed one by one to determine whether the sub-transaction is a large transaction. Specifically, the split sub-transactions can be used as new database query statements one by one, and S101 and S102 are repeatedly executed until all the split sub-transactions are not large transactions, and then S103 is executed.
[0052] S103. Execute multiple sub-transactions.
[0053] One possible implementation method is to control the sub-transaction executor to execute multiple sub-transactions based on the transaction execution manager, and the execution results of the multiple sub-transactions in the database are the execution results of the target transaction in the database.
[0054] In this way, since subtransactions are not large transactions, they will not occupy resources for a long time during execution, and the results of executing multiple subtransactions on the database are consistent with the results of executing the target transaction on the database. Therefore, the execution of the target transaction is achieved while ensuring the execution of other transactions except large transactions.
[0055] The transaction execution method provided in this embodiment determines whether the target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement. If the target transaction is a large transaction, the database is pre-queried according to the database query statement to obtain the data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated, and the multiple sub-transactions are executed. The method of the present application realizes the execution of large transactions by splitting the target transaction determined to be a large transaction into multiple sub-transactions for execution, while ensuring the execution of other transactions except the large transaction.
[0056] The following is an example of how to split a target transaction into multiple sub-transactions according to a data set to be operated in the transaction processing method of the present application.
[0057] Generate multiple first screening conditions according to the first field in the data set to be operated, add each first screening condition to the database query statement, form a first sub-database query statement corresponding to each first screening condition, and the first transaction corresponding to the first sub-database query statement is a sub-transaction of the target transaction.
[0058] The first field is at least one field in the data set to be operated, and the data set to be operated can be segmented based on the first field. The first screening condition is used as a screening condition in the database query statement, and the data to be operated can be screened in the data set to be operated. A corresponding first sub-database query statement can be formed according to each first screening condition, and the screening range of each first screening condition corresponds to a segment after the data set to be operated is segmented according to the first field.
[0059] Taking the operation type of a database query statement as a delete operation as an example, for example, multiple first filtering conditions can be generated based on the first field in the data set to be deleted, and each first filtering condition is added to the database query statement to form a first sub-database query statement corresponding to each first filtering condition. The first transaction corresponding to the first sub-database query statement is a sub-transaction of the target transaction, and each sub-database query statement indicates that a delete operation is to be performed on part of the data in the data set to be deleted.
[0060] For example, taking the first field in the data set to be deleted as field A, and field A in each row of data in the data set to be deleted as 1 or 2, the first first screening condition can be a condition that field A is equal to 1 represented in the format of a database query statement, and the second first screening condition can be a condition that field A is equal to 2 represented in the format of a database query statement. Each first screening condition is added to the database query statement to form two first sub-database query statements. The screening range of the first first screening condition corresponds to the row data in which field A is equal to 1 in the data set to be operated, and the screening range of the second first screening condition corresponds to the row data in which field A is equal to 2 in the data set to be operated, that is, the screening range of each first screening condition corresponds to a segment after the data set to be operated is segmented by field A.
[0061] The transaction execution method provided in this embodiment splits the target transaction into multiple sub-transactions according to the data set to be operated, and executes the multiple sub-transactions obtained by splitting based on the data set to be operated. The execution result is not affected by the execution order of the multiple sub-transactions, and the execution results of the multiple sub-transactions can be made consistent with the execution result of the target transaction. On the basis of realizing the execution of large transactions, the accuracy of the results of database processing transactions is improved.
[0062] Optionally, when the operation type is an update operation, multiple second filtering conditions can be generated based on the second field of the table to be queried, and each second filtering condition can be added to the database query statement to form a second sub-database query statement corresponding to each second filtering condition, and the second transaction corresponding to the second sub-database query statement is a sub-transaction of the target transaction.
[0063] The second field is at least one field in the table to be queried, and the table to be queried can be segmented based on the second field. The second screening condition is used as a screening condition in the database query statement, and can screen part of the data in the table to be queried. A corresponding second sub-database query statement can be formed according to each second screening condition, and the screening range of each second screening condition corresponds to a segment after the table to be queried is segmented according to the second field.
[0064] The index is used to speed up the query speed of the table, and the index can be created based on any field in the query table. In this way, for example, the second field can be any index in the table to be queried, and the index can be a common index, a primary key index, or an auto-increment index in the table to be queried. When the operation type is an update operation, multiple second screening conditions can be generated based on any index of the table to be queried, and each second screening condition is added to the database query statement to form a second sub-database query statement corresponding to each second screening condition, and the second transaction corresponding to the second sub-database query statement is a sub-transaction of the target transaction.
[0065] For example, the second field in the table to be queried for update operation is an auto-increment index, and the data in the table to be queried can be segmented by row based on the auto-increment index, and the number of rows of data in each segment is less than the preset row number threshold, and the transaction for updating the data with the preset row number threshold is not a large transaction. In this way, taking the auto-increment index in the table to be queried as 1 to 900 and the preset row number threshold as 500 as an example, the first second screening condition can be a condition that the auto-increment index represented by the format of the database query statement is less than or equal to 500, and the second second screening condition can be a condition that the auto-increment index represented by the format of the database query statement is greater than 500. Each second screening condition is added to the database query statement to form two second sub-database query statements, the screening range of the first second screening condition corresponds to the row data with the auto-increment index of 1 to 500 in the table to be queried, and the screening range of the second second screening condition corresponds to the row data with the auto-increment index of 500 to 900 in the table to be queried, that is, the screening range of each second screening condition corresponds to a segment after the table to be queried is segmented by the auto-increment index.
[0066] The transaction execution method provided in this embodiment, when the operation type is an update operation, splits the target transaction into multiple sub-transactions based on the second field of the table to be queried, which can improve the speed of splitting large transactions.
[0067] Optionally, when the filtering conditions in the database query statement include multiple sub-filtering conditions separated by an OR operator, for any sub-filtering condition, the multiple sub-filtering conditions separated by an OR operator in the database query statement are replaced with any sub-filtering condition to form a third sub-database query statement corresponding to any sub-filtering condition, and the third transaction corresponding to the third sub-database query statement is a sub-transaction of the target transaction.
[0068] The OR operation is used to specify multiple conditions in a database query statement. For example, the filter condition in the database query statement is a sub-filter condition connected by an OR operator, indicating that at least one of the sub-filter conditions is satisfied. Taking a structured query statement as an example, the OR operation may be, for example, "OR".
[0069] Taking the database query statement "UPDATE T SET F=f WHERE A=a OR B=b OR C=c" as an example, the filtering condition is "A=a OR B=b OR C=c". One possible implementation method is to form three third sub-database query statements, wherein the filtering condition in the database query statement is replaced with the sub-filtering condition "A=a" to form the first third sub-database query statement "UPDATE T SET F=f WHERE A=a", the filtering condition in the database query statement is replaced with the sub-filtering condition "B=b" to form the second third sub-database query statement "UPDATE T SET F=f WHERE B=b", and the filtering condition in the database query statement is replaced with the sub-filtering condition "C=c" to form the third third sub-database query statement "UPDATE T SET F=f WHERE C=c".
[0070] The transaction execution method provided in this embodiment can improve the speed of splitting large transactions by splitting the target transaction into multiple sub-transactions based on the sub-filtering conditions when the filter conditions in the database query statement include multiple sub-filtering conditions separated by an OR operator.
[0071] Optionally, when the filtering numerical range in the filtering condition in the database query statement exceeds a preset range, the filtering numerical range is divided into multiple sub-numerical ranges, and for any sub-numerical range, the filtering numerical range in the database query statement is replaced with any sub-numerical range to form a fourth sub-database query statement corresponding to any sub-numerical range, and the third transaction corresponding to the fourth sub-database query statement is a sub-transaction of the target transaction.
[0072] Taking the case where the screening condition in the database query statement includes "the numerical range is greater than or equal to 100 and less than 900" and the preset range is 500 as an example, one possible implementation method is to form two fourth sub-database query statements, wherein the first fourth sub-database query statement includes "the numerical range is greater than or equal to 100 and less than 600", and the second third sub-database query statement includes "the numerical range is greater than or equal to 600 and less than 900".
[0073] The transaction execution method provided in this embodiment divides the filter value range into multiple sub-value ranges when the filter value range in the filter condition in the database query statement exceeds the preset range, and splits the target transaction into multiple sub-transactions based on the sub-value ranges, which can improve the speed of splitting large transactions.
[0074] The above is an example description of a method for splitting a target transaction into multiple sub-transactions according to a data set to be operated provided in an embodiment of the present application.
[0075] Optionally, before determining whether the target transaction corresponding to the database query statement is a large transaction based on the statistical information of the table to be queried in the database query statement and the operation type of the database query statement, the database query statement can also be parsed into a syntax tree, and the table to be queried in the database query statement and the operation type of the database query statement can be determined based on the syntax tree.
[0076] Accordingly, based on the statistical information of the table to be queried in the database query statement and the operation type of the database query statement, it is determined whether the target transaction corresponding to the database query statement is a large transaction. This can be done by determining whether the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type. If the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, it is determined that the target transaction corresponding to the database query statement is a large transaction.
[0077] For example, you can pre-configure a database information table and a large transaction rule table. The database information table can record statistical information of tables in the database that store large amounts of data, such as table name, table data volume, primary key index, unique index, regular index, full-text index, data distribution, number of rows, storage space, etc.
[0078] The large transaction rule table includes the conditions for determining whether a transaction is a large transaction, and the corresponding statistical information threshold can be set for the operation type, for example, the threshold for the number of rows in the table to be queried for the update operation, or the threshold for the amount of data to be updated for the update operation.
[0079] In this way, the database query statement is parsed into a syntax tree, and the table to be queried and the operation type of the database query statement in the database query statement are determined according to the syntax tree tool in the relevant technology. The syntax tree tool can determine the various components in the database query statement according to the syntax tree corresponding to the database query statement. According to the table to be queried, the statistical information of the table to be queried is queried from the database information, and according to the operation type, the statistical information threshold corresponding to the operation type is queried from the large transaction rule table to determine whether the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type. If the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, it is determined that the target transaction corresponding to the database query statement is a large transaction.
[0080] Figure 2 A schematic diagram of a transaction execution method provided in an embodiment of the present application Figure 2 .like Figure 2 As shown, the specific steps include:
[0081] S201, parsing a database query statement into a syntax tree.
[0082] S202: Determine the table to be queried in the database query statement and the operation type of the database query statement according to the syntax tree.
[0083] S203: Determine whether the statistical information of the table to be queried exceeds a statistical information threshold corresponding to the operation type.
[0084] If yes, execute S204;
[0085] If not, execute S206.
[0086] Determine whether the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, that is, determine whether the target transaction corresponding to the database query statement is a large transaction.
[0087] S204: Split the target transaction into multiple sub-transactions.
[0088] If the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, it is determined that the target transaction corresponding to the database query statement is a large transaction.
[0089] Optionally, a database may be pre-queried according to a database query statement to obtain a data set to be operated, and the target transaction may be split into multiple sub-transactions according to the data set to be operated.
[0090] Optionally, when the operation type is an update operation, the target transaction may be split into multiple sub-transactions based on the second field of the table to be queried.
[0091] Optionally, when the filter condition in the database query statement includes multiple sub-filter conditions separated by an OR operator, the target transaction can be split into multiple sub-transactions based on the sub-filter conditions.
[0092] Optionally, when the filter value range in the filter condition in the database query statement exceeds a preset range, the filter value range may be divided into multiple sub-value ranges, and the target transaction may be split into multiple sub-transactions based on the sub-value ranges.
[0093] It should be understood that the target transaction can be split into multiple sub-transactions based on any method of splitting the target transaction in the embodiments of the present application.
[0094] S205: Take the multiple sub-transactions as new target transactions respectively and execute S201.
[0095] The multiple sub-transactions are respectively used as new target transactions, and S201 is executed to confirm whether the new target transaction is a large transaction, that is, whether the sub-transaction is a large transaction. If the sub-transaction is a large transaction, the sub-transaction is split based on the sub-transaction.
[0096] S206: Execute the transaction.
[0097] If the target transaction is determined to be a non-large transaction, the target transaction is executed. Alternatively, after the target transaction is split into multiple sub-transactions, if any sub-transaction is determined to be a non-large transaction as a new target transaction, the multiple sub-transactions are executed.
[0098] The transaction execution method provided in this embodiment determines whether the target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement. If the target transaction is a large transaction, the database is pre-queried according to the database query statement to obtain the data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated, and the multiple sub-transactions are executed. The method of the present application realizes the execution of large transactions by splitting the target transaction determined to be a large transaction into multiple sub-transactions for execution, while ensuring the execution of other transactions except the large transaction.
[0099] Optionally, the target transaction determined to be a large transaction is split, and after the target transaction is split into multiple sub-transactions, the multiple sub-transactions may be added to a message queue to execute the multiple sub-transactions in parallel.
[0100] One possible implementation method is to add multiple subtransactions to a message queue, obtain the multiple subtransactions from the message queue through a transaction execution manager, assign the multiple subtransactions to multiple subtransaction executors, and control the multiple subtransactions to execute corresponding subtransactions in parallel.
[0101] The transaction execution manager may obtain multiple sub-transactions from the message queue, distribute the sub-transactions to multiple sub-transaction executors, and receive execution results from the sub-transaction executors.
[0102] Optionally, when any of the multiple subtransactions fails to execute, it can be determined whether any of the subtransactions meets a condition for stopping execution. The condition for stopping execution includes that the current number of executions of any of the subtransactions reaches a subtransaction retry threshold, and / or that the current execution time of any of the subtransactions reaches a timeout time.
[0103] If any sub-transaction does not meet the conditions for stopping execution, execute any sub-transaction again;
[0104] If any subtransaction meets the condition for stopping execution, the execution result of any subtransaction is determined to be a failure, and the other subtransactions except the subtransaction with the failed execution result are rolled back.
[0105] Figure 3 Schematic diagram of a processing flow of an execution manager provided in an embodiment of the present application. Take the example of splitting a target transaction into two sub-transactions and executing them by two sub-transaction executors. Figure 3 As shown, the transaction execution manager takes out subtransaction 1 and subtransaction 2 from the message queue and distributes them to subtransaction executor 1 and subtransaction executor 2 respectively. When subtransaction executor 1 successfully executes subtransaction 1, it sends the result of successful execution to the transaction execution manager. When subtransaction executor 2 fails to execute subtransaction 2, it sends the result of failed execution to the transaction execution manager. The transaction execution manager controls subtransaction executor 1 to roll back, and subtransaction executor 1 sends the result of successful rollback to the transaction execution manager.
[0106] Optionally, the transaction execution manager may write the execution result of the corresponding sub-transaction in the message queue into the message queue based on the execution result of the sub-transaction executed by the sub-transaction executor, for example, by marking the sub-transaction in the message queue and writing the execution result into the message queue, using different marks to represent the execution success or execution failure. Further, according to the execution result of the sub-transaction in the message queue, it can be determined whether the target transaction is executed successfully.
[0107] Optionally, when the execution results of multiple subtransactions are all successful, the execution result of the target transaction is determined to be successful; when the execution result of any subtransaction among the multiple subtransactions is failure, and the rollback of any other subtransaction except the subtransaction with the failed execution result fails, the execution result of the target transaction is determined to be an execution exception; when the execution result of any subtransaction among the multiple subtransactions is failure, and the rollback of other subtransactions except the subtransaction with the failed execution result is successful, the execution result of the target transaction is determined to be an execution failure.
[0108] Optionally, the transaction execution manager may record the execution result of the target transaction based on the determined execution result of the target transaction. For example, the execution result of the target transaction may be written into a log.
[0109] The transaction execution method provided in this embodiment obtains multiple sub-transactions from a message queue through a transaction execution manager, distributes the multiple sub-transactions to multiple sub-transaction executors, and controls the multiple sub-transactions to execute corresponding sub-transactions in parallel, thereby realizing concurrent execution of multiple sub-transactions. Based on the execution results of the multiple sub-transactions, the execution result of the target transaction can be determined, thereby improving the execution efficiency of the multiple sub-transactions.
[0110] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.
[0111] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0112] The above is a description of the method embodiment of the present application. The following is a description of the business processing device provided in the embodiment of the present application.
[0113] Figure 4 FIG. 1 is a schematic diagram of the structure of a transaction execution device provided in an embodiment of the present application. Figure 4 As shown, the device may include, for example: a determination module 401, a splitting module 402 and an execution module 403, and optionally, may also include a parsing module.
[0114] Determination module 401, used to determine whether the target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement;
[0115] The splitting module 402 is used for, when the target transaction is a large transaction, pre-querying the database according to the database query statement to obtain the data set to be operated, and splitting the target transaction into multiple sub-transactions according to the data set to be operated;
[0116] The execution module 403 is used to execute multiple sub-transactions.
[0117] In one possible implementation, the splitting module is specifically used to generate multiple first filtering conditions based on the first field in the data set to be operated, and each first filtering condition is added to the database query statement to form a first sub-database query statement corresponding to each first filtering condition, the first transaction corresponding to the first sub-database query statement is a sub-transaction of the target transaction, and the filtering range of each first filtering condition corresponds to a segment after the data set to be operated is segmented according to the first field.
[0118] In one possible implementation, the splitting module 402 is also used to generate multiple second filtering conditions based on the second field of the table to be queried when the operation type is an update operation, and add each second filtering condition to the database query statement to form a second sub-database query statement corresponding to each second filtering condition, the second transaction corresponding to the second sub-database query statement is a sub-transaction of the target transaction, and the filtering range of each second filtering condition corresponds to a segment after the table to be queried is segmented according to the second field.
[0119] In a possible implementation, the splitting module 402 is also used to replace the multiple sub-filtering conditions separated by the OR operator in the database query statement with any sub-filtering condition when the filter condition in the database query statement includes multiple sub-filtering conditions separated by the OR operator, so as to form a third sub-database query statement corresponding to any sub-filtering condition, and the third transaction corresponding to the third sub-database query statement is a sub-transaction of the target transaction.
[0120] In a possible implementation, the splitting module 402 is also used to divide the filter value range into multiple sub-value ranges when the filter value range in the filter condition in the database query statement exceeds a preset range. For any sub-value range, the filter value range in the database query statement is replaced with any sub-value range to form a fourth sub-database query statement corresponding to any sub-value range, and the third transaction corresponding to the fourth sub-database query statement is a sub-transaction of the target transaction.
[0121] In a possible implementation, before determining whether a target transaction corresponding to a database query statement is a large transaction based on statistical information of a table to be queried in the database query statement and an operation type of the database query statement, the parsing module is specifically used to parse the database query statement into a syntax tree; and determine the table to be queried in the database query statement and the operation type of the database query statement based on the syntax tree;
[0122] Accordingly, exemplarily, determination module 401 is specifically used to determine whether the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type. If the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, it is determined that the target transaction corresponding to the database query statement is a large transaction.
[0123] In a possible implementation, after splitting the target transaction into multiple sub-transactions according to the data set to be operated, the splitting module 402 is further used to add the multiple sub-transactions to the message queue;
[0124] Accordingly, exemplarily, the execution module is specifically used to obtain multiple sub-transactions from the message queue through the transaction execution manager, distribute the multiple sub-transactions to multiple sub-transaction executors, and control the multiple sub-transactions to execute corresponding sub-transactions in parallel.
[0125] In a possible implementation, the execution module 403 is also used to determine whether any subtransaction meets the condition for stopping execution when the execution of any subtransaction among multiple subtransactions fails. The condition for stopping execution includes: the current number of executions of any subtransaction reaches the subtransaction retry threshold, and / or it is determined that the current execution time of any subtransaction reaches the timeout time; if any subtransaction does not meet the condition for stopping execution, execute any subtransaction again; if any subtransaction meets the condition for stopping execution, determine that the execution result of any subtransaction is failure, and roll back other subtransactions except the subtransaction with a failed execution result.
[0126] In a possible implementation, the execution module 403 is further used to determine that the execution result of the target transaction is a successful execution when the execution results of multiple subtransactions are all successful; determine that the execution result of the target transaction is an abnormal execution when the execution result of any subtransaction among the multiple subtransactions is a failure, and any other subtransaction except the subtransaction with a failed execution result fails to roll back; determine that the execution result of the target transaction is an execution failure when the execution result of any subtransaction among the multiple subtransactions is a failure, and other subtransactions except the subtransaction with a failed execution result are rolled back successfully.
[0127] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.
[0128] Figure 5 Schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include: at least one processor 501 and a memory 502 .
[0129] The memory 502 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.
[0130] The memory 502 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.
[0131] The processor 501 is used to execute the computer-executable instructions stored in the memory 502 to implement the method of the aforementioned method embodiment. The processor 501 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.
[0132] Optionally, the electronic device may further include a communication interface 503. In specific implementation, if the communication interface 503, the memory 502 and the processor 501 are implemented independently, the communication interface 503, the memory 502 and the processor 501 may be interconnected via a bus and communicate with each other.
[0133] Optionally, in a specific implementation, if the communication interface 503, the memory 502 and the processor 501 are integrated on a chip, the communication interface 503, the memory 502 and the processor 501 can communicate through an internal interface.
[0134] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used to implement the actions of the above-mentioned method implementation method.
[0135] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one processor of the electronic device can read the execution instruction from the readable storage medium, and at least one processor executes the execution instruction so that the electronic device implements the actions of the above method implementation.
[0136] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.
[0137] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.
[0138] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0139] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0140] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0141] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A transaction execution method, characterized in that: include: Determine whether a target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement; In the case where the target transaction is a large transaction, a database is pre-queried according to the database query statement to obtain a data set to be operated, and the target transaction is split into multiple sub-transactions according to the data set to be operated; The plurality of sub-transactions are executed.
2. The method according to claim 1, characterized in that: The step of splitting the target transaction into a plurality of sub-transactions according to the data set to be operated includes: A plurality of first filtering conditions are generated according to the first field in the data set to be operated, and each of the first filtering conditions is added to the database query statement to form a first sub-database query statement corresponding to each of the first filtering conditions, wherein the first transaction corresponding to the first sub-database query statement is a sub-transaction of the target transaction, and the filtering range of each of the first filtering conditions corresponds to a segment after the data set to be operated is segmented according to the first field.
3. The method according to claim 1, characterized in that Also includes: In the case where the operation type is an update operation, multiple second filtering conditions are generated based on the second field of the table to be queried, and each of the second filtering conditions is added to the database query statement to form a second sub-database query statement corresponding to each second filtering condition, the second transaction corresponding to the second sub-database query statement is a sub-transaction of the target transaction, and the filtering range of each second filtering condition corresponds to a segment after the table to be queried is segmented according to the second field.
4. The method according to any one of claims 1 to 3, characterized in that: Also includes: In the case where the screening conditions in the database query statement include multiple sub-screening conditions separated by an OR operator, for any sub-screening condition, the multiple sub-screening conditions separated by an OR operator in the database query statement are replaced with the any sub-screening condition to form a third sub-database query statement corresponding to the any sub-screening condition, and the third transaction corresponding to the third sub-database query statement is a sub-transaction of the target transaction.
5. The method according to any one of claims 1 to 3, characterized in that: Also includes: When the filtering numerical range in the filtering condition in the database query statement exceeds the preset range, the filtering numerical range is divided into multiple sub-numerical ranges, and for any sub-numerical range, the filtering numerical range in the database query statement is replaced with any sub-numerical range to form a fourth sub-database query statement corresponding to any sub-numerical range, and the third transaction corresponding to the fourth sub-database query statement is a sub-transaction of the target transaction.
6. The method according to any one of claims 1 to 3, characterized in that: Before determining whether the target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement, the method further includes: Parsing the database query statement into a syntax tree; Determine, according to the syntax tree, a table to be queried in the database query statement and an operation type of the database query statement; The determining, based on the statistical information of the table to be queried in the database query statement and the operation type of the database query statement, whether the target transaction corresponding to the database query statement is a large transaction includes: Determine whether the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type; if the statistical information of the table to be queried exceeds the statistical information threshold corresponding to the operation type, determine that the target transaction corresponding to the database query statement is a large transaction.
7. The method according to any one of claims 1 to 3, characterized in that: After splitting the target transaction into a plurality of sub-transactions according to the data set to be operated, the method further includes: Adding the multiple sub-transactions to a message queue; The executing the plurality of sub-transactions comprises: The transaction execution manager obtains the multiple sub-transactions from the message queue, distributes the multiple sub-transactions to multiple sub-transaction executors, and controls the multiple sub-transactions to execute corresponding sub-transactions in parallel.
8. The method according to any one of claims 1 to 3, characterized in that: Also includes: In the case that the execution of any subtransaction among the multiple subtransactions fails, determining whether the subtransaction satisfies a condition for stopping execution, wherein the condition for stopping execution includes: the current execution count of the subtransaction reaches a subtransaction retry threshold, and / or determining that the current execution time of the subtransaction reaches a timeout time; If any of the sub-transactions does not meet the condition for stopping execution, then execute any of the sub-transactions again; If any of the subtransactions meets the condition for stopping execution, it is determined that the execution result of any of the subtransactions is failure, and the other subtransactions except the subtransaction with the failed execution result are rolled back.
9. The method according to claim 8, characterized in that Also includes: When the execution results of the multiple sub-transactions are all successful, determining that the execution result of the target transaction is successful; If the execution result of any subtransaction among the multiple subtransactions is failure, and any other subtransaction except the subtransaction with the failed execution result fails to be rolled back, then determining that the execution result of the target transaction is execution abnormality; If the execution result of any subtransaction among the multiple subtransactions is failure, and the other subtransactions except the subtransaction with the failed execution result are rolled back successfully, it is determined that the execution result of the target transaction is execution failure.
10. A transaction execution device, comprising: A determination module, used to determine whether a target transaction corresponding to the database query statement is a large transaction according to the statistical information of the table to be queried in the database query statement and the operation type of the database query statement; A splitting module, used for, when the target transaction is a large transaction, pre-querying the database according to the database query statement to obtain a data set to be operated, and splitting the target transaction into multiple sub-transactions according to the data set to be operated; An execution module is used to execute the multiple sub-transactions.
11. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.
13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 9 when being executed by a processor.