Execution method and device of to-be-executed statement, processor and electronic equipment
By monitoring the database memory-to-thread ratio, partitioning and concurrently executing statements, and utilizing a mutex execution statement table and message queue, the problem of insufficient database performance in high-concurrency systems was solved, thereby improving system stability and response speed.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- INDUSTRIAL AND COMMERCIAL BANK OF CHINA
- Filing Date
- 2023-06-07
- Publication Date
- 2026-05-01
AI Technical Summary
In existing technologies, databases in high-concurrency systems are prone to performance issues and unavailability when accessed by a large number of users, resulting in slow or unavailable application systems and failing to effectively improve the execution efficiency of statements to be executed.
By monitoring the ratio of current running memory to the number of threads in the database, statement partitioning is determined, and concurrent execution is performed based on the ratio and comparison results. Statements are stored in partitions using a mutex execution statement table and a message queue, prioritizing the execution of high-priority statements and statements without mutex locks. The database status and the number of idle threads are monitored in real time to adjust the execution strategy.
It improves the concurrent execution efficiency of pending statements, enhances the high availability of the system, reduces the probability of overall system unavailability caused by database unavailability, and ensures the stability and responsiveness of the system.
Smart Images

Figure CN116775704B_ABST
Abstract
Description
Methods and apparatus for executing statements to be executed, processors and electronic devices Technical Field
[0001] This application relates to the field of database technology, and more specifically, to a method and apparatus for executing statements to be executed, a processor, and an electronic device. Background Technology
[0002] Currently, in order to improve the request volume and response speed of high-concurrency systems, system clusters are often built online. Although increasingly mature clustering technology can support system clusters to handle multiple concurrent requests, production application systems still often become unavailable. For example, a large number of users concurrently accessing the system in a production environment can lead to unstable database operation, insufficient performance, and consequently, slow response or unavailability of the entire application system.
[0003] Currently, there is no effective solution to the problem of how to improve the execution efficiency of statements to be executed for multiple requests in related technologies. Summary of the Invention
[0004] The main objective of this application is to provide a method, apparatus, processor, and electronic device for executing statements to be executed, so as to solve the problem in the related art of how to improve the execution efficiency of statements to be executed corresponding to multiple requests.
[0005] To achieve the above objectives, according to one aspect of this application, a method for executing a statement to be executed is provided. The method includes: responding to an operation request from a target object to a database; determining the current running memory value of the database; comparing the current running memory value with a first preset threshold to obtain a comparison result; determining the ratio between the number of running threads and the total number of threads from all threads in the database; and concurrently executing the statement to be executed indicated by the operation request based on the comparison result and the ratio.
[0006] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if the ratio is determined to be less than or equal to the second preset threshold, then determining the statement partition corresponding to the statement to be executed; if multiple statement partitions are determined to exist, concurrently executing all statements to be executed within the multiple statement partitions.
[0007] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is less than the third preset threshold, then determining the statement partition corresponding to the statement to be executed; obtaining the partition priority of the statement partition; if multiple statement partitions are determined to exist, sorting the multiple statement partitions according to the partition priority to obtain a sorting result; obtaining at least two statement partitions with the highest partition priority from the sorting result, and concurrently executing all statements to be executed in the at least two statement partitions.
[0008] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0009] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any mutually exclusive execution statement in any set of mutually exclusive execution statements in the N sets, then the remaining mutually exclusive execution statements in any set are determined; if it is determined that the remaining mutually exclusive execution statements are inconsistent with the currently running execution statement, concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0010] Optionally, the preset mutual exclusion execution statement table is determined by: obtaining executed statements that have been concurrently executed from the historical operation log of the database; if it is determined that the execution time period of the executed statements exceeds a first preset time period, then generating the preset mutual exclusion execution statement table based on the executed statements and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statements.
[0011] Optionally, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
[0012] Optionally, before concurrently executing the statement to be executed based on the comparison result and the ratio, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
[0013] Optionally, the plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value includes at least one of the following: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; determining the statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold as the third .... The execution statement is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target execution statement, and the fourth target execution statement is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is determined as the fifth target execution statement, and the fifth target execution statement is stored in the fifth queue partition, and the fifth queue partition is determined as the statement partition corresponding to the fifth target execution statement.
[0014] Optionally, after concurrently executing all statements to be executed in the at least two statement partitions, the method further includes: obtaining other statement partitions from the sorting result, wherein the other statement partitions represent statement partitions other than the at least two statement partitions among the plurality of statement partitions; and concurrently executing all statements to be executed in the other statement partitions.
[0015] To achieve the above objectives, according to another aspect of this application, an execution apparatus for an executable statement is provided. The apparatus includes: a value determination module, configured to determine the current running memory value of the database in response to an operation request from a target object; a result obtaining module, configured to compare the current running memory value with a first preset threshold to obtain a comparison result; a ratio determination module, configured to determine the ratio between the number of running threads and the total number of threads from all threads in the database; and a statement execution module, configured to concurrently execute the executable statement indicated by the operation request based on the comparison result and the ratio.
[0016] This application employs the following steps: in response to a target object's operation request to the database, a comparison result is obtained between the current running memory value of the database and a first preset threshold. The ratio between the number of currently running threads and the total number of threads in the database is determined. Based on the comparison result and the ratio, the statement to be executed as indicated by the operation request is executed concurrently. By implementing concurrent execution of the statement to be executed, the problem of how to improve the execution efficiency of statements to be executed corresponding to multiple requests in related technologies is solved, thereby improving the concurrent execution efficiency of the statement to be executed and enhancing the high availability of the system. Attached Figure Description
[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0018] Figure 1 is a flowchart of the execution method of the statement to be executed according to an embodiment of this application;
[0019] Figure 2 is a schematic flowchart of obtaining a statement to be executed according to an embodiment of this application;
[0020] Figure 3 is a schematic flowchart of determining the weight value of a statement to be executed according to an embodiment of this application;
[0021] Figure 4 is a schematic flowchart of determining a preset mutually exclusive execution statement table according to an embodiment of this application;
[0022] Figure 5 is a schematic flowchart of a partitioned execution of statements to be executed according to an embodiment of this application;
[0023] Figure 6 is a schematic flowchart of executing a statement to be executed according to an embodiment of this application;
[0024] Figure 7 is a schematic diagram of an execution device for an executable statement provided according to an embodiment of this application;
[0025] Figure 8 is a schematic diagram of an optional electronic device provided according to an embodiment of this application. Detailed Implementation
[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0027] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:
[0030] Mutex (mutual exclusion) is a mechanism used in multithreaded programming to prevent two threads from simultaneously reading and writing to the same shared resource (such as a global variable). The existence of a mutex between different threads means that these threads cannot simultaneously read or write to the same shared resource.
[0031] SQL (Structured Query Language) is a database-oriented programming language used for querying and processing data in relational database systems. SQL statements are the programming language used to manipulate the database.
[0032] The present invention will now be described with reference to preferred implementation steps. Figure 1 is a flowchart of the execution method of the statement to be executed according to an embodiment of the present application. As shown in Figure 1, the method includes the following steps:
[0033] Step S102: Respond to the target object's operation request to the database and determine the current running memory value of the database;
[0034] Step S104: Compare the current running memory value with the first preset threshold to obtain the comparison result;
[0035] Optionally, the first preset threshold can be understood as the running memory value when the database is running normally, and the maximum running memory value when the database is running normally can be determined as the first preset threshold.
[0036] Step S106: Determine the ratio between the number of running threads and the total number of threads from all threads in the database;
[0037] Step S108: Execute the statement to be executed as indicated by the operation request concurrently based on the comparison result and the ratio.
[0038] The execution method for pending statements provided in this application embodiment, in response to the target object's operation request to the database, obtains a comparison result between the current running memory value of the database and a first preset threshold, and determines the ratio between the number of running threads and the total number of threads from all threads in the database. Based on the comparison result and the ratio, the pending statements indicated by the operation request can be executed concurrently. By achieving concurrent execution of pending statements, the problem of how to improve the execution efficiency of pending statements corresponding to multiple requests in related technologies is solved, thereby improving the concurrent execution efficiency of pending statements and enhancing the high availability of the system.
[0039] Optionally, in one embodiment, if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is greater than or equal to a third preset threshold, then the execution of the statement to be executed is stopped; or, if the statement to be executed exists in a preset mutually exclusive execution statement table, and the second weight value of the statement to be executed is less than a fifth preset threshold, then the execution of the statement to be executed is stopped; or, if the statement to be executed does not exist in the preset mutually exclusive execution statement table, but the first weight value of the statement to be executed is greater than a fourth preset threshold, then the execution of the statement to be executed is stopped.
[0040] Optionally, step S108 above may also include the following implementation steps: Step S11, if the comparison result is used to indicate that the current running memory value is less than or equal to the first preset threshold, and if the ratio is determined to be less than or equal to the second preset threshold, then the statement partition corresponding to the statement to be executed is determined; Step S12, if multiple statement partitions are determined to exist, all statements to be executed in the multiple statement partitions are executed concurrently.
[0041] Optionally, the aforementioned second preset threshold can be understood as the ratio between the number of non-running threads (i.e., idle threads) and the total number of threads in the database. For example, it can be 0.5, but it is not limited to this.
[0042] Optionally, step S108 may also include other implementation steps: Step S21, if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is less than the third preset threshold, then the statement partition corresponding to the statement to be executed is determined; Step S22, the partition priority of the statement partition is obtained; Step S23, if multiple statement partitions are determined to exist, the multiple statement partitions are sorted according to the partition priority to obtain a sorting result; Step S24, at least two statement partitions with the highest partition priority are obtained from the sorting result, and all statements to be executed in the at least two statement partitions are executed concurrently.
[0043] Optionally, the second preset threshold is less than the third preset threshold.
[0044] Optionally, a ratio less than a third preset threshold indicates that the number of running threads is less than the total number of threads, meaning that there are enough idle threads. For example, the third preset threshold can be 1, but it is not limited to this.
[0045] Optionally, in this embodiment, the implementation process of concurrent execution of all statements to be executed in the at least two statement partitions can also be described by the following technical solution: obtain N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with mutual exclusion lock relationships, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then all statements to be executed in the at least two statement partitions are executed concurrently.
[0046] Optionally, step S24 may further include: step S2401, obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; step S2402, for each of the statements to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any of the mutually exclusive execution statements in any set of mutually exclusive execution statements in the N sets, then the remaining mutually exclusive execution statements in any set are determined; step S2403, if it is determined that the remaining mutually exclusive execution statements are inconsistent with the execution statement currently running, then all the statements to be executed in the at least two statement partitions are executed concurrently.
[0047] Optionally, in step S2403, if it is determined that the remaining mutually exclusive execution statements are consistent with the execution statement currently running, the concurrent execution of all statements to be executed in the at least two statement partitions is stopped.
[0048] Optionally, in the above embodiments, the preset mutual exclusion execution statement table can be determined by: obtaining executed statements that have been concurrently executed from the historical operation log of the database; if it is determined that the execution time period of the executed statements exceeds a first preset time period, then generating the preset mutual exclusion execution statement table based on the executed statements and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statements.
[0049] Optionally, in this embodiment, if it is determined that the execution time period of the executed statement does not exceed a preset time period, or there is no mutually exclusive execution statement that has a mutual exclusion lock relationship with the executed statement, then the executed statement is not added to the preset mutually exclusive execution statement table.
[0050] Optionally, based on the above embodiments, the statement partition corresponding to the statement to be executed is further determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
[0051] Optionally, before executing the scheme of concurrently executing the statement to be executed based on the comparison result and the ratio in step S108 above, the statement partition corresponding to the statement to be executed can be determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
[0052] Optionally, the process of determining the first weight value of the statement to be executed based on the number of concurrent executions includes the following steps: determining the first initial weight value of the statement to be executed; obtaining the first quotient of the number of concurrent executions and the preset number; and determining the sum of the first quotient and the first initial weight value as the first weight value.
[0053] Optionally, when the preset number of times cannot be divided by the number of concurrent executions, the remainder obtained by dividing the number of concurrent executions by the preset number can be rounded up, and the sum of the rounded result, the first quotient, and the first initial weight value can be determined as the first weight value.
[0054] Optionally, the process of determining the second weight value of the statement to be executed based on the concurrent execution time may include the following steps: determining the second initial weight value of the statement to be executed; obtaining the second quotient of the concurrent execution time and the second preset time period, and determining the sum of the second quotient and the second initial weight value as the second weight value; optionally, when the preset number cannot be divided by the number of concurrent executions, the remainder obtained by dividing the number of concurrent executions by the preset number is rounded up, and the sum of the rounded result, the first quotient, and the first initial weight value is determined as the first weight value.
[0055] Optionally, in this embodiment, the plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; then, for the implementation scheme of determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value, it includes at least one of the following: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; determining the statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold as the third target statement to be executed. The third target statement to be executed is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target statement to be executed; the statement to be executed that has a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target statement to be executed, and the fourth target statement to be executed is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target statement to be executed; the statement to be executed that has a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is determined as the fifth target statement to be executed, and the fifth target statement to be executed is stored in the fifth queue partition, and the fifth queue partition is determined as the statement partition corresponding to the fifth target statement to be executed.
[0056] Optionally, the aforementioned fourth preset threshold can be understood, for example, as the average of the first weight values of all statements to be executed, and the aforementioned fifth preset threshold can be understood, for example, as the average of the second weight values of all statements to be executed, but is not limited thereto.
[0057] Optionally, after executing the scheme of concurrently executing all statements to be executed in the at least two statement partitions in step S2403, other statement partitions can be obtained from the sorting result, wherein the other statement partitions refer to statement partitions other than the at least two statement partitions among the plurality of statement partitions; and all statements to be executed in the other statement partitions are executed concurrently.
[0058] In this embodiment, the process of concurrently executing all statements to be executed in the other statement partitions can refer to the above-described process of concurrently executing all statements to be executed in the at least two statement partitions, and will not be repeated here.
[0059] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0060] This application embodiment also provides an optional method for executing the statement to be executed, in conjunction with Figures 2 to 6, specifically including the following steps:
[0061] Step 1: Save the SQL statements corresponding to the system functions;
[0062] Specifically, this step can be explained with reference to Figure 2. As shown in Figure 2, Figure 2 is a schematic flowchart of obtaining a statement to be executed according to an embodiment of this application, including the following steps:
[0063] Step S201: The test library simulates the production library;
[0064] Step S202: Conduct a comprehensive functional test of the test environment;
[0065] Step S203: Parse the SQL statement log table;
[0066] Step S204: Save the SQL statement to the log table (sql_exec_log);
[0067] Step S205: The relationship between the function and the SQL statement is saved to the relation table (sql_func_log).
[0068] The above steps simulate the data of the online database (i.e., the production database) in the test environment, ensuring data consistency between the test database and the online database. Subsequently, a comprehensive functional test is performed on the test environment system. All SQL statements involved in the concurrent execution of system functions, as well as the relationship between system functions and SQL statements, are output to the SQL log file. This allows for parsing of the SQL log file to obtain the SQL statements and the relationship between system functions and SQL statements. The SQL statements are saved to the log table sql_exec_log, and the relationship between system functions and SQL statements is saved to the relationship table sql_func_log.
[0069] Step 2: Calculate the weight of the SQL statement;
[0070] Specifically, this step can be explained with reference to Figure 3. As shown in Figure 3, Figure 3 is a schematic flowchart of determining the weight value of a statement to be executed according to an embodiment of this application, including the following steps:
[0071] Step S301: Monitor online system user behavior logs in real time;
[0072] Step S302: Analyze the system functions running in the behavior log, determine the system functions that can be executed concurrently, and determine the number of times the system functions are executed concurrently;
[0073] Optionally, the number of times the SQL statements corresponding to the concurrently executed system functions are executed can be determined as the number of times the system functions are executed concurrently;
[0074] Step S303: Save to table func_complicated;
[0075] The above steps, through real-time monitoring and analysis of online system user behavior logs, can determine which system functions support concurrent execution and obtain the number of requests for system functions per day. This information can be recorded in the table func_complicated.
[0076] Step S304: Query the number of concurrent SQL statement executions recorded in the func_complicated table;
[0077] In this step, the func_complicated table can be queried in the test environment to execute the SQL statements corresponding to the concurrent execution function based on the sql_exec_log and sql_func_log tables from steps S204 to S205.
[0078] Step S304 may include the following steps:
[0079] Step S3041: Determine the first weight value of the SQL statement based on the number of concurrent executions of the SQL statement. For example, set the initial value of the first weight value to 0, and increment the first weight value by 1 every 5 executions. The higher the weight value, the more important the system function.
[0080] Step S3042: Record the SQL statement and its first weight value in the table weight_func_use;
[0081] Step S305: Execute the corresponding SQL statements in the tables sql_exec_log and sql_func_log to obtain the concurrent execution time of the SQL statements;
[0082] Step S305 may include the following steps:
[0083] Step S3051: Determine the second weight value of the SQL statement based on the concurrent execution time of the SQL statement. For example, set the initial value of the second weight value to 0, and increment the second weight value by 1 every 5 (or 3, this application does not limit) SQL statements.
[0084] Step S3052: When the SQL statement execution ends, record the SQL statement and its second weight value in the table weight_func_aql.
[0085] Step 3: Analyze the impact of all SQL statements in the sql_exec_log table on database performance;
[0086] In this step, the SQL statements in the queried log table sql_exec_log are executed, the execution time of the SQL statements is recorded, and the lock records involved in the SQL statements are analyzed. The execution time of the SQL statements and the lock records involved are then recorded in the sql_ana_info table (equivalent to the aforementioned preset mutex execution statement table).
[0087] Specifically, this step can be explained with reference to Figure 4. As shown in Figure 4, which is a schematic flowchart of determining a preset mutually exclusive execution statement list according to an embodiment of this application, the steps include:
[0088] Step S401: Query the SQL statements in the sql_exec_log table;
[0089] Step S402: Analyze the execution time of the SQL statement and the mutex lock relationship of the SQL statement;
[0090] Step S403: Determine whether the execution time of the SQL statement exceeds 5 seconds; if yes, proceed to step S405; otherwise, proceed to step S404.
[0091] Step S404: Do not save;
[0092] Step S405: Record the SQL statement execution time and the mutex lock relationships involved in the SQL statement in the sql_ana_info table. If the SQL statement does not have a mutex lock relationship, or if the execution time of the SQL statement does not exceed 5 seconds, then this SQL statement is not recorded in the sql_ana_info table.
[0093] Step 4: After saving the SQL statements to the message queue partitions based on their weight values, execute the SQL statements from different partitions concurrently:
[0094] Specifically, this step can be explained with reference to Figure 5. As shown in Figure 5, Figure 5 is a schematic flowchart of a partitioned execution of statements to be executed according to an embodiment of this application, including the following steps:
[0095] Step S501: After the user sends a request to the application system, determine whether an operation request for an SQL statement has been generated; if yes, proceed to step S502; otherwise, send a message to the user that no operation request for an SQL statement exists.
[0096] Step S502: Calculate the weight of the SQL statement;
[0097] Step S503: Save the SQL statements to different partitions of the message queue according to their weights;
[0098] In this step, when the application system receives a user's request to operate on the data, the application system sends an SQL statement to the message queue. The message queue has multiple partitions, such as partition 0, partition 1, partition 2, and partition 3.
[0099] After step S503 is executed, step S507 can be executed.
[0100] Step S503 further includes the following steps:
[0101] Step S5031: For partition 0 (equivalent to the first queue partition mentioned above), save the SQL statements that are not recorded in the sql_ana_info table; or, in other cases, for partition 0 (equivalent to the second queue partition mentioned above), save the SQL statements whose first weight is less than a preset threshold and whose second weight is less than a preset threshold.
[0102] Step S5032: For partition 1 (i.e., the third queue partition mentioned above), save the SQL statements whose first weight is less than the fourth preset threshold and whose second weight is greater than the fifth preset threshold.
[0103] Step S5033, for partition 2 (i.e. the fourth queue partition mentioned above): save SQL statements where the first weight is greater than the fourth preset threshold and the second weight is greater than the fifth preset threshold;
[0104] Step S5034, for partition 3 (i.e. the fifth queue partition mentioned above): save the SQL statements with the first weight greater than the fourth preset threshold and the second weight less than the fifth preset threshold;
[0105] Step S504: Query the database status and the number of idle threads;
[0106] Step S505: Determine whether the database performance is low and there are no idle threads. If yes, proceed to step S506; otherwise, proceed to step S507.
[0107] Step S506: Pause the operation request to send SQL statements to the database;
[0108] Step S507: Obtain the SQL statements stored in different partitions;
[0109] Specifically, in step S507, the following steps can also be implemented:
[0110] Step S5071: If it is determined that the database is in good condition and there are a large number of idle threads, SQL statements in the four partitions of the message queue can be executed concurrently.
[0111] Step S5072: If it is determined that the database performance is low and there are idle threads, prioritize executing SQL statements in partition 0 and partition 1 concurrently.
[0112] Step S508: Determine whether the current partition SQL (i.e., partition 0,1) and the SQL executed in the data exist in a mutual exclusion table with a mutual exclusion lock relationship; if yes, proceed to step S510; otherwise, proceed to step S509.
[0113] Step S509: Execute SQL statements according to the operation request;
[0114] Step S510: Continue reading the SQL statements in partitions 2 and 3; partitions 2 and 3, which are to be read further, can be understood as the other statement partitions mentioned above.
[0115] Step S511: Determine whether the current partition SQL (i.e., partitions 2 and 3) and the SQL to be executed in the data exist in a mutual exclusion table with a mutual exclusion lock relationship; if yes, proceed to step S507; otherwise, proceed to step S509.
[0116] Step 5: Monitor the data status and the number of idle threads in the database in real time to determine whether to execute the SQL statements in the message queue.
[0117] Specifically, this step can be explained with reference to Figure 6. As shown in Figure 6, Figure 6 is a schematic flowchart of executing a statement to be executed according to an embodiment of this application, including the following steps:
[0118] Step S601, monitoring alarm; by monitoring the database status and the number of idle database threads in real time, it can be determined which partition's SQL statement to execute.
[0119] Step S602: Once the real-time monitoring module is identified, it can be connected to the monitoring alarm.
[0120] Step S603: Query the SQL statements that are currently being executed and that are affecting data performance;
[0121] Step S604: Determine whether the SQL statement queried in step S603 is in the table weight_func_sql; if yes, proceed to step S606; otherwise, proceed to step S605.
[0122] Step S605: If no action is taken, promptly notify frontline maintenance personnel for handling.
[0123] Step S606: Determine whether the SQL statement queried in step S603 is in the table sql_ana_info; if yes, proceed to step S607; otherwise, proceed to step S605.
[0124] Step S607: Determine whether the SQL statement queried in step S603 is in the table func_complicated; if yes, proceed to step S608; otherwise, proceed to step S609.
[0125] Step S608: Then determine whether the weight of the SQL statement is relatively small; if yes, proceed to step S609; if no, proceed to step S605.
[0126] Step S609: Terminate the execution of the SQL statement.
[0127] In the above steps, if the database is in good condition and there are a large number of idle threads, the SQL statements in the four partitions of the message queue from step four will be executed concurrently. If the database performance is currently low but there are idle threads, the SQL statements in partitions 0 and 1 will be executed concurrently first. If the database performance is low and there are no idle threads, sending SQL statement execution requests to the database will be paused.
[0128] Before executing the SQL statement in partition 1, the system first checks if the SQL statement in partition 1 exists in the `sql_ana_info` table, thus determining if a mutual exclusion lock exists between the SQL statement in the current database and the SQL statement in partition 1. If it does, the SQL statement in partition 1 is not executed; otherwise, it is executed. Similarly, after executing the SQL statement in partition 1, the same method is used to determine if a mutual exclusion lock exists between the SQL statement in partitions 2 and 3 and the SQL statement in those partitions. If a mutual exclusion lock exists, the SQL statement in those partitions is not executed; otherwise, it is executed. This implementation can be repeated to determine mutual exclusion lock relationships for SQL statements in different partitions.
[0129] Furthermore, in the above scheme, if the database performance is currently low and there are no idle threads, the sending of SQL statement execution requests to the database can be paused. For example, if the SQL statement (weight value obtained at execution time) in the table weight_func_sql exists in the table sql_ana_info (whether there is a lock after timeout) and the table weight_func_use, and the first weight value retrieved from the table weight_func_use is less than the fourth preset threshold, then the execution of the SQL statement is terminated.
[0130] Alternatively, for SQL statements that do not exist in the sql_ana_info table but are being executed, if the execution time has exceeded the preset duration (e.g., 5 seconds), their execution will be terminated.
[0131] By following the steps above, SQL statements are stored in different partitions of the message queue based on their weight values when requested by the high-concurrency system. This allows for partitioned and concurrent execution of SQL statements within the message queue, reducing the interdependencies of numerous SQL statements during high-concurrency execution, improving system availability and database stability, and thus reducing the probability of system-wide unavailability due to database failure. Simultaneously, real-time monitoring and alerts are implemented. For SQL statements that severely impact performance, their execution is terminated, and information about the terminated SQL statements is sent to operations and development personnel in real time for timely remediation, reducing the workload of operations staff and effectively ensuring the stable operation of the system to respond normally to user requests.
[0132] This application also provides an execution device for an executable statement. It should be noted that the execution device for the executable statement provided in this application can be used to execute the execution method for the executable statement provided in this application. The execution device for the executable statement provided in this application will be described below.
[0133] Figure 7 is a schematic diagram of an execution device for an executable statement according to an embodiment of this application. As shown in Figure 7, the device includes:
[0134] The numerical determination module 72 is used to respond to the target object's operation request to the database and determine the current running memory value of the database;
[0135] Module 74 is used to compare the current running memory value with a first preset threshold to obtain a comparison result.
[0136] The ratio determination module 76 is used to determine the ratio between the number of running threads and the total number of threads from all threads in the database;
[0137] Statement execution module 78 is used to concurrently execute the statement to be executed indicated by the operation request based on the comparison result and the ratio.
[0138] The execution device for the statement to be executed provided in this application embodiment responds to the target object's operation request to the database, obtains a comparison result between the current running memory value of the database and a first preset threshold, and determines the ratio between the number of running threads and the total number of threads from all threads in the database. It can concurrently execute the statement to be executed indicated by the operation request according to the comparison result and the ratio. By realizing the concurrent execution of the statement to be executed, it solves the problem in related technologies of how to improve the execution efficiency of the statement to be executed corresponding to multiple requests, thereby achieving the effect of improving the concurrent execution efficiency of the statement to be executed and improving the high availability of the system.
[0139] Optionally, in one embodiment, the execution device for the statement to be executed further includes a stop execution module, configured to, when determining that the comparison result indicates that the current running memory value is greater than the first preset threshold, stop executing the statement to be executed if the ratio is determined to be greater than or equal to a third preset threshold; or, if the statement to be executed exists in a preset mutually exclusive execution statement table and the second weight value of the statement to be executed is less than a fifth preset threshold, stop executing the statement to be executed; or, if the statement to be executed does not exist in the preset mutually exclusive execution statement table, but the first weight value of the statement to be executed is greater than a fourth preset threshold, stop executing the statement to be executed.
[0140] Optionally, the statement execution module 78 described above is further configured to perform the following implementation steps: Step S11, if it is determined that the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if it is determined that the ratio is less than or equal to the second preset threshold, then the statement partition corresponding to the statement to be executed is determined; Step S12, if it is determined that there are multiple statement partitions, all statements to be executed in the multiple statement partitions are executed concurrently.
[0141] Optionally, the aforementioned second preset threshold can be understood as the ratio between the number of non-running threads (i.e., idle threads) and the total number of threads in the database. For example, it can be 0.5, but it is not limited to this.
[0142] Optionally, the statement execution module 78 described above is further configured to perform other implementation steps: Step S21, if it is determined that the comparison result indicates that the current running memory value is greater than the first preset threshold, and if it is determined that the ratio is less than the third preset threshold, then the statement partition corresponding to the statement to be executed is determined; Step S22, the partition priority of the statement partition is obtained; Step S23, if it is determined that there are multiple statement partitions, the multiple statement partitions are sorted according to the partition priority to obtain a sorting result; Step S24, at least two statement partitions with the highest partition priority are obtained from the sorting result, and all statements to be executed in the at least two statement partitions are executed concurrently.
[0143] Optionally, a ratio less than a third preset threshold indicates that the number of running threads is less than the total number of threads, meaning that there are enough idle threads. For example, the third preset threshold can be 1, but it is not limited to this.
[0144] Optionally, in this embodiment, the statement execution module 78 further includes a first execution unit, configured to obtain N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, wherein N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then all statements to be executed in the at least two statement partitions are executed concurrently.
[0145] Optionally, the statement execution module 78 further includes a second execution unit, used to perform the following steps: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each of the statements to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any of the mutually exclusive execution statements in any set of mutually exclusive execution statements in the N sets, then the remaining mutually exclusive execution statements in any set are determined; if it is determined that the remaining mutually exclusive execution statements are inconsistent with the execution statement currently running, all statements to be executed in the at least two statement partitions are executed concurrently.
[0146] Optionally, the second execution unit is further configured to, upon determining that the remaining mutually exclusive execution statements are consistent with the execution statement currently running, stop the concurrent execution of all statements to be executed in the at least two statement partitions.
[0147] Optionally, the first execution unit or the second execution unit may also determine the preset mutual exclusion execution statement table in the following way: obtain the executed statements that have been executed concurrently from the historical operation log of the database; if it is determined that the execution time period of the executed statements exceeds a first preset time period, then generate the preset mutual exclusion execution statement table based on the executed statements and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statements.
[0148] Optionally, in this embodiment, if it is determined that the execution time period of the executed statement does not exceed a preset time period, or there is no mutually exclusive execution statement that has a mutual exclusion lock relationship with the executed statement, then the executed statement is not added to the preset mutually exclusive execution statement table.
[0149] Optionally, the statement execution module 78 described above is further configured to determine the statement partition corresponding to the statement to be executed by: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
[0150] Optionally, the statement execution module 78 is further configured to determine the statement partition corresponding to the statement to be executed by the following method before concurrently executing the statement to be executed according to the comparison result and the ratio: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; and determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
[0151] Optionally, the statement execution module 78 is further configured to determine a first initial weight value for the statement to be executed; obtain a first quotient of the concurrent execution count and a preset count; and determine the sum of the first quotient and the first initial weight value as the first weight value. Optionally, when the preset count cannot be divided evenly by the concurrent execution count, the remainder obtained by dividing the concurrent execution count by the preset count can be rounded up, and the sum of the rounded result, the first quotient, and the first initial weight value can be determined as the first weight value.
[0152] Optionally, the statement execution module 78 is further configured to determine a second initial weight value for the statement to be executed; obtain a second quotient value between the concurrent execution time and a second preset time period; and determine the sum of the second quotient value and the second initial weight value as the second weight value. Optionally, when the preset number of times cannot be divided evenly by the number of concurrent executions, the remainder obtained by dividing the number of concurrent executions by the preset number is rounded up, and the sum of the rounded result, the first quotient value, and the first initial weight value is determined as the first weight value.
[0153] Optionally, in this embodiment, the plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; then, the statement execution module 78 is further configured to implement a scheme for determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; determining the statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold as the third target statement to be executed. The third target statement to be executed is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target statement to be executed; the statement to be executed that has a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target statement to be executed, and the fourth target statement to be executed is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target statement to be executed; the statement to be executed that has a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is determined as the fifth target statement to be executed, and the fifth target statement to be executed is stored in the fifth queue partition, and the fifth queue partition is determined as the statement partition corresponding to the fifth target statement to be executed.
[0154] Optionally, the statement execution module 78 described above is further configured to, after performing concurrent execution of all statements to be executed in the at least two statement partitions, obtain other statement partitions from the sorting results, wherein the other statement partitions represent statement partitions other than the at least two statement partitions among the plurality of statement partitions; and perform concurrent execution of all statements to be executed in the other statement partitions.
[0155] The execution device for the statement to be executed includes a processor and a memory. The aforementioned first execution unit and the like are all stored in the memory as program units, and the processor executes the aforementioned program units stored in the memory to realize the corresponding functions.
[0156] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of improving the execution efficiency of statements corresponding to multiple requests in related technologies.
[0157] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0158] This invention provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements a method for executing the statement to be executed.
[0159] This invention provides a processor for running a program, wherein the program executes a method for executing the statement to be executed.
[0160] As shown in Figure 8, an embodiment of the present invention provides an electronic device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: responding to an operation request from a target object to a database, determining the current running memory value of the database; comparing the current running memory value with a first preset threshold to obtain a comparison result; determining the ratio between the number of running threads and the total number of threads in the database; and concurrently executing the statements to be executed indicated by the operation request based on the comparison result and the ratio.
[0161] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if the ratio is determined to be less than or equal to the second preset threshold, then determining the statement partition corresponding to the statement to be executed; if multiple statement partitions are determined to exist, concurrently executing all statements to be executed within the multiple statement partitions.
[0162] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is less than the third preset threshold, then determining the statement partition corresponding to the statement to be executed; obtaining the partition priority of the statement partition; if multiple statement partitions are determined to exist, sorting the multiple statement partitions according to the partition priority to obtain a sorting result; obtaining at least two statement partitions with the highest partition priority from the sorting result, and concurrently executing all statements to be executed in the at least two statement partitions.
[0163] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0164] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any mutually exclusive execution statement in any set of mutually exclusive execution statements in the N sets, then the remaining mutually exclusive execution statements in any set are determined; if it is determined that the remaining mutually exclusive execution statements are inconsistent with the currently running execution statement, concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0165] Optionally, the preset mutual exclusion execution statement table is determined by: obtaining executed statements that have been concurrently executed from the historical operation log of the database; if it is determined that the execution time period of the executed statements exceeds a first preset time period, then generating the preset mutual exclusion execution statement table based on the executed statements and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statements.
[0166] Optionally, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
[0167] Optionally, before concurrently executing the statement to be executed based on the comparison result and the ratio, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
[0168] Optionally, the plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value includes at least one of the following: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; determining the statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold as the third .... The execution statement is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target execution statement, and the fourth target execution statement is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is determined as the fifth target execution statement, and the fifth target execution statement is stored in the fifth queue partition, and the fifth queue partition is determined as the statement partition corresponding to the fifth target execution statement.
[0169] Optionally, after concurrently executing all statements to be executed in the at least two statement partitions, the method further includes: obtaining other statement partitions from the sorting result, wherein the other statement partitions represent statement partitions other than the at least two statement partitions among the plurality of statement partitions; and concurrently executing all statements to be executed in the other statement partitions.
[0170] The devices mentioned in this article can be servers, PCs, tablets, mobile phones, etc.
[0171] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: responding to a target object's operation request to a database, determining the current running memory value of the database; comparing the current running memory value with a first preset threshold to obtain a comparison result; determining the ratio between the number of running threads and the total number of threads from all threads in the database; and concurrently executing the statement to be executed indicated by the operation request based on the comparison result and the ratio.
[0172] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if the ratio is determined to be less than or equal to the second preset threshold, then determining the statement partition corresponding to the statement to be executed; if multiple statement partitions are determined to exist, concurrently executing all statements to be executed within the multiple statement partitions.
[0173] Optionally, concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is less than the third preset threshold, then determining the statement partition corresponding to the statement to be executed; obtaining the partition priority of the statement partition; if multiple statement partitions are determined to exist, sorting the multiple statement partitions according to the partition priority to obtain a sorting result; obtaining at least two statement partitions with the highest partition priority from the sorting result, and concurrently executing all statements to be executed in the at least two statement partitions.
[0174] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0175] Optionally, concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any mutually exclusive execution statement in any set of mutually exclusive execution statements in the N sets, then the remaining mutually exclusive execution statements in any set are determined; if it is determined that the remaining mutually exclusive execution statements are inconsistent with the currently running execution statement, concurrent execution of all statements to be executed in the at least two statement partitions is performed.
[0176] Optionally, the preset mutual exclusion execution statement table is determined by: obtaining executed statements that have been concurrently executed from the historical operation log of the database; if it is determined that the execution time period of the executed statements exceeds a first preset time period, then generating the preset mutual exclusion execution statement table based on the executed statements and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statements.
[0177] Optionally, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
[0178] Optionally, before concurrently executing the statement to be executed based on the comparison result and the ratio, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
[0179] Optionally, the plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value includes at least one of the following: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; determining the statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold as the third .... The execution statement is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target execution statement, and the fourth target execution statement is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target execution statement; the execution statement with a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is determined as the fifth target execution statement, and the fifth target execution statement is stored in the fifth queue partition, and the fifth queue partition is determined as the statement partition corresponding to the fifth target execution statement.
[0180] Optionally, after concurrently executing all statements to be executed in the at least two statement partitions, the method further includes: obtaining other statement partitions from the sorting result, wherein the other statement partitions represent statement partitions other than the at least two statement partitions among the plurality of statement partitions; and concurrently executing all statements to be executed in the other statement partitions.
[0181] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0182] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.
[0183] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0184] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0185] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0186] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0187] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0188] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0190] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for executing a statement to be executed, characterized in that, include: In response to the target object's operation request to the database, determine the current running memory value of the database; The current running memory value is compared with a first preset threshold to obtain a comparison result; Determine the ratio between the number of running threads and the total number of threads in the database; The statement to be executed concurrently according to the comparison result and the ratio is executed in accordance with the operation request; wherein, the concurrent execution of the statement to be executed according to the comparison result and the ratio includes: if it is determined that the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if it is determined that the ratio is less than or equal to the second preset threshold, then the statement partition corresponding to the statement to be executed is determined; if it is determined that multiple statement partitions exist, all statements to be executed in the multiple statement partitions are executed concurrently.
2. The method according to claim 1, characterized in that, The method of concurrently executing the statement to be executed based on the comparison result and the ratio includes: if the comparison result indicates that the current running memory value is greater than the first preset threshold, and if the ratio is less than the third preset threshold, then determining the statement partition corresponding to the statement to be executed; obtaining the partition priority of the statement partition; if multiple statement partitions are determined to exist, sorting the multiple statement partitions according to the partition priority to obtain a sorting result; obtaining at least two statement partitions with the highest partition priority from the sorting result, and concurrently executing all statements to be executed in the at least two statement partitions.
3. The method according to claim 2, characterized in that, Concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with mutual exclusion lock relationships, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is inconsistent with each mutually exclusive execution statement in each set of mutually exclusive execution statements in the N sets of mutually exclusive execution statements, then concurrent execution of all statements to be executed in the at least two statement partitions is performed.
4. The method according to claim 2, characterized in that, Concurrent execution of all statements to be executed in the at least two statement partitions includes: obtaining N sets of mutually exclusive execution statements from a preset mutually exclusive execution statement table, wherein each of the N sets of mutually exclusive execution statements contains at least two mutually exclusive execution statements with a mutual exclusion lock relationship, where N is a positive integer; for each statement to be executed in the at least two statement partitions, if it is determined that each statement to be executed is consistent with any mutually exclusive execution statement in any of the N sets of mutually exclusive execution statements, then the remaining mutually exclusive execution statements in any set are determined; if it is determined that the remaining mutually exclusive execution statements are inconsistent with the currently running execution statement, concurrent execution of all statements to be executed in the at least two statement partitions is performed.
5. The method according to any one of claims 3 and 4, characterized in that, The preset mutual exclusion execution statement table is determined as follows: Executed statements that have been concurrently executed are obtained from the historical operation log of the database; if it is determined that the execution time period of the executed statement exceeds a first preset time period, the preset mutual exclusion execution statement table is generated based on the executed statement and the mutual exclusion execution statements that have a mutual exclusion lock relationship with the executed statement.
6. The method according to claim 5, characterized in that, The statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions, and the multiple queue partitions include at least a first queue partition; determining all mutually exclusive execution statements in a preset mutually exclusive execution statement table as the first target statement to be executed; storing the first target statement to be executed in the first queue partition, and determining the first queue partition as the statement partition corresponding to the first target statement to be executed; wherein the preset mutually exclusive statement table is generated based on the executed statements in the database and the mutually exclusive execution statements that have a mutual exclusion lock relationship with the executed statements.
7. The method according to any one of claims 1 to 4, characterized in that, Before concurrently executing the statement to be executed based on the comparison result and the ratio, the statement partition corresponding to the statement to be executed is determined by the following method: obtaining a preset message queue for partitioning and storing the statement to be executed, wherein the preset message queue is configured with multiple queue partitions; obtaining the number of concurrent executions of the statement to be executed in the test environment and the concurrent execution time of the statement to be executed in the test environment; determining a first weight value of the statement to be executed based on the number of concurrent executions, and determining a second weight value of the statement to be executed based on the concurrent execution time; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value.
8. The method according to claim 7, characterized in that, The plurality of queue partitions includes at least one of the following: a second queue partition, a third queue partition, a fourth queue partition, and a fifth queue partition; determining the statement partition corresponding to the statement to be executed based on the first weight value and the second weight value includes at least one of the following: determining the statement to be executed with a first weight value less than a fourth preset threshold and a second weight value less than a fifth preset threshold as the second target statement to be executed, storing the second target statement to be executed in the second queue partition, and determining the second queue partition as the statement partition corresponding to the second target statement to be executed; The statement to be executed with a first weight value less than or equal to a fourth preset threshold and a second weight value greater than or equal to a fifth preset threshold is determined as the third target statement to be executed. The third target statement to be executed is stored in the third queue partition, and the third queue partition is determined as the statement partition corresponding to the third target statement to be executed. The statement to be executed with a first weight value greater than a fourth preset threshold and a second weight value greater than a fifth preset threshold is determined as the fourth target statement to be executed. The fourth target statement to be executed is stored in the fourth queue partition, and the fourth queue partition is determined as the statement partition corresponding to the fourth target statement to be executed. The statement to be executed that has a first weight value greater than a fourth preset threshold and a second weight value less than a fifth preset threshold is identified as the fifth target statement to be executed. The fifth target statement to be executed is stored in the fifth queue partition, and the fifth queue partition is identified as the statement partition corresponding to the fifth target statement to be executed.
9. The method according to claim 2, characterized in that, After concurrently executing all statements to be executed in the at least two statement partitions, the method further includes: obtaining other statement partitions from the sorting result, wherein the other statement partitions represent statement partitions other than the at least two statement partitions among the plurality of statement partitions; and concurrently executing all statements to be executed in the other statement partitions.
10. An execution device for a statement to be executed, characterized in that, include: The numerical determination module is used to respond to the target object's operation request to the database and determine the current running memory value of the database; The result acquisition module is used to compare the current running memory value with a first preset threshold to obtain a comparison result. The ratio determination module is used to determine the ratio between the number of running threads and the total number of threads in the database. The statement execution module is configured to concurrently execute the statement to be executed indicated by the operation request based on the comparison result and the ratio; wherein, the statement execution module is further configured to: if it is determined that the comparison result indicates that the current running memory value is less than or equal to the first preset threshold, and if it is determined that the ratio is less than or equal to the second preset threshold, then determine the statement partition corresponding to the statement to be executed. If multiple statement partitions are determined to exist, all statements to be executed within the multiple statement partitions are executed concurrently.
11. A processor, characterized in that, The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 9 when it runs.
12. An electronic device, characterized in that, The method includes one or more processors and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method of any one of claims 1 to 9.
Citation Information
Patent Citations
Data access method and device
CN108897876A