Method, device and equipment for limiting flow of SQL (Structured Query Language) statement and distributed database

Abstract

The embodiment of the invention provides a method, device and equipment for limiting the flow of SQL statements and a distributed database, and the method for limiting the flow of the SQL statements is applied to a computing node of the distributed database and comprises the steps that the SQL statements in an execution state on a first data storage node of the distributed database are obtained, according to the values of the specified fields in the SQL statements, determining a target SQL statement group containing the quantity of the SQL statements exceeding a preset quantity threshold, wherein the values of the specified fields in the SQL statements contained in the target SQL statement group are the same; then, based on field values of specified fields in the SQL statements contained in the target SQL statement group, an SQL flow limiting rule is generated, and the SQL flow limiting rule is used for limiting the number of the SQL statements which are executed on the distributed database and contain the specified fields, and the values of the specified fields are the field values; and finally, sending the SQL flow limiting rule to a second data storage node of the distributed database, so that the second data storage node performs flow limiting on the SQL statement according to the SQL flow limiting rule.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of database technology, and in particular to a method, apparatus, device, medium, computer program product, and distributed database for limiting the flow of SQL statements. Background Art

[0002] In modern information systems, database performance and stability are crucial. This is especially true in highly concurrent query environments, where databases are prone to resource bottlenecks and performance degradation. Traditional monitoring methods struggle to promptly identify and address abnormal Structured Query Language (SQL) statements, leading to slow system responses or even crashes.

[0003] Therefore, it is necessary to provide a solution for a database system to automatically generate throttling rules for identifying high-concurrency or abnormal SQL operations and apply the throttling rules to throttling abnormal SQL statements, so as to improve system reliability and user experience. Summary of the Invention

[0004] In light of this, embodiments of this specification provide a method for limiting the flow of SQL statements. One or more embodiments of this specification also relate to an apparatus for limiting the flow of SQL statements, a computing device, a computer-readable storage medium, and a computer program product, as well as a distributed database, to address technical deficiencies in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a method for limiting the flow of SQL statements is provided, which is applied to a computing node of a distributed database, including: Obtaining an SQL statement in execution state on a first data storage node of a distributed database; Determining a target SQL statement group, the number of which exceeds a preset threshold, based on the value of a specified field in the SQL statement in execution; wherein the SQL statements included in the target SQL statement group have the same value of the specified field; Generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statement that meets the specific conditions is an SQL statement that includes the specified field and the value of the specified field is the field value; The SQL current limiting rule is sent to a second data storage node of the distributed database, so that the second data storage node limits the SQL statement according to the SQL current limiting rule.

[0006] According to a second aspect of an embodiment of this specification, a device for limiting the flow of SQL statements is provided, which is applied to a computing node of a distributed database, including: An SQL statement acquisition module, configured to acquire an SQL statement in execution state on a first data storage node of a distributed database; a target SQL statement group determining module, configured to determine, based on the value of a specified field in the SQL statement in execution, a target SQL statement group including SQL statements whose number exceeds a preset number threshold; wherein the SQL statements included in the target SQL statement group have the same value of the specified field; An SQL flow limiting rule generation module, configured to generate an SQL flow limiting rule based on the field value of the designated field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is configured to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that include the designated field and whose value is the field value; The SQL current limiting rule sending module is used to send the SQL current limiting rule to the second data storage node of the distributed database, so that the second data storage node limits the SQL statement according to the SQL current limiting rule.

[0007] According to a third aspect of an embodiment of this specification, there is provided a distributed database, including computing nodes and data storage nodes; The computing node is configured to obtain an SQL statement in execution state on a first data storage node of a distributed database; determine a target SQL statement group containing a number of SQL statements exceeding a preset number threshold according to the value of a specified field in the SQL statement in execution state, the SQL statements contained in the target SQL statement group having the same value of the specified field; generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements contained in the target SQL statement group, the SQL flow limiting rule being used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database, the SQL statements that meet the specific conditions being SQL statements that include the specified field and whose value is the field value; and send the SQL flow limiting rule to a second data storage node of the distributed database, so that the second data storage node limits the SQL statements according to the SQL flow limiting rule; A first data storage node among the data storage nodes is configured to provide an SQL statement in an execution state to the computing node; The second data storage node in the data storage nodes is configured to receive the SQL current limiting rule sent by the computing node; and limit the SQL statement according to the SQL current limiting rule.

[0008] According to a fourth aspect of the embodiments of this specification, a computing device is provided, including: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above-mentioned method for limiting the flow of SQL statements are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for limiting the flow of SQL statements.

[0010] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned method for limiting the flow of SQL statements.

[0011] One embodiment of the present specification can at least achieve the following beneficial effects: by monitoring the SQL statements in execution state on the data storage node in real time, and grouping the SQL statements in execution state according to the values of the specified fields in the SQL statements, then determining a target SQL statement group whose number of SQL statements exceeds a preset number threshold, and then based on the field values of the specified fields in the SQL statements included in the target SQL statement group, generating an SQL flow limiting rule for limiting the number of SQL statements that meet specific conditions and are executed on the distributed database including the data storage node, wherein the SQL statement that meets the specific conditions is an SQL statement that contains the specified field and the value of the specified field is the field value, thereby achieving automatic and timely discovery of highly concurrent or abnormal SQL operations, which is beneficial to improving the reliability and user experience of the distributed database system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of a scenario of a method for limiting the flow of SQL statements provided in an embodiment of this specification; Figure 2 A flowchart of a method for limiting the flow of SQL statements provided in an embodiment of this specification; Figure 3 A schematic diagram of a solution framework for detecting and eliminating abnormal SQL in an actual application scenario provided by an embodiment of this specification; Figure 4 The embodiments of this specification provide corresponding Figure 2 A structural diagram of a device for limiting the flow of SQL statements; Figure 5 An embodiment of this specification provides a structural block diagram of a computing device 500. DETAILED DESCRIPTION

[0013] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0014] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0016] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0017] First, the terms involved in one or more embodiments of this specification are explained.

[0018] A distributed database is a logically unified database management system composed of multiple physical database nodes connected by a computer network. Compared to stand-alone databases, distributed databases often offer better scalability, increasing the overall computing and storage performance of the database by adding nodes, without being constrained by the hardware configuration of a single physical node.

[0019] A distributed database consists of coordinator nodes and data nodes. Coordinator nodes, also commonly known as compute nodes, are responsible for coordinating and managing the entire database cluster. Data nodes, also known as data storage nodes, are nodes that store and process actual data. Each data storage node contains a portion of the database's data and corresponding processing capabilities. These data storage nodes can be distributed across different physical servers or compute nodes, forming a distributed database cluster.

[0020] Specifically, compute nodes are responsible for receiving access requests from clients and returning execution results to them. They are also responsible for breaking down tasks and scheduling task shards for parallel execution across data storage nodes. Compute nodes typically include a query optimizer and execution engine to optimize query performance and execute complex data processing tasks. Data storage nodes receive data read and write requests from applications or other nodes and are responsible for performing the actual data storage and retrieval operations. Data storage nodes are typically organized in a distributed manner, with each node storing a portion of the data. Sharding technology is used to achieve horizontal data scalability and load balancing.

[0021] To scale computing performance, a distributed database system can include multiple compute nodes, load balancing devices, and multiple storage nodes. Based on load balancing, the load balancing device distributes received SQL statements to a specific compute node among these multiple compute nodes, which then executes the SQL statement. Specifically, a compute node can receive SQL statements from a user. The user in this context can be an individual or an organization. A user can register an account in the database system and log in to that account on their device cluster. Devices in the cluster can then send SQL statements to the database system based on pre-set task logic to perform corresponding data operations.

[0022] In modern information systems, the database serves as the underlying engine that carries core task data. Its performance and stability are directly related to the service quality and user experience of the entire system. With the accelerated advancement of enterprise digital transformation, the average daily amount of data processed by the system is growing exponentially. Especially in high-concurrency scenarios such as e-commerce flash sales, financial transaction settlement, and real-time data processing of the Internet of Things, the database cluster needs to respond to tens of thousands of query requests per second. This continuous high-pressure working state is very likely to cause resource competition problems - multiple query threads compete fiercely for CPU computing resources, memory buffer pools, and disk I / O channels, often leading to chain reactions such as lock contention, deadlock, and query blocking. Even more troublesome is that some unoptimized complex joined-table queries or full-table scan SQL statements will quickly exhaust system resources like an avalanche effect when executed concurrently, causing the performance of the entire database service to plummet.

[0023] When a database experiences sudden traffic spikes that cause a sharp increase in SQL statement concurrency, some SQL statements occupy a large amount of database resources, or a large number of SQL statements without indexes are called, impacting normal tasks, it is necessary to limit the concurrency of problematic SQL statements to ensure the normal operation of most tasks. In related technologies, typically after discovering a significant increase in the query-per-second (QPS) rate of a database system, problematic SQL statements are sequentially checked for data storage nodes in the database system and corresponding throttling rules are set. Specifically, this requires manual judgment on the timing of setting throttling rules and manual configuration of which SQL statements with certain characteristics require throttling (for example, manually determining specific throttling parameters). This creates a lag and impacts database reliability.

[0024] In this specification, it is believed that problematic SQL statements that cause slow database system responses or crashes in actual applications typically have two characteristics: the first is that the execution time of a single problematic SQL statement is too long, and the second is that the number of problematic SQL statements is too large (i.e., the concurrency is too high). When the database system already has excessive resource utilization and a large number of active sessions, the execution time of all SQL statements may be unreasonably extended, making it difficult to identify problematic SQL statements based on the execution time of the SQL statements. Therefore, concurrency statistics can be performed on the executing SQL statements in a certain dimension to identify high-concurrency SQL statements as high-risk SQL statements (high-risk SQL statements, i.e., SQL statements with a high probability of being problematic SQL statements). Flow control can then be performed on these high-concurrency SQL statements. In this way, high-risk SQL statements can be discovered and throttled in a timely manner, reducing the probability of the database system experiencing problems such as slow response or crashes caused by problematic SQL statements.

[0025] To this end, the technical solution provided in the embodiments of this specification is to set up a management module for performing automatic detection and killing in the computing node. The management module can extract the SQL statements in execution state on each data storage node, and determine the SQL statements that meet specific conditions and whose number exceeds a preset number threshold according to the value of the specified field in the extracted SQL statement. Then, SQL flow limiting rules can be automatically generated based on the field value in the SQL statement that meets the specific conditions, thereby facilitating the timely implementation of flow limiting of SQL statements that meet the specific conditions and are executed on the data storage nodes of the distributed database.

[0026] In this specification, a method for limiting the flow of SQL statements is provided. This specification also relates to an apparatus for limiting the flow of SQL statements, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0027] Figure 1 A schematic diagram of a scenario of a method for limiting the flow of SQL statements provided in an embodiment of this specification.

[0028] In such Figure 1 In the illustrated application scenario, the database server 10 can be connected to one or more user clients 20 via a local area network (LAN), a wide area network (WAN), the Internet, or other types of data networks. User clients 20 herein may include, but are not limited to, smartphones, tablet computers, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. In actual applications, the users of user clients 20 may be database users. Database users can access the database server 10 through user clients 20 and, specifically, perform data processing operations, including reading and writing data on the database server 10.

[0029] The database server 10 can be connected to one or more management clients 30 via a local area network (LAN), a wide area network (WAN), the Internet, or other types of data networks. The management clients 30 herein may include, but are not limited to, smartphones, tablet computers, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. In actual applications, users of the management clients 30 may be database management and maintenance personnel, etc. Database management and maintenance personnel, etc., can use the management clients 30 to maintain the database system, thereby implementing the methods provided in the embodiments of this specification.

[0030] In an embodiment of the present specification, the database server 10 can execute the following steps in response to the management of the management client 30: obtain the SQL statements in execution state on the first data storage node of the distributed database; divide the SQL statements into SQL statement groups according to the value of the specified field in the SQL statements in execution state, and the value of the specified field in the SQL statements of the same SQL statement group is the same; determine from the multiple SQL statement groups a target SQL statement group containing a number of SQL statements exceeding a preset number threshold; generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements contained in the target SQL statement group, and the SQL flow limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database, and the SQL statement that meets the specific conditions is an SQL statement that contains the specified field and the value of the specified field is the field value.

[0031] Figure 2 A flowchart of a method for limiting the flow of SQL statements provided in an embodiment of this specification.

[0032] From a program perspective, the execution subject of the process can be a program installed on a server, or further, a program installed on a computing node of a distributed database, and more specifically, a rule manager running on a computing node in a distributed database system.

[0033] like Figure 2 As shown, the process may include the following steps: Step 202: Obtain an SQL statement in execution state on a first data storage node of a distributed database.

[0034] In practical applications, a distributed database system may include several computing nodes and multiple data storage nodes. Figure 2 The modules of the method shown are deployed on a master node among the several computing nodes.

[0035] Considering that in real applications, when a database system encounters resource bottlenecks or performance degradation, problems often occur in data storage nodes, data storage nodes can be regularly monitored. In step 202, a rule manager deployed on a compute node (e.g., a master node) can continuously inspect one or more data storage nodes to identify any anomalies.

[0036] exist Figure 2In the method embodiment shown, for each data storage node in the database system, the SQL statements in execution state on the data storage node can be obtained, so as to identify SQL statements with high risks for the database system by analyzing these SQL statements in execution state, and then generate flow limiting rules for these high-risk SQL statements.

[0037] An executing SQL statement can be one that's currently occupying CPU resources. Specifically, when a SQL statement is scheduled for execution on the CPU, its corresponding process or thread is in the Running state. In other words, an executing SQL statement can be one that's already executing within a process or thread. For example, in the MySQL database, you can use the SHOW PROCESSLIST command to view currently active SQL connections. Processes with "executing" displayed in the State column represent SQL statements in the executing state.

[0038] Step 204: Determine a target SQL statement group containing SQL statements whose number exceeds a preset threshold value based on the value of the specified field in the SQL statements in execution state; the SQL statements in the target SQL statement group contain the same value of the specified field.

[0039] First, the SQL statements in the execution state may be divided into SQL statement groups according to the values of the designated fields in the SQL statements in the execution state; the values of the designated fields in the SQL statements in the same SQL statement group are the same.

[0040] In step 202 , what is actually extracted may be the SQL statement containing the designated field being executed on the first data storage node.

[0041] The designated fields may be fields that are empirically determined to appear in problematic SQL statements. For example, the characteristics of problematic SQL statements may be analyzed based on information such as the historical rate limiting and historical fault handling of the database system, thereby determining the fields typically included in problematic SQL statements as designated fields. It is understood that the high-risk SQL statements (or problematic SQL statements) determined based on the embodiments of this specification need to include the designated fields. However, SQL statements that include the designated fields are not necessarily high-risk SQL statements (or problematic SQL statements).

[0042] In actual application, the designated field can be updateable. For example, when the first designated field is set and the following is executed Figure 2After the method is repeated several times or over a period of time, the first designated field can be evaluated based on the execution effect (for example, whether the problematic SQL statement is correctly identified, etc.); if the evaluation result meets the standard, the first designated field will continue to be used; if the evaluation result does not meet the standard, the first designated field will be updated to the second designated field and the method will be executed again. Figure 2 The method described.

[0043] Furthermore, the designated field may specifically include one or more designated fields.

[0044] When the designated field specifically includes multiple designated fields, optionally, in step 204, the SQL statements are grouped according to the value of the designated field so that the value of the designated field in the SQL statements of the same SQL statement group is the same, and the values of the multiple designated fields may be the same. For example, if the designated field includes a first designated field and a third designated field, step 204 specifically includes dividing the SQL statements into SQL statement groups according to the value of the first designated field and the value of the third designated field in the SQL statements, and the SQL statements of the same SQL statement group have the same value of the first designated field and the same value of the third designated field. As an example, if the designated fields include userId and companyId, then SQL statements with the same userId value and the same companyId value can be divided into the same SQL statement group.

[0045] When the designated field specifically includes multiple designated fields, optionally, in step 204, the SQL statements are grouped based on the values of the designated fields so that the values of the designated fields in the SQL statements within the same SQL statement group are the same. This may be the case where the values of at least some of the multiple designated fields are the same. For example, if the designated fields include a first designated field and a third designated field, step 204 may specifically include grouping the SQL statements into SQL statement groups based on the value of the first designated field in the SQL statements, where the values of the first designated field in the SQL statements within the same SQL statement group are the same; or, grouping the SQL statements into SQL statement groups based on the value of the third designated field in the SQL statements, where the values of the third designated field in the SQL statements within the same SQL statement group are the same. For example, if the designated fields include userId and companyId, then SQL statements with the same userId value (regardless of whether the companyId value is the same) may be grouped into the same SQL statement group; or, alternatively, SQL statements with the same companyId value (regardless of whether the userId value is the same) may be grouped into the same SQL statement group. Specific grouping rules can be set and adjusted based on actual needs.

[0046] Then, a target SQL statement group containing SQL statements exceeding a preset number threshold may be determined from the SQL statement groups.

[0047] In embodiments of this specification, the purpose of setting a designated field may be to identify SQL statements with high concurrency in the designated field dimension after grouping the SQL statements extracted from the first data storage node based on the value of the designated field. In other words, the designated field may be a conditional field determined based on historical data and used to filter out high-risk SQL statements. For ease of subsequent description, in embodiments of this specification, a SQL statement group among multiple SQL statement groups that contains more than a preset threshold number of SQL statements is referred to as a target SQL statement group.

[0048] The preset threshold can be determined based on the concurrency of SQL statements under normal use of the database system. Normal conditions refer to situations where the database system's resource utilization, number of active sessions, and other factors are within normal ranges. For example, if the concurrency of SQL statements typically does not exceed 10 under normal use of the database system, then 10 can be set as the preset threshold. The value 10 given here is merely an example of the preset threshold; in actual applications, the preset threshold can be set based on actual circumstances.

[0049] In the embodiments of this specification, the target SQL statement group may include one or more. In actual applications, in addition to using a preset number threshold as a screening condition when determining the target SQL statement group, the number of filtered target SQL statement groups may also be limited. In actual applications, the flow limiting rules can be generated and implemented by repeatedly executing the identification process multiple times by limiting the flow of several groups of SQL statements each time.

[0050] Specifically, the concurrency of each SQL statement group can be calculated based on the number of SQL statements contained in each SQL statement group, that is, the concurrency can be the number of SQL statements contained in the SQL statement group, or can be a value calculated according to a preset calculation rule; then, the SQL statement groups are sorted from large to small according to the concurrency to obtain a SQL statement group sequence; then, a preset number of SQL statement groups that meet a preset concurrency threshold and have relatively high concurrency are selected from the SQL statement group sequence as target SQL statement groups. Optionally, after calculating the concurrency of each SQL statement group based on the number of SQL statements contained in each SQL statement group, alternative SQL statement groups whose concurrency meets a preset number threshold can be selected from each SQL statement group; then, the SQL statement groups are sorted based on the number of SQL statements contained in each alternative SQL statement group to obtain a SQL statement group sequence, and then a preset number of SQL statement groups with relatively high concurrency in the sequence are selected as target SQL statement groups.

[0051] Step 206: Generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is used to limit the number of SQL statements that meet specific conditions executed on the distributed database; the SQL statement that meets the specific conditions is an SQL statement that contains the specified field and the value of the specified field is the field value.

[0052] In the embodiments of this specification, for the convenience of subsequent description, the value of the specified field in the SQL statement included in the target SQL statement group is referred to as a field value.

[0053] As an example, in step 202, the SQL statements containing the specified field being executed on the first data storage node can be extracted. Assume that the specified field is userId. In step 204, the extracted SQL statements can be grouped according to the value of the specified field. Assume that the values of the specified field in the extracted SQL statements include userId=001, userId=003, userId=006, etc. After the SQL statements are grouped according to the value of the specified field, the value of the specified field in the SQL statements of the first SQL statement group is userId=001, the value of the specified field in the SQL statements of the second SQL statement group is userId=003, the value of the specified field in the SQL statements of the third SQL statement group is userId=006, etc. Then, the SQL statements can be grouped according to the value of the specified field in each SQL statement group. The number of SQL statements included is used to determine the target SQL statement group from the SQL statement groups. Assume that the number of SQL statements in the first SQL statement group that include the filter condition userId=001 is 13, the number of SQL statements in the second SQL statement group that include the filter condition userId=003 is 1, the number of SQL statements in the third SQL statement group that include the filter condition userId=006 is 3, and so on. Assume that the preset number threshold is 10, then the first SQL statement group can be determined as the target SQL statement group; in step 206, based on the filter condition userId=001 of the field value of the specified field in the target SQL statement group, an SQL flow limiting rule can be generated to limit the flow of SQL statements that include the filter condition userId=001.

[0054] Among them, the SQL current limiting rule can be a rule for limiting the number of SQL statements executed concurrently on the data storage node. Specifically, based on Figure 2 The SQL current limiting rule automatically generated by the method can be used to limit the number of SQL statements concurrently executed on the data storage node that contain the specified field and whose value is the field value.

[0055] In actual applications, the SQL flow limiting rule may also include a SQL flow limiting quantity. Optionally, the SQL flow limiting quantity may be equal to the preset quantity threshold. Alternatively, the SQL flow limiting quantity may be greater than or less than the preset quantity threshold. Continuing with the above example, if the preset quantity threshold is 10, then when setting the SQL flow limiting quantity, it may also be set to 10.

[0056] Continuing with the above example, when generating an SQL flow limiting rule for limiting the flow of SQL statements containing the filter condition userId=001, it can be set to generate such a rule: if the SQL statement to be executed contains the filter condition userId=001, and the number of SQL statements containing the filter condition userId=001 being executed on the first data storage node is less than the SQL flow limiting number 10, then the SQL statement to be executed can be executed; and if the SQL statement to be executed contains the filter condition userId=001, and the number of SQL statements containing the filter condition userId=001 being executed on the first data storage node reaches the SQL flow limiting number 10, it is necessary to wait for at least part of the SQL statements containing the filter condition userId=001 being executed to be completed, so that the number of at least part of the SQL statements containing the filter condition userId=001 in execution state on the first data storage node is less than the SQL flow limiting number 10 before the SQL statement to be executed can be executed. Therefore, based on this current limiting rule, the concurrency of identified high-risk SQL statements can be limited to a controllable range (for example, limited to 10 in the above example), thereby reducing the probability of risks such as slow database system response or crash.

[0057] As described above, the designated fields, their field values, and the number of SQL flow limits can be referred to as flow limit parameters in the SQL flow limit rule. In embodiments of this specification, at least some of the flow limit parameters can be determined based on the methods provided in embodiments of this specification. For example, the field values of the designated fields can be determined by statistical analysis of SQL statements executed on the data storage node. Some of the parameters in the flow limit rule can be pre-set by database management and maintenance personnel. For example, the designated fields, the number of SQL flow limits, etc. can be set based on actual needs or task experience.

[0058] Step 208: Send the SQL current limiting rule to the second data storage node of the distributed database, so that the second data storage node limits the SQL statement according to the SQL current limiting rule.

[0059] The second data storage node may include the first data storage node.

[0060] In one or more embodiments of the present specification, a rule manager deployed in a computing node may generate SQL throttling rules. Subsequently, the SQL throttling rules may be distributed to data storage nodes in a distributed database, so that the data storage nodes can implement SQL throttling according to the SQL throttling rules in subsequent data processing flows. Specifically, the SQL throttling rules may be distributed to one or more data storage nodes in the distributed database, so that the one or more data storage nodes can implement SQL throttling according to the SQL throttling rules.

[0061] In the embodiments of the present specification, considering that a database system usually includes multiple data storage nodes, after determining the field value of the specified field in the target SQL statement group for the first data storage node and generating the corresponding SQL current limiting rule, the SQL current limiting rule can be applied on the first data storage node; the SQL current limiting rule can also be sent to one or more other data storage nodes in the database system so that the SQL current limiting rule can be applied on one or more other data storage nodes in the distributed database (for example, all data storage nodes).

[0062] It should be understood that in the methods described in one or more embodiments of this specification, the order of some steps can be adjusted according to actual needs, or some steps can be omitted.

[0063] Figure 2 The method monitors SQL statements in execution state on a first data storage node in a distributed database in real time, groups SQL statements according to the values of specified fields in the SQL statements in execution state, and then determines a target SQL statement group whose number of included SQL statements exceeds a preset number threshold. Then, based on the field values of the specified fields in the SQL statements included in the target SQL statement group, generates an SQL throttling rule for limiting the number of SQL statements that meet specific conditions and are executed on the distributed database, wherein the SQL statements that meet the specific conditions are SQL statements that include the specified field and whose value is the field value. Thus, automatic and timely discovery of highly concurrent or abnormal SQL operations is achieved, which is beneficial to improving the reliability and user experience of the distributed database system.

[0064] based on Figure 2 The present specification also provides some improved implementation methods of the method, which are described below.

[0065] In one or more embodiments of this specification, SQL statements with potential risks can be commented out in advance, so that when executing Figure 2When the method for limiting the flow of SQL statements is described, only the SQL statements with comment marks running on the data storage nodes can be analyzed, which not only saves computing resources but also improves analysis efficiency.

[0066] Specifically, before obtaining the SQL statement in execution state on the first data storage node of the distributed database, the method may further include: if the condition field of the original SQL statement requested to be executed by the client includes the specified field, generating SQL comment information including the specified field; and sending the SQL comment information to the first data storage node by carrying the SQL statement generated based on the original SQL statement. Accordingly, the method of determining the target SQL statement group including the number of SQL statements exceeding the preset number threshold based on the value of the specified field in the SQL statement in execution state may specifically include: based on the specified field contained in the SQL comment information carried by the SQL statement in execution state, determining the target SQL statement group including the number of SQL statements exceeding the preset number threshold based on the value of the specified field in the SQL statement in execution state.

[0067] Among them, the generating of SQL comment information containing the specified field can specifically include: obtaining the original SQL statement requested to be executed by the client; then, on the one hand, based on the original SQL statement, determining the SQL statement for sending to the first data storage node; on the other hand, if the condition field of the original SQL statement contains the specified field, generating SQL comment information containing the specified field; then, the SQL statement carrying the SQL comment information can be sent to the first data storage node.

[0068] The client can be a database user's client, for example, Figure 1 The user client 20 in.

[0069] Optionally, obtaining the original SQL statement requested by the client may be obtaining a SQL statement directly sent by the client as the original SQL statement. Alternatively, obtaining the original SQL statement requested by the client may be obtaining query parameters sent by the client and then generating a SQL statement based on the query parameters as the original SQL statement.

[0070] In actual applications, after receiving the original SQL statement, the compute node in the database system can parse and disassemble the original SQL statement, then generate an execution plan and distribute the task to one or more data storage nodes. In other words, the compute node can generate SQL statements based on the original SQL statement for distribution to one or more data storage nodes. As a result, one or more data storage nodes can parse the received SQL statement and perform data processing operations on their corresponding disks, such as performing data queries and returning the query data results to the compute node. The compute node can then further process and encapsulate the data returned by the one or more data storage nodes and return it to the client requesting execution of the original SQL statement.

[0071] In one or more embodiments of the present specification, database management and maintenance personnel may pre-create SQL marking rules, which may specify the characteristics of original SQL statements to be detected. As a result, a rule manager on a computing node in the database system may, based on the pre-created SQL marking rules, identify original SQL statements that meet the characteristics specified by the SQL marking rules from a large number of original SQL statements that database users request the database system to execute, and then generate SQL comment information for the original SQL statements to serve as a marker.

[0072] Specifically, the method for limiting the flow of SQL statements may further include: obtaining pre-set comment indication information; the comment indication information is used to indicate that SQL comment information needs to be generated for the original SQL statement containing the specified field. Accordingly, if the condition field of the original SQL statement requested by the client includes the specified field, then generating SQL comment information including the specified field may specifically include: based on the specified field targeted by the comment indication, if the condition field of the original SQL statement requested by the client includes the specified field, then generating SQL comment information including the specified field.

[0073] Furthermore, based on the comment indication information, the computing node can know that it is necessary to generate SQL comment information for the original SQL statement containing the specified field. Therefore, for any original SQL statement obtained, the computing node can determine whether the original SQL statement contains the specified field. If the condition field of the original SQL statement contains the specified field, SQL comment information containing the specified field is generated; if the condition field of the original SQL statement does not contain the specified field, there is no need to generate SQL comment information containing the specified field.

[0074] Further optionally, the comment indication information set in the SQL marking rule may only include a specified field identifier. In this case, the comment indication information is used to indicate that when the condition field of an original SQL statement includes the specified field, SQL comment information needs to be generated for the original SQL statement; or optionally, the comment indication information set in the SQL marking rule may include a specified table identifier and a specified field identifier. In this case, the comment indication information is used to indicate that when the data processing object of an original SQL statement is the specified field in the specified table, SQL comment information needs to be generated for the original SQL statement; or optionally, the comment indication information set in the SQL marking rule may include a specified database identifier, a specified table identifier and a specified field identifier. In this case, the comment indication information is used to indicate that when the data processing object of an original SQL statement is the specified field in the specified table in the specified database, SQL comment information needs to be generated for the original SQL statement.

[0075] In actual application, the management and maintenance personnel of the database can pre-create one or more SQL marking rules. When multiple SQL marking rules are set, after obtaining the original SQL statement requested by the client to be executed, the original SQL statements that meet each SQL marking rule can be identified and the corresponding SQL comment information can be generated. As an example, if two SQL marking rules are pre-created, the first comment indication information set by the first SQL marking rule is used to indicate that SQL comment information needs to be generated for the original SQL statement containing the first specified field, and the second comment indication information of the second detection rule is used to indicate that SQL comment information needs to be generated for the original SQL statement containing the second specified field. Then, for any original SQL statement, it is necessary to determine whether it contains the first specified field and whether it contains the second specified field. If the original SQL statement contains the first specified field but does not contain the second specified field, SQL comment information containing the first specified field is generated; if the original SQL statement does not contain the first specified field but contains the second specified field, SQL comment information containing the second specified field is generated; if the original SQL statement contains both the first specified field and the second specified field, SQL comment information containing the first specified field and the second specified field is generated.

[0076] In an embodiment of the present specification, the generation of SQL comment information for the original SQL statement containing the specified field may actually be the addition of SQL comment information to the SQL statement generated based on the original SQL statement for sending to the data storage node, that is, if the condition field of the original SQL statement contains the specified field, SQL comment information containing the specified field is generated, and the SQL comment information is carried to the SQL statement generated based on the original SQL statement for sending to the data storage node.

[0077] As an example, the SQL comment information may specifically include a HINT comment. A HINT comment is a special SQL comment used to convey instructions to the database optimizer to generate a specified execution plan. In actual applications, the SQL comment information may also use other comment forms that can meet the requirements.

[0078] Based on at least some embodiments of the present specification, in the setting stage, the management and maintenance personnel of the database can pre-create SQL marking rules to specify which original SQL statements containing specified fields need to generate SQL comment information; in the application stage, when the database system obtains the original SQL statement requested to be executed by the user client, if the original SQL statement contains the specified fields set in the SQL marking rules, SQL comment information containing the specified fields is generated, and the SQL comment information is used as the comment information of the SQL statement generated based on the original SQL statement, and then the SQL statement carrying the SQL comment information is sent to the data storage node. As a result, when inspecting the SQL statements in the execution state on the data storage node, only the SQL statements carrying specific SQL comment information can be obtained for statistical analysis, without having to analyze the structure of the SQL statements in the execution state one by one to identify which SQL statements should be statistically analyzed, which not only saves computing resources but also improves data analysis efficiency and the accuracy of analysis results.

[0079] In one or more embodiments of the present specification, in order to further improve the accuracy of high-risk SQL screening, based on the concept of grouping SQL based on specified fields to determine high-risk SQL with high concurrency, the structural information of SQL statements is also combined. In actual application, SQL statements with similar structures, containing specified fields and having the same values of the specified fields can be divided into the same SQL statement group to improve the accuracy of screening high-risk SQL based on the concurrency of SQL within the group.

[0080] Furthermore, considering that in an actual database system, the computing nodes will optimize the original SQL statements that reflect the task requirements based on the need for load balancing and then generate SQL statements for sending to the data storage nodes, therefore, when using the structural information of the SQL statements to divide the SQL statement groups, the SQL statement groups can be divided by referring to the structural information of the original SQL statements used to generate the SQL statements.

[0081] Specifically, the method for limiting the flow of SQL statements may further include: generating a template identifier based on the original SQL statement requested to be executed by the client; the template identifier is used to identify the structure of the remaining SQL statements in the original SQL statement except the value of the condition field.

[0082] Correspondingly, the generating of SQL comment information including the designated field specifically includes: generating SQL comment information including the template identifier and the designated field.

[0083] Correspondingly, determining the target SQL statement group including a number of SQL statements exceeding a preset number threshold based on the value of the specified field in the SQL statement in the execution state specifically includes: determining the target SQL statement group including a number of SQL statements exceeding the preset number threshold based on the template identifier of the SQL statement in the execution state and the value of the specified field; the SQL statements included in the target SQL statement group have the same template identifier and the same value of the specified field.

[0084] The template identifier and the value of the designated field can be considered as characteristic information of the SQL statement; the characteristic information is the same for SQL statements in the same SQL statement group. Accordingly, an SQL flow limiting rule can be generated based on the characteristic information; the SQL flow limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database containing the first data storage node; the SQL statements that meet the specific conditions are SQL statements that meet the characteristic information.

[0085] SQL statements containing the same template identifier in the SQL comment information are generated based on original SQL statements with the same statement structure. SQL statements containing the same feature information in the SQL comment information are not only generated based on original SQL statements with the same statement structure, but also have the same values for designated fields in the SQL statements.

[0086] In the embodiments of this specification, from one perspective, the template identifier is generated based on the structure of the original SQL statement and is used to identify the structure of the original SQL statement. From another perspective, the template identifier is carried in the SQL comment information of the SQL statement sent to the data storage node and is used to identify the structure of the original SQL statement from which the SQL statement is generated.

[0087] In actual applications, different template identifiers are used to identify different structures in original SQL statements. Original SQL statements corresponding to the same template identifier are identical except for the values of the condition fields. Specifically, original SQL statements corresponding to the same template identifier can have the same keywords, table names, and field names. In actual applications, SQL statements with the same template identifier can correspond to the same backend query statement.

[0088] As an example, the template identifier can be generated based on information other than the value of the condition field in the original SQL statement. Furthermore, the template identifier can be a hash value obtained by performing a hash operation on the information other than the value of the condition field in the original SQL statement. Thus, different template identifiers can be used to identify portions of the original SQL statement other than the value of the condition field.

[0089] In a specific example, assuming that the template identifier templateId of the first original SQL statement "select * from tb where userId = 1" is 001, then the second original SQL statement "select * from tb where userId = 2" is identical to the first original SQL statement "select * from tb where userId" except for the value of the condition field. Therefore, the template identifier templateId of the second original SQL statement is also 001. The third original SQL statement "select * from tb where companyId = 200" is not identical to the first original SQL statement "select * from tb where userId" except for the value of the condition field. Therefore, the template identifier templateId of the third original SQL statement is not 001 and can be other values.

[0090] As described above, the template identifier, the designated fields and their field values, and the number of SQL flow limits can be referred to as flow limit parameters in the SQL flow limit rules. In an embodiment of the present specification, at least some of the flow limit parameters can be determined based on the method provided in the embodiment of the present specification. For example, the field value of the designated field can be determined by statistically analyzing the SQL statement running on the first data storage node. For another example, the template identifier can be obtained by calculating the statement structure of the original SQL statement obtained by the computing node. Some of the parameters in the flow limit rules can be pre-set by the management and maintenance personnel of the database. For example, the designated fields, the number of SQL flow limits, etc. can be set according to actual task requirements or task experience.

[0091] Based on at least some embodiments of the present specification, on the one hand, according to the pre-set SQL marking rules, the original SQL statement whose condition field contains the specified field set in the SQL marking rules can be identified; on the other hand, based on the statement structure of the original SQL statement, a template identifier for reflecting the statement structure of the original SQL statement can be generated; further, SQL comment information containing the template identifier and the specified field can be generated for the original SQL statement and added to the SQL statement generated based on the original SQL statement for sending to the data storage node, thereby making it possible for the SQL statement in the execution state on the data storage node to be executed as follows: Figure 2 When using the aforementioned method for limiting SQL statements, the data combination of [template identifier - value of a specified field] can be used as characteristic information for grouping SQL statements. Compared to grouping SQL statements based solely on the value of a specified field, using characteristic information to group SQL statements takes into account the consistency of the original SQL statements. That is, the SQL statements grouped into the same SQL statement group are generated based on original SQL statements with the same statement structure. As a result, the number of SQL statements included in such grouping results can more accurately reflect the SQL concurrency situation from a task perspective, improving the accuracy of the grouping results and the accuracy of the SQL throttling rules generated based on identifying SQL statements that meet specific conditions based on the grouping results. This can improve the SQL throttling effect and enhance the stability of the database system.

[0092] It's important to note that the accuracy of an SQL throttling rule reflects the probability that the original SQL statement targeted by the SQL throttling rule is a problematic SQL statement. A higher accuracy in an SQL throttling rule indicates a greater probability of throttling problematic SQL statements.

[0093] In one or more embodiments of the present specification, considering that the original SQL statements to be executed obtained by the database system have different requesting parties and different risk levels, for example, some original SQL statements may be initiated by clients with a higher security level and a relatively low risk, for example, Figure 1 The other part of the original SQL statements may be initiated by clients with relatively low security levels and relatively high risks, for example, Figure 1 As shown, the user client initiates the task. Therefore, in order to save computing resources and improve the execution efficiency of the method for limiting the flow of SQL statements, only the execution of data processing tasks initiated by clients with relatively low security levels and relatively high risks can be analyzed.

[0094] Specifically, generating SQL comment information containing the designated field includes generating SQL comment information containing a first SQL identifier and the designated field; the first SQL identifier is used to indicate that the original SQL statement was obtained based on a client request. Correspondingly, obtaining an SQL statement in execution on the first data storage node of the distributed database includes obtaining, based on the SQL comment information carried by the executed SQL statement, an SQL statement in execution that contains the first SQL identifier in its SQL comment information carried by the first data storage node of the distributed database.

[0095] Among them, the first SQL identifier can be used to mark the data processing tasks initiated by the client with relatively low security level and relatively high risk. In actual application, it can be based on Figure 1 Specifically, if the original SQL statement is obtained based on the request of the database user, it can be the original SQL statement sent by the database user, or it can be the original SQL statement generated based on the parameters sent by the database user.

[0096] In an embodiment of the present specification, optionally, after obtaining the original SQL statement, on the one hand, it can be identified that the original SQL statement is obtained based on the request of the database user, on the other hand, the original SQL statement containing the specified field in the condition field can be identified, and on the other hand, a template identifier can be generated to reflect the statement structure of the original SQL statement. Thus, for the original SQL statement containing the specified field obtained based on the request of the database user, SQL comment information containing the first SQL identifier, the template identifier and the specified field can be generated, and the SQL comment information and the SQL statement generated based on the original SQL statement can be sent to the data storage node at the same time. As a result, when the SQL statement in the execution state on the data storage node is executed as shown below, Figure 2When using the method for limiting the flow of SQL statements, only the SQL statements carrying the first SQL identifier in the SQL comment information can be obtained to perform subsequent analysis. Since there is no need to include the original SQL statements initiated by clients with a higher security level and relatively low risk in the analysis scope, it can greatly improve the analysis efficiency, save computing resources, and reduce the risk of accidentally damaging normal SQL.

[0097] In one or more embodiments of the present specification, before executing the method for generating SQL current limiting rules as described in the embodiments of the present specification, the operation status of the data storage node can be monitored first. When it is detected that the operation status of the data storage node is poor, the method for generating SQL current limiting rules can be executed.

[0098] Specifically, before obtaining the SQL statements in execution state on the first data storage node of the distributed database, the method further includes: obtaining the first operation data of the first data storage node; the first operation data includes at least one of the number of SQL statements in execution state and resource occupancy data. Accordingly, the method of obtaining the SQL statements in execution state on the first data storage node of the distributed database specifically includes: if the first operation data meets the preset trigger condition, obtaining the SQL statements in execution state on the first data storage node. Further, if the first operation data meets the preset trigger condition, obtaining the SQL statements in execution state on the first data storage node; the preset trigger condition includes, the number of SQL statements is greater than or equal to the first SQL number threshold; or, the resource occupancy rate is greater than or equal to the first resource occupancy rate threshold; or, the number of SQL statements is greater than or equal to the second SQL number threshold and the resource occupancy rate is greater than or equal to the second resource occupancy rate threshold.

[0099] Among them, the number of SQL statements in execution state may optionally refer to the total number of all SQL statements being executed on the first data storage node; or, optionally, may refer to the number of SQL statements carrying the first SQL identifier being executed on the first data storage node.

[0100] Among them, resource usage data can be used to reflect the usage of node resources. Specifically, node resources may include CPU resources, memory resources, IO resources, etc. Correspondingly, the resource usage data may include CPU utilization, memory utilization, IO resource utilization, etc.

[0101] The preset trigger conditions may include a first trigger condition for the number of SQL statements and a second trigger condition for the resource occupancy data.

[0102] Optionally, if the first operating data includes the number of SQL statements, it may be determined whether the number of SQL statements satisfies a first trigger condition, and if so, the method for generating SQL throttling rules may be executed. For example, it may be determined whether the number of SQL statements is greater than or equal to a preset SQL number threshold, and if so, the method for generating SQL throttling rules may be executed.

[0103] If the first operating data includes resource occupancy data, it can be determined whether the resource occupancy data meets the second trigger condition. If so, the method for generating SQL current limiting rules is executed. For example, it can be determined whether the CPU utilization is greater than or equal to the preset CPU utilization threshold. If so, the method for generating SQL current limiting rules is executed. For another example, it can be determined whether the memory utilization is greater than or equal to the preset memory utilization threshold. If so, the method for generating SQL current limiting rules is executed. For another example, it can be determined whether the IO resource utilization is greater than or equal to the preset IO resource utilization threshold. If so, the method for generating SQL current limiting rules is executed. In actual application, a comprehensive judgment can also be made for multiple resource occupancy data. For example, when one or more resource occupancy data reaches the corresponding preset threshold, the method for generating SQL current limiting rules is executed.

[0104] If the first operation data includes the number of SQL statements and resource usage data, it can be determined whether at least one of the following conditions is met: the number of SQL statements meets the first trigger condition and the resource usage data meets the second trigger condition; if so, the method for generating SQL throttling rules is executed. For example, if the first operation data includes the number of SQL statements and CPU utilization, it can optionally be determined whether the number of SQL statements is greater than or equal to a first SQL quantity threshold and the CPU utilization is greater than or equal to a first CPU utilization threshold. If so, the method for generating SQL throttling rules is executed; or alternatively, it can be determined whether the number of SQL statements is greater than or equal to a second SQL quantity threshold. If so, the method for generating SQL throttling rules is executed, wherein the second SQL quantity threshold can be greater than the first SQL quantity threshold; or alternatively, it can be determined whether the CPU utilization is greater than or equal to a second CPU utilization threshold. If so, the method for generating SQL throttling rules is executed, wherein the second CPU utilization threshold can be greater than the first CPU utilization threshold. As an example, the first CPU utilization can be 95% and the second CPU utilization can be 99%.

[0105] A distributed database system usually includes multiple data storage nodes. In actual applications, considering that the data distribution, query request distribution, etc. of one or more data storage nodes may be uneven, in order to timely discover potential high-risk SQL on the data storage nodes, the above-mentioned detection process can be executed for one or more data storage nodes in the distributed database system to identify the data storage nodes that meet the preset trigger conditions, and then the method of generating SQL current limiting rules is executed for the data storage nodes that meet the preset trigger conditions.

[0106] Furthermore, for each data storage node, periodic detection can be performed according to a preset detection period, for example, every few seconds, every minute, or every few minutes. In this way, high-risk SQL statements that may have problems running on the data storage node can be discovered in a timely manner.

[0107] In one or more embodiments of the present specification, considering that temporary high-concurrency queries of normal SQL may also occur during the actual use of the database system, in order to avoid misidentifying normal temporary high-concurrency situations as requiring the execution of the method for generating SQL current limiting rules, the trigger conditions can be further optimized.

[0108] Specifically, if the first operating data satisfies the preset trigger condition, the SQL statement in execution state on the first data storage node is obtained, specifically including: if the second operating data satisfies the preset trigger condition after the first operating data satisfies the preset trigger condition for a preset interval, the SQL statement in execution state on the first data storage node is obtained; the second operating data includes at least one of the number of SQL statements in execution state updated on the first data storage node after the preset interval and the updated resource occupancy data.

[0109] Furthermore, if the first operating data meets the preset trigger condition, an abnormal alarm message is sent; if the second operating data meets the preset trigger condition after a preset interval after sending the abnormal alarm message, the SQL statement in execution state on the first data storage node is obtained.

[0110] The abnormal alarm information may be alarm log information, specifically including, but not limited to, the alarm item, alarm name, alarm time, alarm severity, and alarm annotation information. The alarm annotation information may include, but not limited to, information reflecting the content of the alarm, such as abnormal statement information, abnormal indicator information, and abnormal time information. In actual applications, the content of the abnormal alarm information can be customized based on task requirements.

[0111] Furthermore, the abnormal alarm information can be sent to the instant messaging page of the database management and maintenance personnel, so that the management and maintenance personnel can find the problem in time and take corresponding maintenance measures to maintain the stability of the database system.

[0112] The preset interval duration can be determined based on the actual task situation. For example, if the typical duration of temporary high-concurrency queries is calculated based on historical operation data, the preset interval duration can be set to be slightly longer than the duration. As an example, in practice, the preset interval duration can be set to 1 minute or several minutes.

[0113] Based on at least some embodiments of the present specification, it is possible to perform inspections on data storage nodes in a distributed database system, and determine whether the data storage nodes meet preset trigger conditions based on the operating data of the data storage nodes; if so, the trigger conditions are met, indicating that the operating status of the data storage node is abnormal, and the method for generating SQL flow limiting rules and the method for flow limiting SQL statements provided in the embodiments of the present specification are automatically executed. It can be seen that based on the scheme of the embodiments of the present specification, by inspecting the data storage nodes, it is possible to automatically identify data storage nodes with poor operating status, automatically trigger statistical analysis of SQL statements in execution on the data storage nodes with poor operating status, and then analyze high-concurrency, high-risk SQL statements, and then automatically generate SQL flow limiting rules based on the analysis results. As a result, in actual application, there is no need for database management and maintenance personnel to manually identify the timing of setting SQL flow limiting rules, nor is there a need to manually determine how to set specific parameters in the SQL flow limiting rules. This breaks the traditional method of manually setting flow limiting rules, improves the timeliness and efficiency of flow limiting rule setting, and can promptly flow limit identified high-risk SQL statements, which is conducive to maintaining the long-term stable operation of the distributed database system and improving the database user experience.

[0114] In one or more embodiments of the present specification, a self-healing mechanism is further provided. Specifically, after the computing node issues the SQL flow limiting rules and the data storage node executes the SQL flow limiting rules, the issued and executed SQL flow limiting rules can be canceled or deleted if certain conditions are met.

[0115] In an optional embodiment, after sending the SQL current limiting rule to the second data storage node of the distributed database, it also includes: obtaining the execution time of several SQL statements that comply with the SQL current limiting rule and are executed on the second data storage node after the second data storage node limits the SQL statement according to the SQL current limiting rule; if the execution time of the several SQL statements is less than a preset time threshold, canceling the SQL current limiting rule.

[0116] The duration threshold is used to reflect the typical execution time of non-abnormal SQL statements (normal SQL statements). In actual applications, the duration threshold can be a characteristic value obtained based on historical SQL execution data statistics on the database system, or it can be a value set based on expert experience. In practice, the setting of the duration threshold may vary depending on information such as the hardware parameters, software parameters, and task parameters of the database system. As an example, in some databases, the execution time of query-type SQL statements usually does not exceed 0.3s, so the duration threshold can be set to 0.3s.

[0117] Furthermore, the execution status of SQL statements in the execution state that meet the characteristic information specified in the SQL throttling rule but do not exceed the SQL throttling limit can be continuously monitored. For example, the execution time of SQL statements in the execution state that do not exceed the SQL throttling limit can be continuously monitored. If the execution time of these SQL statements in the execution state does not exceed a preset time threshold, the SQL throttling rule can be canceled.

[0118] Furthermore, in order to improve the accuracy of the operation of canceling the SQL flow limiting rule, a preset number of SQL statements in the execution state that meet the characteristic information specified in the SQL flow limiting rule after the SQL flow limiting is executed can be detected. If the execution time or characteristic execution time (for example, the average execution time, the maximum execution time, etc.) of the detected SQL statement does not exceed the time threshold, the SQL flow limiting rule is canceled.

[0119] In actual applications, the preset number of SQL statements involved in the detection can be the first several SQL statements after the execution of SQL throttling; or, it can be several SQL statements after the execution of SQL throttling for a period of time. In practice, the period of time can be the time during which the overall operating status of the database system improves due to the execution of the throttling rule. The period of time can be determined based on historical statistical data or expert experience, and is not specifically limited to this.

[0120] In addition, in actual application, the preset number of SQL statements involved in the detection can be several SQL statements executed continuously; or, it can be some SQL statements extracted from the continuously executed SQL statements according to certain extraction rules. The specific extraction method can be determined according to task requirements or expert experience.

[0121] Furthermore, the several SQL statements may specifically include SQL statements that comply with the SQL current limiting rule and are executed N times consecutively on the second data storage node after the SQL statements are current limited according to the SQL current limiting rule, where N is an integer greater than 1.

[0122] Based on the method of the embodiments of this specification, considering that after all high-risk SQL statements with high concurrency are throttled, the operating status of the second data storage node will be restored. At this time, the SQL statements executed on the second data storage node can reflect the actual execution time of each SQL statement (the actual execution time can refer to the execution time when computer resources are sufficient). At this time, it is possible to determine whether the high-risk SQL statement being throttled is a problematic SQL by determining whether the actual execution time of the throttled SQL statement is less than or equal to a preset time threshold. For example, if the actual execution time of the throttled SQL statement is less than or equal to the preset time threshold, it may indicate that the throttled SQL statement is not a problematic SQL statement, and the corresponding throttling rule can be canceled. In this way, it is possible to promptly discover SQL statements that are erroneously throttled and promptly restore the execution of such normal SQL statements to avoid a negative user experience. For another example, if the actual execution time of the throttled SQL statement is greater than the preset time threshold, it may indicate that the throttled SQL statement is a problematic SQL statement, and there is no need to cancel the corresponding SQL throttling rule.

[0123] In an optional embodiment, the sending of the SQL current limiting rule to one or more data storage nodes in the distributed database so that the one or more data storage nodes execute SQL current limiting according to the SQL current limiting rule further includes: canceling the SQL current limiting rule after sending the SQL current limiting rule for a preset time.

[0124] In practical applications, consider that after the compute node generates and sends SQL throttling rules to the second data storage node, and the second data storage node executes throttling according to the SQL throttling rules, if a client's data processing request hits the running SQL throttling rules, the client will receive various throttling alerts, such as request timeouts and errors. The problematic SQL can then be corrected by the database user rewriting the request statement. In this case, there's no need to continue executing the SQL throttling rules. Therefore, a certain expiration time can be set for the generated SQL throttling rules, and when this expiration time is reached, they can be automatically canceled. Based on this solution, throttling rules can be canceled conveniently without complex calculations and judgments, making it universally applicable and conserving computing resources.

[0125] Based on the solution of the embodiment of this specification, a fast self-healing solution can be implemented to detect the execution time of SQL statements in the execution state to promptly discover the SQL statements that are incorrectly restricted and cancel the corresponding SQL flow limiting rules, or a slow self-healing solution can be implemented to automatically cancel the SQL flow limiting rules after a preset time, or a combination of the two can be used. Overall, by setting up a self-healing mechanism for automatically canceling the SQL flow limiting rules that have been generated and executed, the refined management of SQL flow limiting is improved and resource waste is avoided; by combining the self-healing mechanism with the detection and alarm mechanism, the automatic detection and killing mechanism, etc., a fully automated operation and maintenance management of the distributed database system is achieved, and a solution for achieving stable operation and maintenance of the database system with low manpower consumption, low cost, high efficiency and high accuracy is achieved.

[0126] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, the arbitrary combination of the various technical features in the above embodiments also falls within the scope of disclosure of this specification.

[0127] According to the above description, the embodiment of this specification provides a schematic diagram of the scheme framework of the abnormal SQL detection and elimination scheme in an actual application scenario, as shown in FIG. Figure 3 shown.

[0128] like Figure 3 As shown, first, an administrative user can set an SQL marking rule containing a specified field through a user management client.

[0129] As an example, Figure 3 As shown, an administrative user can use the CREATE CCL_DETECT command to create a detection task named 'name'. For example, execute the statement CREATE CCL_DETECT 'name'. Furthermore, in the detection task named 'name', you can set the specified database (db_name) and database table (table_name), and set the root column (ROOT_COLUMN) to "column_name". For example, execute the statement CREATE CCL_DETECT 'name' ON 'db_name'.'table_name' WITH ROOT_COLUMN= ''column_name''. Among them, 'name', 'db_name', 'table_name', and ''column_name'' can be set according to actual conditions. For example, column_name can be userId.

[0130] Thus, the SQL statements containing the column ''column_name'' in the data table ''table_name'' in the database ''db_name'' in the received filter conditions can be identified and marked.

[0131] The aforementioned information set through the user management client can be sent to the computing nodes of the distributed database system.

[0132] Second, the computing node generates an SQL statement based on the original SQL statement for sending to the data storage node, and can add HINT comments to the SQL statement based on the previously set SQL marking rules before sending it to the data storage node.

[0133] In actual applications, when a user initiates a data processing request, such as an SQL query request, the computing node receives the SQL query request and generates a HINT comment for the original SQL statement that meets the conditions, and sends the HINT comment and the SQL statement generated based on the original SQL statement to the data storage node at the same time.

[0134] As an example, Figure 3 As shown, it is assumed that the original SQL statement is SELECT * from tb where userId =?, and it is assumed that based on the original SQL statement, a first SQL statement SELECT *from tb_1 where userId =? for sending to data storage node 1, a second SQL statement SELECT * fromtb_2 where userId =? for sending to data storage node 2, and a third SQL statement SELECT * fromtb_3 where userId =? for sending to data storage node 3 can be generated.

[0135] For this example, based on the solution of the embodiment of this specification, a HINT comment / *CCL;TemplateId;ColumnName / * / can also be generated for being sent to the data storage node simultaneously with the first SQL statement, the second SQL statement and the third SQL statement. Among them, CCL can be used to indicate that the original SQL statement corresponding to the first SQL statement / the second SQL statement / the third SQL statement is obtained based on the request of the database user. In actual application, the SQL statement marked with CCL will be scanned and analyzed by the rule manager deployed on the database computing node. TemplateId can be used to identify the statement structure of the original SQL statement corresponding to the first SQL statement / the second SQL statement / the third SQL statement, and more specifically, to identify the part of the original SQL statement except the value of the condition field. ColumnName can be used to indicate that the condition field of the original SQL statement corresponding to the first SQL statement / the second SQL statement / the third SQL statement contains the specified field set in the SQL marking rule created previously.

[0136] Third, while the data storage nodes are executing SQL statements, the rule manager can inspect the running status of each data storage node in the background.

[0137] In actual application, the rule manager may be a functional module deployed in a computing node.

[0138] For each data storage node, anomaly detection can first be performed. Specifically, the rule manager can obtain the data storage node's operational data, for example, at least one of the number of SQL statements being executed on the data storage node and resource usage data (e.g., CPU utilization). Based on the operational data, the rule manager can determine whether the data storage node is operating abnormally.

[0139] Then, if one or more data storage nodes are operating abnormally, a log alarm can be generated. For example, the log alarm information can be pushed to the instant messaging page of the database management and maintenance personnel to prompt manual intervention.

[0140] Furthermore, anomaly detection can be repeated after a preset interval (for example, every three minutes). If two or more consecutive abnormality detection results are obtained, the subsequent automatic detection and elimination process will be further executed. By triggering automatic elimination based on more than one abnormality detection result, false triggers can be effectively avoided, reducing the waste of computing resources caused by false elimination and lowering the false elimination rate.

[0141] Fourth, the rule manager can generate SQL flow control rules and send them to the data storage node when the running status of the data storage node meets the preset conditions.

[0142] First, for the data storage node that consistently experiences abnormal operations, using the previous example, the rule manager can extract the SQL statements marked with the CCL tag from the data storage node and group them according to their characteristic information. Furthermore, the data combination of [template identifier - specified field value] can be used as characteristic information to group the extracted SQL statements.

[0143] Then, for the multiple SQL statement groups obtained, based on the number of SQL statements contained in each SQL statement group, a preset number of SQL statement groups containing a relatively large number of SQL statements and having a number of SQL statements exceeding a preset number threshold can be determined as target SQL statement groups.

[0144] Furthermore, SQL flow limiting rules can be generated based on the feature information of the SQL statements included in the filtered target SQL statement group.

[0145] Specifically, if the characteristic information formed by the data combination of [template identifier-specified field value] is recorded as key, and different characteristic information is recorded as key1, key2, etc., then different target SQL statement groups correspond to different keys.

[0146] As an example, you can generate an SQL throttling rule by executing the statement "Call dbms_ccl.add_ccl_rule('SELECT',20,'key1'). In this statement, "call" indicates calling a stored procedure, "dbms_ccl" indicates the package name of the stored procedure, and "add_ccl_rule" indicates the stored procedure name. In actual use, dbms_ccl and "add_ccl_rule" can be configured based on actual circumstances. "SELECT", "20", and "key1" are throttling parameters. The "key1" parameter specifies that throttling will be applied to SQL statements that meet the characteristics indicated by "key1"; the "SELECT" parameter specifies that throttling will be applied to SQL statements of the SELECT type; and the "20" parameter specifies that throttling will be applied if the number of SQL statements that meet the other parameter conditions exceeds 20. For example, subsequent SQL statements will be queued for execution or will return an error.

[0147] It should be noted that the purpose of setting the parameter 'SELECT' when generating SQL throttling rules is to: Consider that in actual applications, the type of problematic SQL that causes problems such as database slowness or crashes is usually SELECT type. By limiting the SQL type to SELECT type when setting SQL throttling rules, the data processing volume when executing SQL throttling rules can be effectively reduced, the execution efficiency of throttling rules can be improved, and the loss of computer resources caused by executing SQL throttling rules can be reduced.

[0148] Fifth, the rule manager executes a self-healing mechanism to automatically cancel SQL flow limiting rules.

[0149] Optionally, a fast self-healing mechanism can be implemented. Specifically, if a SQL statement that meets certain characteristic information is executed with throttling, the execution time of the corresponding SQL statement after throttling can be detected. If the execution time after throttling meets the preset conditions, the previously set SQL throttling rule can be canceled.

[0150] Optionally, a slow self-healing mechanism may be implemented. Specifically, the SQL flow limiting rule may be canceled after the SQL flow limiting has been implemented for a preset period of time.

[0151] For example, corresponding to the above example, the SQL throttling rule generated by executing the statement Call dbms_ccl.add_ccl_rule('SELECT',20,'key1') can be canceled by executing the statement Call dbms_ccl.del_ccl_rule(''key1').

[0152] Based on the same idea, the embodiments of this specification also provide a device corresponding to the above method.

[0153] Figure 4 The embodiments of this specification provide corresponding Figure 2 A structural diagram of a device for limiting the flow of SQL statements.

[0154] like Figure 4 As shown, the device is applied to a computing node of a distributed database and may include: An SQL statement acquisition module 402 is configured to acquire an SQL statement in execution state on a first data storage node of a distributed database; a target SQL statement group determining module 404 configured to determine, based on the value of a specified field in the executed SQL statement, a target SQL statement group including SQL statements whose number exceeds a preset threshold; wherein the SQL statements included in the target SQL statement group have the same value of the specified field; An SQL flow limiting rule generation module 406 is configured to generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is configured to limit the number of SQL statements that meet a specific condition and are executed on the distributed database; the SQL statement that meets the specific condition is an SQL statement that includes the specified field and whose value is the field value; The SQL current limiting rule sending module 408 is used to send the SQL current limiting rule to the second data storage node of the distributed database, so that the second data storage node can limit the SQL statement according to the SQL current limiting rule.

[0155] based on Figure 4 The present specification also provides some specific implementation plans of the method, which are described below.

[0156] Optionally, the device further comprises: An SQL comment information generating module, configured to generate SQL comment information including the specified field if the condition field of the original SQL statement requested to be executed by the client includes the specified field; An SQL statement sending module is configured to send the SQL comment information to the first data storage node by carrying it in an SQL statement generated based on the original SQL statement.

[0157] Correspondingly, the target SQL statement group determination module 404 is specifically used to: based on the specified field contained in the SQL comment information carried by the SQL statement in the execution state, and according to the value of the specified field in the SQL statement in the execution state, determine the target SQL statement group whose number of SQL statements exceeds a preset number threshold.

[0158] Optionally, the apparatus further includes a comment indication information acquisition module configured to acquire pre-set comment indication information, wherein the comment indication information indicates a need to generate SQL comment information for an original SQL statement containing the specified field. Accordingly, the SQL comment information generation module is configured to generate SQL comment information containing the specified field, based on the specified field targeted by the comment indication information, if a condition field of the original SQL statement requested by the client contains the specified field.

[0159] Optionally, the apparatus further comprises a template identifier generating module, configured to generate a template identifier based on the original SQL statement executed by the client request; the template identifier is used to identify the structure of the remaining SQL statements in the original SQL statement except the value of the condition field; The SQL comment information generating module is further configured to generate SQL comment information including the template identifier and the designated field; The target SQL statement group determination module 404 is specifically used to determine a target SQL statement group containing SQL statements whose number exceeds a preset number threshold based on the template identifier of the SQL statement in the execution state and the value of the specified field; the SQL statements contained in the target SQL statement group have the same template identifier and the same value of the specified field.

[0160] Optionally, the SQL comment information generating module is specifically configured to generate SQL comment information including a first SQL identifier and the designated field; the first SQL identifier is used to indicate that the original SQL statement is obtained based on a request from a client; The SQL statement acquisition module 402 is specifically configured to acquire, based on the SQL comment information carried by the SQL statement in execution state, an SQL statement in execution state, the SQL comment information of which is carried on the first data storage node of the distributed database and contains the first SQL identifier.

[0161] Optionally, the device further includes an operation data acquisition module, configured to acquire first operation data of the first data storage node; the first operation data includes at least one of the number of SQL statements in execution state and resource usage data; The SQL statement acquisition module 402 is specifically used to obtain the SQL statement in execution state on the first data storage node if the first running data meets the preset trigger condition; the preset trigger condition includes that the number of SQL statements is greater than or equal to the first SQL number threshold; or, the resource occupancy rate is greater than or equal to the first resource occupancy rate threshold; or, the number of SQL statements is greater than or equal to the second SQL number threshold and the resource occupancy rate is greater than or equal to the second resource occupancy rate threshold.

[0162] Optionally, the SQL statement acquisition module 402 is specifically used to obtain the SQL statement in execution state on the first data storage node if the second operation data meets the preset trigger condition after the first operation data meets the preset trigger condition for a preset interval period; the second operation data includes at least one of the number of SQL statements in execution state updated on the first data storage node after the preset interval period and the updated resource occupancy data.

[0163] Optionally, the device is further configured to: send abnormal alarm information if the first operating data meets the preset trigger condition.

[0164] Optionally, the second data storage node includes the first data storage node.

[0165] Optionally, the target SQL statement group determination module 404 is specifically used to: divide the SQL statements in the execution state into SQL statement groups according to the value of the specified field in the SQL statements in the execution state; the value of the specified field in the SQL statements in the same SQL statement group is the same; and determine, from the SQL statement groups, a target SQL statement group containing SQL statements whose number exceeds a preset number threshold.

[0166] Optionally, the device also includes a first self-healing module, which is used to obtain the execution time of several SQL statements that comply with the SQL current limiting rule and are executed on the second data storage node after the second data storage node limits the SQL statement according to the SQL current limiting rule; if the execution time of the several SQL statements is less than a preset time threshold, the SQL current limiting rule is canceled.

[0167] Optionally, the several SQL statements specifically include SQL statements that comply with the SQL current limiting rule and are executed N times consecutively on the second data storage node after the SQL statements are current limited according to the SQL current limiting rule, where N is an integer greater than 1.

[0168] Optionally, the device further includes a second self-healing module, configured to cancel the SQL current limiting rule after the SQL current limiting rule has been sent for a preset time period.

[0169] The above is a schematic diagram of a device for limiting the flow of SQL statements according to this embodiment. It should be noted that the technical solution of this device for limiting the flow of SQL statements is based on the same concept as the technical solution of the method for limiting the flow of SQL statements described above. For details not described in detail in the technical solution of the device for limiting the flow of SQL statements, please refer to the description of the technical solution of the method for limiting the flow of SQL statements described above.

[0170] Based on the same idea, the embodiments of this specification also provide a distributed database corresponding to the above method and device.

[0171] Specifically, the distributed database provided in the embodiments of this specification includes computing nodes and data storage nodes; The computing node is configured to obtain an SQL statement in execution state on a first data storage node of a distributed database; determine a target SQL statement group containing a number of SQL statements exceeding a preset number threshold according to the value of a specified field in the SQL statement in execution state, the SQL statements contained in the target SQL statement group having the same value of the specified field; generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements contained in the target SQL statement group, the SQL flow limiting rule being used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database, the SQL statements that meet the specific conditions being SQL statements that include the specified field and whose value is the field value; and send the SQL flow limiting rule to a second data storage node of the distributed database, so that the second data storage node limits the SQL statements according to the SQL flow limiting rule; A first data storage node among the data storage nodes is configured to provide an SQL statement in an execution state to the computing node; The second data storage node in the data storage nodes is configured to receive the SQL current limiting rule sent by the computing node; and limit the SQL statement according to the SQL current limiting rule.

[0172] Optionally, the second data storage node includes the first data storage node.

[0173] The above is a schematic diagram of a distributed database solution of this embodiment. It should be noted that the technical solution of this distributed database and the technical solution of the method for limiting the flow of SQL statements described above are based on the same concept. For details not described in detail in the technical solution of the distributed database, please refer to the description of the technical solution of the method for limiting the flow of SQL statements described above.

[0174] Figure 5 An embodiment of this specification provides a structural block diagram of a computing device 500.

[0175] The computing device 500 includes: Memory 510 and processor 520; The memory 510 is used to store computer programs / instructions, and the processor 520 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 520, the steps of the method for limiting the flow of SQL statements are implemented.

[0176] In one or more embodiments of this specification, the computing device can be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), an all-in-one model machine, a mobile phone, a tablet computer or other portable intelligent terminal, etc., and the computing device can be pre-installed with the model described in the above embodiments of this application.

[0177] In addition, the components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 and the memory 510 may be connected via a bus.

[0178] The computing device 500 may also include an access device that enables the computing device 500 to communicate with a database storing data via one or more networks. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless type, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, or a near field communication (NFC) interface.

[0179] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 5 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0180] Computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or personal computer (PC). Computing device 500 can also be a mobile or stationary server.

[0181] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the method for limiting the flow of SQL statements described above are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the method for limiting the flow of SQL statements described above.

[0182] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the above-mentioned method for limiting the flow of SQL statements are implemented.

[0183] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium is based on the same concept as the technical solution of the method for limiting the flow of SQL statements described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the method for limiting the flow of SQL statements described above.

[0184] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned method for limiting the flow of SQL statements when executed by a processor.

[0185] The above is an illustrative solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product shares the same concept as the technical solution of the aforementioned method for limiting the flow of SQL statements. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the aforementioned method for limiting the flow of SQL statements.

[0186] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0187] The computer program / instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0188] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0189] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0190] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for limiting the flow of SQL statements, applied to a computing node of a distributed database, comprising: Obtaining an SQL statement in execution state on a first data storage node of a distributed database; Determining a target SQL statement group, the number of which exceeds a preset number threshold, based on a value of a specified field in the SQL statement in execution state; The values of the designated fields in the SQL statements included in the target SQL statement group are the same; Generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statement that meets the specific conditions is an SQL statement that includes the specified field and the value of the specified field is the field value; The SQL current limiting rule is sent to a second data storage node of the distributed database, so that the second data storage node limits the SQL statement according to the SQL current limiting rule.

2. The method according to claim 1, before obtaining the SQL statement in execution state on the first data storage node of the distributed database, further comprising: If the condition field of the original SQL statement requested to be executed by the client includes the specified field, then generate SQL comment information including the specified field; The SQL comment information is carried by the SQL statement generated based on the original SQL statement and sent to the first data storage node; The determining, based on the value of the specified field in the SQL statement in execution state, a target SQL statement group including a number of SQL statements exceeding a preset number threshold, specifically includes: Based on the designated field contained in the SQL comment information carried by the SQL statement in execution state, and according to the value of the designated field in the SQL statement in execution state, a target SQL statement group including a number of SQL statements exceeding a preset number threshold is determined.

3. The method of claim 2, further comprising: Get pre-set annotation instructions; The comment indication information is used to indicate that SQL comment information needs to be generated for the original SQL statement containing the specified field; If the condition field of the original SQL statement requested to be executed by the client includes the specified field, then generating SQL comment information including the specified field, specifically including: Based on the designated field targeted by the comment indication information, if the condition field of the original SQL statement requested to be executed by the client includes the designated field, SQL comment information including the designated field is generated.

4. The method of claim 2, further comprising: Generate a template identifier based on the original SQL statement executed by the client request; The template identifier is used to identify the structure of the remaining SQL statements except the value of the condition field in the original SQL statement; The generating of SQL comment information including the specified field specifically includes: Generate SQL comment information including the template identifier and the specified field; The determining, based on the value of the specified field in the SQL statement in execution state, a target SQL statement group including a number of SQL statements exceeding a preset number threshold, specifically includes: Based on the template identifier of the SQL statement in execution state and the value of the designated field, a target SQL statement group including a number of SQL statements exceeding a preset number threshold is determined; the SQL statements included in the target SQL statement group have the same template identifier and the same value of the designated field.

5. The method according to claim 2, wherein generating SQL comment information including the designated field specifically comprises: Generate SQL comment information including the first SQL identifier and the designated field; The first SQL identifier is used to indicate that the original SQL statement is obtained based on a request from a client; The obtaining of an SQL statement in execution state on a first data storage node of a distributed database specifically includes: Based on the SQL comment information carried by the SQL statement in execution state, an SQL statement in execution state, which contains the first SQL identifier in the SQL comment information carried by the first data storage node of the distributed database, is obtained.

6. The method according to claim 1, before obtaining the SQL statement in execution state on the first data storage node of the distributed database, further comprising: Acquire first operating data of the first data storage node; The first operation data includes at least one of the number of SQL statements in execution state and resource occupancy data; The obtaining of an SQL statement in execution state on a first data storage node of a distributed database specifically includes: If the first running data meets the preset trigger conditions, the SQL statements in execution state on the first data storage node are obtained; the preset trigger conditions include that the number of SQL statements is greater than or equal to the first SQL number threshold; or, the resource occupancy rate is greater than or equal to the first resource occupancy rate threshold; or, the number of SQL statements is greater than or equal to the second SQL number threshold and the resource occupancy rate is greater than or equal to the second resource occupancy rate threshold.

7. The method according to claim 6, wherein if the first running data satisfies a preset trigger condition, obtaining the SQL statement in execution state on the first data storage node specifically comprises: If the second operating data meets the preset trigger condition after the first operating data meets the preset trigger condition for a preset interval, the SQL statement in execution state on the first data storage node is obtained; the second operating data includes at least one of the number of SQL statements in execution state and updated resource occupancy data on the first data storage node after the preset interval.

8. The method of claim 6, further comprising: If the first operating data meets the preset trigger condition, an abnormal alarm message is sent.

9. The method of claim 1, wherein: The second data storage node includes the first data storage node.

10. The method of claim 1, wherein determining a target SQL statement group including a number of SQL statements exceeding a preset number threshold based on the value of a specified field in the SQL statement in execution state specifically comprises: Dividing the SQL statements in the execution state into SQL statement groups according to the values of designated fields in the SQL statements in the execution state; The values of the specified fields in the SQL statements of the same SQL statement group are the same; A target SQL statement group is determined from the SQL statement groups, the target SQL statement group including a number of SQL statements exceeding a preset number threshold.

11. The method according to claim 1, further comprising: after sending the SQL flow limiting rule to the second data storage node of the distributed database; Obtaining execution durations of several SQL statements that comply with the SQL current limiting rule and are executed on the second data storage node after the second data storage node limits the SQL statements according to the SQL current limiting rule; If the execution times of the multiple SQL statements are all less than the preset time threshold, the SQL current limiting rule is canceled.

12. The method of claim 11, wherein: The multiple SQL statements specifically include SQL statements that comply with the SQL current limiting rule and are executed N times consecutively on the second data storage node after the SQL statements are current limited according to the SQL current limiting rule, where N is an integer greater than 1.

13. The method according to claim 1, further comprising: after sending the SQL flow limiting rule to the second data storage node of the distributed database; After the SQL current limiting rule is sent for a preset time, the SQL current limiting rule is canceled.

14. A device for limiting the flow of SQL statements, applied to a computing node of a distributed database, comprising: An SQL statement acquisition module, configured to acquire an SQL statement in execution state on a first data storage node of a distributed database; a target SQL statement group determination module, configured to determine a target SQL statement group including a number of SQL statements exceeding a preset number threshold according to a value of a specified field in the SQL statement in execution; The values of the designated fields in the SQL statements included in the target SQL statement group are the same; An SQL flow limiting rule generation module, configured to generate an SQL flow limiting rule based on the field value of the designated field in the SQL statements included in the target SQL statement group; the SQL flow limiting rule is configured to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that include the designated field and whose value is the field value; The SQL current limiting rule sending module is used to send the SQL current limiting rule to the second data storage node of the distributed database, so that the second data storage node limits the SQL statement according to the SQL current limiting rule.

15. A distributed database comprising computing nodes and data storage nodes; The computing node is configured to obtain an SQL statement in execution state on a first data storage node of a distributed database; determine a target SQL statement group containing a number of SQL statements exceeding a preset number threshold according to the value of a specified field in the SQL statement in execution state, the SQL statements contained in the target SQL statement group having the same value of the specified field; generate an SQL flow limiting rule based on the field value of the specified field in the SQL statements contained in the target SQL statement group, the SQL flow limiting rule being used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database, the SQL statements that meet the specific conditions being SQL statements that include the specified field and whose value is the field value; and send the SQL flow limiting rule to a second data storage node of the distributed database, so that the second data storage node limits the SQL statements according to the SQL flow limiting rule; A first data storage node among the data storage nodes is configured to provide an SQL statement in an execution state to the computing node; The second data storage node in the data storage nodes is configured to receive the SQL current limiting rule sent by the computing node; and limit the SQL statement according to the SQL current limiting rule.

16. The distributed database according to claim 15, wherein: The second data storage node includes the first data storage node.

17. A computing device comprising: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 13 are implemented.

18. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction is executed by a processor to implement the steps of the method according to any one of claims 1 to 13.

19. A computer program product comprising a computer program / instructions, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 13.

Images