Methods, devices, equipment, and distributed databases for rate limiting of SQL statements

Abstract

The embodiments of this specification provide a method, apparatus, device, and distributed database for rate limiting of SQL statements. The method for rate limiting SQL statements is applied to the computing nodes of the distributed database and includes: acquiring SQL statements currently in execution on a first data storage node of the distributed database, and determining a target SQL statement group containing more SQL statements than a preset threshold based on the values ​​of specified fields in the SQL statements, wherein the specified fields in the SQL statements within the target SQL statement group have the same value; then, generating SQL rate limiting rules based on the field values ​​of the specified fields in the SQL statements within the target SQL statement group, the SQL rate limiting rules being used to limit the number of SQL statements executed on the distributed database that contain specified fields and whose values ​​are field values; finally, sending the SQL rate limiting rules to a second data storage node of the distributed database so that the second data storage node can rate limit the SQL statements according to the SQL rate limiting rules.

Description

Technical Field

[0001] This specification relates to the field of database technology, and in particular to a method, apparatus, device, medium, computer program product, and distributed database for rate limiting of SQL statements. Background Technology

[0002] In modern information systems, database performance and stability are crucial, especially in high-concurrency query environments where databases are prone to resource bottlenecks and performance degradation. Traditional monitoring methods struggle to identify and handle abnormal Structured Query Language (SQL) statements in a timely manner, leading to slower system response or crashes.

[0003] Therefore, there is a need to provide a solution for database systems to automatically generate rate limiting rules for identifying high-concurrency or abnormal SQL operations and apply these rules to rate limit abnormal SQL statements, in order to improve system reliability and user experience. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method for rate limiting SQL statements. One or more embodiments of this specification also relate to an apparatus for rate limiting SQL statements, a computing device, a computer-readable storage medium, and a computer program product, as well as a distributed database, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a method for rate limiting of SQL statements is provided, applied to a computing node of a distributed database, comprising:

[0006] Retrieve the SQL statements that are currently in execution on the first data storage node of the distributed database;

[0007] Based on the values ​​of specified fields in the SQL statements that are in the execution state, a target SQL statement group is determined where the number of SQL statements exceeds a preset threshold; the specified fields in the SQL statements contained in the target SQL statement group are the same.

[0008] Based on the field values ​​of the specified fields in the SQL statements contained in the target SQL statement group, an SQL rate limiting rule is generated; the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose values ​​are the specified fields.

[0009] The SQL rate limiting rule is sent to the second data storage node of the distributed database so that the second data storage node can limit the rate of SQL statements according to the SQL rate limiting rule.

[0010] According to a second aspect of the embodiments of this specification, an apparatus for rate limiting SQL statements is provided, applied to a computing node of a distributed database, comprising:

[0011] The SQL statement retrieval module is used to retrieve SQL statements that are in execution state on the first data storage node of the distributed database.

[0012] The target SQL statement group determination module is used to determine a target SQL statement group containing more than a preset number threshold based on the values ​​of specified fields in the SQL statements that are in the execution state; the values ​​of the specified fields in the SQL statements contained in the target SQL statement group are the same.

[0013] The SQL rate limiting rule generation module is used to generate SQL rate limiting rules based on the field values ​​of specified fields in the SQL statements contained in the target SQL statement group; the SQL rate limiting rules are used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that contain the specified field and whose value is the field value.

[0014] The SQL rate limiting rule distribution module is used to send the SQL rate limiting rule to the second data storage node of the distributed database, so that the second data storage node can limit the rate of SQL statements according to the SQL rate limiting rule.

[0015] According to a third aspect of the embodiments of this specification, a distributed database is provided, including computing nodes and data storage nodes;

[0016] The computing node is configured to: acquire SQL statements in execution state on a first data storage node of the distributed database; determine a target SQL statement group containing more SQL statements than a preset threshold based on the values ​​of specified fields in the SQL statements in execution state, wherein the specified fields in the SQL statements in the target SQL statement group have the same value; generate an SQL rate limiting rule based on the field values ​​of the specified fields in the SQL statements in the target SQL statement group, wherein the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions executed on the distributed database, wherein the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose specified fields are the specified field values; and send the SQL rate limiting rule to a second data storage node of the distributed database so that the second data storage node can rate limit the SQL statements according to the SQL rate limiting rule.

[0017] The first data storage node in the data storage nodes is configured to provide the computing node with SQL statements in an execution state;

[0018] The second data storage node in the data storage node is configured to receive SQL rate limiting rules sent by the computing node; and to rate limit SQL statements according to the SQL rate limiting rules.

[0019] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0020] Memory and processor;

[0021] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the above-mentioned method for rate limiting of SQL statements.

[0022] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method for rate limiting of SQL statements.

[0023] According to a sixth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method for rate limiting of SQL statements.

[0024] One embodiment of this specification can achieve at least the following beneficial effects: by real-time monitoring of SQL statements in the execution state on data storage nodes, and grouping the SQL statements in the execution state according to the values ​​of specified fields in the SQL statements, a target SQL statement group containing more than a preset threshold number of SQL statements is determined. Then, based on the field values ​​of the specified fields in the SQL statements contained in the target SQL statement group, an SQL rate limiting rule is generated to limit the number of SQL statements that meet specific conditions that are executed on the distributed database containing the data storage node. The SQL statements that meet the specific conditions are SQL statements that contain the specified fields and whose values ​​are the specified fields. Thus, the automatic and timely detection of high-concurrency or abnormal SQL operations is achieved, which is beneficial to improving the reliability of the distributed database system and the user experience. Attached Figure Description

[0025] Figure 1 This is a schematic diagram illustrating a method for rate limiting of SQL statements provided in an embodiment of this specification.

[0026] Figure 2 A flowchart illustrating a method for rate limiting of SQL statements provided in an embodiment of this specification;

[0027] Figure 3 This is a schematic diagram of the framework for a detection and removal scheme for abnormal SQL in a practical application scenario provided in the embodiments of this specification.

[0028] Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of a device for rate limiting SQL statements;

[0029] Figure 5 This specification provides a structural block diagram of a computing device 500 in an embodiment. Detailed Implementation

[0030] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0031] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0033] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0034] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0035] A distributed database is a logically unified database management system that uses a computer network to connect multiple physical database nodes. Compared to single-machine databases, distributed databases often have better scalability, and can improve the overall computing and storage performance of the database by adding nodes, without being limited by the hardware configuration of a single physical node.

[0036] A distributed database consists of a coordinator node and data nodes. The coordinator node, also known as a compute node, is responsible for coordinating and managing the entire database cluster. Data nodes, also called data storage nodes, are responsible for storing and processing the actual data. Each data storage node contains a portion of the database data and corresponding processing capabilities. These data storage nodes can be distributed across different physical servers or compute nodes, forming a distributed database cluster.

[0037] Specifically, compute nodes are responsible for receiving access requests from clients and returning execution results; they are also responsible for decomposing tasks and scheduling task shards for parallel execution across various data storage nodes. Compute nodes typically contain query optimizers and execution engines to optimize query performance and execute complex data processing tasks. Data storage nodes receive data read and write requests from applications or other nodes and are responsible for performing the actual data storage and retrieval operations. Data storage nodes are typically organized in a distributed manner, with each node storing a portion of the data, and horizontal scaling and load balancing are achieved through sharding technology.

[0038] To maximize computing performance, a distributed database system can include multiple compute nodes, a load balancer, and multiple storage nodes. The load balancer distributes received SQL statements to one of these compute nodes for execution. Specifically, compute nodes can receive SQL statements from users. These users can be individual or institutional users. Users can register an account in the database system and log in to that account on their device cluster. Devices in the cluster can then send SQL statements to the database system according to pre-defined task logic to perform corresponding data operations.

[0039] In modern information systems, the database, as the underlying engine carrying core task data, directly impacts the service quality and user experience of the entire system. With the accelerated pace of enterprise digital transformation, the daily data volume processed by systems is growing exponentially. Especially in high-concurrency scenarios such as e-commerce flash sales, financial transaction settlements, and real-time IoT data processing, database clusters need to respond to tens of thousands of query requests per second. This continuous high-pressure working state easily triggers resource contention issues—the intense competition among multiple query threads for CPU computing resources, memory buffers, and disk I / O channels often leads to a chain reaction of lock contention, deadlocks, and query blocking. Even more problematic is that some unoptimized complex join queries or full table scan SQL statements, when executed concurrently, can rapidly exhaust system resources like an avalanche, causing a precipitous drop in the performance of the entire database service.

[0040] When a database experiences sudden surges in traffic leading to a sharp increase in SQL statement concurrency, with some SQL statements consuming excessive database resources, or a large number of SQL statements without indexes being invoked, impacting normal tasks, it's necessary to limit the concurrency of problematic SQL statements to ensure the normal operation of the majority of tasks. In related technologies, this is typically done after a significant increase in the database system's queries per second (QPS) is detected. Problematic SQL statements are then identified by sequentially checking the data storage nodes in the database system, and corresponding rate-limiting rules are set. Specifically, this requires manual judgment regarding the timing of rate-limiting rules and manual configuration of which SQL statements with specific characteristics should be subject to traffic restrictions (e.g., manually determining specific rate-limiting parameters), which is inherently lagging and affects database reliability.

[0041] In this specification, considering that problematic SQL statements that cause slow database system response or crashes in practical applications typically have two characteristics: first, the execution time of a single problematic SQL statement is excessively long; second, the number of problematic SQL statements is excessive (i.e., excessively high concurrency). When the database system already has excessively high resource utilization and too many active sessions, the execution time of all SQL statements may be unreasonably extended, making it difficult to identify problematic SQL statements based on their execution time. Therefore, by performing concurrency statistics on SQL statements in execution across certain dimensions, high-concurrency SQL statements can be identified as high-risk SQL statements (i.e., SQL statements with a high probability of being problematic SQL statements). Rate limiting can then be applied to these high-concurrency SQL statements. In this way, high-risk SQL statements can be detected and rate-limited in a timely manner, reducing the probability of slow response or crashes in the database system caused by problematic SQL statements.

[0042] Therefore, the technical solution provided in this specification embodiment is to set up a management module in the computing node for performing automatic detection and removal. This management module can extract the SQL statements that are in the execution state on each data storage node, and determine the SQL statements that meet specific conditions and whose number exceeds a preset threshold according to the values ​​of specified fields in the extracted SQL statements. Then, it can automatically generate SQL rate limiting rules based on the field values ​​in the SQL statements that meet the specific conditions. This facilitates timely traffic limiting of SQL statements that meet the specific conditions executed on the data storage nodes of the distributed database.

[0043] This specification provides a method for rate limiting SQL statements. It also relates to an apparatus for rate limiting SQL statements, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0044] Figure 1 This is a schematic diagram illustrating a method for rate limiting of SQL statements provided in an embodiment of this specification.

[0045] In such Figure 1 In the application scenario shown, the database server 10 can connect to one or more user clients 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These user clients 20 can include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. In practical applications, the user client 20 can be a database user. Database users can access the database server 10 through the user client 20, specifically performing data processing operations, including reading and writing, on the database server 10.

[0046] Database server 10 can connect to one or more management clients 30 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. These management clients 30 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. In practical applications, users of the management client 30 can be database management and maintenance personnel. These personnel can use the management client 30 to maintain and manage the database system, thereby implementing the methods provided in the embodiments of this specification.

[0047] In this embodiment of the specification, the database server 10 may, in response to the management of the management client 30, perform the following steps: obtain SQL statements in execution state on the first data storage node of the distributed database; divide the SQL statements into SQL statement groups according to the values ​​of specified fields in the SQL statements in execution state, wherein the values ​​of the specified fields in the SQL statements of the same SQL statement group are the same; determine, from the plurality of SQL statement groups, a target SQL statement group containing more than a preset number threshold of SQL statements; generate an SQL rate limiting rule based on the field values ​​of the specified fields in the SQL statements contained in the target SQL statement group, wherein the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database, wherein the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose values ​​are the specified field values.

[0048] Figure 2 This is a flowchart illustrating a method for rate limiting of SQL statements provided in an embodiment of this specification.

[0049] From a programming perspective, the entity executing the process can be a program hosted on a server, or more specifically, a program hosted on a computing node of a distributed database. More specifically, it can be a rule manager running on a computing node within a distributed database system.

[0050] like Figure 2 As shown, the process may include the following steps:

[0051] Step 202: Obtain the SQL statements that are in execution state on the first data storage node of the distributed database.

[0052] In practical applications, a distributed database system may include several computing nodes and multiple data storage nodes. In the embodiments of this specification, nodes used for performing tasks such as... Figure 2 The modules of the method shown are deployed on the master node among the plurality of computing nodes.

[0053] Considering that in practical applications, when database systems encounter resource bottlenecks or performance degradation, these issues typically occur at data storage nodes, regular monitoring of these nodes is necessary. In step 202, the rule manager deployed on the compute node (e.g., the master node) can continuously inspect one or more data storage nodes to identify any anomalies.

[0054] exist Figure 2In the method embodiment shown, for each data storage node in the database system, the SQL statements in the execution state on that data storage node can be obtained. In order to identify high-risk SQL statements for the database system by analyzing these SQL statements in the execution state, and then generate rate limiting rules for these high-risk SQL statements.

[0055] An SQL statement in the execution state can be any SQL statement currently consuming CPU resources. Specifically, when an SQL statement is scheduled for execution on the CPU, its corresponding process or thread is in the Running state. In other words, an SQL statement in the execution state can be an SQL statement running within an execution process or thread. Taking MySQL as an example, the SHOW PROCESSLIST command can be used to view currently active SQL connections; processes displaying "executing" in the State column represent SQL statements in the execution state.

[0056] Step 204: Based on the values ​​of specified fields in the SQL statements that are in the execution state, determine the target SQL statement group whose number of SQL statements exceeds a preset threshold; the specified fields in the SQL statements contained in the target SQL statement group are the same.

[0057] First, the SQL statements in the execution state can be divided into SQL statement groups based on the values ​​of specified fields in the SQL statements in the execution state; the values ​​of the specified fields are the same in the SQL statements in the same SQL statement group.

[0058] In step 202, what is actually extracted may be the SQL statement containing the specified field that is being executed on the first data storage node.

[0059] The specified fields can be fields that are determined empirically to appear in problematic SQL statements. For example, the characteristics of problematic SQL statements can be analyzed based on information such as the historical rate limiting and fault handling of the database system, thereby determining the fields that are usually included in problematic SQL statements as specified fields. It is understood that high-risk SQL statements (or problematic SQL statements) identified based on the embodiments of this specification need to include the specified fields; however, SQL statements that include the specified fields are not necessarily high-risk SQL statements (or problematic SQL statements).

[0060] In practical applications, the specified field can be updatable. For example, when the first specified field is set, and an operation is performed as follows... Figure 2After the method has been used several times or over a period of time, the first specified field can be evaluated based on the execution results (e.g., whether problematic SQL statements were correctly identified). If the evaluation result is satisfactory, the first specified field continues to be used; if the evaluation result is unsatisfactory, the first specified field is updated to the second specified field, and the process is repeated. Figure 2 The method described.

[0061] Furthermore, the specified field may specifically include one or more specified fields.

[0062] When the specified fields specifically include multiple specified fields, optionally, in step 204, the SQL statements are grouped according to the values ​​of the specified fields so that the values ​​of the specified fields are the same in the SQL statements within the same SQL statement group. This can be achieved by having all the specified fields have the same value. For example, if the specified fields include a first specified field and a third specified field, step 204 specifically includes dividing the SQL statements into SQL statement groups based on the values ​​of the first and third specified fields in the SQL statements. In the same SQL statement group, the values ​​of the first and third specified fields are the same. As an example, if the specified fields include userId and companyId, then SQL statements with the same userId value and the same companyId value can be grouped into the same SQL statement group.

[0063] When the specified fields specifically include multiple specified fields, optionally, in step 204, the SQL statements are grouped according to the values ​​of the specified fields so that the values ​​of the specified fields in the SQL statements within the same SQL statement group are the same. This can be achieved by having at least some of the specified fields have the same value. For example, if the specified fields include a first specified field and a third specified field, step 204 may specifically include: dividing the SQL statements into SQL statement groups based on the value of the first specified field, where the values ​​of the first specified field are the same in the SQL statements within the same SQL statement group; or, dividing the SQL statements into SQL statement groups based on the value of the third specified field, where the values ​​of the third specified field are the same in the SQL statements within the same SQL statement group. As an example, if the specified fields include userId and companyId, then SQL statements with the same userId value (regardless of whether the companyId value is the same) can be grouped into the same SQL statement group; or, SQL statements with the same companyId value (regardless of whether the userId value is the same) can be grouped into the same SQL statement group. The specific grouping rules can be set and adjusted according to actual needs.

[0064] Then, from the SQL statement group, target SQL statement groups containing more than a preset number threshold can be identified.

[0065] In the embodiments of this specification, the purpose of setting a specified field is to identify high-concurrency SQL statements based on the specified field dimension after grouping the SQL statements extracted from the first data storage node according to the value of the specified field. In other words, the specified field can be a conditional field determined based on historical data to filter out high-risk SQL statements. For ease of explanation later, in the embodiments of this specification, a group of SQL statements containing more than a preset threshold number of SQL statements is referred to as a target SQL statement group.

[0066] The preset threshold can be determined based on the concurrency of SQL statements under normal database system usage. Normal usage refers to a situation where the database system's resource utilization, number of active sessions, etc., are within normal ranges. For example, if the concurrency of SQL statements typically does not exceed 10 under normal database system usage, then 10 can be set as the preset threshold. The value of 10 given here is merely an example of a preset threshold; in actual applications, the preset threshold can be set according to the specific circumstances.

[0067] In the embodiments of this specification, the target SQL statement group may include one or more. In practical applications, in the process of determining the target SQL statement group, in addition to using a preset quantity threshold as a filtering condition, the number of selected target SQL statement groups can also be limited. In practical applications, the scheme of generating and executing rate limiting rules can be implemented by repeatedly executing the identification process multiple times for each group of SQL statements.

[0068] Specifically, the concurrency of each SQL statement group can be calculated first based on the number of SQL statements it contains. This concurrency can be the number of SQL statements in the group, or a value calculated according to a preset rule. Then, the SQL statement groups are sorted from highest to lowest concurrency to obtain a sequence. Next, a preset number of SQL statement groups that meet a preset concurrency threshold and have relatively high concurrency are selected from the sequence as target SQL statement groups. Alternatively, after calculating the concurrency of each SQL statement group based on the number of SQL statements it contains, candidate SQL statement groups whose concurrency meets a preset threshold can be selected first. Then, the candidate SQL statement groups are sorted based on the number of SQL statements they contain to obtain a sequence, and a preset number of SQL statement groups with relatively high concurrency in the sequence are selected as target SQL statement groups.

[0069] Step 206: Generate an SQL rate limiting rule based on the field value of the specified field in the SQL statements contained in the target SQL statement group; the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statement that meets specific conditions is an SQL statement that contains the specified field and the value of the specified field is the field value.

[0070] In the embodiments of this specification, for ease of subsequent explanation, the values ​​of the specified fields in the SQL statements contained in the target SQL statement group are referred to as field values.

[0071] As an example, in step 202, the SQL statements currently being executed on the first data storage node that contain the specified field can be extracted. Assume the specified field is userId. In step 204, the extracted SQL statements can be grouped according to the value of the specified field. Assume the values ​​of the specified field in the extracted SQL statements include userId=001, userId=003, userId=006, etc. After grouping the SQL statements according to the value of the specified field, the first SQL statement group has a specified field value of userId=001, the second SQL statement group has a specified field value of userId=003, the third SQL statement group has a specified field value of userId=006, etc. Then, the SQL statements in each SQL statement group can be further grouped according to their values. The number of SQL statements included is used to determine the target SQL statement group from each SQL statement group. For example, if the number of SQL statements containing the filter condition userId=001 in the first SQL statement group is 13, the number of SQL statements containing the filter condition userId=003 in the second SQL statement group is 1, the number of SQL statements containing the filter condition userId=006 in the third SQL statement group is 3, and so on, and assuming that the preset number threshold is 10, then the first SQL statement group can be determined as the target SQL statement group. In step 206, based on the field value userId=001 of the specified field in the target SQL statement group, an SQL rate limiting rule can be generated to rate limit the SQL statements containing the filter condition userId=001.

[0072] Among them, SQL rate limiting rules can be rules used to limit the number of SQL statements executed concurrently on data storage nodes. Specifically, based on Figure 2 The method automatically generates SQL rate limiting rules, which can be used to limit the number of SQL statements that are executed concurrently on data storage nodes and contain the specified field, where the value of the specified field is the value of the specified field.

[0073] In practical applications, the SQL rate limiting rule may also include the number of SQL rate limits. Optionally, the number of SQL rate limits may be equal to the preset threshold. Alternatively, the number of SQL rate limits may be greater than or less than the preset threshold. Continuing with the previous example, if the preset threshold is 10, then the number of SQL rate limits can also be set to 10.

[0074] Continuing with the previous example, when generating SQL rate limiting rules for SQL statements containing the filter condition userId=001, the rules can be set as follows: If the SQL statement to be executed contains the filter condition userId=001, and the number of SQL statements containing the filter condition userId=001 currently being executed on the first data storage node is less than the SQL rate limiting limit of 10, then the SQL statement to be executed can be executed; however, if the SQL statement to be executed contains the filter condition userId=001, and the number of SQL statements containing the filter condition userId=001 currently being executed on the first data storage node reaches the SQL rate limiting limit of 10, then it is necessary to wait for at least some of the SQL statements containing the filter condition userId=001 to be executed to complete, so that the number of at least some SQL statements containing the filter condition userId=001 currently being executed on the first data storage node is less than the SQL rate limiting limit of 10, before the SQL statement to be executed can be executed. Therefore, based on this rate limiting rule, the concurrency of identified high-risk SQL statements can be limited to a controllable range (for example, limited to 10 in the above example), thereby reducing the probability of risks such as slow database system response or crashes.

[0075] As described above, the specified field and its value, as well as the SQL rate limit quantity, can be referred to as the rate limiting parameters in the SQL rate limiting rule. In the embodiments of this specification, at least some of the rate limiting parameters can be determined based on the methods provided in the embodiments of this specification. For example, the field value of the specified field can be determined by statistical analysis of the SQL statements running on the data storage node. Some parameters in the rate limiting rule can be preset by the database management and maintenance personnel. For example, the specified field and the SQL rate limit quantity can be set according to actual needs or task experience.

[0076] Step 208: Send the SQL rate limiting rule to the second data storage node of the distributed database so that the second data storage node can rate limit the SQL statements according to the SQL rate limiting rule.

[0077] The second data storage node may include the first data storage node.

[0078] In one or more embodiments of this specification, SQL rate limiting rules can be generated by a rule manager deployed on a compute node. These rules can then be distributed to data storage nodes in a distributed database, enabling the data storage nodes to implement SQL rate limiting according to the rules in subsequent data processing. Specifically, the SQL rate limiting rules can be sent to one or more data storage nodes in the distributed database, allowing these nodes to execute SQL rate limiting based on the rules.

[0079] In the embodiments of this specification, considering that a database system typically includes multiple data storage nodes, after determining the field value of a specified field in the target SQL statement group for the first data storage node and generating the corresponding SQL rate limiting rule, the SQL rate limiting rule can be applied on the first data storage node; alternatively, the SQL rate limiting rule can be distributed to one or more other data storage nodes in the database system so that the SQL rate limiting rule can be applied on one or more other data storage nodes of the distributed database (e.g., all data storage nodes).

[0080] It should be understood that in the methods described in one or more embodiments of this specification, the order of some steps may be adjusted according to actual needs, or some steps may be omitted.

[0081] Figure 2 The method described herein involves real-time monitoring of SQL statements in execution on the first data storage node of a distributed database. SQL statements are grouped according to the values ​​of specified fields within these statements. Target SQL statement groups containing more than a preset threshold are identified. Based on the values ​​of the specified fields in the SQL statements within these target groups, SQL rate limiting rules are generated to restrict the number of SQL statements that meet specific conditions and are executed on the distributed database. These SQL statements meet the specific conditions by containing the specified fields and having the specified field value as stated in the field value. This approach enables automatic and timely detection of high-concurrency or abnormal SQL operations, improving the reliability of the distributed database system and enhancing the user experience.

[0082] based on Figure 2 In addition to the method described herein, this specification also provides some improved implementation methods, which will be described below.

[0083] In one or more embodiments of this specification, potentially risky SQL statements may be pre-commented and marked, thereby preventing them from being executed when... Figure 2The method for rate limiting SQL statements described above can analyze only the SQL statements marked with comments that run on the data storage node, which saves computing resources and improves analysis efficiency.

[0084] Specifically, before obtaining the SQL statements in execution state on the first data storage node of the distributed database, the process may further include: if the condition field of the original SQL statement requested by the client contains the specified field, then generating SQL comment information containing the specified field; and sending the SQL comment information to the first data storage node by an SQL statement generated based on the original SQL statement. Correspondingly, determining the target SQL statement group containing more than a preset threshold number of SQL statements based on the value of the specified field in the SQL statements in execution state may specifically include: determining the target SQL statement group containing more than a preset threshold number of SQL statements based on the specified field contained in the SQL comment information carried by the SQL statements in execution state, and based on the value of the specified field in the SQL statements in execution state.

[0085] Specifically, generating SQL comment information containing the specified field may include: obtaining the original SQL statement requested by the client; then, on the one hand, determining the SQL statement to be sent to the first data storage node based on the original SQL statement; on the other hand, if the condition field of the original SQL statement contains the specified field, generating SQL comment information containing the specified field; and then, sending the SQL statement carrying the SQL comment information to the first data storage node.

[0086] The client can be the database user's client, for example, it could be... Figure 1 User client 20.

[0087] Optionally, obtaining the original SQL statement requested by the client can be done by obtaining the SQL statement directly sent by the client. Alternatively, obtaining the original SQL statement requested by the client can be done by obtaining the query parameters sent by the client, and then generating an SQL statement based on the query parameters as the original SQL statement.

[0088] In practical applications, after obtaining the original SQL statement, the compute nodes in a database system can parse and decompose it, then generate an execution plan and distribute the task to one or more data storage nodes. That is, the compute nodes can generate SQL statements to be sent to one or more data storage nodes based on the original SQL statement. Subsequently, one or more data storage nodes can parse the received SQL statement and perform data processing operations on their corresponding disks, such as data queries, and send the queried results back to the compute nodes. Furthermore, the compute nodes can further process and encapsulate the data returned by the one or more data storage nodes before returning it to the client that requested the execution of the original SQL statement.

[0089] In one or more embodiments of this specification, database administrators can pre-create SQL tagging rules, which can specify which characteristics of raw SQL statements should be detected. Thus, the rule manager on the computing node of the database system can identify raw SQL statements that conform to the characteristics specified by the SQL tagging rules from a large number of raw SQL statements requested by database users to be executed by the database system, based on the pre-created SQL tagging rules, and then generate SQL comment information for the raw SQL statements as tags.

[0090] Specifically, the method for rate limiting SQL statements may further include: obtaining pre-set comment indication information; the comment indication information is used to indicate that SQL comment information needs to be generated for the original SQL statement containing the specified field. Correspondingly, generating SQL comment information containing the specified field if the condition field of the original SQL statement requested by the client contains the specified field may specifically include: based on the specified field targeted by the comment indication information, if the condition field of the original SQL statement requested by the client contains the specified field, then generating SQL comment information containing the specified field.

[0091] Furthermore, based on the annotation instruction information, the computing node can determine that SQL annotation information needs to be generated for the original SQL statement containing the specified field. Thus, for any obtained original SQL statement, the computing node can determine whether the original SQL statement contains the specified field. If the condition field of the original SQL statement contains the specified field, then SQL annotation information containing the specified field is generated; if the condition field of the original SQL statement does not contain the specified field, then it is not necessary to generate SQL annotation information containing the specified field.

[0092] Further optionally, the comment indication information set in the SQL tagging rule may only include a specified field identifier. In this case, the comment indication information is used to indicate that when the condition field of an original SQL statement contains the specified field, SQL comment information needs to be generated for the original SQL statement. Alternatively, the comment indication information set in the SQL tagging rule may include a specified table identifier and a specified field identifier. In this case, the comment indication information is used to indicate that when the data processing object of an original SQL statement is the specified field in a specified table, SQL comment information needs to be generated for the original SQL statement. Alternatively, the comment indication information set in the SQL tagging rule may include a specified database identifier, a specified table identifier, and a specified field identifier. In this case, the comment indication information is used to indicate that when the data processing object of an original SQL statement is the specified field in a specified table in a specified database, SQL comment information needs to be generated for the original SQL statement.

[0093] In practical applications, database administrators can pre-create one or more SQL tagging rules. When multiple SQL tagging rules are set, after obtaining the original SQL statement requested by the client, the administrator can identify the original SQL statements that conform to each SQL tagging rule and generate corresponding SQL comment information. For example, if two SQL tagging rules are pre-created, the first SQL tagging rule sets a first comment indication to indicate that SQL comment information needs to be generated for original SQL statements containing a first specified field, and the second detection rule sets a second comment indication to indicate that SQL comment information needs to be generated for original SQL statements containing a second specified field, then for any original SQL statement, it is necessary to determine whether it contains both the first and second specified fields. If the original SQL statement contains the first specified field but not the second specified field, then SQL comment information containing the first specified field is generated; if the original SQL statement does not contain the first specified field but contains the second specified field, then SQL comment information containing the second specified field is generated; if the original SQL statement contains both the first and second specified fields, then SQL comment information containing both the first and second specified fields is generated.

[0094] In the embodiments of this specification, generating SQL comment information for the original SQL statement containing the specified field can actually be adding SQL comment information to the SQL statement generated based on the original SQL statement for distribution to the data storage node. That is, if the condition field of the original SQL statement contains the specified field, then SQL comment information containing the specified field is generated, and the SQL comment information is carried to the SQL statement generated based on the original SQL statement for distribution to the data storage node.

[0095] As an example, the SQL comment information may specifically include HINT comments. A HINT comment is a special type of SQL comment used to convey instructions to the database optimizer, thereby generating a specified execution plan. In practice, other comment formats that meet the requirements can also be used.

[0096] Based on at least some embodiments of this specification, during the setup phase, database administrators can pre-create SQL tagging rules to specify which original SQL statements containing specified fields need to generate SQL comment information. During the application phase, when the database system receives the original SQL statement requested by the user client, if the original SQL statement contains the specified fields set in the SQL tagging rules, SQL comment information containing the specified fields is generated. This SQL comment information is then used as the comment information for the SQL statement generated based on the original SQL statement. The SQL statement carrying this SQL comment information is then sent to the data storage node. This allows for statistical analysis of only the SQL statements carrying specific SQL comment information when inspecting SQL statements in execution mode on the data storage node, without needing to analyze the structure of each SQL statement in execution mode to identify which SQL statements should be statistically analyzed. This saves computing resources and improves data analysis efficiency and the accuracy of the analysis results.

[0097] In one or more embodiments of this specification, in order to further improve the accuracy of high-risk SQL screening, based on the concept of grouping SQL based on specified fields to determine high-risk SQL with high concurrency, the structural information of SQL statements is also combined. In practical applications, SQL statements with similar structures, containing specified fields and having the same value of the specified fields can be grouped into the same SQL statement group in order to improve the accuracy of screening high-risk SQL based on the concurrency of SQL within the group.

[0098] Furthermore, considering that in actual database systems, computing nodes will optimize the original SQL statements reflecting task requirements based on load balancing needs to generate SQL statements for distribution to data storage nodes, when using the structural information of SQL statements to divide SQL statement groups, the SQL statement groups can be divided by referring to the structural information of the original SQL statements used to generate the SQL statements.

[0099] Specifically, the method for rate limiting of SQL statements may further include: generating a template identifier based on the original SQL statement executed by the client request; the template identifier is used to identify the structure of the remaining SQL statement in the original SQL statement excluding the values ​​of the condition fields.

[0100] Accordingly, generating SQL comment information containing the specified field specifically includes: generating SQL comment information containing the template identifier and the specified field.

[0101] Accordingly, determining the target SQL statement group containing more than a preset number of SQL statements based on the value of a specified field in the SQL statement in the execution state specifically includes: determining the target SQL statement group containing more than a preset number of SQL statements based on the template identifier of the SQL statement in the execution state and the value of the specified field; wherein the target SQL statement group contains SQL statements with the same template identifier and the same value of the specified field.

[0102] The template identifier and the value of the specified field can be considered as characteristic information of the SQL statement; the characteristic information of SQL statements in the same SQL statement group is the same. Accordingly, SQL rate limiting rules can be generated based on the characteristic information; the SQL rate limiting rules are used to limit the number of SQL statements that meet specific conditions executed on the distributed database containing the first data storage node; the SQL statements that meet the specific conditions are SQL statements that meet the characteristic information.

[0103] Specifically, SQL statements containing the same template identifier in their SQL comment information are generated based on original SQL statements with the same statement structure. SQL statements containing the same characteristic information in their SQL comment information are not only generated based on original SQL statements with the same statement structure, but also have the same values ​​for specified fields.

[0104] In the embodiments of this specification, from one perspective, the template identifier is generated based on the structure of the original SQL statement and is used to identify the structure of the original SQL statement. From another perspective, the template identifier is carried in the SQL comment information of the SQL statement issued to the data storage node and is used to identify the structure of the original SQL statement that generated the SQL statement.

[0105] In practical applications, different template identifiers are used to identify different structures of the original SQL statements; in the original SQL statements corresponding to the same template identifier, everything except the values ​​of the condition fields is the same. Specifically, in the original SQL statements corresponding to the same template identifier, keywords, table names, field names, etc., can all be the same. In practice, SQL statements with the same template identifier can correspond to the same backend query statement.

[0106] As an example, a template identifier can be generated based on information in the original SQL statement other than the values ​​of the condition fields. Furthermore, a template identifier can be a hash value obtained by performing a hash operation on the information in the original SQL statement other than the values ​​of the condition fields. Thus, different template identifiers can be used to identify parts of the original SQL statement other than the values ​​of the condition fields.

[0107] In a specific example, suppose the templateId of the first original SQL statement "select * from tb where userId = 1" is 001. Then, the templateId of the second original SQL statement "select * from tb where userId = 2" is the same as the first original SQL statement except for the value of the condition field "select * from tb where userId". Therefore, the templateId of the third original SQL statement "select * from tb where companyId = 200" is not exactly the same as the first original SQL statement except for the value of the condition field "select * from tb where userId". Therefore, the templateId of the third original SQL statement is not 001 and can be other values.

[0108] As described above, the template identifier, specified fields and their values, and the SQL rate limit can be referred to as rate limiting parameters in the SQL rate limiting rule. In the embodiments of this specification, at least some of the rate limiting parameters can be determined based on the methods provided in the embodiments of this specification. For example, the field value of the specified field can be determined by statistical analysis of the SQL statements running on the first data storage node; similarly, the template identifier can be calculated by analyzing the statement structure of the original SQL statements obtained by the computing node. Some parameters in the rate limiting rule can be preset by the database management and maintenance personnel. For example, the specified fields and the SQL rate limit can be set according to actual task requirements or task experience.

[0109] Based on at least some embodiments of this specification, on the one hand, original SQL statements containing specified fields set in the SQL marking rules can be identified according to pre-set SQL marking rules. On the other hand, a template identifier reflecting the statement structure of the original SQL statement can be generated based on the statement structure of the original SQL statement. Furthermore, SQL comment information containing the template identifier and specified fields for the original SQL statement can be generated and added to the SQL statement generated based on the original SQL statement for distribution to the data storage node. This ensures that when an SQL statement in execution state is executed on the data storage node, as shown in the example... Figure 2 The method for rate limiting SQL statements can use the data combination of [template identifier - value of specified field] as the feature information for dividing SQL statement groups. Compared to dividing SQL statement groups solely based on the value of specified fields, this method of dividing SQL statement groups using feature information takes into account the consistency of the original SQL statements. That is, it ensures that SQL statements grouped into the same SQL statement group are generated from original SQL statements with the same statement structure. Therefore, the number of SQL statements included in this grouping result can more accurately reflect the SQL concurrency from a task perspective. This improves the accuracy of the grouping results and the accuracy of the SQL rate limiting rules generated based on the grouping results to identify SQL statements that meet specific conditions, thereby improving the SQL rate limiting effect and enhancing the stability of the database system.

[0110] It's important to explain that the accuracy of an SQL rate limiting rule reflects the probability that the original SQL statement targeted by that rule is a problematic SQL statement. Higher accuracy means a greater probability of rate limiting problematic SQL statements.

[0111] In one or more embodiments of this specification, considering that the requesters of the raw SQL statements to be executed obtained by the database system may differ and their risk levels may vary, for example, some raw SQL statements may be initiated by clients with higher security levels and relatively lower risks, such as... Figure 1 The management client shown is the initiator; another part of the original SQL statements may be initiated by clients with relatively low security levels and relatively high risks, for example, such as... Figure 1 As shown, the data processing tasks initiated by the user client are analyzed only for data processing tasks initiated by clients with relatively low security levels and relatively high risks, in order to save computing resources and improve the execution efficiency of the method of rate limiting SQL statements.

[0112] Specifically, generating SQL comment information containing the specified field includes: generating SQL comment information containing a first SQL identifier and the specified field; the first SQL identifier indicates that the original SQL statement was obtained based on a client request. Correspondingly, obtaining the SQL statement in execution state on the first data storage node of the distributed database specifically includes: based on the SQL comment information carried by the SQL statement in execution state, obtaining the SQL statement in execution state carried on the first data storage node of the distributed database that contains the first SQL identifier and is in execution state.

[0113] The first SQL identifier can be used to mark data processing tasks initiated by clients with relatively low security levels and relatively high risks. In practical applications, it can be based on, for example... Figure 1 The example shown is the original SQL statement executed by the user client's request. Specifically, if the original SQL statement is obtained based on the database user's request, it could be the original SQL statement sent by the database user, or it could be the original SQL statement generated based on the parameters sent by the database user.

[0114] In the embodiments of this specification, optionally, after obtaining the original SQL statement, on the one hand, it can be identified that the original SQL statement was obtained based on a database user's request; on the other hand, it can be identified that the original SQL statement contains a specified field in the condition fields; furthermore, a template identifier reflecting the statement structure of the original SQL statement can be generated. Thus, for the original SQL statement obtained based on a database user's request and containing a specified field, SQL comment information containing a first SQL identifier, a template identifier, and the specified field can be generated, and the SQL comment information and the SQL statement generated based on the original SQL statement are simultaneously sent to the data storage node. This allows for the execution of SQL statements that are in an execution state on the data storage node, such as... Figure 2When performing rate limiting on SQL statements, the method described above can only obtain SQL statements that carry the first SQL identifier in the SQL comment information for subsequent analysis. Since it is not necessary to include the original SQL statements initiated by clients with higher security levels and relatively lower risks in the analysis scope, it can greatly improve analysis efficiency, save computing resources, and reduce the risk of accidentally damaging normal SQL statements.

[0115] In one or more embodiments of this specification, before executing the method for generating SQL rate limiting rules as described in the embodiments of this specification, the operation of the data storage node can be monitored first. When the operation of the data storage node is found to be poor, the method for generating SQL rate limiting rules is then executed.

[0116] Specifically, before obtaining the SQL statements in execution state on the first data storage node of the distributed database, the method further includes: obtaining first running data of the first data storage node; the first running data includes at least one of the number of SQL statements in execution state and resource usage data. Correspondingly, obtaining the SQL statements in execution state on the first data storage node of the distributed database specifically includes: if the first running data meets a preset triggering condition, then obtaining the SQL statements in execution state on the first data storage node. Further, if the first running data meets a preset triggering condition, then obtaining the SQL statements in execution state on the first data storage node; the preset triggering condition includes: the number of SQL statements is greater than or equal to a first SQL number threshold; or, the resource usage rate is greater than or equal to a first resource usage rate threshold; or, the number of SQL statements is greater than or equal to a second SQL number threshold and the resource usage rate is greater than or equal to a second resource usage rate threshold.

[0117] The number of SQL statements in execution state may optionally refer to the total number of all SQL statements being executed on the first data storage node; or, optionally, may refer to the number of SQL statements carrying the first SQL identifier being executed on the first data storage node.

[0118] Resource usage data reflects the utilization of node resources. Specifically, node resources may include CPU resources, memory resources, and I / O resources. Correspondingly, the resource usage data may include CPU utilization, memory utilization, and I / O resource utilization.

[0119] The preset triggering conditions may include a first triggering condition based on the number of SQL statements and a second triggering condition based on the resource usage data.

[0120] Optionally, if the first running data includes the number of SQL statements, it can be determined whether the number of SQL statements meets the first triggering condition. If so, the method for generating SQL rate limiting rules is executed. For example, it can be determined whether the number of SQL statements is greater than or equal to a preset SQL number threshold. If so, the method for generating SQL rate limiting rules is executed.

[0121] If the first running data includes resource usage data, it can be determined whether the resource usage data meets the second triggering condition. If so, the method for generating SQL rate limiting rules is executed. For example, it can be determined whether the CPU utilization is greater than or equal to a preset CPU utilization threshold. If so, the method for generating SQL rate limiting rules is executed. Similarly, it can be determined whether the memory utilization is greater than or equal to a preset memory utilization threshold. If so, the method for generating SQL rate limiting rules is executed. Furthermore, it can be determined whether the IO resource utilization is greater than or equal to a preset IO resource utilization threshold. If so, the method for generating SQL rate limiting rules is executed. In practical applications, a comprehensive judgment can also be made based on multiple resource usage data. For example, when one or more resource usage data reach a corresponding preset threshold, the method for generating SQL rate limiting rules is executed.

[0122] If the first running data includes the number of SQL statements and resource usage data, it can be determined whether at least one of the following is met: the number of SQL statements meets a first triggering condition, and the resource usage data meets a second triggering condition; if so, the method for generating SQL rate limiting rules is executed. For example, if the first running data includes the number of SQL statements and CPU utilization, it can be optionally determined whether the number of SQL statements is greater than or equal to a first SQL number threshold and whether the CPU utilization is greater than or equal to a first CPU utilization threshold; if so, the method for generating SQL rate limiting rules is executed. Alternatively, it can be optionally determined whether the number of SQL statements is greater than or equal to a second SQL number threshold; if so, the method for generating SQL rate limiting rules is executed, wherein the second SQL number threshold may be greater than the first SQL number threshold. Alternatively, it can be optionally determined whether the CPU utilization is greater than or equal to a second CPU utilization threshold; if so, the method for generating SQL rate limiting rules is executed, wherein the second CPU utilization threshold may be greater than the first CPU utilization threshold. As an example, the first CPU utilization may be 95%, and the second CPU utilization may be 99%.

[0123] Distributed database systems typically contain multiple data storage nodes. In practical applications, considering that the data distribution and query request distribution of one or more data storage nodes may be uneven, in order to promptly detect potentially high-risk SQL on data storage nodes, the aforementioned detection process can be performed on one or more data storage nodes in the distributed database system to identify data storage nodes that meet preset trigger conditions. Then, the method for generating SQL rate limiting rules can be executed on the data storage nodes that meet the preset trigger conditions.

[0124] Furthermore, for each data storage node, periodic checks can be performed according to a preset detection cycle, such as every few seconds, every minute, or every few minutes. This allows for the timely detection of high-risk SQL statements that may pose problems while running on the data storage node.

[0125] In one or more embodiments of this specification, considering that there may be temporary high-concurrency queries of normal SQL in the actual use of the database system, the triggering conditions can be further optimized in order to avoid misidentifying normal temporary high-concurrency situations as requiring the execution of the SQL generation rate limiting rule method.

[0126] Specifically, the step of obtaining the SQL statements in the execution state on the first data storage node if the first running data meets the preset triggering condition includes: if the second running data meets the preset triggering condition after a preset interval, then obtaining the SQL statements in the execution state on the first data storage node; the second running data includes at least one of the updated number of SQL statements in the execution state on the first data storage node after the preset interval and the updated resource usage data.

[0127] Furthermore, if the first running data meets the preset triggering condition, an abnormal alarm message is sent; if the second running data meets the preset triggering condition after a preset interval following the sending of the abnormal alarm message, the SQL statement in the execution state on the first data storage node is obtained.

[0128] The abnormal alarm information can be alarm log information, specifically including alarm item, alarm name, alarm time, alarm severity, alarm label information, etc., but not limited to these. The alarm label information can specifically include information reflecting the content addressed by the alarm, such as abnormal statement information, abnormal indicator information, abnormal time information, etc., but not limited to these. In practical applications, the content of the abnormal alarm information can be set according to task requirements.

[0129] Furthermore, the abnormal alarm information can be sent to the instant messaging page of the database management and maintenance personnel, so that the management and maintenance personnel can promptly discover the problem and take corresponding maintenance measures to maintain the stability of the database system.

[0130] The preset interval duration can be determined based on the actual task situation. For example, if the typical duration of temporary high-concurrency queries is determined based on historical operational data, the preset interval duration can be set to be slightly longer than the duration. As an example, in practice, the preset interval duration can be set to 1 minute or several minutes.

[0131] Based on at least some embodiments of this specification, data storage nodes in a distributed database system can be inspected. The operational data of these nodes is used to determine whether they meet preset triggering conditions. If they do, the meeting of the triggering conditions indicates an abnormal operational state of the data storage node, and the method for generating SQL rate limiting rules and rate limiting SQL statements, as provided in the embodiments of this specification, is automatically executed. Therefore, the solution based on the embodiments of this specification, by inspecting data storage nodes, can automatically identify data storage nodes with poor operational status, automatically triggering statistical analysis of SQL statements in execution on these nodes. This analysis identifies high-concurrency, high-risk SQL statements, and automatically generates SQL rate limiting rules based on the analysis results. This eliminates the need for database management and maintenance personnel to manually identify the timing and parameters for setting SQL rate limiting rules in practical applications. It breaks away from the traditional method of manually setting rate limiting rules, improving the timeliness and efficiency of rule setting. It enables timely rate limiting of identified high-risk SQL statements, which is beneficial for maintaining the long-term stable operation of the distributed database system and improving the database user experience.

[0132] In one or more embodiments of this specification, a self-healing mechanism is further provided. Specifically, after the computing node issues SQL rate limiting rules and the data storage node executes SQL rate limiting rules, the issued and executed SQL rate limiting rules can be canceled or deleted under certain conditions.

[0133] In an optional embodiment, after sending the SQL rate limiting rule to the second data storage node of the distributed database, the method further includes: obtaining the execution time of several SQL statements that conform to the SQL rate limiting rule executed on the second data storage node after the SQL statements are rate-limited according to the SQL rate limiting rule; if the execution time of the several SQL statements is less than a preset time threshold, then the SQL rate limiting rule is canceled.

[0134] The duration threshold is used to reflect the typical execution time of non-abnormal SQL statements (normal SQL statements). In practical applications, the duration threshold can be a characteristic value obtained from statistical analysis of historical SQL execution data on the database system, or it can be a value set based on expert experience. In practice, the setting of the duration threshold may vary due to factors such as the hardware parameters, software parameters, and task parameters of the database system. As an example, in some databases, the execution time of query-type SQL statements typically does not exceed 0.3 seconds; therefore, the duration threshold can be set to 0.3 seconds.

[0135] Furthermore, for SQL statements that meet the characteristics specified in the SQL rate limiting rules but do not exceed the SQL rate limiting limit and are currently in execution, their execution status can be continuously monitored. For example, the execution time of SQL statements that are currently in execution and do not exceed the SQL rate limiting limit can be continuously monitored. If the execution time of these SQL statements does not exceed a preset time threshold, the SQL rate limiting rule can be canceled.

[0136] Furthermore, to improve the accuracy of canceling SQL rate limiting rules, a preset number of SQL statements that meet the characteristic information specified in the SQL rate limiting rules and are in the execution state after SQL rate limiting is implemented can be detected. If the execution time or characteristic execution time (e.g., average execution time, maximum execution time, etc.) of the detected SQL statements does not exceed the time threshold, then the SQL rate limiting rule is canceled.

[0137] In practical applications, the preset number of SQL statements participating in the detection can be the first few SQL statements after SQL rate limiting is implemented; or, it can be a number of SQL statements after a period of time has elapsed since SQL rate limiting was implemented. In practice, this period of time can be the time during which the overall operating status of the database system improves due to the implementation of the rate limiting rules. This period of time can be determined based on historical statistical data or expert experience, and is not specifically limited thereto.

[0138] In addition, in practical applications, the preset number of SQL statements participating in the detection can be a number of continuously executed SQL statements; or, it can be a portion of SQL statements extracted from continuously executed SQL statements according to certain extraction rules. The specific extraction method can be determined according to task requirements or expert experience.

[0139] Furthermore, the plurality of SQL statements may specifically include SQL statements that conform to the SQL rate limiting rules and are executed N consecutively on the second data storage node after the SQL statements are rate-limited according to the SQL rate limiting rules, where N is an integer greater than 1.

[0140] Based on the method described in this specification, considering that after rate limiting is applied to high-risk SQL statements with high concurrency, the running state of the second data storage node will be restored, the SQL statements executed on the second data storage node at this time can reflect the actual execution time of each SQL statement (actual execution time can refer to the execution time when computer resources are sufficient). Therefore, it is possible to determine whether the rate-limited high-risk SQL statement is a problematic SQL statement by judging whether the actual execution time of the rate-limited SQL statement is less than or equal to a preset time threshold. For example, if the actual execution time of the rate-limited SQL statement is less than or equal to the preset time threshold, it may indicate that the rate-limited SQL statement is not a problematic SQL statement, and the corresponding rate limiting rule can be canceled. This allows for the timely detection of incorrectly rate-limited SQL statements and the timely restoration of the execution of such normal SQL statements, avoiding negative user experience. Conversely, if the actual execution time of the rate-limited SQL statement is greater than the preset time threshold, it may indicate that the rate-limited SQL statement is a problematic SQL statement, and it is not necessary to cancel the corresponding SQL rate limiting rule.

[0141] In an optional embodiment, after sending the SQL rate limiting rule to one or more data storage nodes in the distributed database so that the one or more data storage nodes execute SQL rate limiting according to the SQL rate limiting rule, the method further includes: canceling the SQL rate limiting rule after a preset time has elapsed since the SQL rate limiting rule was sent.

[0142] In practical applications, considering that when a compute node generates and distributes SQL rate-limiting rules to a second data storage node, and the second data storage node executes rate-limiting according to these rules, if a client's data processing request hits the running SQL rate-limiting rule, the client will receive various forms of alerts related to rate-limiting, such as request timeout or error. Subsequently, the problematic SQL can be corrected by the database user rewriting the request statement. In this case, it is unnecessary to continue executing the SQL rate-limiting rule. Therefore, a certain lifespan can be set for the generated SQL rate-limiting rules, after which they can be automatically canceled. Based on this scheme, the cancellation of rate-limiting rules can be easily performed without complex calculations and judgments, offering universality and saving computing resources.

[0143] Based on the embodiments of this specification, a rapid self-healing solution can be implemented by detecting the execution duration of SQL statements in the execution state to promptly identify SQL statements that are incorrectly restricted and cancel the corresponding SQL rate limiting rules; alternatively, a slow self-healing solution can be implemented by automatically canceling SQL rate limiting rules after a preset duration; or a combination of both can be used. Overall, by setting a self-healing mechanism that automatically cancels generated and executed SQL rate limiting rules, the fine-grained management of SQL rate limiting is improved, avoiding resource waste; by combining the self-healing mechanism with detection and alarm mechanisms, automatic detection and removal mechanisms, etc., fully automated operation and maintenance management of distributed database systems is achieved, realizing a solution for stable operation and maintenance of database systems in a low-manpower, low-cost, high-efficiency, and high-accuracy manner.

[0144] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they have not been described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0145] Based on the above description, the schematic diagram of the scheme framework for detecting and eliminating abnormal SQL in a practical application scenario provided in the embodiments of this specification is as follows: Figure 3 As shown.

[0146] like Figure 3 As shown, firstly, users can set SQL tagging rules that include specified fields through the user management client.

[0147] As an example, such as Figure 3 As shown, administrators can use the `CREATE CCL_DETECT` command to create a detection task named 'name', for example, by executing the statement `CREATE CCL_DETECT 'name'`. Furthermore, within this detection task named 'name', a database (db_name) and a table (table_name) can be specified, and the root column (ROOT_COLUMN) can be set to "column_name", for example, by executing the statement `CREATE CCL_DETECT 'name' ON 'db_name'.'table_name' WITH ROOT_COLUMN= ''column_name''`. Here, 'name', 'db_name', 'table_name', and ''column_name' can all be set according to actual needs. For example, column_name can be userId.

[0148] Therefore, it is possible to identify and mark SQL statements that contain filtering conditions targeting the column represented by 'column_name' in the table represented by 'table_name' in the database represented by 'db_name'.

[0149] The aforementioned information, set through the user management client, can be sent to the computing nodes of the distributed database system.

[0150] Second, the compute nodes generate SQL statements for distribution to the data storage nodes based on the original SQL statements, and can add HINT comments to the SQL statements based on the previously set SQL marking rules before distributing them to the data storage nodes.

[0151] In practical applications, when a user initiates a data processing request, such as an SQL query request, the compute node receives the SQL query request, generates a hint annotation for the original SQL statement that meets the conditions, and sends the hint annotation and the SQL statement generated based on the original SQL statement to the data storage node at the same time.

[0152] As an example, such as Figure 3 As shown, assume the original SQL statement is SELECT * from tb where userId =?, and assume that based on this original SQL statement, the following SQL statements can be generated: SELECT * from tb_1 where userId = ? for sending to data storage node 1, SELECT * from tb_2 where userId = ? for sending to data storage node 2, and SELECT * from tb_3 where userId = ? for sending to data storage node 3.

[0153] For this example, based on the solution of the embodiments in this specification, a hint annotation / *CCL;TemplateId;ColumnName / * / can also be generated for simultaneous distribution to the data storage node along with the first, second, and third SQL statements. Here, CCL indicates that the original SQL statement corresponding to the first / second / third SQL statement was obtained based on a database user's request. In practical applications, SQL statements marked with CCL are scanned and analyzed by the rule manager deployed on the database compute node. TemplateId identifies the statement structure of the original SQL statement corresponding to the first / second / third SQL statement; more specifically, it identifies the part of the original SQL statement excluding the values ​​of the condition fields. ColumnName indicates that the condition fields of the original SQL statement corresponding to the first / second / third SQL statement contain a specified field set in the previously created SQL tagging rule.

[0154] Third, during the execution of SQL statements on the data storage nodes, the rule manager can monitor the running status of each data storage node in the background.

[0155] In practical applications, the rule manager can be a functional module deployed in the compute node.

[0156] For each data storage node, anomaly detection can be performed first. Specifically, the rule manager can obtain the operational data of the data storage node, such as at least one of the following: the number of SQL statements in execution state on the data storage node and resource usage data (e.g., CPU utilization), and determine whether the data storage node is operating abnormally based on the operational data.

[0157] Then, if one or more data storage nodes malfunction, log alerts can be generated. For example, log alert information can be pushed to the instant messaging page of database management and maintenance personnel to remind them to intervene manually.

[0158] Furthermore, anomaly detection can be performed again after a preset interval (e.g., 3 minutes). If two or more consecutive anomaly detection results are obtained, the subsequent automatic removal process will be executed. By triggering automatic removal based on more than one anomaly detection result, false triggers can be effectively avoided, reducing the waste of computing resources caused by false removals and lowering the false removal rate.

[0159] Fourth, the rule manager can generate SQL rate limiting rules and distribute them to the data storage nodes when the running status of the data storage nodes meets preset conditions.

[0160] First, for the data storage node with continuous operational anomalies detected earlier, following the previous example, the rule manager can extract the SQL statements carrying CCL tags from that data storage node and group the SQL statements according to their characteristic information. Furthermore, the data combination [template identifier - value of specified field] can be used as characteristic information to group the extracted SQL statements.

[0161] Then, for the multiple SQL statement groups obtained, a preset number of SQL statement groups containing more SQL statements than a preset threshold can be determined as the target SQL statement groups based on the number of SQL statements contained in each SQL statement group.

[0162] Furthermore, SQL rate limiting rules can be generated based on the characteristic information of the SQL statements contained in the selected target SQL statement group.

[0163] Specifically, if the feature information consisting of the data combination of [template identifier - value of specified field] is denoted as key, and different feature information is denoted as key1, key2, etc., then different target SQL statement groups correspond to different keys.

[0164] As an example, you can generate SQL rate limiting rules by executing the statement `Call dbms_ccl.add_ccl_rule('SELECT',20,'key1')`. In this statement, `call` indicates calling the stored procedure, `dbms_ccl` represents the package name of the stored procedure, and `add_ccl_rule` represents the stored procedure name. In practice, `dbms_ccl` and `add_ccl_rule` can be set according to the actual situation. `'SELECT'`, `20`, and `'key1'` are rate limiting parameters. The parameter `'key1'` indicates that rate limiting will be applied to SQL statements that match the characteristics represented by `'key1'`; the parameter `'SELECT'` indicates that rate limiting will be applied to SQL statements of type `SELECT`; and the parameter `20` indicates that when the number of SQL statements that meet the other parameter conditions exceeds 20, traffic will be limited; for example, subsequent SQL statements will need to be queued for execution or return an error.

[0165] It should be noted that the purpose of setting the parameter 'SELECT' when generating SQL rate limiting rules is as follows: In practical applications, the SQL type that causes problems such as slow database or crashes is usually SELECT. By restricting the SQL type to SELECT when setting SQL rate limiting rules, the amount of data processed when executing SQL rate limiting rules can be effectively reduced, the execution efficiency of rate limiting rules can be improved, and the consumption of computer resources caused by executing SQL rate limiting rules can be reduced.

[0166] Fifth, the rule manager implements a self-healing mechanism to automatically cancel SQL rate limiting rules.

[0167] Optionally, a rapid self-healing mechanism can be implemented. Specifically, if an SQL statement that meets certain characteristics is rate-limited, the execution time of the corresponding SQL statement after rate limiting can be detected. If the execution time after rate limiting meets preset conditions, the previously set SQL rate limiting rules can be canceled.

[0168] Optionally, a slow self-healing mechanism can be implemented. Specifically, the SQL rate limiting rule can be canceled after a preset duration has elapsed.

[0169] For example, corresponding to the example above, you can cancel the SQL rate limiting rule generated by executing the statement Call dbms_ccl.add_ccl_rule('SELECT',20,'key1') by executing the statement Call dbms_ccl.del_ccl_rule('key1').

[0170] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods.

[0171] Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of a device for rate limiting SQL statements.

[0172] like Figure 4 As shown, this device is applied to the computing nodes of a distributed database and may include:

[0173] SQL statement retrieval module 402 is used to retrieve SQL statements that are in execution state on the first data storage node of the distributed database;

[0174] The target SQL statement group determination module 404 is used to determine a target SQL statement group containing more than a preset number threshold based on the values ​​of specified fields in the SQL statements that are in the execution state; the values ​​of the specified fields in the SQL statements contained in the target SQL statement group are the same.

[0175] The SQL rate limiting rule generation module 406 is used to generate SQL rate limiting rules based on the field values ​​of the specified fields in the SQL statements contained in the target SQL statement group; the SQL rate limiting rules are used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose values ​​are the specified fields.

[0176] The SQL rate limiting rule distribution module 408 is used to send the SQL rate limiting rule to the second data storage node of the distributed database, so that the second data storage node can limit the rate of SQL statements according to the SQL rate limiting rule.

[0177] based on Figure 4 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0178] Optionally, the device further includes:

[0179] The SQL comment information generation module is used to generate SQL comment information containing the specified field if the condition field of the original SQL statement requested by the client contains the specified field.

[0180] The SQL statement delivery module is used to deliver the SQL comment information to the first data storage node by carrying the SQL statement generated based on the original SQL statement.

[0181] Accordingly, the target SQL statement group determination module 404 is specifically used to: determine a target SQL statement group containing more than a preset number of SQL statements based on the specified fields contained in the SQL comment information carried by the SQL statement in the execution state, according to the values ​​of the specified fields in the SQL statement in the execution state.

[0182] Optionally, the device further includes a comment indication information acquisition module, configured to: acquire pre-set comment indication information; the comment indication information indicates that SQL comment information needs to be generated for the original SQL statement containing the specified field. Accordingly, the SQL comment information generation module is specifically configured to: based on the specified field targeted by the comment indication information, if the condition field of the original SQL statement requested by the client contains the specified field, then generate SQL comment information containing the specified field.

[0183] Optionally, the apparatus further includes a template identifier generation module, used to generate a template identifier based on the original SQL statement executed by the client request; the template identifier is used to identify the structure of the remaining SQL statement in the original SQL statement excluding the values ​​of the condition fields;

[0184] The SQL comment information generation module is also used to generate SQL comment information containing the template identifier and the specified field;

[0185] The target SQL statement group determination module 404 is specifically used to determine a target SQL statement group containing more than a preset number threshold based on the template identifier of the SQL statement in the execution state and the value of the specified field; the target SQL statement group contains SQL statements with the same template identifier and the same value of the specified field.

[0186] Optionally, the SQL comment information generation module is specifically used to generate SQL comment information containing a first SQL identifier and the specified field; the first SQL identifier is used to indicate that the original SQL statement was obtained based on a client request;

[0187] The SQL statement acquisition module 402 is specifically used to acquire, based on the SQL comment information carried by the SQL statement in the execution state, an SQL statement that contains the first SQL identifier and is in the execution state on the first data storage node of the distributed database.

[0188] Optionally, the device further includes a running data acquisition module, used to acquire first running data of the first data storage node; the first running data includes at least one of the number of SQL statements in execution state and resource usage data;

[0189] The SQL statement acquisition module 402 is specifically used to acquire SQL statements that are in execution state on the first data storage node if the first running data meets a preset triggering condition; the preset triggering condition includes that the number of SQL statements is greater than or equal to a first SQL number threshold; or that the resource utilization rate is greater than or equal to a first resource utilization rate threshold; or that the number of SQL statements is greater than or equal to a second SQL number threshold and the resource utilization rate is greater than or equal to a second resource utilization rate threshold.

[0190] Optionally, the SQL statement acquisition module 402 is specifically used to acquire the SQL statements in the execution state on the first data storage node if the second running data meets the preset triggering condition after the first running data meets the preset triggering condition for a preset interval; the second running data includes at least one of the updated number of SQL statements in the execution state on the first data storage node after the preset interval and the updated resource usage data.

[0191] Optionally, the device is further configured to: send an abnormal alarm message if the first operating data meets the preset triggering condition.

[0192] Optionally, the second data storage node includes the first data storage node.

[0193] Optionally, the target SQL statement group determination module 404 is specifically used to: divide the SQL statements in the execution state into SQL statement groups according to the values ​​of specified fields in the SQL statements in the execution state; the values ​​of the specified fields in the SQL statements in the same SQL statement group are the same; and determine the target SQL statement group from the SQL statement group that contains more than a preset number threshold of SQL statements.

[0194] Optionally, the device further includes a first self-healing module, configured to obtain the execution time of several SQL statements that conform to the SQL rate limiting rules executed on the second data storage node after the second data storage node rate limits the SQL statements according to the SQL rate limiting rules; if the execution time of the several SQL statements is less than a preset time threshold, then the SQL rate limiting rules are cancelled.

[0195] Optionally, the plurality of SQL statements specifically includes SQL statements that conform to the SQL rate limiting rules and are executed N consecutively on the second data storage node after the SQL statements are rate-limited according to the SQL rate limiting rules, where N is an integer greater than 1.

[0196] Optionally, the device further includes a second self-healing module, used to cancel the SQL rate limiting rule after the SQL rate limiting rule has been sent for a preset time.

[0197] The above is an illustrative scheme of an apparatus for rate limiting SQL statements according to this embodiment. It should be noted that the technical solution of this apparatus for rate limiting SQL statements belongs to the same concept as the technical solution of the method for rate limiting SQL statements described above. For details not described in detail in the technical solution of the apparatus for rate limiting SQL statements, please refer to the description of the technical solution of the method for rate limiting SQL statements described above.

[0198] Based on the same idea, embodiments of this specification also provide a distributed database corresponding to the above-described methods and apparatus.

[0199] Specifically, the distributed database provided in the embodiments of this specification includes computing nodes and data storage nodes;

[0200] The computing node is configured to: acquire SQL statements in execution state on a first data storage node of the distributed database; determine a target SQL statement group containing more SQL statements than a preset threshold based on the values ​​of specified fields in the SQL statements in execution state, wherein the specified fields in the SQL statements in the target SQL statement group have the same value; generate an SQL rate limiting rule based on the field values ​​of the specified fields in the SQL statements in the target SQL statement group, wherein the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions executed on the distributed database, wherein the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose specified fields are the specified field values; and send the SQL rate limiting rule to a second data storage node of the distributed database so that the second data storage node can rate limit the SQL statements according to the SQL rate limiting rule.

[0201] The first data storage node in the data storage nodes is configured to provide the computing node with SQL statements in an execution state;

[0202] The second data storage node in the data storage node is configured to receive SQL rate limiting rules sent by the computing node; and to rate limit SQL statements according to the SQL rate limiting rules.

[0203] Optionally, the second data storage node includes the first data storage node.

[0204] The above is an illustrative scheme of a distributed database according to this embodiment. It should be noted that the technical solution of this distributed database and the technical solution of the method for rate limiting SQL statements described above belong to the same concept. For details not described in detail in the technical solution of the distributed database, please refer to the description of the technical solution of the method for rate limiting SQL statements described above.

[0205] Figure 5 This specification provides a structural block diagram of a computing device 500 in an embodiment.

[0206] The computing device 500 includes:

[0207] Memory 510 and processor 520;

[0208] The memory 510 is used to store computer programs / instructions, and the processor 520 is used to execute the computer programs / instructions, which, when executed by the processor 520, implement the steps of the method for rate limiting of SQL statements.

[0209] In one or more embodiments of this specification, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, PC (Personal Computer), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0210] Furthermore, the components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 and the memory 510 can be connected via a bus.

[0211] The computing device 500 may also include an access device that enables the computing device 500 to communicate with a database storing data via one or more networks. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0212] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0213] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0214] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the method for rate limiting SQL statements described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the method for rate limiting SQL statements described above.

[0215] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described method for rate limiting of SQL statements.

[0216] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the method for rate limiting SQL statements described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the method for rate limiting SQL statements described above.

[0217] An embodiment of this specification also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the above-described method for rate limiting of SQL statements.

[0218] The above is an illustrative solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above-described method for rate limiting of SQL statements belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described method for rate limiting of SQL statements.

[0219] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0220] The computer program / instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0221] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0222] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0223] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for rate limiting SQL statements, applied to the computing nodes of a distributed database, comprising: Retrieve the SQL statements that are currently in execution on the first data storage node of the distributed database; The SQL statement in the execution state carries SQL comment information; The SQL comment information includes a template identifier and specified fields; the template identifier is used to identify the structure of the original SQL statement that generated the SQL statement in the execution state; Based on the template identifier of the SQL statement in the execution state and the value of the specified field, determine the target SQL statement group containing more than a preset number threshold. The target SQL statement group contains SQL statements with the same template identifier and the same value for the specified field; Based on the field values ​​of the specified fields in the SQL statements contained in the target SQL statement group, an SQL rate limiting rule is generated; the SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that contain the specified fields and whose values ​​are the specified fields. The SQL rate limiting rule is sent to the second data storage node of the distributed database so that the second data storage node can limit the rate of SQL statements according to the SQL rate limiting rule.

2. The method as described in claim 1, further comprising, before obtaining the SQL statement in execution state on the first data storage node of the distributed database: If the condition field of the original SQL statement requested by the client contains the specified field, then SQL comment information containing the specified field is generated; The SQL comment information is carried by the SQL statement generated based on the original SQL statement and sent to the first data storage node.

3. The method of claim 2, further comprising: Retrieve pre-set annotation instructions; The annotation indication information is used to indicate that SQL annotation information needs to be generated for the original SQL statement containing the specified field; If the condition field of the original SQL statement requested by the client contains the specified field, then SQL comment information containing the specified field is generated, specifically including: Based on the specified field targeted by the annotation indication information, if the condition field of the original SQL statement requested by the client contains the specified field, then SQL annotation information containing the specified field is generated.

4. The method of claim 2, further comprising: Generate template identifiers based on the original SQL statements executed in the client request; The template identifier is used to identify the structure of the remaining SQL statement in the original SQL statement, excluding the values ​​of the condition fields; The generation of SQL comment information containing the specified fields specifically includes: Generate SQL comment information containing the template identifier and the specified field.

5. The method as described in claim 2, wherein generating SQL comment information containing the specified field specifically includes: Generate SQL comment information containing the first SQL identifier and the specified field; The first SQL identifier is used to indicate that the original SQL statement was obtained based on a client request; The process of obtaining the SQL statements that are in execution state on the first data storage node of the distributed database specifically includes: Based on the SQL comment information carried by the SQL statement in the execution state, obtain the SQL statement in the SQL comment information carried by the first data storage node of the distributed database that contains the first SQL identifier and is in the execution state.

6. The method as described in claim 1, further comprising, before obtaining the SQL statement in execution state on the first data storage node of the distributed database: Obtain the first running data from the first data storage node; The first running data includes at least one of the following: the number of SQL statements in execution state and resource usage data; The step of obtaining the SQL statements that are in execution state on the first data storage node of the distributed database specifically includes: If the first running data meets the preset triggering conditions, then the SQL statements in the execution state on the first data storage node are obtained; the preset triggering conditions include that the number of SQL statements is greater than or equal to a first SQL number threshold; or that the resource utilization rate is greater than or equal to a first resource utilization rate threshold; or that the number of SQL statements is greater than or equal to a second SQL number threshold and the resource utilization rate is greater than or equal to a second resource utilization rate threshold.

7. The method as described in claim 6, wherein if the first running data satisfies a preset triggering condition, then obtaining the SQL statement in the execution state on the first data storage node specifically includes: If the first running data meets the preset triggering condition after a preset interval, and the second running data meets the preset triggering condition, then the SQL statements in the execution state on the first data storage node are obtained; the second running data includes at least one of the updated number of SQL statements in the execution state on the first data storage node after the preset interval and the updated resource usage data.

8. The method of claim 6, further comprising: If the first running data meets the preset triggering condition, an abnormal alarm message is sent.

9. The method of claim 1, wherein, The second data storage node includes the first data storage node.

10. The method as described in claim 1, wherein determining the target SQL statement group containing more than a preset number of SQL statements based on the value of a specified field in the SQL statement in the execution state specifically includes: Based on the values ​​of specified fields in the SQL statements that are in the execution state, the SQL statements in the execution state are divided into SQL statement groups; The specified fields in the SQL statements of the same SQL statement group have the same value; From the SQL statement group, identify the target SQL statement group whose number of SQL statements exceeds a preset threshold.

11. The method of claim 1, further comprising, after sending the SQL rate limiting rule to the second data storage node of the distributed database: The execution time of several SQL statements that conform to the SQL rate limiting rules and are executed on the second data storage node after the SQL statement rate limiting rules are applied to the second data storage node is obtained. If the execution time of the aforementioned SQL statements is less than the preset time threshold, then the SQL rate limiting rule is cancelled.

12. The method of claim 11, wherein, The aforementioned SQL statements specifically include SQL statements that conform to the SQL rate limiting rules and are executed N consecutively on the second data storage node after the SQL statements are rate-limited according to the SQL rate limiting rules, where N is an integer greater than 1.

13. The method of claim 1, further comprising, after sending the SQL rate limiting rule to the second data storage node of the distributed database: After the SQL rate limiting rule has been sent for a preset period of time, the SQL rate limiting rule is cancelled.

14. A device for rate limiting SQL statements, applied to a computing node of a distributed database, comprising: The SQL statement retrieval module is used to retrieve SQL statements that are in execution state on the first data storage node of the distributed database. The SQL statement in the execution state carries SQL comment information; The SQL comment information includes a template identifier and specified fields; the template identifier is used to identify the structure of the original SQL statement that generated the SQL statement in the execution state; The target SQL statement group determination module is used to determine a target SQL statement group containing more than a preset number threshold based on the template identifier of the SQL statement in the execution state and the value of the specified field. The target SQL statement group contains SQL statements with the same template identifier and the same value for the specified field; The SQL rate limiting rule generation module is used to generate SQL rate limiting rules based on the field values ​​of specified fields in the SQL statements contained in the target SQL statement group; the SQL rate limiting rules are used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database; the SQL statements that meet specific conditions are SQL statements that contain the specified field and whose value is the field value. The SQL rate limiting rule distribution module is used to send the SQL rate limiting rule to the second data storage node of the distributed database, so that the second data storage node can limit the rate of SQL statements according to the SQL rate limiting rule.

15. A distributed database, comprising computing nodes and data storage nodes; The computing node is configured to acquire SQL statements in the execution state from the first data storage node of the distributed database; the SQL statements in the execution state carry SQL comment information. The SQL comment information includes a template identifier and a specified field. The template identifier is used to identify the structure of the original SQL statement that generated the SQL statement in the execution state. Based on the template identifier of the SQL statement in the execution state and the value of the specified field, a target SQL statement group containing more than a preset threshold number of SQL statements is determined. The SQL statements in the target SQL statement group have the same template identifier and the same value of the specified field. Based on the field value of the specified field in the SQL statements in the target SQL statement group, an SQL rate limiting rule is generated. The SQL rate limiting rule is used to limit the number of SQL statements that meet specific conditions and are executed on the distributed database. The SQL statements that meet specific conditions are SQL statements that contain the specified field and whose value is the specified field value. The SQL rate limiting rule is sent to the second data storage node of the distributed database so that the second data storage node can rate limit the SQL statements according to the SQL rate limiting rule. The first data storage node in the data storage nodes is configured to provide the computing node with SQL statements in an execution state; The second data storage node in the data storage node is configured to receive SQL rate limiting rules sent by the computing node; and to rate limit SQL statements according to the SQL rate limiting rules.

16. The distributed database as described in claim 15, wherein, The second data storage node includes the first data storage node.

17. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

18. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

19. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images