Query processing method, electronic device, storage medium and program product

Abstract

A query processing method includes, within a first scheduling period, obtaining service quality parameters of a database cluster and obtaining a first token quantity based on the service quality parameters; wherein the service quality parameters include: service quality parameters within a preceding scheduling period; the preceding scheduling period includes: one or more scheduling periods prior to the first scheduling period; the first token quantity includes: the token quantity corresponding to a subsequent scheduling period; the subsequent scheduling period includes: one or more scheduling periods after the first scheduling period; and within a second scheduling period, in response to receiving a query request, determining a second token quantity corresponding to the second scheduling period based on the first token quantity, and distributing the query request to the database cluster based on the second token quantity; wherein the second scheduling period is the first scheduling period among the subsequent scheduling periods, and the distribution of the query request in the database cluster is completed based on an adaptively adjusted token quantity, ensuring the stable operation of the database cluster system and efficient resource utilization under various load conditions. Correspondingly, electronic devices, computer-readable storage media, and computer program products are also disclosed.

Description

Technical Field

[0001] This relates to the field of information technology, and in particular to a query processing method, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Currently, database cluster systems used for big data analysis face numerous technical challenges when handling high-concurrency query tasks due to the lack of effective concurrency control mechanisms, such as low resource utilization and low query success rate. As a result, their service performance and reliability are difficult to meet the needs of practical applications. Summary of the Invention

[0003] In view of this, a query processing method, an electronic device, a computer-readable storage medium, and a computer program product are provided to solve or partially solve the above problems.

[0004] In some cases, the above query processing method may include: within a first scheduling period, obtaining service quality parameters of the database cluster and obtaining a first token quantity based on the service quality parameters; wherein the service quality parameters include: service quality parameters within a preceding scheduling period; the preceding scheduling period includes: one or more scheduling periods prior to the first scheduling period; the first token quantity includes: the token quantity corresponding to a subsequent scheduling period; the subsequent scheduling period includes: one or more scheduling periods after the first scheduling period; and within a second scheduling period, in response to receiving a query request, determining a second token quantity corresponding to the second scheduling period based on the first token quantity, and distributing the query request to the database cluster based on the second token quantity; wherein the second scheduling period belongs to the first scheduling period among the subsequent scheduling periods.

[0005] In some cases, the query processing apparatus described above may include:

[0006] The data acquisition module is used to acquire the service quality parameters of the database cluster during the first scheduling period; wherein, the service quality parameters may include: service quality parameters during the preceding scheduling period; the preceding scheduling period may include: one or more scheduling periods before the first scheduling period; The scheduling plan module is used to obtain a first token quantity based on service quality parameters; wherein, the first token quantity may include: the token quantity corresponding to the subsequent scheduling period; the subsequent scheduling period may include: one or more scheduling periods after the first scheduling period; The token configuration module is used, within the second scheduling period, in response to a received query request, to determine the number of second tokens corresponding to the second scheduling period based on the number of first tokens; wherein the second scheduling period belongs to the first scheduling period in the subsequent scheduling periods; and The scheduling module is used to distribute query requests to the database cluster based on the number of second tokens.

[0007] In addition, the aforementioned electronic device may include: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned query processing method.

[0008] The aforementioned non-transitory computer-readable storage medium stores computer instructions that are used to cause a computer to execute the aforementioned query processing method.

[0009] The aforementioned computer program product includes computer program instructions that, when executed on a computer, cause the computer to perform the aforementioned query processing method.

[0010] It can be seen that the above query processing method, electronic device, computer-readable storage medium and computer program product can periodically and adaptively adjust the number of tokens corresponding to one or more subsequent scheduling cycles according to the real-time service quality parameters of the database cluster system, and can complete the distribution of query requests in the database cluster based on the adaptively adjusted number of tokens, thereby ensuring the stable operation of the database cluster system and the efficient utilization of resources under various load conditions. Attached Figure Description

[0011] To more clearly illustrate the technical solutions, the accompanying drawings used in the examples or related technical descriptions will be briefly introduced below. Obviously, the accompanying drawings described below are merely examples, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0012] Figure 1 This shows an example of an application scenario for the query processor.

[0013] Figure 2 This shows an example of the query processing method implementation flow.

[0014] Figure 3 This shows an example of a method flow for obtaining the first number of tokens based on quality of service parameters.

[0015] Figure 4 This example demonstrates an implementation of a method to dynamically adjust the number of tokens for one or more scheduling cycles based on success rate trends.

[0016] Figure 5 This shows an example of the internal structure of a query processing unit.

[0017] Figure 6An example of an electronic device hardware structure for implementing a query processing method is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages clearer, the technical solutions will be further explained in detail below with reference to specific examples and accompanying drawings.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used should have the common meaning understood by a person with general skills in the field. The terms "first," "second," and similar words used in the examples do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Words such as "including" or "contains" mean that the element or object preceding the word covers the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; these relative positions may change accordingly when the absolute position of the described object changes.

[0020] It is understandable that before using the technical solutions in each example, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0021] For example, upon receiving a user's proactive request, a prompt message can be sent to the user, explicitly informing them that the requested operation will require the acquisition and use of their personal information. This allows the user to choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the technical solution.

[0022] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0023] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the described implementation method. Other methods that comply with relevant laws and regulations may also be applied to the described implementation method.

[0024] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.

[0025] As mentioned earlier, current database cluster systems, due to the lack of effective concurrency control mechanisms, face numerous technical challenges when handling high-concurrency query tasks, such as low resource utilization and low query success rate, making it difficult for their service performance and reliability to meet the needs of practical applications.

[0026] Figure 1 This shows an example application scenario of a database cluster system 100. For example... Figure 1 As shown, the database cluster system 100 may include: a business client 110, a query proxy layer 120, and a database cluster 130. The business client 110 can be a terminal application or service process that interfaces with the distributed database cluster and initiates legitimate read / write requests from the upper-layer business system; it is the sole business initiator for data access. The business client 110 only follows standard database communication protocols and authorization rules for access, without needing to be aware of the underlying sharding, replicas, node topology, data routing, or storage architecture. The business client 110 can focus on implementing its own business logic without participating in cluster scheduling, metadata management, or data storage. The query proxy layer 120 can be an independent intermediate gateway component deployed between the business client 110 and the database cluster 130 (which typically includes coordination nodes, compute nodes, or storage nodes). The query proxy layer 120 serves as the unified external access point for the database cluster 130, handling all client connections and query requests, implementing traffic aggregation, protocol forwarding, traffic governance, and access control, completely shielding the details of the backend database cluster node topology, sharding, replicas, and scaling up / down, thus decoupling the business side from the underlying database architecture. The aforementioned database cluster 130 can be an organic whole composed of multiple independent database service nodes, relying on unified metadata collaboration, communication protocols, and scheduling mechanisms. The database cluster 130 can provide a single logical database service entry point externally, and internally implement data sharding and distributed storage, multi-replica redundancy and disaster recovery, read / write load balancing, and horizontal resource scaling. It possesses high availability, high performance, high concurrency, and linear scalability capabilities, and is used to support large-capacity, high-throughput, and complex query business scenarios that cannot be supported by a single-machine database.

[0027] Because database system 100 lacks an effective concurrency control mechanism, when faced with a sudden surge in query requests, all query tasks are directly submitted to the database cluster by the query proxy layer 120. This causes the computing resources of database cluster 130, such as the CPU and memory, to be rapidly exhausted, leading to overload of database cluster 130. The direct consequences are a significant decrease in query success rate, a sharp increase in response time, and even the potential for database cluster 130 nodes to crash, severely damaging the overall availability and stability of the service.

[0028] Furthermore, in mixed query load scenarios where query requests can correspond to different priorities, since the current database cluster system 100 cannot provide effective resource guarantees for high-priority query requests, when the database cluster 130 is overloaded, high-priority critical query requests have to compete with a large number of low-priority query requests for limited resources. This results in the execution success rate and response time of high-priority query requests not being effectively guaranteed, which may adversely affect the normal operation of core businesses.

[0029] To address the aforementioned technical issues, a query processing method that can be implemented by the query proxy layer 120 is proposed, which can achieve adaptive concurrency control based on the service quality parameters of the database cluster 130.

[0030] The specific process of the above query processing method will be further explained in detail below with reference to the accompanying drawings and examples. Specifically, Figure 2 This shows an example of the query processing method implementation flow. For example... Figure 2 As shown, in some cases, the above query processing method may specifically include the following steps.

[0031] In step 210, within the first scheduling period, the service quality parameters of the database cluster are obtained.

[0032] In some cases, the aforementioned quality of service parameters may include quality of service parameters within the preceding scheduling period. The preceding scheduling period may include one or more scheduling periods prior to the first scheduling period.

[0033] In step 220, the first number of tokens is obtained based on the quality of service parameters.

[0034] In some cases, the aforementioned first token quantity may include the token quantity corresponding to a subsequent scheduling period. The aforementioned subsequent scheduling period may include one or more scheduling periods following the first scheduling period.

[0035] In step 230, within the second scheduling period, in response to receiving a query request, the number of second tokens corresponding to the second scheduling period is determined based on the number of first tokens.

[0036] In some cases, the aforementioned second scheduling cycle is the first scheduling cycle among the aforementioned subsequent scheduling cycles.

[0037] In step 240, query requests are distributed to the database cluster based on the number of second tokens.

[0038] In some cases, the second scheduling period can typically be the next scheduling period after the first scheduling period. That is, assuming the first scheduling period is the current scheduling period, the method can determine the token quantity for one or more subsequent scheduling periods within the current scheduling period, and distribute query requests based on the token quantity determined in the next scheduling period. Conversely, assuming the second scheduling period is the current scheduling period, the method can determine the token quantity for the current scheduling period within the preceding scheduling period, and distribute query requests based on the token quantity determined in the preceding scheduling period.

[0039] It should be noted that the above query processing method can be executed repeatedly according to the above scheduling period. For example, in some cases, the above scheduling period can be preset to 1 minute. In other words, the above scheduling period can be preset, so that the query proxy layer can execute the above query processing method once in each scheduling period to periodically update the token count for one or more scheduling periods after the first scheduling period.

[0040] As can be seen, the above query processing method can periodically and adaptively adjust the number of tokens in subsequent scheduling cycles based on the real-time service quality parameters of the database cluster system, and can complete the distribution of query requests in the database cluster based on the adaptively adjusted number of tokens. Thus, it can achieve adaptive concurrency control based on the health status of the database cluster system, ensuring the stable operation of the database cluster system and the efficient utilization of resources under various load conditions.

[0041] The specific implementation methods of each step in the above query processing method will be explained in detail below.

[0042] Regarding step 210 above, in some cases, the aforementioned service quality parameters may include: the total number of queries and the number of successful queries in the database cluster; and / or, the query success rate of the database cluster. In some cases, the aforementioned total number of queries, the number of successful queries, and / or the query success rate may be the total number of queries, the number of successful queries, and / or the query success rate with the aforementioned scheduling period as the time granularity. For example, assuming the aforementioned scheduling period is 1 minute, then the aforementioned total number of queries, the number of successful queries, and / or the query success rate may refer to the total number of queries per minute, the number of successful queries per minute, and / or the query success rate per minute of the database cluster, respectively. It should be noted that the aforementioned 1 minute is only a specific example, and in applications, the aforementioned scheduling period can be flexibly set according to actual needs or experience, and the specific value of the aforementioned scheduling period is not limited here.

[0043] As mentioned earlier, the query proxy layer can execute the query processing method once per scheduling cycle. Based on this, in some cases, the query proxy layer can extract the service quality parameters of the database cluster from the system tables or management system of the database cluster within the previous scheduling cycle, and cache the extracted service quality parameters in a buffer for use in subsequent operations. For example, the query proxy layer can extract the service quality parameters of the database cluster from the system tables or management system of the database cluster within the previous minute every minute, and cache the extracted service quality parameters in a buffer according to the timestamp of the service quality parameters. Thus, during operation, the query proxy layer can obtain service quality parameters within any time interval before the current scheduling cycle. Furthermore, in some cases, it is not necessary to cache all historical service quality parameters, but only the service quality parameters of a predetermined number of scheduling cycles before the current scheduling cycle. This method of storing data for a predetermined number of scheduling cycles does not affect the execution of the above method and also saves storage resources.

[0044] Regarding step 220 above, the method for obtaining the first token quantity based on service quality parameters involves the production process of the scheduling plan, which can be specifically described as follows: Figure 3 As shown, it includes the following steps. Furthermore, it can be understood that step 220 is also executed within the aforementioned first scheduling cycle.

[0045] In step 310, the query success rate metric is obtained based on the service quality parameters.

[0046] In step 320, the first number of tokens is obtained based on the pre-established mapping relationship.

[0047] In some cases, the above mapping relationship can include a positive correlation between the query success rate metric and the number of first tokens. This positive correlation between the number of first tokens and the query success rate metric, which is based on quality of service parameters, allows the database cluster system's real-time quality of service parameters to periodically and adaptively adjust the number of tokens in subsequent scheduling cycles. This enables better adaptive concurrency control based on the health status of the database cluster system.

[0048] The following will provide a detailed explanation of the specific implementation methods of steps 310 and 320 above, using concrete examples.

[0049] Regarding step 310 above, as mentioned earlier, in some cases, the aforementioned service quality parameters may include: the total number of queries and the number of successful queries in the database cluster; and / or, the query success rate of the database cluster, etc. In this case, the specific method for obtaining the query success rate indicator based on the service quality parameters described in step 310 above may include the following multiple steps.

[0050] First, based on the aforementioned service quality parameters, obtain the query success rate parameters within the aforementioned pre-scheduling period.

[0051] Specifically, assuming the aforementioned quality of service parameters include the total number of queries and the number of successful queries for the database cluster, the query proxy layer can extract the total number of queries and the number of successful queries for one or more scheduling periods from the buffer, and then calculate the quotient of the number of successful queries and the total number of queries for each scheduling period to obtain the query success rate parameter for one or more preceding scheduling periods. Alternatively, assuming the aforementioned quality of service parameters include the query success rate of the database cluster, the query proxy layer can directly extract the query success rate for one or more scheduling periods from the buffer as the query success rate parameter for one or more preceding scheduling periods.

[0052] Then, based on the pre-set filtering conditions, one or more valid query success rate parameters are further extracted from the obtained query success rate parameters.

[0053] In some cases, the above filtering conditions may include: the total number of queries must be greater than a pre-set total threshold. In this case, query success rate parameters within scheduling periods where the total number of queries exceeds the pre-set total threshold will be filtered as valid query success rate parameters and thus retained; while query success rate parameters within scheduling periods where the total number of queries is lower than the aforementioned total threshold will be identified as invalid query success rate parameters and will be filtered out or corrected to a pre-set default value. It can be understood that filtering the validity of the obtained query success rate parameters can effectively avoid query success rate deviations caused by insufficient query volume, thereby avoiding the adverse effects of query success rate deviations on concurrency control.

[0054] Finally, the query success rate metric is obtained based on the one or more valid query success rate parameters.

[0055] In some cases, the query success rate parameter closest to the first scheduling period among the aforementioned one or more effective query success rates can be used as the query success rate metric. In other cases, a weighted average of the aforementioned one or more effective query success rate parameters can be used as the query success rate metric. Specifically, when using the weighted average method, the weight coefficients corresponding to each effective query success rate can be uneven, and the weight coefficients for parameters closer to the first scheduling period can be set larger. It can be seen that regardless of the method used, the obtained query success rate metric can accurately reflect the real-time query success rate of the database cluster system, that is, the real-time service quality.

[0056] Regarding step 320 above, as mentioned earlier, the mapping relationship may include a positive correlation between the query success rate metric and the number of first tokens.

[0057] Specifically, in some cases, the above mapping relationship may include a pre-defined positively correlated functional relationship, such as y=f(x). Here, the query success rate index is the independent variable x of the functional relationship, and the first token quantity can be the dependent variable y of the functional relationship. Generally, since the functional relationship is positively correlated, a higher query success rate index usually results in a larger first token quantity determined based on the functional relationship. Furthermore, as mentioned earlier, the first token quantity may include the token quantity corresponding to subsequent scheduling periods. And as mentioned earlier, the subsequent scheduling periods may include one or more scheduling periods after the first scheduling period. Therefore, the token quantity corresponding to the subsequent scheduling periods may include: one or more token quantities; each token quantity corresponds to one scheduling period. In this case, the one or more token quantities may be the same, or they may be dynamically adjusted according to the success rate trend. Specifically, the scheme for dynamically adjusting the one or more token quantities corresponding to the subsequent scheduling periods according to the success rate trend will be described in detail later and will be omitted here.

[0058] In other cases, the above mapping relationship can also include a pre-defined positively correlated segmented mapping relationship. For example, when the query success rate indicator is greater than or equal to a first success rate threshold, the token quantity can be set to a first quantity; when the query success rate indicator is less than the first success rate threshold but greater than or equal to a second success rate threshold, the token quantity can be set to a second quantity; and when the query success rate indicator is less than the second success rate threshold, the token quantity can be set to a third quantity. Wherein, the first success rate threshold is greater than the second success rate threshold; the first quantity is greater than the second quantity; and the second quantity is greater than the third quantity. As mentioned earlier, the first token quantity can include one or more token quantities corresponding to subsequent scheduling cycles. In this case, the one or more token quantities can be the same, or they can be dynamically adjusted according to the success rate trend. The scheme for dynamically adjusting one or more token quantities corresponding to subsequent scheduling cycles according to the success rate trend will be described in detail later and will be omitted here.

[0059] Specifically, in mixed query load scenarios where query requests can correspond to different priorities, in some cases, the first token quantity can also include the token quantity corresponding to each different priority. That is, the first token quantity not only includes one or more token quantities corresponding to different scheduling periods, but also, within each scheduling period, further includes token quantities corresponding to different priorities. In this case, different mapping relationships between query success rate metrics and the first token quantity can be pre-set for different priorities. These mapping relationships should generally be positively correlated. Thus, during the scheduling plan generation process, the first token quantity corresponding to each priority can be independently obtained based on the aforementioned mapping relationships between different query success rate metrics and the first token quantity. Moreover, it can be understood that, to prioritize the service quality of high-priority query requests, the first token quantity corresponding to high-priority requests should generally be greater than the first token quantity corresponding to low-priority requests. For example, suppose a query request includes three priorities P0, P1, and P2, where P0 > P1 > P2. Different mapping relationships can be pre-set for these three priorities, and these mapping relationships can include the aforementioned functional relationship or the aforementioned segmented mapping relationship. In a specific example, taking a segmented mapping relationship, the segmented mapping relationship can be set as follows: when the query success rate is greater than or equal to 95%, the token quantity corresponding to priority P0 is 100, the token quantity corresponding to priority P1 is 50, and the token quantity corresponding to priority P2 is 30. When the query success rate is less than 95% but greater than or equal to 80%, the token quantity corresponding to priority P0 is 50, the token quantity corresponding to priority P1 is 20, and the token quantity corresponding to priority P2 is 10. When the query success rate is less than 80%, the token quantity corresponding to priority P0 is 10, the token quantity corresponding to priority P1 is 5, and the token quantity corresponding to priority P2 is 2. Based on the above segmented mapping relationship, after determining the query success rate, the first token quantity corresponding to different priorities can be determined. For example, if the determined query success rate is 85%, the determined first token quantity can include: in each scheduling period after the first scheduling period, the token quantity corresponding to priority P0 is 50, the token quantity corresponding to priority P1 is 20, and the token quantity corresponding to priority P2 is 10. As mentioned earlier, the number of tokens corresponding to each of the above scheduling cycles can be the same, or it can be dynamically adjusted according to the success rate trend.

[0060] The following section details the scheme for dynamically adjusting the number of one or more tokens based on success rate trends. For mixed query load scenarios where query requests can correspond to different priorities, the above scheme applies to each priority level; that is, it can be executed separately for each priority level. Figure 4This shows an example of the implementation process for dynamically adjusting the number of one or more tokens based on success rate trends. For example... Figure 4 As shown, the above method may include: In step 410, the success rate trend is identified based on the above service quality parameters.

[0061] In some cases, the aforementioned success rate trends may include: a first trend, a second trend, and a third trend.

[0062] In step 420, the adjustment coefficient is obtained based on the success rate trend.

[0063] In some cases, the adjustment coefficient corresponding to the first trend can be greater than 1; the adjustment coefficient corresponding to the second trend can be less than 1; and the adjustment coefficient corresponding to the third trend can be equal to 1.

[0064] In step 430, the number of the first tokens is updated based on the adjustment coefficient.

[0065] In some cases, when the success rate trend is the first trend, the number of the first tokens can increase over time, that is, the number of tokens corresponding to each scheduling period after the first scheduling period can increase over time; when the success rate trend is the second trend, the number of the first tokens can decrease over time, that is, the number of tokens corresponding to each scheduling period after the first scheduling period can decrease over time; when the success rate trend is the third trend, the number of the first tokens can remain unchanged, that is, the number of tokens corresponding to each scheduling period after the first scheduling period can remain unchanged over time.

[0066] In this way, success rate trends are identified based on service quality parameters, and adjustment coefficients are obtained for different success rate trends. The number of tokens is then adjusted so that the number of tokens varies with different success rate trends, thereby achieving better adaptive concurrency control.

[0067] The following section will provide a detailed explanation of the specific implementation methods for steps 410-430 above, using concrete examples.

[0068] In some cases, in step 410 above, the effective query success rate within multiple scheduling periods prior to the first scheduling period can be obtained based on the service quality parameters. The specific method for step 410 can be found in the implementation process described in step 310 of the aforementioned embodiments, and will not be repeated here. Then, the difference between the effective query success rate in the last scheduling period and the effective query success rate in the earliest scheduling period is calculated as the success rate trend indicator. Further, the success rate trend indicator is compared with a pre-set upper and / or lower threshold, where the upper threshold is greater than the lower threshold. Typically, the upper threshold is greater than 0, while the lower threshold is less than 0. If the success rate trend indicator is greater than or equal to the upper threshold, the success rate trend can be identified as a first trend, or a recovery trend; if the success rate trend indicator is less than or equal to the lower threshold, the success rate trend can be identified as a second trend, or a declining trend; and if the success rate trend indicator is between the upper and lower thresholds, the success rate trend can be identified as a third trend, or a stable trend.

[0069] Regarding step 420 above, in some cases, different adjustment coefficients can be pre-set for different success rate trends. As mentioned earlier, the adjustment coefficient corresponding to the first trend (recovery trend) can be greater than 1, for example, it can be set to 1.2; the adjustment coefficient corresponding to the second trend (decline trend) can be less than 1, for example, it can be set to 0.8; and the adjustment coefficient corresponding to the third trend (stable trend) can be equal to 1. In this way, the number of tokens can be dynamically adjusted over time according to the trend of the success rate.

[0070] Regarding step 430 above, in some cases, the first token quantity determined above can be used as the baseline token quantity, and the token quantity T corresponding to one or more scheduling cycles can be dynamically adjusted to the baseline token quantity and the adjustment coefficient, respectively. i The product of powers, where i This can be the sequence number corresponding to one or more of the aforementioned scheduling periods, that is, which scheduling period the currently calculated token count corresponds to. Specifically, the following expression can be used to represent the sequence number of the aforementioned one or more scheduling periods. i Number of tokens per scheduling cycle : Where, represents the number of baseline tokens; This represents a fixed adjustment coefficient. It can be seen that, using the above method, when the success rate trend is the first trend, the token quantity can increase over time; when the success rate trend is the second trend, the token quantity can decrease over time; and when the success rate trend is the third trend, the token quantity remains unchanged. In a specific example, assuming the first token quantity can include the token quantity corresponding to 5 scheduling cycles, and the baseline token quantity is 50, with an adjustment coefficient of 1.2, then after dynamic adjustment, the token quantity corresponding to the above 5 scheduling cycles can be 60 (…). ), 72 ( ), 86 ( ), 103 ) and 124 ( ).

[0071] In other cases, an upper limit on the number of tokens can be set based on the processing capacity of the database cluster. If the number of tokens determined through dynamic adjustment already exceeds the upper limit, then dynamic adjustment can be stopped, and the upper limit can be directly set. For example, in the previous example, assuming the upper limit of the number of tokens is 100, and after dynamic adjustment, the number of tokens for the last two scheduling cycles is greater than 100, then, based on the previous example, the number of tokens for the above 5 scheduling cycles can be set to 60 respectively. ), 72 ( ), 86 ( ), 100 (maximum number of tokens) and 100 (maximum number of tokens).

[0072] Similarly, in mixed query load scenarios where query requests can correspond to different priorities, the method described above for dynamically adjusting the number of tokens corresponding to one or more scheduling cycles based on success rate trends can be executed separately for each priority. In other words, the determined baseline number of tokens is dynamically adjusted for each priority. Furthermore, the dynamic adjustment of the number of tokens for each priority can be performed independently, and the process is essentially the same; therefore, it will not be explained separately for each priority here.

[0073] This shows that, through the above dynamic adjustment method, after obtaining the number of tokens corresponding to the real-time service quality parameters of the database cluster, the number of obtained tokens can be adjusted according to the trend of success rate changes. This maximizes the utilization of system resources during the database cluster system recovery process, and ensures the success rate of the database system as much as possible when the system performance degrades, preventing a precipitous drop in system performance.

[0074] In some scenarios, after determining the number of tokens for one or more scheduling cycles following the first scheduling cycle, the token bucket can be initialized, allowing query requests to be distributed based on the token bucket mechanism in future scheduling cycles. Specifically, after determining the number of tokens for each scheduling cycle, the token bucket can be initialized for one or more scheduling cycles following the first scheduling cycle. In a specific example, the number of tokens for each scheduling cycle can be cached first, and the token bucket for the next scheduling cycle can be initialized only, thereby saving system resources. In some cases, the token bucket initialization process may include: creating a token bucket; setting the token bucket's lifespan; where the lifespan may include: an effective time and an expiration time; and setting the initial count value of the token bucket based on the number of tokens corresponding to the token bucket. In particular, to be applicable to the mixed query load scenario described above, the token bucket initialization process may also include: setting the priority of the token bucket.

[0075] Regarding step 230 above, within the second scheduling period, the number of tokens corresponding to the second scheduling period can be determined based on the number of tokens in the subsequent scheduling period after the first scheduling period, which was determined within the first scheduling period.

[0076] As mentioned earlier, in some cases, the second scheduling period can typically be the next scheduling period after the first scheduling period. Therefore, the token quantity corresponding to the first scheduling period in the subsequent scheduling period can be obtained from the first token quantity as the token quantity corresponding to the second scheduling period.

[0077] Regarding step 240 above, query requests from business clients can be distributed based on the number of tokens corresponding to the acquired second scheduling period. It can be understood that step 240 can also be executed within the second scheduling period.

[0078] Specifically, in some cases, after receiving a query request from a business client, the token bucket count value corresponding to the current scheduling period (i.e., the second scheduling period) can be extracted first. Only if the count value is greater than 0 can the received query request be distributed to the database cluster, and the token bucket count value is further decremented by 1 after distribution.

[0079] As can be seen, the token bucket mechanism effectively controls the concurrency of query requests. Furthermore, the initial token bucket count for the current scheduling cycle is dynamically optimized based on the database cluster's service quality parameters in previous scheduling cycles. The implementation process shows that when the database cluster's query success rate is high, the number of tokens allocated for future scheduling cycles will be higher, thus fully utilizing the database cluster's resources to handle more query requests and improve overall throughput. Conversely, when the database cluster's load is too high, causing its query success rate to drop, the number of tokens allocated for future scheduling cycles can be reduced, tightening the concurrency entry point and preventing excessive query requests from overwhelming the database cluster. This maintains the overall query success rate of the database cluster system at a healthier and more stable level, avoiding a precipitous drop in service quality.

[0080] Furthermore, in mixed query load scenarios, the aforementioned token buckets can be configured separately for each priority level, meaning that independent concurrency control can be implemented for query requests of different priorities. Specifically, upon receiving a query request from a business client, the priority of the query request can be extracted first, and then the count value of the token bucket corresponding to that priority in the current scheduling cycle can be extracted. Only if this count value is greater than 0 can the received query request be distributed to the database cluster, and after distribution, the count value of the token bucket is decremented by 1. In this way, when the database cluster is overloaded, high-priority critical tasks will not compete with other low-priority tasks for limited resources, thus prioritizing the success rate and response time of high-priority tasks and ensuring the normal operation of core business.

[0081] The query processing method described above will be further illustrated below with a specific example. In this example, it is assumed that the scheduling period is 1 minute, meaning that the query proxy layer will repeatedly execute the query processing method described above at a fixed time every minute. The specific execution process is described below.

[0082] First, the query proxy layer collects data, which may include extracting and caching the database cluster's service quality parameters from the system tables or management system of the database cluster in the previous minute, such as query success rate.

[0083] In addition, the query broker layer will generate a scheduling plan, which may include the following processes. First, the cached query success rates from the previous 5 minutes are extracted to determine the query success rate metric, such as the most recent effective query success rate, for example, 85%, as the query success rate metric. Next, based on the above query success rate metric of 85%, and based on the mapping relationship between query success rate and token quantity: when the query success rate metric is less than 95% but greater than or equal to 80%, the token quantity corresponding to priority P0 is 50, the token quantity corresponding to priority P1 is 20, and the token quantity corresponding to priority P2 is 10. It can be determined that the baseline token quantity corresponding to the next 5 scheduling cycles after the current scheduling cycle is: priority P0 is 50, priority P1 is 20, and priority P2 is 10. Furthermore, the success rate trend will be determined based on the extracted query success rates from the previous 5 minutes. For example, the difference between the query success rate of the previous 1 minute and the query success rate of the previous 5 minutes, for example, 2%, can be determined as a recovery trend based on the upper limit threshold of 1% based on the 20% difference. Next, based on the adjustment coefficient of 1.2 corresponding to the recovery trend, the number of tokens per minute in the 5 minutes following the current moment can be further adjusted to obtain the number of tokens per minute for each priority. Specifically, for priority P0, the number of tokens per minute in the following 5 minutes are 60, 72, 86, 100, and 100, respectively; for priority P1, the number of tokens per minute are 24, 28, 34, 41, and 49, respectively; and for priority P2, the number of tokens per minute are 12, 14, 17, 20, and 24, respectively.

[0084] Furthermore, the query agent layer will initialize the token buckets based on the generated scheduling plan. Specifically, after determining the number of tokens per minute for the next 5 minutes, the three token buckets corresponding to the three priorities for the next minute can be initialized. Specifically, the lifespan of each of the three token buckets can be set to the start time (e.g., a minutes 01 seconds) and end time (e.g., a minutes 59 seconds) of the next minute. The initial count value of the token bucket corresponding to priority P0 is 60, the initial count value of the token bucket corresponding to priority P1 is 24, and the initial count value of the token bucket corresponding to priority P2 is 12.

[0085] Furthermore, after receiving a query request from a business client, the query proxy layer can first determine the priority of the received query request and extract the count value of the token bucket corresponding to that priority at the current moment. If the count value is greater than 0, the query request can be distributed and the count value is decremented by 1; if the count value is equal to 0, the distribution of the query request is stopped.

[0086] When the next minute arrives, the query proxy layer can repeat the above method to determine the number of tokens per minute for the next 5 minutes. If the number of tokens determined in the next minute corresponds to the same scheduling period as the previously determined number of tokens, the later determined number of tokens can overwrite the previously determined number of tokens. For example, assuming the current time is t (in minutes), the number of tokens with different priorities corresponding to times t+1 to t+5 can be determined at the current time. At this time, the token bucket initial operation will be performed based on the number of tokens at time t+1, while the number of tokens with different priorities corresponding to times t+2 to t+5 will be cached. When time reaches t+1, the above method is repeated to determine the number of tokens with different priorities corresponding to times t+2 to t+6. It can be seen that the number of tokens with different priorities corresponding to times t+2 to t+5 has already been determined and cached in the previous time, resulting in data duplication in the scheduling period. In this scenario, the new token quantities corresponding to different priorities at times t+2 to t+5, determined at time t+1, can overwrite the previously cached token quantities corresponding to different priorities at times t+2 to t+5, and further cache the token quantities corresponding to different priorities at time t+6. It can be seen that this method of updating cached data in real-time can essentially ensure that the determined number of tokens can be dynamically adjusted according to changes in the performance of the actual database system.

[0087] Furthermore, the query processing method described above simultaneously retrieves and caches multiple token counts corresponding to different time points. While this operation may seem redundant, this redundant data storage can resist algorithm failures. Specifically, in the example above, the token counts for different priorities from time t+1 to t+5 can be determined at the current time. At this point, the token bucket initialization operation is performed based on the token counts for different priorities at time t+1, while the token counts for different priorities from time t+2 to t+5 are cached. When time reaches t+1, the above method is repeated. Assuming the algorithm fails at this point and no new token counts for different priorities from time t+2 to t+6 are obtained, the token bucket corresponding to time t+2 can still be initialized based on the cached token counts for different priorities at time t+2. However, if the query processing method only determines the token counts for different priorities at time t+1 at time t, then if an algorithm failure occurs when the query processing method is repeated at time t+1, the token counts for different priorities at time t+2 cannot be determined, thus preventing the token bucket initialization at time t+2 and leading to concurrency control failure or a malfunction. Therefore, obtaining the number of tokens with different priorities for multiple subsequent scheduling cycles simultaneously within each scheduling cycle can further guarantee the performance of concurrency control and reduce the failure rate of concurrency control.

[0088] Corresponding to the above query processing method, a query processing device located in the query proxy layer of the database cluster system is also provided. Figure 5 This shows an example of the internal structure of a query processing unit. For example... Figure 5 As shown, the query processor described above may include: The data acquisition module 510 is used to acquire the service quality parameters of the database cluster during the first scheduling period; wherein the service quality parameters may include the service quality parameters during the preceding scheduling period; the preceding scheduling period may include one or more scheduling periods before the first scheduling period.

[0089] The scheduling plan module 520 is used to obtain a first token quantity based on the quality of service parameters; wherein the first token quantity may include: the token quantity corresponding to the subsequent scheduling period; the subsequent scheduling period may include: one or more scheduling periods after the first scheduling period.

[0090] The token configuration module 530 is used to determine the number of second tokens corresponding to the second scheduling period based on the number of first tokens in response to receiving a query request during the second scheduling period; wherein the second scheduling period belongs to the first scheduling period in the subsequent scheduling periods.

[0091] The scheduling module 540 is used to distribute query requests to the database cluster based on the number of second tokens.

[0092] It should be noted that each module of the above query processing device executes the above... Figure 2-4 The specific implementation methods of each module for the corresponding query processing methods can be found in the detailed descriptions of each step in the query processing methods in the aforementioned examples. These methods can also achieve the technical effects that the query processing methods in the aforementioned examples can achieve, and will not be repeated here.

[0093] Based on the same concept, corresponding to any of the above methods, an electronic device is also provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above query processing methods.

[0094] Figure 6 A schematic diagram illustrating a more specific hardware structure of an electronic device is shown. This device may include: a processor 2010, a memory 2020, an input / output interface 2030, a communication interface 2040, and a communication interface 2050. The processor 2010, memory 2020, input / output interface 2030, and communication interface 2040 are internally connected to each other via a bus 2050.

[0095] The processor 2010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and achieve the above-mentioned technical solutions.

[0096] The memory 2020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 2020 can store the operating system and other applications. When the above technical solutions are implemented through software or firmware, the relevant program code is stored in the memory 2020 and is called and executed by the processor 2010.

[0097] The input / output interface 2030 is used to connect input / output devices to enable information input and output. These input / output devices can be configured within the device or externally connected to provide corresponding functions. Input devices may include microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0098] The communication interface 2040 is used to connect a communication component (not shown in the figure) to enable communication between this device and other devices. The communication component can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0099] Bus 2050 includes a pathway for transmitting information between various components of the device (e.g., processor 2010, memory 2020, input / output interface 2030, and communication interface 2040).

[0100] It should be noted that although the above-described device only shows the processor 2010, memory 2020, input / output interface 2030, communication interface 2040, and bus 2050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the above scheme, and does not necessarily include all the components shown in the figures.

[0101] The aforementioned electronic device is used to implement the corresponding query processing method in any of the preceding examples, and has the beneficial effects of the corresponding method, which will not be elaborated here.

[0102] Based on the same concept, corresponding to any of the above example methods, in some cases a non-transitory computer-readable storage medium is also provided, which stores computer instructions for causing the computer to perform the query processing method as described above.

[0103] The aforementioned computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program components, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0104] The computer instructions stored in the aforementioned storage medium are used to cause the computer to execute the query processing method described in any of the above examples, and have the beneficial effects of the corresponding method, which will not be elaborated further here.

[0105] In some cases, a computer program product is also provided, including computer program instructions, which, when executed on a computer, cause the computer to perform the above-described query processing method and have the corresponding beneficial effects, which will not be elaborated further here.

[0106] Those skilled in the art should understand that the discussion of any of the above examples is merely illustrative and is not intended to imply that the scope of the claimed protection is limited to these examples; under the disclosed technical concept, the technical features of the above examples or different examples can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the examples, which are not provided in detail for the sake of brevity.

[0107] Additionally, to simplify the description and discussion, and to avoid obscuring the above examples, the provided figures may or may not show well-known power / ground connections to integrated circuit (IC) chips and other components. Furthermore, boards may be shown in block diagram form to avoid obscuring the examples, and this also takes into account the fact that the details of implementation of these block diagram boards are highly dependent on the platform on which these examples will be implemented (i.e., these details should be entirely within the understanding of those skilled in the art). In the context of the specific details (e.g., circuits) set forth to describe these examples, it will be apparent to those skilled in the art that the above examples can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0108] Although various examples have been described, many substitutions, modifications, and variations of these examples will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the examples discussed.

[0109] These examples are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of these examples shall be included within the scope of protection.

Claims

1. A query processing method, comprising: Within the first scheduling period, service quality parameters of the database cluster are obtained, and a first token quantity is obtained based on the service quality parameters; wherein, the service quality parameters include: service quality parameters within the preceding scheduling period; the preceding scheduling period includes: one or more scheduling periods prior to the first scheduling period; the first token quantity includes: the token quantity corresponding to the subsequent scheduling period; the subsequent scheduling period includes: one or more scheduling periods after the first scheduling period; and Within the second scheduling period, in response to receiving a query request, the second token quantity corresponding to the second scheduling period is determined based on the first token quantity, and the query request is distributed to the database cluster based on the second token quantity; wherein, the second scheduling period belongs to the first scheduling period in the subsequent scheduling periods.

2. The query processing method according to claim 1, wherein, The process of obtaining the first token quantity based on the service quality parameters includes: The query success rate metric is obtained based on the aforementioned service quality parameters; and Based on a pre-established mapping relationship, the first token quantity is obtained; wherein the mapping relationship includes: the mapping relationship between the query success rate indicator and the first token quantity; and the mapping relationship is a positive correlation mapping relationship.

3. The query processing method according to claim 2, wherein, The query success rate metric obtained based on the service quality parameters includes: Based on the service quality parameters, obtain the query success rate parameters within the pre-scheduling period; Based on pre-defined filtering conditions, one or more valid query success rate parameters are obtained from the query success rate parameters; and The query success rate metric is obtained based on one or more of the effective query success rate parameters.

4. The method according to claim 3, wherein, The process of obtaining the query success rate metric based on the one or more valid query success rate parameters includes: The query success rate metric is the one or more valid query success rates that is closest to the first scheduling period; or... The weighted average of the one or more valid query success rate parameters is used as the query success rate indicator.

5. The query processing method according to claim 1, wherein, The number of tokens corresponding to subsequent scheduling cycles includes multiple token numbers corresponding to different priorities.

6. The query processing method according to claim 1, further comprising: Success rate trends are identified based on the service quality parameters; wherein, the success rate trends include: a first trend, a second trend, and a third trend; An adjustment coefficient is obtained based on the success rate trend; wherein the adjustment coefficient corresponding to the first trend is greater than 1; the adjustment coefficient corresponding to the second trend is less than 1; and the adjustment coefficient corresponding to the third trend is equal to 1; and The first token quantity is updated based on the adjustment coefficient; wherein, when the success rate trend is the first trend, the first token quantity increases over time; when the success rate trend is the second trend, the first token quantity decreases over time; and when the success rate trend is the third trend, the first token quantity remains unchanged.

7. The query method according to claim 1, further comprising: Initialize a token bucket based on the first number of tokens; wherein, the token bucket includes: a token bucket corresponding to the subsequent scheduling period.

8. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the query processing method as described in any one of claims 1-7.

9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the query processing method according to any one of claims 1-7.

10. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform the query processing method as described in any one of claims 1-7.

Images