Distributed database resource control method and related equipment
Patent Information
- Application Number
- CN202511013683.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-22
- Publication Date
- 2025-11-07
AI Technical Summary
In existing technologies, distributed database resource control methods are complex to manage and configure, introducing additional overhead, especially during version upgrades, making it difficult to achieve fine-grained resource control.
By acquiring the target user's SQL request, quantifying it to obtain the resource requirement, and checking it according to logical resource control rules, if it is insufficient, the corresponding logical resource control strategy is executed to achieve fine-grained resource management.
It enables resource management across different nodes in a distributed database, improving query performance, reducing redundant resource consumption, lowering reliance on high-configuration hardware, and adapting to short-term fluctuations in business traffic.
Smart Images
Figure CN120910099A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a distributed database resource control method, a distributed database resource control device, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology
[0002] For distributed databases that separate storage and computation, a distributed Structured Query Language (SQL) agent is often used as the computation service layer to provide user access and database services.
[0003] In related technologies, control groups (cgroups) can be used to manage resources in distributed databases with separated storage and compute. However, this resource management method suffers from complex management and configuration, introducing additional overhead.
[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0005] This disclosure provides a distributed database resource control method and related equipment, which at least to some extent overcomes the problems of complex management and configuration and the introduction of additional overhead in the distributed database resource control methods of related technologies.
[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.
[0007] According to one aspect of this disclosure, a distributed database resource control method is provided, applied to a target node. The method includes: acquiring structured query language (SQL) requests from target users within a preset time threshold; quantifying the SQL requests to obtain resource requirements; performing logical resource checks based on resource requirements and logical resource control rules; and if insufficient logical resources are acquired, executing corresponding logical resource control strategies on the SQL requests based on the logical resource control rules.
[0008] In one embodiment of this disclosure, the logical resource control rule includes at least one of the following: logical resource control rule name, logical resource threshold, logical resource control rule time threshold, logical resource control user, logical resource control user address, and logical resource control policy.
[0009] In an embodiment of the present disclosure, the target node is a shard node; wherein the quantifying the SQL request to obtain the resource demand amount comprises at least one of the following: when the SQL request is a read request, quantifying a single read request as 1 logical resource; when the SQL request is a read request, quantifying a batch read request as 4 logical resources; when the SQL request is a read request, consuming 1 logical resource for every first preset number of result returns; when the SQL request is a write request, quantifying a single write request as 4 logical resources; when the SQL request is a write request, quantifying a batch write request as 8 logical resources; and when the SQL request is a write request, consuming 1 logical resource for every second preset number of write requests.
[0010] In an embodiment of the present disclosure, the method further comprises: in response to the physical operator interacting with the shard node, the shard node performing logical resource checking according to the resource demand amount and the logical resource control rule.
[0011] In an embodiment of the present disclosure, the target node is a computing node; wherein the quantifying the SQL request to obtain the resource demand amount comprises at least one of the following: for a physical operator with disk write and read, quantifying according to the data request amount of the SQL request, and consuming 1 logical resource for every third preset number of write and read requests; and the demand amount of a central processing unit is consumed at an interval of a preset usage duration.
[0012] In an embodiment of the present disclosure, if sufficient logical resources are not obtained, the SQL request is executed with a corresponding logical resource control strategy based on the logical resource control rule, comprising: if the resource demand amount exceeds the logical resource threshold within the preset time threshold, it is determined that sufficient logical resources are not obtained.
[0013] According to another aspect of the present disclosure, a distributed database resource control apparatus is also provided, applied to a target node, the apparatus comprising: a request obtaining module configured to obtain a structured query language (SQL) request of a target user within a preset time threshold; a resource quantifying module configured to quantize the SQL request to obtain a resource demand amount; and a resource control module configured to perform logical resource checking according to the resource demand amount and a logical resource control rule, and execute a corresponding logical resource control strategy for the SQL request based on the logical resource control rule if sufficient logical resources are not obtained.
[0014] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the above-mentioned distributed database resource control method by executing the executable instructions.
[0015] According to another aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, the computer program being executed by a processor to implement the distributed database resource control method described above.
[0016] According to still another aspect of the present disclosure, a computer program product is provided, which comprises a computer program or computer instructions, the computer program or the computer instructions being loaded and executed by a processor to cause a computer to implement the distributed database resource control method described above.
[0017] In the embodiments of the present disclosure, the target node acquires the SQL requests of the target user within a preset time threshold; the SQL requests are quantitatively processed to obtain resource demand quantity; logical resource checking is performed according to the resource demand quantity and the logical resource management and control rule, and if sufficient logical resources are not acquired, corresponding logical resource management and control strategies are executed on the SQL requests based on the logical resource management and control rule. The present disclosure can statistically count the usage of resources in the form of logical resources, and can simultaneously implement resource management and control of different nodes and implement fine-grained resource management and control.
[0018] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS
[0019] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.
[0020] Figure 1 A structural schematic diagram of a distributed database provided by an embodiment of the present disclosure is shown.
[0021] Figure 2 Another structural schematic diagram of a distributed database provided by an embodiment of the present disclosure is shown.
[0022] Figure 3 A flowchart of a distributed database resource control method provided by an embodiment of the present disclosure is shown.
[0023] Figure 4 Another flowchart of a distributed database resource control method provided by an embodiment of the present disclosure is shown.
[0024] Figure 5 A structural schematic diagram of a distributed database resource control device provided by an embodiment of the present disclosure is shown.
[0025] Figure 6 A structural block diagram of an electronic device provided in an embodiment of this disclosure is shown.
[0026] Figure 7 This diagram illustrates the structure of a computer-readable storage medium provided in an embodiment of the present disclosure. Detailed Implementation
[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0028] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.
[0030] like Figure 1 As shown, for distributed databases with separate storage and computation, a distributed Structured Query Language (SQL) agent is typically used as the computation service layer to provide user access and database services. The storage layer is generally divided into several shards, with a single-machine database instance deployed on each shard. Data is stored on different shards according to certain distribution rules. The number of shards and the distribution rules in the storage layer can be determined based on actual needs.
[0031] When a user issues a SQL query command, the computing node parses and optimizes the SQL query command according to the connection protocol, generates corresponding physical computing power, and generates a corresponding execution plan. The execution plan is executed by the SQL executor. The computing node collects the query results returned on each shard, performs subsequent calculations such as aggregation, sorting, filtering, and the like, and finally returns the final calculation result to the user.
[0032] The SQL parser is used to convert the SQL statement input by the user into an internal data form that can be understood by the distributed database.
[0033] The SQL executor is the core component responsible for executing the SQL query plan. It converts the query plan generated by the optimizer into specific operation steps and coordinates modules such as the storage engine, cache, and lock management to finally return the query result.
[0034] The physical operator is the basic building block of the SQL executor. It is responsible for converting the logical query plan into specific physical operations. The physical operator directly interacts with data, reads data, processes data, and returns results.
[0035] In related technologies, for resource management and control of a distributed database, usually only the computing node or the shard is managed and controlled. For example, for memory resource management and control, usually only the memory resources on the shard or only the memory resources on the computing node are managed and controlled.
[0036] In the resource management and control of a distributed database, it is usually achieved through control groups (cgroup). The configuration and management of the above implementation method are complex and introduce additional overhead, especially when fine-grained resource management and control is required, the complexity is even greater. Secondly, the functions and characteristics of cgroup change with the update of the kernel version, and need to be adjusted according to the actual used kernel version. The above implementation method introduces great difficulty in version upgrade.
[0037] Figure 2 Another structure diagram of a distributed database provided by an embodiment of the present disclosure is shown. As shown in Figure 2 Based on the Figure 1 embodiment, when the computing node obtains the SQL request of the target user within the preset time threshold, the resource demand quantity is obtained by quantitatively processing the SQL request, so that the physical operator and the shard node perform logical resource checking according to the resource demand quantity and the preset logical resource management and control rule, to determine whether to perform corresponding logical resource management and control on the SQL request according to the checking result, thereby realizing fine-grained resource management and control.
[0038] Figure 2 The logical IO management and control in the above embodiment is also referred to as logical resource management and control.
[0039] It should be noted that the number of fragments and the mapping relationship between the fragments and the physical operators are not specifically limited by the present disclosure.
[0040] To at least partially solve the above technical problems, the target node of the present disclosure acquires the SQL request of the target user within a preset time threshold; the SQL request is quantitatively processed to obtain a resource demand quantity; logical resource checking is performed according to the resource demand quantity and the logical resource management and control rule, and if sufficient logical resources are not acquired, the corresponding logical resource management and control strategy is executed on the SQL request based on the logical resource management and control rule. The present disclosure statistically analyzes the use of resources in the form of logical resources, can simultaneously implement resource management and control of different nodes, and implements fine-grained resource management and control.
[0041] It should be noted that the embodiments of the present application and the technical features in the embodiments can be combined with each other without conflict.
[0042] The present example embodiment will be described in detail below in combination with the accompanying drawings and embodiments.
[0043] First, the present disclosure provides a distributed database resource control method, which can be executed by any electronic device with computing and processing capabilities. The distributed database resource control method provided in the present disclosure can be executed by a target node, for example, can be executed by a distributed database resource control device configured on a computing node, or can be executed by a distributed database resource control device configured on a fragment node. The above-mentioned device can be a programmable logic controller (PLC) or a single-chip microcomputer, etc., as long as it can realize the corresponding function.
[0044] Figure 3 A flow chart of a distributed database resource control method provided in an embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the distributed database resource control method provided in the present disclosure is applied to a target node, and the method mainly includes the following steps: Figure 3
[0045] S302, acquiring the SQL request of the target user within a preset time threshold.
[0046] In one embodiment, the target user is a user who initiates the SQL request. The target user can initiate the SQL statement using a corresponding terminal device, and the terminal device has an application client installed therein. The terminal device can be various electronic devices, including but not limited to a smart phone, a tablet computer, a smart watch, a wearable device, etc.
[0047] The SQL request can include a read request, a write request, and the like of a sharding node, and a request related to data query, data insertion, data update, and data deletion. The SQL request can include at least one of a data table name, a column name retrieved in the data table, a query condition, a sorting condition, and a grouping condition.
[0048] For example, the SQL statement is "SELECT id, name FROM users WHERE age > 25 LIMIT 20", which can filter 20 id and name from the users table whose age is greater than 25.
[0049] After the target user's SQL request is obtained by the current computing node, the SQL request can be parsed. The SQL parser checks whether the syntax of the SQL request is correct. If the syntax is correct, the SQL request is converted into an internal data structure.
[0050] The preset time threshold can be configured in the logical resource management and control rule. The value of the preset time threshold can be determined according to actual needs. The unit of the preset time threshold is ms, which is used to meet the scene of sudden appearance of business traffic. The smaller the preset time threshold is set, the more it can prevent the situation of insufficient resources caused by traffic explosion. For example, the preset time threshold is configured as 500 ms, 1000 ms, and the like.
[0051] S304, quantizing the SQL request to obtain a resource demand quantity.
[0052] The resource demand quantity refers to the resource quantity required by the target user to complete the SQL request within the preset time threshold.
[0053] Within the preset time threshold, the target user initiates a plurality of SQL requests, and the resource demand quantity is the total resource quantity required by the plurality of SQL requests.
[0054] The SQL request includes a read request and / or a write request. The read request needs to consume central processing unit (CPU) and memory resources for processing data retrieval and transmission; the write request not only consumes CPU and memory resources, but also generates a write operation, which involves disk input / output (I / O) operation.
[0055] In the distributed database, the sharding node of the storage end and the computing node are not on the same physical machine.
[0056] When the target node is a sharding node, the read request is split according to the sharding dimension. According to the read request mode, it can be divided into streaming read request and non-streaming read request. According to the number of request SQL, it can be divided into batch read or single read.
[0057] When the target node is a computing node, the SQL request needs disk IO or CPU.
[0058] In the present disclosure, by quantifying the SQL request, the resource required by the SQL request is quantified as the resource demand, which facilitates the positioning of high-consumption SQL requests, and then targeted optimization is performed to shorten the response time of the request and improve the query performance. In addition, the system resources can be reasonably allocated, the resource allocation of concurrent requests can be dynamically adjusted, the system resources can be prevented from being exhausted by a single complex query, the redundant resource consumption can be greatly reduced, and the dependence on high-configuration hardware can be reduced.
[0059] S306, logical resource checking is performed according to the resource demand and the logical resource management and control rule. If sufficient logical resources are not obtained, a corresponding logical resource management and control strategy is executed on the SQL request based on the logical resource management and control rule.
[0060] In one embodiment, the logical resource management and control rule includes at least one of a name of the logical resource management and control rule, a logical resource threshold, a time threshold of the logical resource management and control rule, a user of the logical resource management and control, an address of the user of the logical resource management and control, and a logical resource management and control strategy.
[0061] The logical resource management and control rule is used to configure the information defined by the logical resource management and control rule, also known as logical resource management and control rule metadata, which can be configured by a SQL statement.
[0062] The name of the logical resource management and control rule is used to distinguish different logical resource management and control rules. For example, the name of the logical resource management and control rule is defined as r1_name.
[0063] The logical resource threshold quantity is used to configure the upper limit of the resource when the resource management and control is required, also known as the logical resource management and control rule threshold. For example, when the resource demand of the target user reaches the above-mentioned logical resource threshold within the preset time threshold, it is determined that the resource management and control is performed on the SQL request.
[0064] The time threshold elapsed of the logical resource management and control rule is used to determine the above-mentioned preset time threshold, and the unit is ms. This parameter is used to meet the scene of sudden short-term surge of business traffic. The smaller the value of this time threshold is, the more it can prevent the situation of resource shortage caused by traffic surge.
[0065] The user of the logical resource management and control user is used to determine the user defined by the logical resource management and control rule. Only when the username of the target user matches the value of the user field of the logical resource management and control user, the resource management and control is performed.
[0066] The user address host of the logical resource management and control, used to determine the user address defined by the logical resource management and control rule. The user address can be the IP supported by the user and the IP network segment. Only when the user address of the target user matches the value of the user address host field of the logical resource management and control, the resource management and control will be performed.
[0067] The logical resource management and control strategy action, used to determine the corresponding strategy when the resource management and control is performed on the SQL request of the target user. That is, when the SQL executor executes the SQL request, the behavior taken when the logical resource is insufficient is obtained. The logical resource management and control strategy supports two modes, namely, termination Kill and waiting Wait. Kill is to directly terminate the SQL request of the target user when the resource is insufficient; Wait is to wait for the request until sufficient logical resources are obtained when the resource is insufficient.
[0068] The SQL syntax of the logical resource management and control rule can be described as:
[0069] alter vschema create resource_limit if not exist r1_name with
[0070] quantity=1000,
[0071] elapsed=500,
[0072] user='user_1,user_2′,
[0073] host='10.10.192.1,10.10.10.0 / 1',
[0074] action='kill′
[0075] Among them, "alter vschema create resource_limit if not exist r1_name with" is used to indicate the keyword of the SQL syntax of the logical resource management and control rule, indicating that the SQL syntax is a statement about the logical management and control rule. r1_name is the name of the logical resource management and control rule, the time threshold is 1000 ms, the resource threshold is 500, the users for resource management and control include user_1 and user_2, and the user addresses for resource management and control include 10.10.192.1 and 10.10.10.0 / 1; When the logical resource is insufficient, the logical resource management and control strategy adopted is kill.
[0076] In an embodiment, the logical resource check according to the resource demand and the logical resource management rule can include a matching check on the time threshold in the resource demand and the logical rule management rule and the logical resource threshold.
[0077] In an embodiment, the logical resource check according to the resource demand and the logical resource management rule can also include a matching process between the remaining logical resource of the target node and the resource demand, and when the remaining logical resource is less than the resource demand, it is determined that sufficient logical resource is not obtained.
[0078] In the embodiment of the present disclosure, the target node obtains the SQL request of the target user within a preset time threshold; the SQL request is quantitatively processed to obtain a resource demand; a logical resource check is performed according to the resource demand and a logical resource management rule, and if sufficient logical resource is not obtained, a corresponding logical resource management strategy is executed on the SQL request based on the logical resource management rule. The present disclosure statistically processes the use of resources in the form of logical resources, can simultaneously implement resource management of different nodes, and implements fine-grained resource management.
[0079] For a distributed database of storage-computing separation, the shard node and the computing node of the storage end are not on one physical machine, and in the present disclosure, the quantification processing of the SQL request by the shard node and the quantification processing of the SQL request by the computing node are performed respectively.
[0080] For the quantification of the SQL request by the shard node, the SQL request can be divided according to the type of the SQL request, the number of read-write requests, and the size of the data returned by the read request or the size of the data of the write request.
[0081] In an embodiment, the target node can be a shard node. The quantification processing of the SQL request in S304 to obtain the resource demand includes at least one of the following:
[0082] When the SQL request is a read request, a single read request is quantified as 1 logical resource;
[0083] When the SQL request is a read request, a batch read request is quantified as 4 logical resources;
[0084] When the SQL request is a read request, 1 logical resource is consumed for returning a result of a first preset number;
[0085] When the SQL request is a write request, a single write request is quantified as 4 logical resources;
[0086] When the SQL request is a write request, a batch write request is quantified as 8 logical resources;
[0087] When the SQL request is a write request, every second preset number of write requests consumes one logical resource.
[0088] In an embodiment, the values of the first preset number and the second preset number can be determined according to actual needs. For example, when the SQL request is a read request, one logical resource can be consumed for every 32 kb of result returned; when the SQL request is a write request, one logical resource can be consumed for every 1 kb of write request.
[0089] It should be noted that the quantified logical resource quantity for single read request, batch read request, single write request, and batch write request can also be determined according to actual needs, and the present disclosure does not make specific limitations thereon.
[0090] In the embodiments of the present disclosure, by uniformly quantifying the logical resources of the sharding nodes in the storage-computing-separated distributed database, the sharding nodes can be controlled in logical resources, the distributed database can be controlled in fine-grained resources, high-consumption SQL requests can be located, targeted optimization can be performed, the response time of requests can be shortened and the query performance can be improved, system resources can be reasonably allocated, resource allocation for concurrent requests can be dynamically adjusted, and the consumption of redundant resources can be greatly reduced, thereby reducing the dependence on high-configuration hardware.
[0091] In an embodiment, for a sharding node, the sharding node performs logical resource checking according to the resource demand quantity and the logical resource control rule only in response to an interaction behavior between the physical operator and the sharding node.
[0092] The interaction behavior between the physical operator and the sharding node includes a behavior in which the physical operator communicates with the sharding node through RPC or a message queue; and can also include a behavior in which the physical operator determines the sharding node where data is located according to a data sharding rule, the sharding node performs a corresponding task, and a query result is obtained.
[0093] In the embodiments of the present disclosure, the sharding node performs logical resource checking after the interaction behavior between the physical operator and the sharding node, which can realize dynamic load balancing and resource optimization, realize fault tolerance and fault isolation, and the sharding node can dynamically adjust the physical operator execution strategy according to the local resource state.
[0094] In an embodiment, when the target node is a computing node, the quantification of the SQL request in S304 to obtain the resource demand quantity includes at least one of the following:
[0095] For a physical operator with disk writing and reading, the data request quantity of the SQL request is quantified, and every third preset number of write requests and read requests consumes one logical resource.
[0096] The amount of the central processor required is consumed by one logical resource per preset usage time length.
[0097] It should be noted that the values of the third preset number and the preset usage time length can be determined according to actual needs.
[0098] For example, for a physical operator with disk write and read, one logical resource is consumed per 16 kb of write and read requests.
[0099] The central processor consumes one logical resource per 2 ms of usage time.
[0100] In the embodiments of the present disclosure, by uniformly quantifying the logical resources of the computing nodes in the distributed database separated from the storage, the computing nodes can be controlled in logical resources, so that the resources in the distributed database can be controlled in fine granularity.
[0101] Figure 4 A flowchart of another distributed database resource control method in the embodiments of the present disclosure is shown. In Figure 3 Based on the embodiments, S306 is refined to S3062 to limit the case where sufficient logical resources are not obtained. As Figure 4 shown, the distributed database resource control method of the embodiments includes S302-S304 and S3062. If sufficient logical resources are not obtained in S306, a corresponding logical resource control strategy for the SQL request is executed based on the logical resource control rule, including:
[0102] In S3062, if the resource demand amount exceeds the logical resource threshold within the preset time threshold, it is determined that sufficient logical resources are not obtained.
[0103] It should be noted that the specific implementation of S302-S304 of the present embodiment is the same as that of S302-S304 in the foregoing embodiments, which will not be described here.
[0104] The preset time threshold and the logical resource threshold are the same as the time threshold and the logical resource threshold in the logical resource control rule.
[0105] When the resource demand amount within the preset time threshold is greater than the logical resource threshold, it indicates that the business traffic corresponding to the SQL request suddenly rises temporarily, and it is determined that the distributed database is in a resource insufficient state, and the corresponding logical resource control strategy in the logical resource control rule needs to be executed for the SQL request of the target user.
[0106] When the resource demand amount within the preset time threshold is less than or equal to the logical resource threshold, it indicates that the distributed database has sufficient logical resources to support the execution of the SQL request, and it is determined that the resources of the distributed database are sufficient.
[0107] It should be noted that in various embodiments of the present disclosure, the resource management and control on the SQL request of the target user is performed on the premise that the username and / or user address of the target user matches the username and / or user address configured in the logical resource management and control rule.
[0108] In the embodiments of the present disclosure, by setting the preset time threshold and the logical resource threshold, the scenario of sudden short-term surge of business traffic can be adapted, the impact of the short-term surge of traffic on the distributed database can be effectively avoided, and the reliability of the distributed database can be greatly improved.
[0109] In the present disclosure, a resource management and control method based on logical resources is proposed. By abstracting the system resources of the distributed database, the usage of the resources is counted in the form of logical resources, the resource management and control of the computing nodes and the shard nodes can be realized at the same time, and the configuration and management of the logical resource management and control rules can be easily realized through the SQL statements, and the fine-grained resource management and control can be realized.
[0110] Based on the same inventive concept, the present disclosure also provides a distributed database resource control device in the embodiments, as follows. Since the principles of the device embodiments for solving problems are similar to the above-mentioned method embodiments, the implementation of the device embodiments can be referred to the implementation of the above-mentioned method embodiments, and the repeated parts will not be repeated.
[0111] Figure 5 A structure schematic diagram of a distributed database resource control device provided by an embodiment of the present disclosure is shown. As shown in Figure 5 The distributed database resource control device of the embodiment of the present disclosure is applied to a target node, and the device includes a request acquisition module 510, a resource quantification module 520, and a resource management and control module 530.
[0112] The request acquisition module 510 is configured to acquire the SQL requests of a target user within a preset time threshold.
[0113] The resource quantification module 520 is configured to quantitatively process the SQL requests to obtain resource demand amounts.
[0114] The resource management and control module 530 is configured to perform logical resource checking according to the resource demand amounts and the logical resource management and control rules, and if sufficient logical resources are not acquired, perform corresponding logical resource management and control strategies on the SQL requests based on the logical resource management and control rules.
[0115] It should be noted that the above request obtaining module 510, resource quantification module 520 and resource management module 530 correspond to S302-S306 in the method embodiment, and the above modules and the examples and application scenarios implemented by the corresponding steps are the same, but are not limited to the contents disclosed in the above method embodiment. It should be noted that the above modules as part of the device can be executed in a computer system such as a group of computer executable instructions.
[0116] It should be noted that the logical resource management rule includes at least one of the name of the logical resource management rule, the logical resource threshold, the time threshold of the logical resource management rule, the user of the logical resource management, the address of the user of the logical resource management, and the logical resource management strategy.
[0117] In one embodiment, the target node is a shard node; the resource quantification module 520 is configured to at least one of: quantifying a single read request as 1 logical resource when the SQL request is a read request; quantifying a batch read request as 4 logical resources when the SQL request is a read request; consuming 1 logical resource for every first preset number of result returns when the SQL request is a read request; quantifying a single write request as 4 logical resources when the SQL request is a write request; quantifying a batch write request as 8 logical resources when the SQL request is a write request; and consuming 1 logical resource for every second preset number of write requests when the SQL request is a write request.
[0118] In one embodiment, the resource management module 530 is configured to, in response to the physical operator interacting with the shard node, the shard node performing logical resource checking according to the resource demand and the logical resource management rule.
[0119] In one embodiment, the target node is a computing node; the resource quantification module 520 is configured to at least one of: for a physical operator with disk write and read, quantifying according to the data request amount of the SQL request, consuming one logical resource for every third preset number of write and read requests; and the demand amount of the central processing unit consuming one logical resource every preset usage time interval.
[0120] In one embodiment, the resource management module 530 is configured to determine that sufficient logical resources have not been obtained if the resource demand exceeds the logical resource threshold within a preset time threshold.
[0121] In the embodiment of the present disclosure, the target node acquires the SQL request of the target user within a preset time threshold; the SQL request is quantitatively processed to obtain a resource demand amount; logical resource checking is performed according to the resource demand amount and a logical resource management and control rule; if sufficient logical resources are not acquired, a corresponding logical resource management and control strategy is executed on the SQL request based on the logical resource management and control rule. The present disclosure can statistically count the use of resources in the form of logical resources, can simultaneously implement resource management and control of different nodes, and can implement fine-grained resource management and control.
[0122] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be specifically implemented as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combined with hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.
[0123] The electronic device 600 according to this embodiment of the present application will be described below with reference to Figure 6 Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functions and use range of the embodiments of the present application.
[0124] In one embodiment, the electronic device 600 includes a processor and a memory for storing executable instructions of the processor, and the processor is configured to execute the distributed database resource control method of the above-mentioned embodiments by executing the executable instructions.
[0125] As shown in Figure 6 , the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 can include, but are not limited to, the above-mentioned at least one processing unit 610, the above-mentioned at least one storage unit 620, and a bus 630 connecting different system components, including the storage unit 620 and the processing unit 610.
[0126] The storage unit 620 stores program code, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present application described in the above "Exemplary Method" section of the specification. For example, the processing unit 610 can acquire the SQL request of the target user within a preset time threshold as shown in Figure 3 ; quantitatively process the SQL request to obtain a resource demand amount; perform logical resource checking according to the resource demand amount and a logical resource management and control rule; if sufficient logical resources are not acquired, execute a corresponding logical resource management and control strategy on the SQL request based on the logical resource management and control rule.
[0127] The storage unit 620 can include a readable medium in the form of volatile storage such as random access memory (RAM) 6201 and / or cache memory 6202, and also can include a non-volatile storage such as read only memory (ROM) 6203.
[0128] The storage unit 620 also can include a program / utility 6204 having a set of program modules 6205 such as an operating system, one or more application programs, other program modules, and program data, each of which can give the electronic device 600 its functionality, at least in part. The program modules 6205 can include, but are not limited to, a network environment implementation, either alone or in some combination.
[0129] The bus 630 can represent one or more of several types of bus structures, including a storage bus or bus for storage controller, a peripheral bus, an accelerated graphics port, an electronic device bus, or a local bus using any of a variety of bus architectures.
[0130] The electronic device 600 also can communicate with one or more external devices 640 such as a keyboard or pointing device, a Bluetooth device, etc.; other devices such as devices enabling a user to interact with the system; and / or any devices (e.g., routers, modems, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 650. Still yet, such communication can occur electronically over a network 660 such as a local area network (LAN) and / or a wide area network (WAN) such as the Internet. As an example, the network 660 can be enabled by a network adapter 610. Figure 6 The network adapter 610 can communicate with the other components of the electronic device 600 via the bus 630. It should be understood that although not shown, other hardware and / or software components that are Figure 6 It should be understood that, although not shown explicitly in the
[0131] Those skilled in the art will readily recognize that the example embodiments described herein can be implemented using software and / or hardware in combination with software. Thus, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.
[0132] Figure 7A schematic diagram of a computer readable storage medium in an embodiment of the present disclosure is shown. As shown in Figure 7 The computer readable storage medium 700 of the embodiment of the present disclosure, on which a computer program is stored, is implemented by a processor to realize the distributed database resource control method of the above-mentioned embodiment.
[0133] In the exemplary embodiments of the present disclosure, a computer readable storage medium 700 is also provided, which can be a readable signal medium or a readable storage medium. In some possible implementations, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for performing the steps described in the above "Exemplary Method" section according to various exemplary embodiments of the present disclosure when the program product is run on a terminal device.
[0134] More specific examples of the computer readable storage medium in the present disclosure can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0135] In the present disclosure, the computer readable storage medium can include a data signal carried in the baseband or as a part of a carrier wave, in which readable program codes are borne. Such a propagated data signal can take multiple forms, including but not limited to electro-magnetic signal, optical signal, or any suitable combination of the foregoing. The readable signal medium can also be any readable medium that can send, propagate or transfer program for use by or in connection with an instruction execution system, apparatus or device, other than the readable storage medium.
[0136] Optionally, the program codes contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0137] In particular embodiments, the program code utilized by the program code instructions can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The application programming interface (API) server 106 can be used to facilitate the connection between the client device 102 and the remote computing device.
[0138] It should be noted that, although several modules or units of the devices for action execution are mentioned in the foregoing detailed description, such division is not mandatory. Indeed, features and functionalities of two or more modules or units described above can be embodied in one module or unit according to embodiments of the present disclosure. Conversely, features and functionalities of one module or unit described above can be further divided into multiple modules or units.
[0139] Furthermore, although the various steps of the methods in the present disclosure are described in a particular order in the figures, this is not required or implied in any way as to the order of the steps or that all illustrated steps be performed to achieve desirable results. Additionally or alternatively, certain steps can be omitted, combined into fewer steps, performed in an order other than that described, and / or split into multiple steps, etc.
[0140] From the above description of the embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium, such as a CD-ROM, a USB flash disk, a mobile hard disk, or the like, or on a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) execute the methods according to the embodiments of the present disclosure.
[0141] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the disclosure be construed as including any patents, patent applications, publications, publications, or other disclosure of complementary subject matter that is within the scope of the disclosure. It is intended that the specification and examples be considered exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.
Claims
1. A method for distributed database resource control, the method comprising: The method is applied to a target node, and the method comprises: obtaining a structured query language (SQL) request of a target user within a preset time threshold; performing quantitative processing on the SQL request to obtain a resource demand quantity; performing logical resource checking according to the resource demand quantity and a logical resource management rule, and if sufficient logical resources are not obtained, performing a corresponding logical resource management strategy on the SQL request based on the logical resource management rule.
2. The method of claim 1, wherein, The logical resource management rule comprises at least one of a name of the logical resource management rule, a logical resource threshold, a time threshold of the logical resource management rule, a user of logical resource management, an address of the user of logical resource management, and a logical resource management strategy.
3. The method of claim 2, wherein, The target node is a shard node. The quantitative processing on the SQL request to obtain the resource demand quantity comprises at least one of the following: when the SQL request is a read request, quantifying a single read request as one logical resource; when the SQL request is a read request, quantifying batch read requests as four logical resources; when the SQL request is a read request, consuming one logical resource for every first preset number of result returns; when the SQL request is a write request, quantifying a single write request as four logical resources; when the SQL request is a write request, quantifying batch write requests as eight logical resources; when the SQL request is a write request, consuming one logical resource for every second preset number of write requests.
4. The method of claim 3, wherein, The method further comprises: in response to a physical operator interacting with the shard node, the shard node performing logical resource checking according to the resource demand quantity and the logical resource management rule.
5. The method of claim 2, wherein, The target node is a computing node. The quantitative processing on the SQL request to obtain the resource demand quantity comprises at least one of the following: for a physical operator with disk writing and reading, quantifying according to a data request quantity of the SQL request, and consuming one logical resource for every third preset number of write requests and read requests; a demand quantity of a central processing unit (CPU) consumes one logical resource every preset usage duration.
6. The method of claim 2, wherein, If the resource demand quantity exceeds the logical resource threshold within the preset time threshold, it is determined that sufficient logical resources are not obtained. The apparatus is applied to a target node, and the apparatus comprises:
7. A distributed database resource control apparatus characterized by comprising: a request obtaining module configured to obtain a structured query language (SQL) request of a target user within a preset time threshold; a resource quantification module configured to perform quantitative processing on the SQL request to obtain a resource demand quantity; a resource management module configured to perform logical resource checking according to the resource demand quantity and a logical resource management rule, and if sufficient logical resources are not obtained, performing a corresponding logical resource management strategy on the SQL request based on the logical resource management rule. The apparatus comprises:
8. An electronic device, comprising: a processor; and a memory configured to store executable instructions of the processor; wherein the processor is configured to execute the distributed database resource control method according to any one of claims 1-6 by executing the executable instructions. 9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the distributed database resource control method as claimed in any one of claims 1-6.
10. A computer program product, characterised in that, The computer program product comprises a computer program or computer instructions, which are loaded and executed by a processor to enable a computer to implement the distributed database resource control method as claimed in any one of claims 1-6.