Request processing method and device based on distributed database

Abstract

The present invention provides a request processing method and device based on a distributed database, which mainly relates to the field of computer technology, and in particular to the field of database technology. The method includes: in response to a user's access request to a distributed database, determining an execution tree structure according to a query statement included in the access request; creating a predecessor node for the target execution node in at least one of the execution nodes to obtain a target execution tree structure when the operation type of the target execution node is a data migration operation type; when the number of the predecessor nodes is less than or equal to a node number threshold, determining the execution cost of the target execution tree structure based on the amount of data read by each predecessor node and the amount of resource consumption of each execution node executing the query statement; when the execution cost of the target execution tree structure is greater than or equal to a resource consumption threshold, performing current limiting processing on the access request based on a current limiting strategy.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, in particular to the field of database technology, and more specifically to a request processing method and device based on a distributed database. Background Art

[0002] In a distributed database system, it is crucial to ensure system stability and reliability when processing query requests. However, in high-concurrency scenarios, if multiple users or applications submit a large number of query requests at the same time, or if some query requests containing complex logic cause the system to be overloaded, the overloaded system may cause increased response delays in the database system or even system crashes, thus affecting the stability and reliability of the database service. Summary of the invention

[0003] In view of this, the present invention provides a request processing method and device based on a distributed database.

[0004] One aspect of the present invention provides a request processing method based on a distributed database, comprising: in response to a user's access request to a distributed database, determining an execution tree structure according to a query statement included in the access request, wherein the execution tree structure includes at least one execution node; in a case where the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, creating a predecessor node for the target execution node in the execution tree structure to obtain a target execution tree structure, wherein the predecessor node is used to read the data to be migrated corresponding to the target execution node; in a case where the number of the predecessor nodes is less than or equal to a node number threshold, determining the execution cost of the target execution tree structure based on the amount of data read by each predecessor node and the amount of resource consumption of each execution node executing the query statement; in a case where the execution cost of the target execution tree structure is greater than or equal to a resource consumption threshold, performing flow limiting processing on the access request based on a flow limiting strategy, wherein the resource consumption threshold is determined based on the remaining amount of resources of the distributed database at the current moment.

[0005] According to an embodiment of the present invention, the method further includes: when the number of the preceding nodes is greater than the node number threshold, performing current limiting processing on the access request based on the current limiting strategy.

[0006] According to an embodiment of the present invention, the above-mentioned determination of the execution cost of the target execution tree structure based on the amount of data read by each of the above-mentioned predecessor nodes and the amount of resource consumption of each of the above-mentioned execution nodes in executing the query statement includes: when the number of the above-mentioned predecessor nodes is less than or equal to the above-mentioned node number threshold, determining the execution cost of the above-mentioned predecessor nodes in executing the data reading operation based on the amount of data read by each of the above-mentioned predecessor nodes and the number of the above-mentioned predecessor nodes; when the execution cost of the above-mentioned predecessor nodes in executing the data reading operation is less than or equal to the above-mentioned resource consumption threshold, determining the execution cost of the above-mentioned target execution tree structure based on the execution cost of the above-mentioned predecessor nodes in executing the data reading operation, the number of the above-mentioned execution nodes and the amount of resource consumption of each of the above-mentioned execution nodes in executing the query operation.

[0007] According to an embodiment of the present invention, the method further includes: when the execution cost of the data reading operation performed by the front node is greater than the resource consumption threshold, performing current limiting processing on the access request based on the current limiting strategy.

[0008] According to an embodiment of the present invention, the current limiting processing of the access request based on the current limiting strategy includes: adding the access request to a waiting queue; or generating an instruction to refuse to execute the access request, so as to perform current limiting processing on the access request.

[0009] According to an embodiment of the present invention, in response to a user's access request to a distributed database, determining the execution tree structure according to the query statement included in the access request includes: parsing the query statement to obtain multiple query statement subsequences and query logic between the multiple query statement subsequences, wherein each of the query statement subsequences represents a query operation or query condition; mapping the query statement subsequences representing the query operations to at least one of the execution nodes respectively; and connecting at least one of the execution nodes into the execution tree structure based on the query logic between the multiple query statement subsequences.

[0010] According to an embodiment of the present invention, in the case where the operation type of the target execution node in at least one of the above-mentioned execution nodes is a data migration operation type, a predecessor node is created for the above-mentioned target execution node in the above-mentioned execution tree structure, and obtaining the target execution tree structure includes: in the case where the operation type of the target execution node in at least one of the above-mentioned execution nodes is a data migration operation type, based on the operation sequence of the data migration operation, determining the creation position of the above-mentioned predecessor node; based on the creation position of the above-mentioned predecessor node, creating the above-mentioned predecessor node for the above-mentioned target execution node in the above-mentioned execution tree structure; establishing a connection relationship between the above-mentioned predecessor node and the above-mentioned target execution node; based on the above-mentioned predecessor node and the above-mentioned connection relationship, updating the above-mentioned execution tree structure to obtain the above-mentioned target execution tree structure.

[0011] According to an embodiment of the present invention, the method further includes: traversing the execution tree structure to obtain the operation type of at least one of the execution nodes; and determining the execution node whose operation type is a data migration operation type as the target execution node.

[0012] According to an embodiment of the present invention, the method further includes: executing the query statement according to the target execution tree structure when the execution cost of the target execution tree structure is less than the resource consumption threshold.

[0013] Another aspect of the present invention provides a request processing device based on a distributed database, comprising: a first determination module, for responding to a user's access request to a distributed database, and determining an execution tree structure according to a query statement included in the access request, wherein the execution tree structure includes at least one execution node; a second determination module, for creating a predecessor node for the target execution node in the execution tree structure to obtain a target execution tree structure when the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, wherein the predecessor node is used to read the data to be migrated corresponding to the target execution node; a third determination module, for determining the execution cost of the target execution tree structure based on the amount of data read by each predecessor node and the amount of resource consumption of each execution node executing the query statement when the number of the predecessor nodes is less than or equal to a node number threshold; a first current limiting module, for performing current limiting processing on the access request based on a current limiting strategy when the execution cost of the target execution tree structure is greater than or equal to a resource consumption threshold, wherein the resource consumption threshold is determined based on the remaining amount of resources of the distributed database at the current moment.

[0014] Another aspect of the present invention provides an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method.

[0015] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above method when executed.

[0016] Another aspect of the present invention provides a computer program product, the computer program product comprising computer executable instructions, and the instructions are used to implement the above method when executed.

[0017] According to an embodiment of the present invention, an execution tree structure is constructed according to the query statement included in the access request, and a predecessor node is created for the target execution node in the execution tree structure, and the number of predecessor nodes is compared with a node number threshold to prevent the system from being overloaded due to too many predecessor nodes; when the number of predecessor nodes meets the conditions, the execution cost of the target execution tree structure is evaluated to reasonably allocate database resources, and when the resource consumption threshold is exceeded, the access request is subjected to current limiting processing based on the current limiting strategy, thereby effectively controlling the access request flow of the database, preventing the access request from consuming a large amount of resources at one time and causing the system to crash, and improving the stability and reliability of the database. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0019] Figure 1 An exemplary system architecture to which a request processing method and apparatus based on a distributed database can be applied according to an embodiment of the present invention is shown.

[0020] Figure 2 A flow chart of a request processing method based on a distributed database according to an embodiment of the present invention is shown.

[0021] Figure 3 A flow chart of a method for determining an execution cost of a target execution tree structure according to an embodiment of the present invention is shown.

[0022] Figure 4 A schematic diagram of an execution tree structure according to a specific embodiment of the present invention is shown.

[0023] Figure 5 A schematic diagram of a target execution tree structure according to a specific embodiment of the present invention is shown.

[0024] Figure 6 A flow chart of a request processing method based on a distributed database according to a specific embodiment of the present invention is shown.

[0025] Figure 7 The diagram shows an architecture for executing a request processing method based on a distributed database according to a specific embodiment of the present invention.

[0026] Figure 8 A block diagram of a request processing device based on a distributed database according to an embodiment of the present invention is shown.

[0027] Fig. 9 A block diagram of an electronic device suitable for implementing a request processing method based on a distributed database according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0028] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0032] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, invention and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present invention provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0034] In a distributed database system, it is crucial to ensure system stability and reliability when processing query requests. However, in high-concurrency scenarios, if multiple users or applications submit a large number of query requests at the same time, or if the system load is too high due to some query requests containing complex logic, it may cause increased response delays in the database system or even system crashes, thus affecting the stability and reliability of the database service.

[0035] In view of this, an embodiment of the present invention constructs an execution tree structure through the query statement included in the access request, creates a predecessor node for the target execution node in the execution tree structure, and compares the number of predecessor nodes with the node number threshold to prevent the system from being overloaded due to too many predecessor nodes; when the number of predecessor nodes meets the conditions, the execution cost of the target execution tree structure is evaluated to reasonably allocate database resources, and when the resource consumption threshold is exceeded, the access request is subjected to current limiting processing based on the current limiting strategy, thereby effectively controlling the access request flow of the database, preventing the access request from consuming a large amount of resources at one time and causing the system to crash, and improving the stability and reliability of the database.

[0036] Specifically, an embodiment of the present invention provides a request processing method based on a distributed database, including: in response to a user's access request to a distributed database, determining an execution tree structure according to a query statement included in the access request; in a case where the operation type of the target execution node in at least one execution node is a data migration operation type, creating a predecessor node for the target execution node in the execution tree structure to obtain a target execution tree structure; in a case where the number of predecessor nodes is less than or equal to a node number threshold, determining the execution cost of the target execution tree structure based on the amount of data read by each predecessor node and the resource consumption of each execution node executing the query statement; in a case where the execution cost of the target execution tree structure is greater than or equal to the resource consumption threshold, performing flow limiting processing on the access request based on a flow limiting strategy.

[0037] It should be noted that the request processing method and device based on a distributed database determined in the embodiment of the present invention can be used in the field of computer technology, such as the field of database technology. The request processing method and device based on a distributed database determined in the embodiment of the present invention can also be used in any field other than the field of computer technology, such as the field of computer science and information technology, the field of cloud computing technology, etc. The application field of the request processing method and device based on a distributed database determined in the embodiment of the present invention is not limited.

[0038] Figure 1 The exemplary system architecture of the request processing method and device based on the distributed database according to the embodiment of the present invention is shown. It should be noted that: Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present invention may be applied, to help those skilled in the art understand the technical content of the present invention, but do not mean that the embodiments of the present invention may not be used in other devices, systems, environments or scenarios.

[0039] like Figure 1 As shown, the system architecture according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, a server 105, and a first database node 106, a second database node 107, and a third database node 108. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0040] The user may use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send access requests, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only for example).

[0041] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0042] The server 105 may be a server that provides various services, such as a background management server (only as an example) that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the access request received from the user, and feed back the processing result (such as the web page, information, or data obtained or generated according to the user request) to the terminal device, such as calling the data in the distributed database as needed. The server may also perform other tasks, such as user authentication, authority management, data processing, etc.

[0043] The server 105 may include a distributed database, which may include a first database node 106, a second database node 107, and a third database node 108. The distributed database stores data in multiple database nodes in a dispersed manner to improve the availability, reliability, and scalability of the data. Each database node may independently process data access requests and communicate and synchronize data with other nodes through a network.

[0044] It should be noted that the request processing method based on a distributed database provided in the embodiment of the present invention can generally be executed by the server 105. Accordingly, the request processing device based on a distributed database provided in the embodiment of the present invention can generally be set in the server 105. The request processing method based on a distributed database provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the request processing device based on a distributed database provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0045] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks, servers and database nodes may be provided according to the implementation requirements.

[0046] Figure 2 A flow chart of a request processing method based on a distributed database according to an embodiment of the present invention is shown.

[0047] like Figure 2 As shown, the method includes operations S201 to S204.

[0048] In operation S201, in response to a user's access request to a distributed database, an execution tree structure is determined according to a query statement included in the access request.

[0049] According to an embodiment of the present invention, a user's access request to a distributed database may include a query statement related to a service that the user needs to handle, and the query statement may be a query statement based on Structured Query Language (SQL).

[0050] According to an embodiment of the present invention, when a distributed database system receives a user's access request to the database, it parses the query statement included in the access request and constructs an execution tree structure according to the semantics of the query statement and query operation information.

[0051] According to an embodiment of the present invention, each execution node included in the execution tree structure can be used to execute a query statement operation, wherein the operation may include data selection, projection, grouping, sorting, connection, aggregation, etc. The connection relationship between each of the multiple execution nodes included in the execution tree structure can be determined based on the execution logic of the query statement.

[0052] In operation S202, when the operation type of the target execution node in at least one execution node is a data migration operation type, a predecessor node is created for the target execution node in the execution tree structure to obtain a target execution tree structure.

[0053] According to an embodiment of the present invention, the target execution node is used to represent an execution node whose operation type is a data migration operation type. The operations performed by the execution node with the data migration operation type may include operations such as connection and aggregation that need to be performed across data nodes.

[0054] According to an embodiment of the present invention, in a distributed database, data is stored on multiple data nodes. When the operation performed by the target execution node involves data transmission across data nodes, if the required data is not on the same data node, it is necessary to create a predecessor node for the target execution node to perform data reading operations on the data to be migrated indicated by the target execution node.

[0055] For example, a target execution node is used to perform a join operation on two tables. If the data of the two tables are stored on two different data nodes, the data of one table needs to be redistributed to the data node where the other table is located, or both tables need to be redistributed to a common data node. In this case, a predecessor node can be created for the target execution node. The predecessor node is used to read the data to be migrated from the original data node, and the required data can be redistributed on different data nodes through the predecessor node and the target execution node.

[0056] According to an embodiment of the present invention, based on a plurality of execution nodes included in the execution tree structure, the created predecessor nodes, and the connection relationship between the nodes, a target tree structure is reconstructed.

[0057] In operation S203, when the number of predecessor nodes is less than or equal to the node number threshold, the execution cost of the target execution tree structure is determined based on the amount of data read by each predecessor node and the amount of resources consumed by each execution node to execute the query statement.

[0058] According to an embodiment of the present invention, since the predecessor node may read a large amount of data, the current limiting rules can be first determined based on the number of predecessor nodes. Specifically, the node number threshold can be determined based on the system's resource limitations, query complexity and other conditions. When the number of predecessor nodes is greater than the node number threshold, the access request can be current limited based on the current limiting strategy. When the number of predecessor nodes is less than or equal to the node number threshold, the execution cost of the target execution tree structure can be further evaluated.

[0059] According to an embodiment of the present invention, the execution cost of the target execution tree structure can be used to measure the cost such as resources and time required to execute the query task corresponding to the query statement. The execution cost of the target execution tree structure may include processor cost, disk scanning cost, communication cost, hardware calculation cost of heterogeneous hardware, etc. For example, the cost of a central processing unit (CPU) calculating a row of data is 0.01, and the cost of scanning the input / output port of a page is 1.0, etc.

[0060] In an embodiment of the present invention, the execution cost of the target execution tree structure can be calculated based on the amount of data read by each predecessor node and the amount of resource consumption by each execution node when performing the operation.

[0061] In operation S204, when the execution cost of the target execution tree structure is greater than or equal to the resource consumption threshold, a flow limiting process is performed on the access request based on the flow limiting policy.

[0062] According to an embodiment of the present invention, the resource consumption threshold can be determined based on the remaining resources of the distributed database at the current moment. When the execution cost of the target execution tree structure is greater than or equal to the resource consumption threshold, it means that the query statement executing the access request may consume too many system resources, thereby affecting the execution of other query statements or the stability of the entire system. At this time, a current limiting strategy can be adopted to limit the access request.

[0063] An embodiment of the present invention constructs an execution tree structure according to the query statement included in the access request, creates a predecessor node for the target execution node in the execution tree structure, and compares the number of predecessor nodes with a node number threshold to prevent the system from being overloaded due to too many predecessor nodes; when the number of predecessor nodes meets the conditions, the execution cost of the target execution tree structure is evaluated to reasonably allocate database resources, and when the resource consumption threshold is exceeded, the access request is subjected to flow limiting processing based on the flow limiting strategy, thereby effectively controlling the access request flow of the database, preventing the access request from consuming a large amount of resources at one time and causing the system to crash, and improving the stability and reliability of the database.

[0064] Figure 3 A flow chart of a method for determining an execution cost of a target execution tree structure according to an embodiment of the present invention is shown.

[0065] like Figure 3 As shown, the method includes operations S301 to S306.

[0066] In operation S301, the number of predecessor nodes is obtained.

[0067] In operation S302, it is determined whether the number of preceding nodes is less than or equal to a node number threshold.

[0068] If yes, then operation S303 is executed to determine the execution cost of the data reading operation performed by the predecessor node based on the amount of data read by each predecessor node and the number of predecessor nodes. If no, then operation S304 is executed to perform current limiting processing on the access request based on the current limiting strategy.

[0069] In operation S305 , it is determined whether the execution cost of the data reading operation performed by the predecessor node is less than or equal to a resource consumption threshold.

[0070] If yes, then operation S306 is executed to determine the execution cost of the target execution tree structure based on the execution cost of the data read operation executed by the predecessor node, the number of execution nodes, and the resource consumption of each execution node executing the query operation. If no, then operation S304 is executed to perform flow limiting processing on the access request based on the flow limiting strategy.

[0071] According to an embodiment of the present invention, when the number of predecessor nodes meets the conditions, it is also possible to determine whether the execution cost of all predecessor nodes exceeds the resource cost threshold based on the amount of data read by each predecessor node in the target execution tree structure. The execution cost of the predecessor node can be determined based on the amount of data performed by each predecessor node and the number of predecessor nodes. For example, the data reading cost of each predecessor node can be added to obtain the execution cost of all predecessor nodes in the target execution tree structure; it is also possible to perform a weighted average based on the amount of data read by each predecessor node, network delay, and communication cost between nodes to obtain the execution cost of all predecessor nodes.

[0072] According to an embodiment of the present invention, when the execution costs of all the predecessor nodes in the target execution tree structure do not exceed the resource cost threshold, the execution cost of the target execution tree structure is calculated again. The execution cost of the target execution tree structure includes not only the execution cost of the predecessor node, but also the execution cost of the remaining execution nodes in the target execution tree structure.

[0073] According to an embodiment of the present invention, the request processing method based on a distributed database also includes: when the remaining resources of the distributed database are sufficient at the current moment, executing the target execution tree structure corresponding to the access request from the user, and feeding back the query result for the access request to the user.

[0074] Before calculating the execution cost of the target tree structure, an embodiment of the present invention pre-determines whether the number of predecessor nodes meets the threshold condition. If the number of predecessor nodes does not meet the condition, the access request is directly limited to prevent individual access requests from occupying large resources due to too many predecessor nodes. If the number of predecessor nodes meets the condition, the execution cost of the predecessor node is determined to reduce the access response time, improve the user experience and achieve more refined resource management.

[0075] According to an embodiment of the present invention, performing current limiting processing on access requests based on a current limiting strategy includes: adding the access request to a waiting queue, or generating an instruction to refuse to execute the access request, so as to perform current limiting processing on the access request.

[0076] According to the embodiment of the present invention, the current limiting strategy may include a direct processing strategy, a concurrency control strategy, a token bucket strategy, etc. In the embodiment of the present invention, the current limiting strategy may also be any other strategy, which is not specifically limited.

[0077] According to an embodiment of the present invention, the direct processing strategy can be configured to directly generate an instruction to refuse to execute the access request, such as by reporting an error or directly exiting, based on the query information included in the query statement of the access request, if it is identified that the access request needs to be directly subjected to flow control processing. In a specific embodiment of the present invention, if the identifiers such as the access address or query keywords of the access request match the identifiers stored in the blacklist, the access request can be directly refused to execute.

[0078] According to an embodiment of the present invention, the concurrency control strategy can be implemented using a concurrency counter. Specifically, each time a query statement of an access request starts to execute, the concurrency counter will be increased by 1, and each time a query statement of an access request is completed, the concurrency counter will be reduced by 1. If the value of the concurrency counter exceeds the preset threshold, it means that the access requests received by the current system exceed the maximum number that can be processed at present, and the access request will be added to the waiting queue until other access requests are executed, so that the count value of the concurrency counter is less than the preset threshold, and there is enough concurrent execution space again.

[0079] According to an embodiment of the present invention, the implementation method of the token bucket strategy may include using a token storage queue to store a fixed number of tokens. In addition, a thread pool is used to periodically generate tokens and store them in the queue. Each time an access request is received, a token is taken out of the queue. If there are tokens in the token storage queue, the access request can be executed. If there are no tokens in the token storage queue, the access request will be added to the waiting queue until there are remaining tokens in the token storage queue again.

[0080] According to an embodiment of the present invention, the waiting queue may include multiple access requests to be executed, and the multiple access requests to be executed may be arranged in sequence in the waiting queue according to the time sequence of initiation. For the multiple access requests to be executed in the waiting queue, the system may periodically check the remaining resources of the distributed database to determine whether there are sufficient remaining resources at the current moment to provide these access requests to be executed.

[0081] The embodiments of the present invention perform flow limiting processing on access requests based on multiple flow limiting strategies. For safe access requests, a buffer space is provided for access requests to be executed, so that the system can still receive access requests during peak loads instead of directly rejecting them, thereby improving the flexibility and scalability of the system and reducing the degradation of user experience caused by directly rejecting requests. At the same time, for access requests for malicious access, execution is directly rejected to improve the response speed of the system and simplify the flow limiting strategy.

[0082] According to an embodiment of the present invention, in response to a user's access request to a distributed database, determining an execution tree structure based on a query statement included in the access request includes: parsing the query statement to obtain multiple query statement subsequences and query logic between the multiple query statement subsequences, wherein each query statement subsequence represents a query operation or query condition; mapping the query statement subsequences representing the query operation to at least one execution node respectively; and connecting at least one execution node into an execution tree structure based on the query logic between the multiple query statement subsequences.

[0083] According to an embodiment of the present invention, by parsing the query statement included in the access request, the text of the query statement can be decomposed into multiple query statement subsequences, and the query logic between the multiple query statement subsequences can be identified. Each query statement subsequence can represent a query operation or query condition. For example, these query statement subsequences can include keywords (such as selection operation SELECT, connection operation JOIN, filter condition WHERE), identifiers (such as table name, column name), characters (such as numbers, strings), etc.

[0084] According to an embodiment of the present invention, query statement subsequences representing query operations are respectively mapped to multiple execution nodes, and multiple execution nodes are connected according to the query logic between the query statement subsequences to construct an execution tree structure. Each execution node can be used to execute a query operation, such as a table scan operation, an index search operation, a join operation, an aggregation operation, etc. The connection relationship between multiple execution nodes can be used to represent the execution order and dependency relationship of the operation, and the execution order and dependency relationship of the operation can be determined based on the query logic.

[0085] According to a specific embodiment of the present invention, the query statement included in the access request can also be parsed by converting the text of the query statement into a tree data structure to obtain a query tree structure, and based on the query tree structure, an execution tree structure can be obtained. The query tree structure can be used to structure the query syntax in the query statement. Specifically, each query node on the query tree structure can represent a query operation or a query condition. Multiple query nodes are connected based on the grammatical rules of the query statement to construct a query tree structure. In the query tree structure, the root query node can represent the entire query statement, and each sub-query node represents a query statement subsequence. The connection relationship between each query node can be used to represent the query logic between each of the multiple query statement subsequences. The query tree structure is traversed, and based on multiple execution strategies, the query nodes used to execute the query operation are reconstructed to obtain an execution tree structure including multiple execution nodes.

[0086] Reference below Figure 4~Figure 5, the construction method of the execution tree structure is further explained in conjunction with a specific embodiment.

[0087] Figure 4 A schematic diagram of an execution tree structure according to a specific embodiment of the present invention is shown.

[0088] For a distributed database, sales data of different regions are stored on multiple database nodes. If the query statement included in the access request is "SELECT SUM(amount) AS total_sales FROM sales WHEREregion IN ('East', 'West')", the query statement means: select data with region names "East" and "West" from the data table named "sales", calculate the sum of the "amount" field values ​​in these data, and then name this sum "total_sales". Figure 4 As shown, the execution tree structure obtained based on the query statement includes execution node 1 to execution node 5.

[0089] The above query statement is parsed to obtain multiple query statement subsequences, including: an operation subsequence "SELECTSUM(amount) AS total_sales", a query condition subsequence "FROM sales", and a filter condition subsequence "WHEREregion IN ('East', 'West')".

[0090] Specifically, execution node 1 is used as the root node of the execution plan, and is used to indicate that the execution plan of the query statement starts from this node.

[0091] For the operation subsequence "SELECT SUM(amount) AS total_sales", the subsequence can be mapped to execution node 2, where execution node 2 is a child node of execution node 1. Execution node 2 can represent "AGGREGATE(SUM(amount) AS total_sales)", that is, calculate the sum of the amount field and name the result total_sales, where "AGGREGATE" represents an aggregation operation, and the field can be converted based on the subsequence "SUM" in query node 2.

[0092] Execution nodes 4 and 5 are child nodes of execution node 2. Based on the query condition subsequence "FROMsales" and the filter condition subsequence "WHERE region IN ('East', 'West')", a scan operation is generated to implement the filter condition, that is, "FROM sales" and "region IN 'East'" are mapped to execution node 4 that performs the scan operation, which is used to scan the "sales" data table stored on the database node in the eastern region. "FROM sales" and "region IN 'West'" are mapped to execution node 5 that performs the scan operation, which is used to scan the "sales" data table stored on the database node in the western region.

[0093] In this embodiment, an execution node 3 for outputting the final calculation result (ie, total_sales) to the user and an execution node 1 as the root node of the execution tree structure are also included. Among them, the execution node 3 is a child node of the execution node 1, and is used to output the final calculation result (ie, total_sales) to the user.

[0094] Based on the query logic between multiple query statement subsequences, execution node 4 and execution node 5 are used as child nodes of execution node 3, and execution node 2 and execution node 3 are used as child nodes of execution node 1. The following is obtained by connecting them: Figure 4 The execution tree structure shown.

[0095] The embodiments of the present invention decompose a query statement into multiple subsequences to construct an execution tree structure. Based on the dependencies and execution order between multiple operations in the query statement, it is easier to process complex access requests, which is beneficial to improving the calculation efficiency of subsequent execution costs and the execution efficiency of current limiting operations.

[0096] According to an embodiment of the present invention, the request processing method based on a distributed database also includes: traversing the execution tree structure to obtain the operation type of at least one execution node; and determining the execution node whose operation type is a data migration operation type as the target execution node.

[0097] According to an embodiment of the present invention, in a distributed database system, since data of different partitions or shards may be stored on different database nodes, during the execution of a query statement, the execution tree structure can be traversed to identify an execution node whose operation type is a data migration operation type. Specifically, during the traversal of the execution tree structure, each execution node can be accessed, and the execution node whose operation type is a data migration operation type can be identified by checking whether the operation type identifier of the execution node matches the preset operation type identifier, and the execution node can be used as the target execution node. Among them, the operations used to characterize the data migration operation type may include operations such as join, aggregation, and grouping.

[0098] According to an embodiment of the present invention, when the operation type of the target execution node in at least one execution node is a data migration operation type, a predecessor node is created for the target execution node in the execution tree structure, and obtaining the target execution tree structure includes: when the operation type of the target execution node in at least one execution node is a data migration operation type, based on the operation order of the data migration operation, determining the creation position of the predecessor node; based on the creation position of the predecessor node, creating a predecessor node for the target execution node in the execution tree structure; establishing a connection relationship between the predecessor node and the target execution node; based on the predecessor node and the connection relationship between the predecessor node and the target execution node, updating the execution tree structure to obtain the target execution tree structure.

[0099] According to an embodiment of the present invention, the front-end node may be responsible for executing actual data transmission operations, for example, reading data from a source database node, sending data to other database nodes through a network, and the like.

[0100] According to an embodiment of the present invention, when the target execution node is determined, the created predecessor node can be placed before the target execution node that needs to read data from other database nodes, or the predecessor node can be placed after the target execution node that needs to send data to other database nodes.

[0101] According to an embodiment of the present invention, after creating a predecessor node, it is necessary to establish a connection relationship between the predecessor node and the target execution node. Among them, these connection relationships can be used to specify the path of data migration, for example, which database nodes the predecessor node needs to perform data reading operations on. Based on the connection relationship between the predecessor node and the target execution node and the creation position of the predecessor node, the execution tree structure is updated to obtain the target execution tree structure.

[0102] Figure 5 A schematic diagram of a target execution tree structure according to a specific embodiment of the present invention is shown.

[0103] like Figure 5 As shown, based on Figure 4 The execution tree structure shown in the figure and the target execution tree structure obtained by updating the created predecessor node are as follows Figure 5 As shown, it includes execution nodes 1 to 5 and a preceding node.

[0104] Specifically, since execution node 2 represents "AGGREGATE (SUM(amount) AS total_sales)", where "AGGREGATE" represents an aggregation operation, and the operation type of the aggregation operation is a data migration type, execution node 2 can be used as the target execution node, and a predecessor node is created at the child node of execution node 2. The predecessor node is used to read the data of "sales" in the eastern region and the data of "sales" in the western region scanned by execution node 3 and execution node 4, or to move the data of "sales" in the eastern region and the data of "sales" in the western region scanned by execution node 3 and execution node 4 to the same node. The predecessor node is connected to execution node 2 at the previous level, and to execution node 4 and execution node 5 at the next level, so as to use execution node 2 to perform an aggregation operation on the data in the data table of "sales" in the eastern region and the data in the data table of "sales" in the western region.

[0105] The embodiments of the present invention create a predecessor node for the target execution node, and read, transmit, and redistribute data stored on different database nodes based on the predecessor node and the target execution node to optimize the data transmission volume during the query process, thereby reducing network latency, improving query response speed, and expanding the data processing capabilities of the distributed database system.

[0106] According to an embodiment of the present invention, a request processing method based on a distributed database also includes: performing hash calculation on the query operation information and / or query condition information included in each of the multiple query nodes in the query tree structure to obtain a hash calculation result; determining the hash calculation result as the query identification value of the query tree structure; matching the query identification value with the multiple query identification values ​​included in the current limiting rule list to obtain a query matching result; and when the query matching result representation matches, performing current limiting processing on the access request based on the current limiting strategy.

[0107] According to an embodiment of the present invention, for the same access request, even if the query statement text input is different in case, space, field order, etc. or parameters are different, the query tree structure generated after standardization is the same, so the query statement can be defined using the query tree structure. However, since the amount of comparison operations between multiple query tree structures is large, the query operation information and / or query condition information included in each of the multiple query nodes in the query tree structure can be hashed to generate a hash calculation result, and the hash calculation result is determined as the query identification value of the query tree structure. The generated query identification value can be associated with information such as user name, timestamp, etc. and stored in a query identification table, and the query identification value is marked based on the user name and access address, and the query identification value of the access request initiated by the user or the user with malicious access or potential security risks is added to the current limiting rule list. In response to receiving a new access request, the query statement included in the access request is parsed, and a query identification value is generated, and the query identification value is matched with multiple historical query identification values ​​included in the current limiting rule list. When the query matching result characterizes a match, the access request is directly subjected to current limiting processing based on the current limiting strategy.

[0108] According to an embodiment of the present invention, when the execution cost of the target execution tree structure is less than the resource consumption threshold, after executing the operation structure corresponding to the query statement according to the target execution tree structure, the request processing method based on the distributed database also includes: obtaining the actual amount of resources consumed by executing the operation structure corresponding to the query statement according to the target execution tree structure; when the actual amount of resources consumed is greater than the resource consumption threshold, updating the current limiting rule list based on the query identification value corresponding to the target execution tree structure to obtain an updated current limiting rule list.

[0109] According to an embodiment of the present invention, after execution is completed, reasonable thresholds can be set for different types of query statements. These preset thresholds can be adjusted according to factors such as system load, historical data, and business needs. The actual resource consumption of the query statement is compared with the preset threshold to determine whether the query statement needs to be limited. When the actual resource consumption of a query statement exceeds the preset threshold, the system automatically triggers the generation of a current limiting rule. For example, the query identification value of the query statement is added to the current limiting rule list. Before a new query statement is executed, the query identification value of the new query statement can be matched with the identification value stored in the current limiting rule list to determine whether the current query statement needs to be limited. If the query matches the current limiting rule, the access request is current limited according to the current limiting strategy, such as delaying execution, refusing execution, or adding the access request to the waiting queue.

[0110] Figure 6A flow chart of a request processing method based on a distributed database according to a specific embodiment of the present invention is shown.

[0111] like Figure 6 As shown, the method includes operations S601 to S608.

[0112] In operation S601, the query statement included in the access request is parsed to obtain a query tree structure.

[0113] In operation S602, a hash calculation is performed on the query tree structure to generate a query identification value.

[0114] In operation S603, it is determined whether the query identification value matches the historical query identification value in the current limiting rule list. If yes, operation S604 is executed to perform current limiting processing on the access request based on the current limiting strategy. If no, operation S605 is executed.

[0115] In operation S605 , based on the query tree structure, a target execution tree structure is determined.

[0116] In operation S606, it is determined whether the number of preceding nodes in the target execution tree structure exceeds the node number threshold. If yes, operation S604 is executed to perform current limiting processing on the access request based on the current limiting strategy. If no, operation S607 is executed.

[0117] In operation S607, it is determined whether the execution cost of the target execution tree exceeds the resource consumption threshold. If yes, operation S604 is executed to perform flow limiting processing on the access request based on the flow limiting strategy. If no, operation S608 is executed.

[0118] In operation S608, the access request is executed based on the target execution tree structure.

[0119] Before operation S607, it may also include determining whether the execution cost of the data reading operation executed by the front node exceeds the resource consumption threshold, if yes, then executing operation S604 to perform current limiting processing on the access request based on the current limiting strategy. If no, then executing operation S607.

[0120] After operation S608, it may also include obtaining the actual amount of resources consumed by executing the access request, and determining whether the actual amount of resources consumed exceeds a preset threshold. If yes, the query identification value of the query statement included in the access request is added to the current limiting rule list to update the current limiting rule list.

[0121] Figure 7 The diagram shows an architecture for executing a request processing method based on a distributed database according to a specific embodiment of the present invention.

[0122] like Figure 7As shown, in a distributed database, a coordinator can be used to parse, optimize, aggregate data and manage resources for query statements, wherein the coordinator may include a parser, an optimizer, an executor and a current limiter. The parser may be used to parse the query statement to obtain a query tree structure, the current limiter may be used to store a current limiting rule list and a current limiting strategy, and perform a hash operation on the query tree structure to generate a query identification value as an index for retrieving the current limiting rule list. In the process of processing the query statement, the current limiter interface may be called through a hook function to perform current limiting judgment and current limiting processing. For example, whether the query statement is allowed to continue to execute, or to perform rate limiting, queuing and other operations. The optimizer may be used to convert the query tree structure into an execution tree structure, and based on the operation type of the execution node, generate a pre-node, and update the optimal target execution tree structure. The executor may be used to execute the operation structure corresponding to the query statement according to the target execution tree structure, and output the final result to the user. In addition, the executor may also be used to count the actual resource consumption in the process of executing the operation structure corresponding to the query statement according to the target execution tree structure.

[0123] like Figure 7 As shown, the database node can be used to store data in the database, which may be distributed on multiple database nodes to achieve distributed storage and parallel processing of data. In addition, the database node can also be used to perform corresponding data access, calculation and other operations according to the target execution tree structure from the coordinator, and return the execution results to the coordinator for summary.

[0124] According to an embodiment of the present invention, the query types of different query statements will affect the optimizer to generate different execution plan types. For example, in a distributed database system, according to the attribute category of the query statement and the characteristics of distributed storage, the optimizer may choose different execution strategies such as directly pushing down the query statement, pushing down the execution plan, or pulling up the data. The selection of these strategies directly affects the generation and execution mode of the execution plan. Among them, the execution plan can be implemented based on the target execution tree structure.

[0125] According to an embodiment of the present invention, the query statement push-down method can be used to indicate that the query tree structure obtained by parsing the query statement by the parser is directly pushed down to the database node for execution. Since this method does not generate a complete execution plan, it is impossible to use the current limiting rules that depend on the cost information in the execution plan. However, current limiting rules that do not depend on the execution plan can be used, such as determining whether the query identification value matches the historical query identification value in the query rule list, or determining whether the actual resource consumption value meets the preset threshold after the execution is completed.

[0126] According to an embodiment of the present invention, the execution plan push-down method can be used to indicate that the query tree structure is obtained by parsing the query statement through the parser, and then converted into a complete target execution tree structure through the optimizer, and pushed down to the database node for execution. The execution plan can include multiple execution node information and execution cost information, so the flow limiting rules that depend on the execution plan can be used, and the flow limiting rules that do not depend on the execution plan can also be used.

[0127] According to an embodiment of the present invention, the data pulling method can be used to indicate that the query tree structure is obtained by parsing the query statement through the parser, and the data is pulled from the database node to the coordinator for calculation through the coordinator. This method can use the flow limiting rules that depend on the execution plan, or can also use the flow limiting rules that do not depend on the execution plan.

[0128] Figure 8 A block diagram of a request processing device based on a distributed database according to an embodiment of the present invention is shown.

[0129] like Figure 8 As shown, the request processing device based on the distributed database includes a first determination module 810 , a second determination module 820 , a third determination module 830 and a first current limiting module 840 .

[0130] The first determination module 810 is used to determine the execution tree structure in response to a user's access request to the distributed database according to a query statement included in the access request.

[0131] The second determination module 820 is configured to create a preceding node for the target execution node in the execution tree structure to obtain a target execution tree structure when the operation type of the target execution node in at least one execution node is a data migration operation type.

[0132] The third determination module 830 is used to determine the execution cost of the target execution tree structure based on the amount of data read by each predecessor node and the resource consumption of each execution node executing the query statement when the number of predecessor nodes is less than or equal to the node number threshold.

[0133] The first current limiting module 840 is used to perform current limiting processing on the access request based on the current limiting strategy when the execution cost of the target execution tree structure is greater than or equal to the resource consumption threshold.

[0134] According to an embodiment of the present invention, the request processing device based on a distributed database further includes a second current limiting module.

[0135] The second current limiting module is used to perform current limiting processing on the access request based on the current limiting strategy when the number of the front nodes is greater than the node number threshold.

[0136] According to the embodiment of the present invention, the third determination module 830 includes a first determination unit and a second determination unit.

[0137] The first determining unit is used to determine the execution cost of the data reading operation performed by the predecessor node based on the amount of data read by each predecessor node and the number of predecessor nodes when the number of predecessor nodes is less than or equal to the node number threshold.

[0138] The second determination unit is used to determine the execution cost of the target execution tree structure based on the execution cost of the predecessor node to execute the data read operation, the number of execution nodes, and the resource consumption of each execution node to execute the query operation when the execution cost of the predecessor node to execute the data read operation is less than or equal to the resource consumption threshold.

[0139] According to an embodiment of the present invention, the third determining module 830 further includes a first current limiting unit.

[0140] The first current limiting unit is used to perform current limiting processing on the access request based on the current limiting strategy when the execution cost of the data reading operation performed by the front node is greater than the resource consumption threshold.

[0141] According to an embodiment of the present invention, the first current limiting module 840 includes a current limiting processing unit.

[0142] The current limiting processing unit adds the access request to the waiting queue; or generates an instruction to refuse to execute the access request to perform current limiting processing on the access request.

[0143] According to an embodiment of the present invention, the first determination module 810 includes a parsing unit, a mapping processing unit, and a third determination unit.

[0144] The parsing unit is used to parse the query statement to obtain multiple query statement subsequences and query logic between the multiple query statement subsequences.

[0145] The mapping processing unit is used to map the query statement subsequences representing the query operation to at least one execution node respectively.

[0146] The third determining unit is used to connect at least one execution node into an execution tree structure based on the query logic between the multiple query statement subsequences.

[0147] According to an embodiment of the present invention, the second determining module 820 includes a fourth determining unit, a creating unit, a connecting unit, and an updating unit.

[0148] The fourth determining unit is used to determine the creation position of the predecessor node based on the operation sequence of the data migration operation when the operation type of the target execution node in at least one execution node is a data migration operation type.

[0149] The creation unit is used to create a predecessor node for a target execution node in the execution tree structure based on the creation position of the predecessor node.

[0150] The connection unit is used to establish a connection relationship between the predecessor node and the target execution node.

[0151] The updating unit is used to update the execution tree structure based on the preceding nodes and the connection relationship to obtain the target execution tree structure.

[0152] According to an embodiment of the present invention, the request processing device based on a distributed database further includes a traversal processing module and a fourth determination module.

[0153] The traversal processing module is used to traverse the execution tree structure to obtain the operation type of at least one execution node.

[0154] The fourth determination module determines an execution node whose operation type is a data migration operation type as a target execution node.

[0155] According to an embodiment of the present invention, the request processing device based on a distributed database further includes an execution module.

[0156] The execution module is used to execute the query statement according to the target execution tree structure when the execution cost of the target execution tree structure is less than the resource consumption threshold.

[0157] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a proper combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be executed.

[0158] For example, any multiple of the first determination module 810, the second determination module 820, the third determination module 830 and the first current limiting module 840 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present invention, at least one of the first determination module 810, the second determination module 820, the third determination module 830 and the first current limiting module 840 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by a suitable combination of any of them. Alternatively, at least one of the first determination module 810 , the second determination module 820 , the third determination module 830 and the first current limiting module 840 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be executed.

[0159] It should be noted that the request processing device part based on a distributed database in the embodiment of the present invention corresponds to the request processing method part based on a distributed database in the embodiment of the present invention. The description of the request processing device part based on a distributed database specifically refers to the request processing method part based on a distributed database, which will not be repeated here.

[0160] Fig. 9 A block diagram of an electronic device suitable for implementing a request processing method based on a distributed database according to an embodiment of the present invention is shown. Fig. 9 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0161] like Fig. 9As shown, the electronic device according to an embodiment of the present invention includes a processor 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory ROM 902 or the program loaded from the storage part 908 to the random access memory RAM 903. The processor 901 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include an on-board memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0162] In RAM 903, various programs and data required for the operation of the electronic device are stored. The processor 901, ROM 902 and RAM 903 are connected to each other via a bus 904. The processor 901 performs various operations of the method flow according to the embodiment of the present invention by executing the programs in ROM 902 and / or RAM 903. It should be noted that the program can also be stored in one or more memories other than ROM 902 and RAM 903. The processor 901 can also perform various operations of the method flow according to the embodiment of the present invention by executing the programs stored in the one or more memories.

[0163] According to an embodiment of the present invention, the electronic device may further include an input / output (I / O) interface 905, which is also connected to the bus 904. The electronic device may further include one or more of the following components connected to the input / output (I / O) interface 905: an input portion 906 including a keyboard, a mouse, etc.; an output portion 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 908 including a hard disk, etc.; and a communication portion 909 including a network interface card such as a LAN card, a modem, etc. The communication portion 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the input / output (I / O) interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage portion 908 as needed.

[0164] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0165] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0166] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device.

[0167] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 902 and / or the RAM 903 described above and / or one or more memories other than the ROM 902 and the RAM 903 .

[0168] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code for executing the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the request processing method based on a distributed database provided by the embodiment of the present invention.

[0169] When the computer program is executed by the processor 901, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0170] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 909, and / or installed from a removable medium 911. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0171] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All these combinations and / or combinations fall within the scope of the present invention.

[0173] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A request processing method based on a distributed database, characterized in that: The method comprises: In response to a user's access request to a distributed database, determining an execution tree structure according to a query statement included in the access request, wherein the execution tree structure includes at least one execution node; In a case where the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, creating a predecessor node for the target execution node in the execution tree structure to obtain a target execution tree structure, wherein the predecessor node is used to read the data to be migrated corresponding to the target execution node; When the number of the predecessor nodes is less than or equal to the node number threshold, determining the execution cost of the target execution tree structure based on the amount of data read by each of the predecessor nodes and the amount of resources consumed by each of the execution nodes to execute the query statement; When the execution cost of the target execution tree structure is greater than or equal to a resource consumption threshold, the access request is subjected to current limiting processing based on a current limiting strategy, wherein the resource consumption threshold is determined based on the remaining amount of resources of the distributed database at the current moment.

2. The method according to claim 1, characterized in that The method further comprises: When the number of the preceding nodes is greater than the node number threshold, the access request is subjected to current limiting processing based on the current limiting strategy.

3. The method according to claim 1, characterized in that: The determining the execution cost of the target execution tree structure based on the amount of data read by each of the preceding nodes and the amount of resources consumed by each of the execution nodes in executing the query statement comprises: When the number of the predecessor nodes is less than or equal to the node number threshold, determining the execution cost of the data read operation performed by the predecessor node based on the amount of data read by each of the predecessor nodes and the number of the predecessor nodes; When the execution cost of the predecessor node performing the data reading operation is less than or equal to the resource consumption threshold, the execution cost of the target execution tree structure is determined based on the execution cost of the predecessor node performing the data reading operation, the number of the execution nodes, and the resource consumption of each of the execution nodes performing the query operation.

4. The method according to claim 3, characterized in that The method further comprises: When the execution cost of the data reading operation performed by the front node is greater than the resource consumption threshold, the access request is subjected to current limiting processing based on the current limiting strategy.

5. The method according to claim 1, 2 or 4, characterized in that: The performing current limiting processing on the access request based on the current limiting strategy includes: adding the access request to a waiting queue; or, An instruction for refusing to execute the access request is generated to perform current limiting processing on the access request.

6. The method according to claim 1, characterized in that In response to a user's access request to a distributed database, determining the execution tree structure according to a query statement included in the access request includes: Parsing the query statement to obtain a plurality of query statement subsequences and query logic between the plurality of query statement subsequences, wherein each query statement subsequence represents a query operation or a query condition; Mapping query statement subsequences representing the query operation to at least one of the execution nodes respectively; Based on the query logic between the plurality of query statement subsequences, at least one of the execution nodes is connected to form the execution tree structure.

7. The method according to claim 1, characterized in that In the case where the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, creating a predecessor node for the target execution node in the execution tree structure to obtain the target execution tree structure includes: In a case where the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, determining a creation position of the predecessor node based on an operation sequence of the data migration operation; Creating the predecessor node for the target execution node in the execution tree structure based on the creation position of the predecessor node; Establishing a connection relationship between the preceding node and the target execution node; Based on the preceding node and the connection relationship, the execution tree structure is updated to obtain the target execution tree structure.

8. The method according to claim 1, characterized in that The method further comprises: Traversing the execution tree structure to obtain an operation type of at least one execution node; An execution node whose operation type is a data migration operation type is determined as a target execution node.

9. The method according to claim 1, characterized in that: The method further comprises: When the execution cost of the target execution tree structure is less than the resource consumption threshold, the query statement is executed according to the target execution tree structure.

10. A request processing device based on a distributed database, characterized in that: The device comprises: A first determination module, configured to determine, in response to a user's access request to a distributed database, an execution tree structure according to a query statement included in the access request, wherein the execution tree structure includes at least one execution node; A second determination module, when the operation type of the target execution node in at least one of the execution nodes is a data migration operation type, creates a predecessor node for the target execution node in the execution tree structure to obtain a target execution tree structure, wherein the predecessor node is used to read the data to be migrated corresponding to the target execution node; A third determination module is used to determine the execution cost of the target execution tree structure based on the amount of data read by each of the predecessor nodes and the amount of resources consumed by each of the execution nodes to execute the query statement when the number of the predecessor nodes is less than or equal to the node number threshold; The first current limiting module is used to perform current limiting processing on the access request based on the current limiting strategy when the execution cost of the target execution tree structure is greater than or equal to the resource consumption threshold, wherein the resource consumption threshold is determined based on the remaining resources of the distributed database at the current moment.

Images