Database cluster data processing method, device, system, medium and electronic device

Abstract

The present disclosure relates to a database cluster data processing method, device, system, medium and electronic device, wherein the database cluster includes at least one computing node and multiple storage nodes, and the method is applied to the computing node, including: generating an execution plan based on a data query request, wherein the data query request comes from a client, and the execution plan is used to instruct the storage node to perform a data query operation; sending the execution plan to the storage node; receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result; and sending the data query result to the client. In the implementation scheme of the present disclosure, the computing pressure is partially balanced to the storage node, so as to achieve a certain degree of distributed computing, relieve the pressure of the computing node, and improve the data processing speed.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of database technology, and in particular to a database cluster data processing method, a database cluster data processing method device, a database system, and a computer-readable storage medium and an electronic device for implementing the database cluster data processing method. Background Art

[0002] The open source distributed relational database supports unlimited horizontal expansion and has strong consistency and high availability. It is suitable for application scenarios with high availability, high requirements for strong consistency, and large data scale.

[0003] With the rapid growth of data scale, in order to provide long-term and stable services, the related technology has proposed an open source distributed relational database cluster solution. The cluster usually contains three basic components: computing nodes, storage nodes, and management nodes to form a database service. The management node is the management module of the entire cluster, and its main tasks are: storage cluster metadata, scheduling, and load balancing. The storage node is responsible for storing data. The computing node itself does not store data, but is only responsible for computing, such as receiving SQL requests and processing SQL-related logic. It can find the address of the storage node where the data required for calculation is located through the management node, and then pull the required data from the corresponding storage node for calculation.

[0004] However, current database clusters have some problems, such as all computing pressure is concentrated on computing nodes and the data processing speed needs to be further improved. Summary of the invention

[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a database cluster data processing method, a database cluster data processing device, a database system, and a computer-readable storage medium and an electronic device for implementing the database cluster data processing method.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for processing data in a database cluster, wherein the database cluster includes at least one computing node and multiple storage nodes, and the method is applied to the computing node, including:

[0007] Generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation;

[0008] Sending the execution plan to the storage node;

[0009] Receiving an operation result obtained by the storage node performing a data query operation to obtain a data query result;

[0010] The data query result is sent to the client.

[0011] Optionally, in some embodiments of the present disclosure, the receiving an operation result obtained by the storage node performing a data query operation to obtain a data query result includes:

[0012] Receiving a table connection operation result returned by the storage node;

[0013] A data query result is generated based on the table connection operation result returned by each storage node.

[0014] Optionally, in some embodiments of the present disclosure, the method further includes:

[0015] Determine a first target storage node and a second target storage node from the plurality of storage nodes;

[0016] The sending the execution plan to the storage node includes:

[0017] Sending the execution plan to the first target storage node and the second target storage node, so that the first target storage node copies the first target data table stored in it to the second target storage node based on the execution plan, and the second target storage node performs a table join operation on the second target data table originally stored in it and the first target data table based on the execution plan;

[0018] The receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result includes:

[0019] receiving a table connection operation result returned by the second target storage node;

[0020] The data query result is generated based on the table connection operation result.

[0021] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold.

[0022] In a second aspect, an embodiment of the present disclosure provides a method for processing data in a database cluster, wherein the database cluster includes at least one computing node and multiple storage nodes. The method is applied to the storage node and includes:

[0023] receiving an execution plan sent by the computing node, wherein the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client;

[0024] Execute the data query operation according to the execution plan to obtain the operation result;

[0025] The operation result is returned to the computing node, so that the computing node obtains a data query result based on the operation result, and returns the data query result to the client.

[0026] Optionally, in some embodiments of the present disclosure, executing the data query operation according to the execution plan to obtain the operation result includes:

[0027] Determine a target data table according to the execution plan;

[0028] Performing a table join operation on the target data table to obtain a table join operation result;

[0029] The returning the operation result to the computing node so that the computing node obtains the data query result based on the operation result includes:

[0030] The table connection operation result is returned to the computing node, so that the computing node generates a data query result based on the table connection operation result returned by each storage node.

[0031] Optionally, in some embodiments of the present disclosure, the method further includes:

[0032] Determine a first target storage node from the plurality of storage nodes;

[0033] According to the execution plan, the first target storage node synchronizes the first target data table copied by the first target storage node to other storage nodes among the multiple storage nodes except the first target storage node;

[0034] The performing of the data query operation according to the execution plan to obtain the operation result includes:

[0035] For each storage node, based on the execution plan, performing a table join operation on the originally stored second target data table and the first target data table to obtain a table join operation result;

[0036] The returning the operation result to the computing node so that the computing node obtains the data query result based on the operation result includes:

[0037] The table connection operation result is returned to the computing node, so that the computing node generates the data query result based on the table connection operation result returned by each storage node.

[0038] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold.

[0039] In a third aspect, an embodiment of the present disclosure provides a database cluster data processing device, wherein the database cluster includes at least one computing node and multiple storage nodes, and the device is applied to the computing node, including:

[0040] A plan generation module, used to generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation;

[0041] A plan sending module, used for sending the execution plan to the storage node;

[0042] A result determination module, used for receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result;

[0043] A result feedback module is used to send the data query result to the client.

[0044] In a fourth aspect, an embodiment of the present disclosure provides a database cluster data processing device, wherein the database cluster includes at least one computing node and multiple storage nodes, and the device is applied to the storage node, including:

[0045] A plan receiving module, used for receiving an execution plan sent by the computing node, wherein the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client;

[0046] A data processing module, used to execute a data query operation according to the execution plan to obtain an operation result;

[0047] The result feedback module is used to return the operation result to the computing node, so that the computing node obtains the data query result based on the operation result, and returns the data query result to the client.

[0048] In a fifth aspect, an embodiment of the present disclosure provides a database system, comprising at least one computing node and multiple storage nodes:

[0049] The computing node is used to generate an execution plan based on a data query request and send the execution plan to the storage node, wherein the data query request comes from a client;

[0050] The storage node is used to perform a data query operation based on the execution plan, obtain an operation result, and return the operation result to the computing node;

[0051] The computing node is further configured to obtain a data query result based on the operation result, and send the data query result to the client.

[0052] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the database cluster data processing method described in any of the above embodiments.

[0053] In a seventh aspect, an embodiment of the present disclosure provides an electronic device, including:

[0054] Processor; and

[0055] Memory for storing computer programs;

[0056] The processor is configured to execute the steps of the database cluster data processing method described in any of the above embodiments by executing the computer program.

[0057] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0058] The database cluster data processing method, device, database system, storage medium and electronic device provided by the embodiments of the present disclosure, the computing node generates an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct the storage node to perform a data query operation; the execution plan is sent to the storage node, and the operation result obtained by the storage node performing the data query operation is received to obtain the data query result; the data query result is sent to the client. In this way, in the scheme of this embodiment, the storage node performs data query operation processing, and then the computing node determines the final data query result based on the operation result returned by the storage node, so that the computing pressure can be at least partially balanced to the storage node, and the computing power of the storage node is used to realize distributed computing, thereby alleviating the pressure on the computing node, and further improving the data processing speed of the database cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 This is a flow chart of a database cluster data processing method according to an embodiment of the present disclosure;

[0062] Figure 2 This is a schematic diagram of the data processing process of the database cluster according to the embodiment of the present disclosure;

[0063] Figure 3 This is a flow chart of a database cluster data processing method according to another embodiment of the present disclosure;

[0064] Figure 4 This is a schematic diagram of a database cluster data processing device according to an embodiment of the present disclosure;

[0065] Figure 5 This is a schematic diagram of a database cluster data processing device according to another embodiment of the present disclosure;

[0066] Figure 6 This is a schematic diagram of a database system according to an embodiment of the present disclosure;

[0067] Figure 7 A schematic diagram of an electronic device for implementing the database cluster data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0070] It should be understood that, in the following, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0071] Figure 1 This is a flow chart of a database cluster data processing method according to an embodiment of the present disclosure. The database cluster includes at least one computing node and multiple storage nodes. The method is applied to the computing nodes in the database cluster.

[0072] The method may specifically include the following steps:

[0073] Step S101: Generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation.

[0074] Exemplarily, the client may be a computer, a similar computing device, or an application deployed on a computer. The client may receive a query instruction from a user through an input device, thereby forming a data query request, and sending the data query request to a computing node. The computing node may be, for example, a database server, which generates an execution plan based on the data query request after receiving the data query request sent by the client. The specific generation process of the execution plan may be understood by referring to the relevant technology, and will not be described in detail here. The generated execution plan is used to instruct the storage node to perform a data query operation.

[0075] For example, if there are 100 storage nodes in the database cluster, and the data queried by the data query request is located on, for example, two storage nodes, then these two storage nodes are the target storage nodes. The execution plan may instruct each target storage node to perform a data query operation respectively.

[0076] Step S102: Send the execution plan to the storage node.

[0077] The storage node is, for example, a server for data storage in a database cluster, which can be connected to the computing node in communication. The computing node sends the execution plan to the storage node based on the communication connection relationship between the two.

[0078] In some embodiments, the computing node may send the execution plan to all storage nodes in the database cluster, so that the storage nodes execute the execution plan. In other embodiments, the computing node may only send the execution plan to some storage nodes in the data cluster.

[0079] For example, there are 100 storage nodes in the database cluster, and two storage nodes are determined from the 100 storage nodes as target storage nodes. The computing node sends the execution plan to the two target storage nodes respectively. Then each target storage node executes the data query operation based on the execution plan to obtain the operation result.

[0080] Step S103: receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result.

[0081] After receiving the operation result returned by the storage node (or the target storage node), the computing node can perform data processing on all the received operation results to generate a data query result. Exemplarily, all the operation results can be summarized into a data table, and then the data table is used as the data query result. For example, the operation result returned by each storage node (or the target storage node) can be, for example, a slice of any data table, and the computing node can summarize these slices into a data table to generate a data query result.

[0082] In other embodiments, the operation result returned by each storage node (or target storage node) may be, for example, any data table. After receiving the data tables returned by all storage nodes, the computing node may establish a connection relationship with all received databases, thereby using all the data tables with established connection relationships as data query results.

[0083] Step S104: Send the data query result to the client.

[0084] Specifically, the computing node returns the finalized data query result to the client.

[0085] In the database cluster data processing method of the disclosed embodiment, the computing node generates an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct the storage node to perform a data query operation; the execution plan is sent to the storage node, and the operation result obtained by the storage node performing the data query operation is received to obtain the data query result; the data query result is sent to the client. In this way, in the scheme of this embodiment, the storage node performs data query operation processing, and then the computing node determines the final data query result based on the operation result returned by the storage node, so that the computing pressure can be at least partially balanced to the storage node, and the computing power of the storage node is used to achieve a certain degree of distributed computing, thereby alleviating the pressure on the computing node and improving the overall data processing speed.

[0086] Optionally, in some embodiments of the present disclosure, receiving an operation result obtained by the storage node performing a data query operation in step S103 to obtain a data query result includes:

[0087] Step i): receiving the table connection operation result returned by the storage node.

[0088] Exemplarily, the data to be queried may be located in two or more data tables on the target storage node. Therefore, each target storage node may perform a data query operation by performing a table join operation, that is, joining the associated data in two data tables, and then obtaining the table join operation result, and returning the table join operation result to the computing node, that is, the computing node receives the table join operation result returned by the target storage node.

[0089] Step ii): Generate a data query result based on the table connection operation result returned by each storage node.

[0090] Exemplarily, the computing node may aggregate and generate a final data query result based on the table connection operation result returned by each target storage node.

[0091] In this embodiment, the storage node performs a table join operation on at least two data tables related to the query, and then the computing node determines the final data query result based on the table join operation result returned by the storage node, so that the computing pressure can be at least partially balanced to the storage node, and the computing power of the storage node is used to achieve a certain degree of distributed computing, thereby alleviating the pressure on the computing node and further improving the overall data processing speed. Figure 2 As shown in , the method may also include the following steps:

[0092] Step a): determining a first target storage node and a second target storage node from the plurality of storage nodes.

[0093] Exemplarily, the computing node can determine a first target storage node such as target storage node A and a second target storage node such as target storage node B from multiple storage nodes based on a data query request, that is, the data to be queried is stored on target storage node A and target storage node B.

[0094] Correspondingly, in step S102, sending the execution plan to the storage node includes: sending the execution plan to the first target storage node and the second target storage node, so that the first target storage node copies the first target data table stored therein to the second target storage node based on the execution plan, and enabling the second target storage node to perform a table join operation on the second target data table originally stored therein and the first target data table based on the execution plan.

[0095] Exemplarily, the computing node sends the execution plan to the target storage node A and the target storage node B. Based on the execution plan, the target storage node A copies the first target data table stored in itself, such as Table 1, to the target storage node B outside itself. The target storage node B performs a table join operation on the second target data table stored in itself, such as Table 2', with the first target data table, such as Table 1, to obtain the first table join operation result; and / or, the target storage node B copies the first target data table stored in itself, such as Table 1', to the target storage node A outside itself. The target storage node A performs a table join operation on the second target data table stored in itself, such as Table 2, with the first target data table, such as Table 1', to obtain the second table join operation result.

[0096] Exemplarily, the first target data table may include at least one sub-table (not shown), and in this case, the target storage node A copies each sub-table of the first target data table, such as Table 1, to a target storage node B outside itself.

[0097] In step S103, an operation result obtained by the storage node performing a data query operation is received to obtain a data query result, including: receiving a table connection operation result returned by the second target storage node; and generating the data query result based on the table connection operation result.

[0098] Exemplarily, the computing node receives a first table connection operation result returned by the target storage node B, and / or the computing node receives a second table connection operation result returned by the target storage node A, and the computing node generates a final data query result based on the first table connection operation result and / or the second table connection operation result.

[0099] In this embodiment, each target storage node first performs a table join operation on the data table where the data to be queried is located, and then the computing node determines the final data query result based on the table join operation result returned by the target storage node. In this way, the computing pressure can be at least partially balanced to the storage node, and the computing power of the storage node is utilized to achieve a certain degree of distributed computing, thereby alleviating the pressure on the computing node and further improving the overall data processing speed.

[0100] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold. Exemplarily, the preset data volume threshold may be less than 50M, but is not limited thereto, and may be determined based on, but not limited to, the hardware configuration and performance parameters of the database cluster.

[0101] In some embodiments, the first target storage node, such as the target storage node A, may include at least two target data tables, that is, the data to be queried is at least partially located in at least two target data tables stored on the target storage node A. At this time, when the target storage node A copies the first target data table stored in it to the second target storage node, such as the target storage node B, based on the execution plan, it may choose to copy only the target data table with a small amount of data in the at least two target data tables on the target storage node A as the first target data table to the target storage node B, and then the target storage node B performs a table connection operation based on the first target data table and its own second target data table, and returns the table connection operation result. At the same time, the target storage node A may return the query results of its remaining target data tables, such as the table connection operation result, and the computing node determines the final data query result based on the table connection operation result returned by the target storage node B and the query result returned by the target storage node A.

[0102] It should be noted that the database cluster in the related technology pulls data from one or more target storage nodes where the data to be queried is located to the computing nodes for unified calculation. When the amount of data is large, this increases the time consumption of data transmission and reduces the data processing speed.

[0103] Therefore, in this embodiment, while the computing pressure is at least partially balanced to the storage nodes and the computing power of the storage nodes is utilized to achieve a certain degree of distributed computing, during the specific processing, the target data table with a small amount of data on each or part of the target storage nodes is copied to the remaining target storage nodes. This reduces the overall data transmission time to a certain extent, thereby further improving the data processing speed.

[0104] The embodiment of the present disclosure also provides a database cluster data processing method, wherein the database cluster includes at least one computing node and multiple storage nodes. Figure 3 As shown, the method is applied to a storage node and includes the following steps:

[0105] Step S301: receiving an execution plan sent by the computing node, wherein the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client.

[0106] The storage node and the computing node can be connected in communication. The storage node receives the execution plan sent by the computing node based on the communication connection relationship.

[0107] The computing node sends the generated execution plan to the storage node. The database cluster may include one or more storage nodes. In some embodiments, all storage nodes in the database cluster receive the execution plan. In other embodiments, only some storage nodes in the database cluster may receive the execution plan.

[0108] For example, if there are 100 storage nodes in the cluster, and the data queried by this data query request is located on, for example, two storage nodes, then these two storage nodes are the target storage nodes. The execution plan can instruct each target storage node to perform a data query operation separately. At this time, each target storage node can receive the execution plan sent by the computing node.

[0109] Step S302: Execute a data query operation according to the execution plan to obtain an operation result.

[0110] Exemplarily, for example, if the two storage nodes are determined to be target storage nodes, the computing node sends the execution plan to each target storage node, and then each target storage node executes the data query operation based on the execution plan to obtain the operation result.

[0111] Step S303: Return the operation result to the computing node, so that the computing node obtains a data query result based on the operation result, and returns the data query result to the client.

[0112] Specifically, the computing node determines the final data query result based on the operation result returned by each target storage node, for example, by aggregating the operation result returned by each target storage node to obtain the data query result. The computing node then returns the finalized data query result to the client.

[0113] In the database cluster data processing method of the embodiment of the present disclosure, the storage node receives the execution plan sent by the computing node, executes the data query operation to obtain the operation result, and returns the operation result to the computing node, so that the computing node obtains the data query result based on the operation result, and returns the data query result to the client. In this way, the computing pressure can be at least partially balanced to the storage node, and the computing power of the storage node is used to achieve a certain degree of distributed computing, thereby alleviating the pressure on the computing node and further improving the overall data processing speed.

[0114] Optionally, in some embodiments of the present disclosure, in step S302, performing a data query operation according to the execution plan to obtain an operation result includes: determining a target data table according to the execution plan; performing a table join operation on the target data table to obtain a table join operation result. In step S303, returning the operation result to the computing node so that the computing node obtains a data query result based on the operation result includes: returning the table join operation result to the computing node so that the computing node generates a data query result based on the table join operation result returned by each storage node.

[0115] Exemplarily, the data to be queried may be located in two or more target data tables on the target storage node, so the target data table on the target storage node can be determined first according to the execution plan, for example, the unique identifier of the target data table and other information can be determined. Usually, the data query request carries the identifier of the target storage node where the data to be queried is located and the identifier of the target data table on the target storage node. The execution plan is generated based on the data query request, so the execution plan can also carry the identifier of the target data table, so the target data table can be determined based on the identifier of the target data table in the execution plan. Afterwards, each target storage node performs a data query operation on the determined target data table, which can be a table join operation, that is, for example, the associated data in two target data tables are joined (Join) to obtain the table join operation result, and the table join operation result is returned to the computing node, that is, the computing node receives the table join operation result returned by the target storage node. The computing node can summarize and generate the final data query result based on the table join operation result returned by each target storage node.

[0116] Optionally, in some embodiments of the present disclosure, the method further includes the following steps: determining a first target storage node from the multiple storage nodes; and synchronizing the first target data table copied by the first target storage node to other storage nodes among the multiple storage nodes except the first target storage node according to the execution plan.

[0117] Exemplary, combined Figure 2 As shown, the computing node can determine the first target storage node from multiple storage nodes, such as target storage node A. The computing node sends the execution plan to the target storage node A, and the target storage node A copies and synchronizes the first target data table stored in itself, such as Table 1, to the target storage node B outside itself based on the execution plan.

[0118] Correspondingly, in step S302, a data query operation is performed according to the execution plan to obtain an operation result, including: for each storage node, a table connection operation is performed on the originally stored second target data table and the first target data table based on the execution plan to obtain a table connection operation result.

[0119] Exemplarily, for each storage node such as target storage node A and target storage node B, target storage node B performs a table join operation on a second target data table such as Table 2' stored in itself with a first target data table such as Table 1 based on the execution plan to obtain a first table join operation result. Target storage node A performs a table join operation on a second target data table such as Table 2 stored in itself with a first target data table such as Table 1' based on the execution plan to obtain a second table join operation result.

[0120] In step S303, the operation result is returned to the computing node so that the computing node obtains the data query result based on the operation result, including: returning the table connection operation result to the computing node so that the computing node generates the data query result based on the table connection operation result returned by each storage node.

[0121] Exemplarily, the target storage node B returns the first table connection operation result to the computing node, and the target storage node A returns the second table connection operation result to the computing node, and the computing node generates a final data query result based on the first table connection operation result and the second table connection operation result.

[0122] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold. Exemplarily, the preset data volume threshold may be less than 50M, but is not limited thereto, and may be determined based on, but not limited to, the hardware configuration and performance parameters of the database cluster.

[0123] In some embodiments, the first target storage node, such as the target storage node A, may include at least two target data tables, that is, the data to be queried is at least partially located in at least two target data tables stored on the target storage node A. At this time, when the target storage node A copies the first target data table stored in it to the second target storage node, such as the target storage node B, based on the execution plan, it may choose to copy only the target data table with a small amount of data in the at least two target data tables on the target storage node A as the first target data table to the target storage node B, and then the target storage node B performs a table connection operation based on the first target data table and its own second target data table, and returns the table connection operation result. At the same time, the target storage node A may return the query results of its remaining target data tables, such as the table connection operation result, and the computing node determines the final data query result based on the table connection operation result returned by the target storage node B and the query result returned by the target storage node A.

[0124] It should be noted that the database cluster in the related technology pulls data from one or more target storage nodes where the data to be queried is located to the computing nodes for unified calculation. When the amount of data is large, this increases the time consumption of data transmission and reduces the data processing speed.

[0125] Therefore, in this embodiment, while the computing pressure is at least partially balanced to the storage nodes and the computing power of the storage nodes is utilized to achieve a certain degree of distributed computing, during the specific processing, the target data table with a small amount of data on each or part of the target storage nodes is copied to the remaining target storage nodes. This reduces the overall data transmission time to a certain extent, thereby further improving the data processing speed.

[0126] Although the steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc. In addition, it is also easy to understand that these steps may be, for example, performed synchronously or asynchronously in multiple modules / processes / threads.

[0127] The present disclosure provides a database cluster data processing device, wherein the database cluster includes at least one computing node and multiple storage nodes. The device is applied to the computing node, such as Figure 4 The database cluster data processing device shown includes:

[0128] A plan generation module 401 is used to generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation;

[0129] A plan sending module 402, used to send the execution plan to the storage node;

[0130] A result determination module 403 is used to receive the operation result obtained by the storage node performing the data query operation to obtain the data query result;

[0131] The result feedback module 404 is used to send the data query result to the client.

[0132] Optionally, in some embodiments of the present disclosure, the result determination module 403 is used to receive the table connection operation result returned by the storage node; and generate a data query result based on the table connection operation result returned by each storage node.

[0133] Optionally, in some embodiments of the present disclosure, the device further includes a node determination module, which is used to determine a first target storage node and a second target storage node from the multiple storage nodes. The plan sending module 402 is used to send the execution plan to the first target storage node and the second target storage node, so that the first target storage node copies its stored first target data table to the second target storage node based on the execution plan, and the second target storage node performs a table connection operation on its originally stored second target data table and the first target data table based on the execution plan. The result determination module 403 is used to receive the table connection operation result returned by the second target storage node; and generate the data query result based on the table connection operation result.

[0134] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold.

[0135] The embodiment of the present disclosure also provides a database cluster data processing device, wherein the database cluster includes at least one computing node and multiple storage nodes. The device is applied to the storage nodes, such as Figure 5 The database cluster data processing device shown includes:

[0136] The plan receiving module 501 is used to receive the execution plan sent by the computing node, where the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client.

[0137] The data processing module 502 is used to execute the data query operation according to the execution plan to obtain the operation result.

[0138] The result feedback module 503 is used to return the operation result to the computing node, so that the computing node obtains a data query result based on the operation result, and returns the data query result to the client.

[0139] Optionally, in some embodiments of the present disclosure, the data processing module 502 is used to determine a target data table according to the execution plan; perform a table join operation on the target data table to obtain a table join operation result. The result feedback module 503 is used to return the table join operation result to the computing node, so that the computing node generates a data query result based on the table join operation result returned by each storage node.

[0140] Optionally, in some embodiments of the present disclosure, the device may further include a node determination module and a data synchronization module, wherein the node determination module is used to determine a first target storage node from the multiple storage nodes. The data synchronization module is used to enable the first target storage node to synchronize the first target data table copied by it to other storage nodes among the multiple storage nodes except the first target storage node according to the execution plan. The data processing module 502 is used to perform a table connection operation on the second target data table originally stored and the first target data table based on the execution plan for each storage node to obtain a table connection operation result. The result feedback module 503 is used to return the table connection operation result to the computing node, so that the computing node generates the data query result based on the table connection operation result returned by each storage node.

[0141] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold.

[0142] The present disclosure provides a database system. Figure 6 The database system shown may include at least one computing node 601 and multiple storage nodes 602:

[0143] The computing node 601 is used to generate an execution plan based on a data query request and send the execution plan to the storage node, wherein the data query request comes from a client;

[0144] The storage node 602 is used to perform a data query operation based on the execution plan to obtain an operation result, and return the operation result to the computing node;

[0145] The computing node 601 is further configured to obtain a data query result based on the operation result, and send the data query result to the client.

[0146] Optionally, in some embodiments, the storage node 602 is used to determine a target data table based on the execution plan, perform a table join operation on the target data table to obtain a table join operation result, and return the table join operation result to the computing node 601. The computing node 601 is used to generate a data query result based on the table join operation result returned by each of the storage nodes 602.

[0147] Optionally, in some embodiments of the present disclosure, the computing node 601 is used to determine a first target storage node and a second target storage node from the multiple storage nodes 602. The execution plan is sent to the first target storage node and the second target storage node. The first target storage node is used to copy the first target data table stored in it to the second target storage node based on the execution plan, and the second target storage node is used to perform a table join operation on the second target data table originally stored in it and the first target data table based on the execution plan. The computing node 601 is used to receive the table join operation result returned by the second target storage node, and generate the data query result based on the table join operation result.

[0148] Optionally, in some embodiments of the present disclosure, the computing node 601 is used to determine a first target storage node from the multiple storage nodes. The first target storage node synchronizes the first target data table copied to other storage nodes in the multiple storage nodes except the first target storage node according to the execution plan. Each of the storage nodes 602 performs a table join operation on the originally stored second target data table and the first target data table based on the execution plan, obtains a table join operation result, and returns the table join operation result to the computing node 601. The computing node 601 generates the data query result based on the table join operation result returned by each of the storage nodes 602.

[0149] Optionally, in some embodiments of the present disclosure, the data volume of the first target data table is less than a preset data volume threshold.

[0150] Regarding the devices and systems in the above embodiments, the specific manner in which each module performs operations and the corresponding technical effects brought about have been described in detail in the corresponding method embodiments and will not be elaborated here.

[0151] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiment of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of a module or unit described above can be further divided into multiple modules or units for concretization. The components displayed as modules or units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the disclosed scheme. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0152] The embodiments of the present disclosure further provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the database cluster data processing method described in any of the above embodiments are implemented.

[0153] Exemplarily, the readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0154] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0155] The present disclosure also provides an electronic device, such as Figure 7 The electronic device shown includes a processor 701 and a memory 702, wherein the memory 702 is used to store a computer program. The processor 701 is configured to execute the steps of the database cluster data processing method in any of the above embodiments by executing the computer program.

[0156] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above-mentioned database cluster data processing method according to the implementation of the present disclosure.

[0157] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0158] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A database cluster data processing method, characterized in that: The database cluster includes at least one computing node and multiple storage nodes. The method is applied to the computing node and includes: Generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation; Sending the execution plan to the storage node; Receiving an operation result obtained by the storage node performing a data query operation to obtain a data query result; Sending the data query result to the client; The method further comprises: Determine a first target storage node and a second target storage node from the plurality of storage nodes; The sending the execution plan to the storage node includes: The execution plan is sent to the first target storage node and the second target storage node, so that the first target storage node copies the first target data table stored therein to the second target storage node based on the execution plan, and the second target storage node performs a table join operation on the second target data table originally stored therein and the first target data table based on the execution plan.

2. The data processing method according to claim 1, characterized in that: The receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result includes: Receiving a table connection operation result returned by the storage node; A data query result is generated based on the table connection operation result returned by each storage node.

3. The data processing method according to claim 1, characterized in that: The receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result includes: receiving a table connection operation result returned by the second target storage node; The data query result is generated based on the table connection operation result.

4. The data processing method according to claim 1, characterized in that: The data volume of the first target data table is less than a preset data volume threshold.

5. A database cluster data processing method, characterized in that: The database cluster includes at least one computing node and multiple storage nodes. The method is applied to the storage nodes and includes: receiving an execution plan sent by the computing node, wherein the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client; Execute the data query operation according to the execution plan to obtain the operation result; Returning the operation result to the computing node, so that the computing node obtains a data query result based on the operation result, and returns the data query result to the client; The method further comprises: Determine a first target storage node from the plurality of storage nodes; According to the execution plan, the first target storage node synchronizes the first target data table copied by the first target storage node to other storage nodes among the multiple storage nodes except the first target storage node; The performing of the data query operation according to the execution plan to obtain the operation result includes: for each storage node, performing a table connection operation on the originally stored second target data table and the first target data table based on the execution plan to obtain the table connection operation result.

6. The data processing method according to claim 5, characterized in that: The performing of the data query operation according to the execution plan to obtain the operation result includes: Determine a target data table according to the execution plan; Performing a table join operation on the target data table to obtain a table join operation result; The returning the operation result to the computing node so that the computing node obtains the data query result based on the operation result includes: The table connection operation result is returned to the computing node, so that the computing node generates a data query result based on the table connection operation result returned by each storage node.

7. The data processing method according to claim 5, characterized in that: The returning the operation result to the computing node so that the computing node obtains the data query result based on the operation result includes: The table connection operation result is returned to the computing node, so that the computing node generates the data query result based on the table connection operation result returned by each storage node.

8. The data processing method according to claim 5, characterized in that: The data volume of the first target data table is less than a preset data volume threshold.

9. A database cluster data processing device, characterized in that: The database cluster includes at least one computing node and multiple storage nodes. The device is applied to the computing node and includes: A plan generation module, used to generate an execution plan based on a data query request, the data query request comes from a client, and the execution plan is used to instruct a storage node to perform a data query operation; A plan sending module, used for sending the execution plan to the storage node; A result determination module, used for receiving the operation result obtained by the storage node performing the data query operation to obtain the data query result; A result feedback module, used to send the data query result to the client; The device further comprises a node determination module, which is used to determine a first target storage node and a second target storage node from the plurality of storage nodes; The plan sending module is used to send the execution plan to the first target storage node and the second target storage node, so that the first target storage node copies the first target data table stored in it to the second target storage node based on the execution plan, and enables the second target storage node to perform a table join operation on the second target data table originally stored in it and the first target data table based on the execution plan.

10. A database cluster data processing device, characterized in that: The database cluster includes at least one computing node and multiple storage nodes. The device is applied to the storage node and includes: A plan receiving module, used for receiving an execution plan sent by the computing node, wherein the execution plan is generated by the computing node based on a data query request, and the data query request comes from a client; A data processing module, used to execute a data query operation according to the execution plan to obtain an operation result; A result feedback module, used for returning the operation result to the computing node, so that the computing node obtains a data query result based on the operation result, and returns the data query result to the client; The device further comprises a node determination module and a data synchronization module, wherein the node determination module is used to determine a first target storage node from the plurality of storage nodes; The data synchronization module is used to enable the first target storage node to synchronize the first target data table copied by the first target storage node to other storage nodes among the multiple storage nodes except the first target storage node according to the execution plan; The data processing module is used for performing a table connection operation on the originally stored second target data table and the first target data table based on the execution plan for each storage node to obtain a table connection operation result.

11. A database system, characterized in that: Includes at least one computing node and multiple storage nodes: The computing node is used to generate an execution plan based on a data query request and send the execution plan to the storage node, wherein the data query request comes from a client; The storage node is used to perform a data query operation according to the execution plan to obtain an operation result, and return the operation result to the computing node; The computing node is further configured to obtain a data query result based on the operation result, and send the data query result to the client; The computing node is further used to determine a first target storage node and a second target storage node from the plurality of storage nodes; Sending the execution plan to the first target storage node and the second target storage node; The first target storage node is used to copy the first target data table stored in it to the second target storage node based on the execution plan, and the second target storage node is used to perform a table join operation on the second target data table originally stored in it and the first target data table based on the execution plan.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the database cluster data processing method described in any one of claims 1 to 8 are implemented.

13. An electronic device, characterized in that: include: processor; as well as Memory for storing computer programs; The processor is configured to execute the steps of the database cluster data processing method according to any one of claims 1 to 8 by executing the computer program.

Images