A distributed database query splitting method and device

Abstract

The application discloses a distributed database query splitting method and device, relates to the technical field of distributed databases, and applies distributed data and comprises the following steps: S1, data sharding storage: table data sharding storage: respectively existing on multiple nodes, selecting hash distribution or random distribution; S2, query splitting steps; comprising: S2.1, generating a query execution plan; S2.2, splitting data into data slots, when a client is connected to a data slot service of a specified number, reading to a data transmission channel; S2.3. Grouping data slots: the coordinator operation in the query plan obtains data from the specified data slot; S3, cluster expansion: expanding the distributed database cluster. The mechanism of the data slot is used to unbond the redistribution data and the nodes and only associate the data slot. Because the data slot can be associated to any node in the cluster, the optimal node can be selected according to the machine load to execute a task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed database, in particular to a distributed database query splitting method and device. BACKGROUND

[0002] The distributed database system usually uses small computer systems, each computer can be placed separately in a place, each computer can have a complete copy of DBMS or a partial copy, and has its own local database, and many computers located in different places are connected through a network to form a complete, global, logically centralized and physically distributed large database.

[0003] 1. Unreasonable resource use: according to the distributed database cluster distribution strategy function, when the data redistribution is performed, the specific data value can only be transmitted to the fixed node. The query task needs to be executed on the data transmission destination node, and when the redistributed data value is uneven, the idle and busy degrees of different nodes are different, and the resource use is unreasonable.

[0004] 2. Fixed data splitting quantity, limited parallel degree: when the data redistribution is performed, the data splitting quantity needs to be consistent with the node quantity in the distributed database cluster, and the splitting granularity cannot be adjusted according to the actual data, so that the parallel degree is limited, and the execution efficiency is affected.

[0005] 3. Lack of flexibility: when the distributed data system is expanded or contracted, the data needs to be moved between nodes, and the database can continue to be used after the data redistribution is completed, and the flexibility is not enough. SUMMARY

[0006] (I) Technical problems solved

[0007] In view of the defects of the prior art, the present application provides a distributed database query splitting method and device, which solves the technical problems proposed in the background art.

[0008] (II) Technical solutions

[0009] In order to achieve the above purpose, the present application is realized by the following technical solutions: a distributed database query splitting method applied to distributed data, the distributed database is composed of a management node, a computing node and a data node, and is connected through a gigabit Ethernet; a server includes a computing node and a data node, the management node is separately installed on another server, and includes the following steps:

[0010] S1, data sharding storage: table data sharding storage: respectively exists on multiple nodes, selects hash distribution or random distribution;

[0011] S2, query splitting step; comprising: S2.1, generating a query execution plan; S2.2, splitting data into data slots, when a client connects to a data slot service with a specified number, reading to a data transmission channel; S2.3. Grouping data slots: coordinator operations in the query plan obtain data from specified data slots, connect data with the same join key column, and divide them into the same data segment; S2.4 Selecting computing nodes and establishing data slot correspondence, referring to the comprehensive load pressure of each node, selecting nodes with small pressure to execute tasks; S2.5 Parallel execution of query tasks, returning the results to the client; on all nodes in the cluster, execute the assigned tasks. Open the data connection and read the data from the corresponding data slot group to complete the query task;

[0012] S3, cluster expansion: when expanding the distributed database cluster, the new nodes can participate in the calculation without redistributing the original data.

[0013] Further, the distributed database system is composed of a management node server, a plurality of computing node and storage node servers, and the servers can access each other through a gigabit Ethernet; the client connects to the management node through a local area network and sends a query request to the management node; the management node generates a query plan after receiving the query request and sends the execution plan to all computing nodes; the computing nodes read data from the data nodes and execute operations according to the received execution plan; the management node receives the results returned by each computing node and returns the results to the client.

[0014] Further, S2.1 includes: for a query request, first generate an execution plan; the execution plan is: after completing the data reading operation TableScan, aggregate data with the same DEPTNO value to the same node, and add a SplitMotion operation.

[0015] Further, in step S2.1, the Coordinator operation is responsible for reading the data generated by the Split Motion operation; stages without dependency are executed simultaneously; the execution plan is sent to all nodes for parallel execution.

[0016] Further, in step S2.2, the data slot number naming rule is: query serial number_stage serial number_distribution serial number_split serial number; the meanings of each serial number are: query serial number: for each query request, there is a unique serial number in the database system, which starts from 1 and increases; stage serial number: for a query request, the query is divided into multiple stages (Stage) according to the distribution action, and each stage corresponds to a stage serial number, which starts from 0 and increases;

[0017] Distribution sequence number: each distribution action corresponds to a distribution sequence number, which is incremented from 1; segment sequence number: for a distribution action, the data is divided into different segments according to a certain distribution rule, and the segment sequence number is incremented from 0.

[0018] Further, in step S2.2, the above-mentioned sequence numbers are connected by underlining, and a unique data slot label is generated for each data segment; when the corresponding data segment needs to be obtained, the segment data is read by connecting the corresponding number of data slots.

[0019] Further, in step S2.4, the comprehensive load pressure of each computing node is calculated respectively:

[0020] Pressure = CPUuse * CPUweight + MEMuse * MEMweight + IO * IOweight + NETWORK * NETWORKweight

[0021] The comprehensive load of the computing node is sorted from low to high, and the node with the lowest comprehensive load is selected to execute the query task of this data slot group; the comprehensive load pressure of each node is recalculated, and the node with the lowest comprehensive load is selected to execute the query task of the next data slot group; and the task of reading data through the coordinator is allocated to a certain computing node.

[0022] Further, after step S2.5, it further includes: S2.6 query splitting execution process specifically includes the following steps: 1.1, receiving a query request; 1.2, generating a query plan; 1.3, dividing the data that needs to be redistributed into data slots; 1.4, selecting the corresponding data slot as a data source for redistribution; 1.5, selecting a node with small comprehensive load to execute the task on the node; and 1.6, returning the result.

[0023] A distributed database query splitting method and device, applied to distributed data, the distributed database is composed of a management node, a computing node and a data node, and is connected through a gigabit Ethernet; a server includes a computing node and a data node, and the management node is separately installed on another server, characterized in that the distributed database query splitting method is applied.

[0024] (Three) beneficial effects

[0025] The application provides a distributed database query splitting method and device. The following beneficial effects are provided:

[0026] For the query task needing to do data redistribution, using the existing method, the specific data value can only be sent to the fixed corresponding node. The method adopts the mechanism of data slot, and the redistributed data is not bound to the node, but is only associated with the data slot. Because the data slot can be associated to any node in the cluster, the optimal node can be selected according to the machine load to execute the task, and the cluster resources are fully utilized.

[0027] The method can manually set the data split quantity, adjust the task parallel granularity, make the task fully parallel, and improve the execution efficiency; when doing distributed database expansion, the newly added node can participate in the calculation without waiting for the data redistribution to be completed, and the use is flexible. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 The figure is a schematic diagram of a typical distributed database system architecture of the application;

[0029] Figure 2 The figure is a schematic diagram of a distributed data cluster node structure of the application;

[0030] Figure 3 The figure is a schematic diagram of a distributed database query execution plan flow structure of the application;

[0031] Figure 4 The figure is a schematic diagram of a node data structure after executing the query plan of the application;

[0032] Figure 5 The figure is a schematic diagram of a typical distributed database system architecture of the application;

[0033] Figure 6 The figure is a schematic diagram of a data storage structure on each node of the application;

[0034] Figure 7 The figure is a schematic diagram of a query execution plan execution flow of the application;

[0035] Figure 8 The figure is a schematic diagram of a query split execution flow of the application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0037] REFERENCE Figures 1 to 4As shown, a distributed database is a database system with multiple nodes. These nodes are independent of each other and connected by a network, and multiple processing nodes can cooperate to complete large-scale computing work. The purpose of establishing a multi-node distributed database system is to obtain stronger computing power at a smaller cost. In a distributed database, a user's query request is often divided into multiple tasks, which can be distributed to multiple nodes of the system for parallel running, thereby maximizing the use of the computing resources of each processing node. Through this parallel running manner, the time for completing the entire application program can be greatly shortened.

[0038] As shown in the prior art typical distributed database system architecture Figure 1 As shown, the distributed database parallel query technology is mainly designed to improve the query performance in multi-node parallel query.

[0039] The process is as follows:

[0040] Step 101, data is evenly stored on each node of the cluster (using hash distribution or random distribution, etc.); Step 102, the management node accepts the client query request; Step 103, analyze the request and generate a query plan; Step 104, distribute the execution plan to the computing nodes and concurrently query Step 105, each computing node obtains the calculation result and returns it to the management node; Step 106, the management node aggregates the calculation result and returns it to the client.

[0041] Taking the most common join operation in the database as an example, if the two tables are equal value connection on the sharding field, each computing node can obtain the query result according to the sharding data it is responsible for (the management node will directly send the query to each relevant computing node for execution); otherwise, each computing node must rely on the data of other computing nodes to complete the query. At this time, the system needs to insert a data transmission operator (motion) in the execution plan tree to be responsible for transmitting data between different computing instances.

[0042] If the associated data is on different nodes, it is impossible to perform a connection. The associated data needs to be transmitted to a node for calculation through the network, so data migration needs to occur. Data migration usually adopts a redistribution method.

[0043] As follows, an example is given to create two data tables EMP and DEPT. The data is stored randomly, that is, the table data is randomly and uniformly stored on each node.

[0044] CREATE TABLE EMP(EMPNO INTEGER NOT NULL, ENAME VARCHAR(20) NOT NULL, DEPTNO INTEGER NOT NULL);

[0045] CREATE TABLE DEPT(DEPTNO INTEGER NOT NULL, DNAME VARCHAR(20) NOT NULL);

[0046] Insert data into two data tables.

[0047] INSERT INTO EMP VALUES();

[0048] INSERT INTO DEPT VALUES();

[0049] Data table EMP data

[0050]

[0051]

[0052] Data table DEPT data

[0053] DEPTNO DNAME 1 Marketing Department 2 Sales Department 3 Research and Development Department 4 Administration Department

[0054] Suppose that the distributed data cluster has four nodes, and each node stores data processing as shown in Table 1. Figure 2

[0055] Execute a query statement

[0056] SELECT * FROM emp JOIN dept ON emp.DEPTNO = dept.DEPTNO;

[0057] Referring to Figure 3 , the query execution plan is as follows:

[0058] ​When the join condition (emp.DEPTNO, dept.DEPTNO is not a distribution key) of the table emp, dept is not a distribution key, the column emp.DEPTNO data of the table emp is randomly distributed, and the column dept.DEPTNO data of the table dept is also randomly distributed, that is, the data rows with the same value on the DEPTNO column may exist on multiple nodes respectively, and the join operation cannot be directly performed. The table emp, dept can be hashed according to the column emp.DEPTNO, dept.DEPTNO respectively, and a data transmission operation is added in the query table emp table dept execution plan respectively, and the data is redistributed (redistribution Redistribute. Each produces a redistribution, and a new stage is cut out). Thus, the data of each computing node is the data after the hash and redistribution, that is, the data rows with the same DEPTNO column value are all sent to the same data node.

[0059] In order to simplify the description, it is assumed that the hash function is f(x)=x, node_index=f(x)%node_count. Based on the join key redistribution, each node data is referred to Figure 4 .

[0060] After redistribution, the data with the same join key is transmitted to the same node, and then the node local join can be performed. The final result is returned to the management service node, the management service node uniformly collects the result, and returns the result to the client. In addition, for the expansion case, after the distributed database cluster is expanded, the original data must be redistributed, so that the new node resources can be utilized.

[0061] The reason is that if the data is not redistributed after expansion, the data still exists on the old node according to the distribution calculation method before expansion. After expansion, the number of nodes changes, and the distribution function calculates the distribution node of the data according to the new number of nodes, which will cause the inconsistency between the original data node and the new calculation result. Therefore, after expansion, the original data must be redistributed according to the distribution function calculation result after expansion.

[0062] Please refer to Figures 5-8 The present application provides a distributed database query splitting method, which is based on the introduction of the prior art and makes a detailed introduction to the technical solution.

[0063] Referring to Figure 5As shown, the distributed database is composed of management nodes, computing nodes, and data nodes, and is connected through Gigabit Ethernet. A server contains a computing node and a data node, and the management node is separately installed on a server. A distributed database system is composed of a server with a management node, and multiple servers with computing and storage nodes, and the servers can access each other through Gigabit Ethernet.

[0064] The client connects to the management node through the local area network and sends a query request to the management node. After receiving the query request, the management node generates a query plan and sends the execution plan to all computing nodes. The computing nodes read data from the data nodes and perform operations such as projection, sorting, aggregation, and connection according to the received execution plan. After receiving the results returned by the computing nodes, the management node aggregates the results and returns them to the client.

[0065] S1, data sharding storage

[0066] Reference Figure 6 As shown, in order to enable parallel processing of data on multiple nodes, the present scheme adopts the storage method of traditional distributed databases and stores table data in shards. The data of a data table exists on multiple nodes respectively, and can be distributed by hash or randomly. In the "Technical Background" section, the storage status of the rows of the table emp and the table dept on the nodes is shown.

[0067] S2, query splitting step

[0068] S2.1, generating a query execution plan

[0069] Reference Figure 7 As shown in the figure, for a query request, an execution plan is first generated. Still taking the query SQL in the "Technical Background" section as an example:

[0070] SELECT * FROM emp JOIN dept ON emp.DEPTNO = dept.DEPTNO;

[0071] Its execution plan is: because the table emp and table dept are stored in random distribution, the data with the same join key joinkey is stored in different nodes, so after the data reading operation TableScan is done, the data with the same DEPTNO value needs to be summarized to the same node, so the Split Motion operation is added. The Coordinator operation is responsible for reading the data generated by the Split Motion operation. Since the Motion operation involves data migration, the execution plan is divided into multiple stages (Stage), each stage contains part of the entire execution plan. For stages without dependencies, such as Stage1 and Stage2, they can be executed simultaneously. The execution plan is sent to all nodes and executed in parallel.

[0072] S2.2, split data into data slots: this step is the key step of the scheme.

[0073] Data slots, i.e. network data transmission channels, provide data reading services to the outside. When the client connects to the data slot service with a specified number, it can read the data in the data transmission channel (data slot).

[0074] Data slot number naming rule: query serial number_stage serial number_distribution serial number_split serial number:

[0075] The meaning of each serial number is:

[0076] Query serial number: for each query request, there is a unique serial number in the database system, which starts from 1 and increases

[0077] Stage serial number: for a query request, according to the distribution action, the query is divided into multiple stages (Stage), each stage corresponds to a stage serial number, which starts from 0 and increases

[0078] Distribution serial number: each distribution action corresponds to a distribution serial number, which starts from 1 and increases

[0079] Fragment serial number: for a distribution action, the data will be divided into different fragments (partitions) according to certain distribution rules (such as hash function), and the fragment serial number starts from 0 and increases. Different distribution actions, the fragment serial number starts from 1. The number of data fragments can be manually set, which can be the same as or different from the number of computing nodes, and is usually set to an integer multiple of the number of computing nodes.

[0080] According to the naming rule, the above serial numbers are connected by underscores, and each data fragment generates a unique data slot label. When the corresponding data fragment needs to be obtained, the corresponding data slot is connected, and the fragment data can be read.

[0081] The Split Motion operation in the query plan is to divide the data segments into corresponding data slots. Taking the data storage in the "Technical Background" chapter as an example, set the data segment number to 4, the data slot divided by each node, and the data slot to which each row of data belongs, as shown below. (Because the amount of data in each node in the example is small, some data slots have no data, which does not affect the actual function)

[0082] Node 1

[0083]

[0084] Node 2

[0085]

[0086]

[0087] Node 3

[0088]

[0089] Node 4

[0090]

[0091] S2.3. Group data slots: This step is the key step of the present scheme.

[0092] The coordinator operation in the query plan is to obtain data from the specified data slot. According to the data distribution rule, the data with the same join key column in the connection condition will be divided into the same data segment. The data slots with the same segment number in the data slot number have data with equal distribution column values. In the query plan segment Stage 3, the data slots connected by the left coordinator and the right coordinator operations are shown in the following table.

[0093] Task 1

[0094]

[0095] Task 2

[0096]

[0097] Task 3

[0098]

[0099] Task 4

[0100]

[0101] S2.4 Select computing nodes and establish data slot correspondence

[0102] Because the coordinator reads data from the data slots through the network, the execution task of reading data from the data slots can be executed on any node in the cluster, unlike the redistribution mode which is limited to be executed on a certain node. Therefore, when selecting the node to execute the task, the comprehensive load pressure of each node can be referred to, and a node with small pressure can be selected to execute the task.

[0103] According to the formula, the comprehensive load pressure of each computing node is calculated respectively

[0104] Pressure = CPUuse * CPUweight + MEMuse * MEMweight + IO * IOweight + NETWORK * NETWORKweight

[0105] The comprehensive load of the computing node is sorted from low to high, and the node with the lowest comprehensive load is selected to execute the query task of the data slot group.

[0106] The comprehensive load pressure of each node is recalculated, the node with the lowest comprehensive load is selected, and the query task of the next data slot group is executed.

[0107] In this way, each task of reading data through the coordinator is allocated to a certain computing node.

[0108] S2.5, the query task is executed in parallel, and the result is returned to the client

[0109] On all nodes in the cluster, the allocated task is executed. Open the data connection, read the data from the corresponding data slot group, and complete the query task.

[0110] The query split execution flow is as shown in Figure 8 , and specifically includes the following steps:

[0111] 1.2, receive the query request

[0112] 1.3, generate a query plan

[0113] 1.4, divide the data that needs to be redistributed into data slots

[0114] 1.5, select the corresponding data slot as the data source for redistribution

[0115] 1.1, select a node with small comprehensive load to execute the task on the node

[0116] 1.1, return the result

[0117] S3, cluster expansion

[0118] When the distributed database cluster is expanded, i.e. the nodes are increased, the new nodes can participate in the calculation without the operation of redistributing the original data.

[0119] This is because, when the data needs to be redistributed, according to the data slot division rule, only the data source needs to be corresponded to the data slot group with the same segment serial number, so that the same data can be divided together. Through the data slot, the query task can obtain non-local data from the network, so the query task does not require to be on the node with data storage. When the query task is assigned to the new node, since the data is obtained through the connection data slot, the original data does not need to be redistributed, and the new node can also execute the query task.

[0120] Please refer to Figures 1-6 The present application provides a kind of distributed database query splitting device, application has the method in embodiment 1.

[0121] In combination with the above contents, the technical solution at least has the following effects:

[0122] 1. For the query task needing to do data redistribution, using the existing method, specific data value can only be sent to the fixed corresponding node. The method uses the mechanism of data slot to bind the redistributed data and node, and only associates with data slot. Because data slot can be associated with any node in the cluster, the optimal node can be selected according to the machine load to execute the task, and the cluster resources are fully utilized.

[0123] 2. The method can manually set the data splitting number, adjust the task parallelism granularity, make the task fully parallel, and improve the execution efficiency.

[0124] 3. When expanding the distributed database, the new node can participate in the calculation without waiting for the data redistribution to be completed, and the use is flexible.

[0125] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (for example, infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium. The semiconductor medium can be a solid state disk.

[0126] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0127] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0128] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division of the distributed database query splitting method and device channel underwater topographic change analysis system and method, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0131] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0133] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A distributed database query splitting method, applied to a distributed database, the distributed database is composed of a management node, a computing node, a data node, and connected through a gigabit Ethernet; a server contains a computing node and a data node, and the management node is separately installed on another server, characterized in that: It comprises the following steps: ​ S1, data slice storage: table data slice storage: respectively exist on multiple nodes, select hash distribution or random distribution; S2, query segmentation step; including: S2.1, generating a query execution plan; S2.2, cutting data into data slots, when the client is connected to the data slot service of the specified number, the data transmission channel is read; S2.

3. Grouping data slots: coordinator operation in the query plan, get data from the specified data slot, connect the data with the same join key column, and divide them into the same data segment; S2.4 Select the computing node, establish the data slot corresponding relationship, refer to the comprehensive load pressure of each node, select the node with small pressure to execute the task; S2.5 Parallel execution of query task, return the result to the client; On all nodes in the cluster, execute the assigned task, open data connection, read data from the corresponding data slot group, and complete the query task; S3, cluster expansion: in the distributed database cluster expansion, without re-distribution operation to the original data, add new nodes to participate in the calculation; The step S2.1 comprises: for a query request, first generate an execution plan; The execution plan is: after the data reading operation TableScan, the data with the same DEPTNO value is summarized to the same node, and the Split Motion operation is added; The coordinator operation is responsible for reading the data generated by the Split Motion operation; For stages without dependency relationship, execute simultaneously; The execution plan is sent to all nodes for parallel execution. 2.The distributed database query partitioning method of claim 1, wherein: The distributed database is composed of a management node server, a plurality of computing node and storage node servers, and the servers can access each other through a gigabit Ethernet; The client connects to the management node through the local area network and sends a query request to the management node; The management node receives the query request, generates a query plan, and sends the execution plan to all computing nodes; The computing node reads data from the data node and executes the operation according to the received execution plan; The management node receives the results returned by each computing node and returns the results to the client.

3. The method of claim 1, wherein: In step S2.2, the data slot number naming rule is: query serial number_stage serial number_distribution serial number_split serial number; The meaning of each serial number is: Query serial number: for each query request, there is a unique serial number in the database system, which increases from 1; Stage serial number: for a query request, according to the distribution action, the query is divided into multiple stages (Stage), each stage corresponds to a stage serial number, which increases from 0; Distribution serial number: each distribution action corresponds to a distribution serial number, which increases from 1; Segment serial number: for a distribution action, the data will be cut into different segments according to a certain distribution rule, and the segment serial number increases from 0.

4. The method of claim 3, wherein: In step S2.2, the above-mentioned serial numbers are connected by underlining, and then a unique data slot number is generated for each data segment; when the corresponding data segment needs to be obtained, the corresponding data slot is connected to read the segment data.

5. The method of claim 1, wherein: In step S2.4, the comprehensive load pressure of each computing node is calculated respectively: Pressure = CPUuse * CPUweight + MEMuse * MEMweight + IO * IOweight + NETWORK * NETWORKweight The comprehensive load of the computing node is sorted from low to high, and the node with the lowest comprehensive load is selected to execute the query task of this data slot group; the comprehensive load pressure of each node is recalculated, the node with the lowest comprehensive load is selected, and the query task of the next data slot group is executed; in this way, each task of reading data through the coordinator is allocated to a certain computing node.

6. The method of claim 1, wherein: After step S2.5, it further includes: S2.6 query splitting execution process, which specifically includes the following steps: 1.1, receiving a query request; 1.2, generating a query plan; 1.3, dividing the data that needs to be redistributed into data slots; 1.4, selecting the corresponding data slot as the data source for redistribution; 1.5, selecting a node with small comprehensive load to execute the task on the node; 1.6, returning the result.

7. A distributed database query splitting device, applied with a distributed database, the distributed database is composed of a management node, a computing node, a data node, and connected through a gigabit Ethernet; a server contains a computing node and a data node, and the management node is separately installed on another server, characterized in that: Apply the distributed database query splitting method as claimed in any one of claims 1-6.

Images