Distributed connection method, device, storage medium and server

Abstract

The embodiment of the present application provides a distributed connection method, device, storage medium and server. In the technical scheme provided by the embodiment of the present application, according to the semantics of a clause in SQL, the SQL is split into a parallel executable subquery and a dependent relationship subquery; a table grouping is obtained by grouping target tables involved in the clause, a table pool connection relationship of the table grouping is converted from a connection relationship in the target tables; a separate node is generated between the table groupings; and by performing separate node execution scheduling, local connection calculation is completed, so that the computing resources of a distributed cluster can be fully utilized, and the stability of a distributed database cluster is improved.

Description

[0001] The present application relates to the technical field of database, and particularly relates to a distributed connection method and device, a storage medium and a server.

[0002] For a distributed database performing data sharding, how to perform flexible, efficient and controllable connection and table association operation on distributed deployed data is one of main challenges in function implementation of the distributed database after the data is distributed deployed. A common distributed database implementation method is centralized connection, which needs to extract connection related data in a structured query language (SQL) statement from a data storage layer to a memory of a computing node or a computing layer of the distributed database, and then uses a memory connection algorithm of a traditional database, such as a conventional nested loop connection, to perform connection operation.

[0003] However, the centralized connection has problems of being unable to fully utilize computing resources of a distributed cluster and poor stability.

[0004] Therefore, the embodiments of the present application provide a distributed connection method, device, storage medium and server, which can fully utilize computing resources of a distributed cluster and improve stability of a distributed database cluster.

[0005] In a first aspect, the embodiments of the present application provide a distributed connection method, and the method comprises the following steps.

[0006] According to semantics of a clause in a SQL, the SQL is split into a parallel executable subquery and a dependency relationship subquery;

[0007] A table grouping is obtained by grouping target tables involved in the clause, and a table pool connection relationship of the table grouping is obtained by converting a connection relationship in the target tables;

[0008] A separate node is generated between the table groupings;

[0009] Local connection calculation is completed by performing separate node execution scheduling.

[0010] Optionally, all tables in a same table grouping have a same data distribution manner on a data storage.

[0011] Optionally, before the step of splitting the SQL into the parallel executable subquery and the dependency relationship subquery according to the semantics of the clause in the SQL, the method further comprises the following steps.

[0012] The semantics of the clause are obtained by parsing the SQL. ​​​

[0013] Optionally, the separation node is used to construct a specific physical execution step.

[0014] In another aspect, the embodiment of the present application provides a distributed connection device, which comprises:

[0015] a splitting module, used for splitting the SQL into a parallel-executable subquery and a dependent relationship subquery according to semantics of a clause in the SQL;

[0016] a conversion module, used for converting a connection relationship in the target table into a table pool connection relationship of the table grouping by grouping the target table involved in the clause;

[0017] a generation module, used for generating a separation node between the table groupings;

[0018] a scheduling module, used for completing local connection calculation by performing separation node execution scheduling.

[0019] Optionally, all tables in the same table grouping are the same in data distribution mode on a data storage.

[0020] Optionally, the device further comprises:

[0021] a parsing module, used for obtaining the semantics of the clause by parsing the SQL.

[0022] Optionally, the separation node is used to construct a specific physical execution step.

[0023] In another aspect, the embodiment of the present application provides a storage medium, which comprises a stored program, wherein when the program runs, the device where the storage medium is located is controlled to perform the distributed connection method.

[0024] In another aspect, the embodiment of the present application provides a server, which comprises a memory and a processor, wherein the memory is used for storing information comprising program instructions, and the processor is used for controlling execution of the program instructions, and the program instructions are loaded and executed by the processor to realize steps of the distributed connection method.

[0025] In the technical scheme of the distributed connection method, device, storage medium and server provided by the embodiment of the present application, the SQL is split into a parallel-executable subquery and a dependent relationship subquery according to semantics of a clause in the SQL; a table grouping is obtained by grouping target tables involved in the clause, and a table pool connection relationship of the table grouping is converted from a connection relationship in the target table; a separation node is generated between the table groupings; and local connection calculation is completed by performing separation node execution scheduling, so that computing resources of a distributed cluster can be fully utilized, and stability of a distributed database cluster can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0027] Figure 1 A flow chart of a distributed connection method provided by an embodiment of the present application;

[0028] Figure 2 A flow chart of a distributed connection method provided by another embodiment of the present application;

[0029] Figure 3 A schematic diagram for executing a subquery and a dependent subquery;

[0030] Figure 4 A schematic diagram for a physical execution step of a split node;

[0031] Figure 5 A schematic diagram for split node execution scheduling;

[0032] Figure 6 A structural schematic diagram of a distributed connection device provided by an embodiment of the present application;

[0033] Figure 7 A schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to better understand the technical solutions of the present application, the embodiments of the present application will be described in detail below with reference to the drawings.

[0035] It should be clear that the described embodiments are only some embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the scope of protection of the present application.

[0036] The terms used in the embodiments of the present application are only for the purpose of describing the specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0037] It should be understood that the term "and / or" used herein is only to describe an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it.

[0038] Figure 1 A flow chart of a distributed connection method provided by an embodiment of the present application is shown in FIG. 1, which includes the following steps. Figure 1

[0039] Step 101, according to the semantics of the clauses in SQL, the SQL is split into subqueries that can be executed in parallel and dependent relationship subqueries.

[0040] Step 102, by grouping the target tables involved in the clauses to obtain table groups, the connection relationship in the target tables is converted into a table pool connection relationship of the table groups.

[0041] Step 103, a separate node is generated between the table groups.

[0042] Step 104, by performing separate node execution scheduling, local connection calculation is completed.

[0043] In the technical scheme of the distributed connection method provided by the embodiment of the present application, according to the semantics of the clauses in SQL, the SQL is split into subqueries that can be executed in parallel and dependent relationship subqueries; by grouping the target tables involved in the clauses to obtain table groups, the connection relationship in the target tables is converted into a table pool connection relationship of the table groups; a separate node is generated between the table groups; by performing separate node execution scheduling, local connection calculation is completed, which can make full use of the computing resources of the distributed cluster and improve the stability of the distributed database cluster.

[0044] Figure 2 A flow chart of a distributed connection method provided by another embodiment of the present application is shown in FIG. 2, which includes the following steps. Figure 2

[0045] Step 201, by parsing the SQL, the semantics of the clauses are obtained.

[0046] In the embodiment of the present application, according to the semantics of the clauses in SQL, the dependent relationship between the clauses is constructed by using a directed acyclic graph, and the parallel execution plan arrangement can be performed between the clauses without dependency.

[0047] Step 202, according to the semantics of the clauses in SQL, the SQL is split into subqueries that can be executed in parallel and dependent relationship subqueries.

[0048] ​​Specifically, clauses with dependencies in the SQL are split into dependent subqueries; clauses without dependencies are split into parallelizable subqueries.

[0049] For example, the SQL is select…from <sub query 1tb1,tb2…col1,col2……>……<subquery 2tb3,tb4…dol1,dol2,dol3>……<sub query 3tb5,tb6…sol1,sol2,sol3><subquery 4tb7,tb8…eol1,eol2,eol3>……<where tab1.col1＝tb3.dol2>……，由于where条件为子查询1(sub query 1)中涉及的tab1(tb1)表的col1列和子查询2(sub query2)中涉及的tab3(tb3)表的dol2列进行关联查询，子查询1和子查询2做连接计算时，需要先执行子查询1的查询，使用其结果执行连接计算，故两者是依赖关系，即子查询2依赖于子查询1。因此，子查询1和子查询2之间存在依赖关系；子查询3和子查询4之间不存在依赖关系，并且子查询3和子查询4分别和子查询1、子查询2之间也不存在依赖关系。因此，如图3所示，子查询1和子查询2为依赖关系子查询，子查询3和子查询4为可并行执行子查询。

[0050] 本发明实施例中，基于子句之间的依赖关系，通过执行计划融合和SQL语句常量条件构建数据库SQL分布式逻辑执行计划。

[0051] 步骤203、通过将子句所涉及的目标表进行分组得到表分组，将目标表中的连接关系转换为表分组的表池连接关系。

[0052] 本发明实施例中，同一表分组的所有表在数据存储上数据分布方式相同，即数据节点相同、数据分片算法相同、并且存在属于"AND”的分片列等值查询条件。例如，查询条件包括t1.c1＝t3.c1，其中c1列是分片列。每一个分组称为表分组。所有子句中的表连接关系转换为表分组的连接关系。本发明实施例基于表分组的概念，简化了分布式连接数据挪动的依赖关系，通过将一种数据分布方式的所述表分组的计算输出作为驱动连接到另一种数据分布方式的所述表分组，实现分布式的循环嵌套连接。

[0053] 例如，数据分布方式包括range模式。

[0054] 需要说明的是，在将子句所涉及的目标表进行分组得到表分组的同时，目标表中的连接关系通过内部算法就自动转换为表分组的表池连接关系。

[0055] 步骤204、在表分组之间生成分离节点。

[0056] 本发明实施例中，通过分离节点衔接各个表分组执行的结果，基于嵌套循环的算法思想，上一个分离节点用于完成本地计算并用于下一个分离节点。连接运算所需的驱动表数据通过临时表的方式传递。

[0057] 其中，分离节点用于构建具体物理执行步骤。每个分离节点的物理执行步骤包括三个步骤：

[0058] 第一步，如图4所示，SQL为select…from<sub query 1tb1,tb2…col1,col2……>……<sub query 2tb3,tb4…dol1,dol2,dol3>……<where tab1.col1＝tb3.dol2>……，基于表分组之间连接运算的语义，将下一个连接所需的上一个表分组，即表分组1的where条件列、最终数据投影所需的列筛选出来，构建一张位于下一个表分组，即表分组2本地的临时表，临时表的数据分布方式与表分组2中表相同。只在表分组1本地运算所需的列是不会被选取。例如where条件t1.c3＝5，这里表分组t1中的c3列将不会被选取；而where条件t1.c4＝t2.c9，这里的表分组t1中的c4列将被选取，因为它将用于对包含了t2表的表分组2的连接运算。

[0059] 本发明实施例中，分布式数据库集群基于SQL语义，将where条件筛选、select列投影尽可能下推到数据节点进行计算，计算节点层 / 协调节点层只进行分布式执行计划的调度、流式的中间结果分发、以及流式的最终结果汇聚处理，计算节点 / 调度节点不直接缓存数据进行连接运算，计算节点 / 调度单一节点的资源不会成为集群连接运算的瓶颈，不会导致计算节点 / 调度单内存溢出，从而增强了数据库集群的稳定性。

[0060] 第二步，为表分组1生成本地数据查询任务，所有只与表分组1本地计算相关的筛选条件将被下推到数据节点进行计算，例如where条件t1.c3＝5。

[0061] 本发明实施例最大程度将数据计算下推到数据节点本地进行，降低了数据抽取的网络消耗，提高了计算效率。

[0062] 第三步，将第二步下推的表分组1的筛选条件应用到临时表上，并创建临时表和表分组2的连接运算任务。

[0063] 例如，表分组a有一列col1,下一个表分组b有一列col2，整体需要执行col1＝col2的连接计算，where col1＝col2即是连接运算的语义，col1即是需要筛选的列。

[0064] 本发明实施例中，面向分布式网络环境进行数据批量操作优化，将数据节点本地计算与临时表级数据批量挪动相结合，实现更加面向网络优化的分布式数据连接方案。

[0065] 同样的，所有表分组的连接将重复上述三个步骤，以分布式表级嵌套循环的方式逐个调度每个表分组的本地查询或连接运算，并将结果以临时表的形式分布到下一个表分组本地以驱动连接运算。

[0066] 本发明实施例中，在分布式数据库连接过程中，执行计划重复考虑分布式系统资源，将数据的抽取、计算、临时结果的重分配进行充分的分布式调度；以嵌套循环的方式，迭代在分布式节点间传递连接的中间结果驱动下一个步骤的连接运算，同时允许不相关子句并行嵌套循环。整个连接运算过程可以完全分布式并行调度，连接计算性能可以随着集群数据节点数的增加而增强，扩展性更好。

[0067] 需要说明的是，分离节点不同于数据节点。数据节点是实际存在的物理数据库服务器，分离节点是一个数据库软件实际运行的逻辑概念，是SQL在执行过程用于承接上下两个表分组计算过程的进程或线程。

[0068] 步骤205、通过进行分离节点执行调度，完成本地连接计算。

[0069] 本发明实施例中，进行分离节点执行调度，即将分离节点的执行任务放到任务队列中进行排队。临时表的创建任务执行时将基于临时表池进行，以应对并发负载场景进行临时表资源的复用。其中，根据配置数据库在启动时事先创建一个临时表池，存储配置规定数量的临时表，当程序运行时不用实时创建临时表，可以直接利用临时表池中的临时表，以提升性能。

[0070] 具体的，如图5所示，进行分离节点执行调度包括3个步骤。第一步，数据筛选智能下推；第二步，按需分布式"挪动”最小数据；第三步，分布式并行计算join结果。

[0071] 本发明实施例中，上一组表分组计算结果的子句输出结构可以是一个常量值或一个表分组。基于上一组表分组的子句输出结构，为逻辑执行计划融合进行实例化构建。无依赖关系的执行计划子树将并行驱动，最终完成顶层查询的执行计划并流式将结果返回给客户端。

[0072] 本发明实施例提供的一种分布式连接方法的技术方案中，通过对SQL进行解析，得到的子句的语义，根据SQL中子句的语义，将SQL拆分为可并行执行子查询及依赖关系子查询；通过将子句所涉及的目标表进行分组得到表分组，将目标表中的连接关系转换为表分组的表池连接关系；在表分组之间生成分离节点；通过进行分离节点执行调度，完成本地连接计算，能够充分利用分布式集群的计算资源，并提高分布式数据库集群稳定性。

[0073] 图6为本发明一实施例提供的一种分布式连接装置的结构示意图，如图6所示，所述装置包括：解析模块31、拆分模块32、转换模块33、生成模块34和调度模块35。

[0074] 解析模块31，用于通过对SQL进行解析，得到的子句的语义。

[0075] 拆分模块32，用于根据SQL中子句的语义，将所述SQL拆分为可并行执行子查询及依赖关系子查询。

[0076] 转换模块33，用于通过将所述子句所涉及的目标表进行分组得到表分组，将所述目标表中的连接关系转换为所述表分组的表池连接关系。

[0077] 本发明实施例中，同一表分组的所有表在数据存储上数据分布方式相同。

[0078] 生成模块34，用于在所述表分组之间生成分离节点。

[0079] 本发明实施例中，分离节点用于构建具体物理执行步骤。

[0080] 调度模块35，用于通过进行分离节点执行调度，完成本地连接计算。

[0081] 本发明实施例提供的分布式连接装置可用于实现上述图1至图2中的分布式连接方法，具体描述可参见上述分布式连接方法的实施例，此处不再重复描述。

[0082] 本发明实施例提供的一种分布式连接装置的技术方案中，通过对SQL进行解析，得到的子句的语义，根据SQL中子句的语义，将SQL拆分为可并行执行子查询及依赖关系子查询；通过将子句所涉及的目标表进行分组得到表分组，将目标表中的连接关系转换为表分组的表池连接关系；在表分组之间生成分离节点；通过进行分离节点执行调度，完成本地连接计算，能够充分利用分布式集群的计算资源，并提高分布式数据库集群稳定性。

[0083] 图7为本说明书服务器一个实施例的结构示意图，如图7所示，上述服务器可以包括至少一个处理器；以及与上述处理器通信连接的至少一个存储器，其中：存储器存储有可被处理器执行的程序指令，上述处理器调用上述程序指令能够执行本说明书图1～图5所示实施例提供的分布式连接方法。

[0084] 图7示出了适于用来实现本说明书实施方式的示例性服务器的框图。图7显示的服务器仅仅是一个示例，不应对本说明书实施例的功能和使用范围带来任何限制。

[0085] 如图7所示，服务器以通用计算设备的形式表现。服务器的组件可以包括但不限于：一个或者多个处理器21，存储器23，连接不同系统组件(包括存储器23和处理单元21)的通信总线24。

[0086] 通信总线24表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(Industry StandardArchitecture；以下简称：ISA)总线，微通道体系结构(Micro Channel Architecture；以下简称：MAC)总线，增强型ISA总线、视频电子标准协会(Video Electronics StandardsAssociation；以下简称：VESA)局域总线以及外围组件互连(Peripheral ComponentInterconnection；以下简称：PCI)总线。

[0087] 服务器典型地包括多种计算机系统可读介质。这些介质可以是任何能够被服务器访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。

[0088] 存储器23可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(Random Access Memory；以下简称：RAM)和 / 或高速缓存存储器。服务器可以进一步包括其它可移动 / 不可移动的、易失性 / 非易失性计算机系统存储介质。存储器23可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本说明书各实施例的功能。

[0089] 具有一组(至少一个)程序模块的程序 / 实用工具，可以存储在存储器23中，这样的程序模块包括——但不限于——操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块通常执行本说明书所描述的实施例中的功能和 / 或方法。

[0090] 处理器21通过运行存储在存储器23中的程序，从而执行各种功能应用以及数据处理，例如实现本说明书图1～图5所示实施例提供的分布式连接方法。

[0091] 本说明书实施例提供一种非暂态计算机可读存储介质，所述非暂态计算机可读存储介质存储计算机指令，所述计算机指令使所述计算机执行本说明书图1～图5所示实施例提供的分布式连接方法。

[0092] 上述非暂态计算机可读存储介质可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(Read Only Memory；以下简称：ROM)、可擦式可编程只读存储器(Erasable ProgrammableRead Only Memory；以下简称：EPROM)或闪存、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。

[0093] 计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括——但不限于——电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。

[0094] 计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于——无线、电线、光缆、RF等等，或者上述的任意合适的组合。

[0095] 可以以一种或多种程序设计语言或其组合来编写用于执行本说明书操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如"C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LocalArea Network；以下简称：LAN)或广域网(Wide Area Network；以下简称：WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。

[0096] 上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下，在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外，在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中，多任务处理和并行处理也是可以的或者可能是有利的。

[0097] 在本说明书的描述中，参考术语"一个实施例”、"一些实施例”、"示例”、"具体示例”、或"一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本说明书的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且，描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外，在不相互矛盾的情况下，本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。

[0098] 此外，术语"第一”、"第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有"第一”、"第二”的特征可以明示或者隐含地包括至少一个该特征。在本说明书的描述中，"多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。

[0099] 流程图中或在此以其他方式描述的任何过程或方法描述可以被理解为，表示包括一个或更多个用于实现定制逻辑功能或过程的步骤的可执行指令的代码的模块、片段或部分，并且本说明书的优选实施方式的范围包括另外的实现，其中可以不按所示出或讨论的顺序，包括根据所涉及的功能按基本同时的方式或按相反的顺序，来执行功能，这应被本说明书的实施例所属技术领域的技术人员所理解。

[0100] 取决于语境，如在此所使用的词语"如果”可以被解释成为"在……时”或"当……时”或"响应于确定”或"响应于检测”。类似地，取决于语境，短语"如果确定”或"如果检测(陈述的条件或事件)”可以被解释成为"当确定时”或"响应于确定”或"当检测(陈述的条件或事件)时”或"响应于检测(陈述的条件或事件)”。

[0101] 需要说明的是，本说明书实施例中所涉及的终端可以包括但不限于个人计算机(Personal Computer；以下简称：PC)、个人数字助理(Personal Digital Assistant；以下简称：PDA)、无线手持设备、平板电脑(Tablet Computer)、手机、MP3播放器、MP4播放器等。

[0102] 在本说明书所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如，多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。

[0103] 另外，在本说明书各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用硬件加软件功能单元的形式实现。

[0104] 上述以软件功能单元的形式实现的集成的单元，可以存储在一个计算机可读取存储介质中。上述软件功能单元存储在一个存储介质中，包括若干指令用以使得一台计算机装置(可以是个人计算机，服务器，或者网络装置等)或处理器(Processor)执行本说明书各个实施例所述方法的部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(Read-Only Memory；以下简称：ROM)、随机存取存储器(Random Access Memory；以下简称：RAM)、磁碟或者光盘等各种可以存储程序代码的介质。

[0105] 以上所述仅为本说明书的较佳实施例而已，并不用以限制本说明书，凡在本说明书的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本说明书保护的范围之内。

Claims

1. A distributed connection method, characterized in that, The method includes: Based on the semantics of the clauses in the SQL, the SQL is split into parallel executable subqueries and dependency subqueries; By grouping the target tables involved in the clauses into table groups, the join relationships in the target tables are converted into table pool join relationships in the table groups; Generate detach nodes between the table groups; Local connection calculations are completed by performing the separate node execution scheduling. All tables in the same table group have the same data distribution method in data storage, that is, the same data nodes, the same data sharding algorithm, and there are sharding column equality query conditions belonging to "AND"; The results of grouping and executing each table are connected through the separation node. Based on the algorithm idea of ​​nested loops, the previous separation node is used to complete local calculations and is used for the next separation node; the driving table data required for the connection operation is passed through temporary tables.

2. The distributed connection method according to claim 1, characterized in that, Before splitting the SQL into parallel-executable subqueries and dependency subqueries based on the semantics of the clauses in the SQL, the process also includes: The semantics of the clause are obtained by parsing the SQL.

3. The distributed connection method according to claim 2, characterized in that, The separate nodes are used to construct specific physical execution steps.

4. A distributed connection device, characterized in that, The device includes: The splitting module is used to split the SQL into parallel executable subqueries and dependency subqueries based on the semantics of the clauses in the SQL. The conversion module is used to group the target tables involved in the clause into table groups, and convert the join relationships in the target tables into table pool join relationships in the table groups. The generation module is used to generate separate nodes between the table groups; The scheduling module is used to complete local connection calculations by performing the execution scheduling of the separated nodes; All tables in the same table group have the same data distribution method in data storage, that is, the same data nodes, the same data sharding algorithm, and there are sharding column equality query conditions belonging to "AND"; The results of grouping and executing each table are connected through the separation node. Based on the algorithm idea of ​​nested loops, the previous separation node is used to complete local calculations and is used for the next separation node; the driving table data required for the connection operation is passed through temporary tables.

5. The apparatus according to claim 4, characterized in that, Also includes: The parsing module is used to parse the SQL to obtain the semantics of the clause.

6. The apparatus according to claim 5, characterized in that, The separate nodes are used to construct specific physical execution steps.

7. A storage medium, characterized in that, The storage medium includes a stored program, wherein, when the program is executed, it controls the device where the storage medium is located to perform the distributed connection method according to any one of claims 1-3.

8. A server comprising a memory and a processor, the memory for storing information including program instructions, and the processor for controlling the execution of the program instructions, characterized in that, When the program instructions are loaded and executed by the processor, they implement the steps of the distributed connection method according to any one of claims 1-3.

Images