Unlock instant, AI-driven research and patent intelligence for your innovation.

Data table connection method and device

A connection method and data table technology, applied in the database field, can solve problems such as consumption of computing resources, and achieve the effect of reducing computing resources and data volume

Active Publication Date: 2021-02-19
ALIBABA GRP HLDG LTD
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In a typical "star" Join scenario, assuming that the data table to be joined includes a main table and n auxiliary tables, and the main table contains M data records, then when performing Join calculations on the main table and n auxiliary tables, shuffle The total amount of data that needs to be processed for sorting includes the amount of data that needs to be processed by the shuffle main table, which is M ＊ The amount of data that needs to be processed by n and shufflen auxiliary tables, which will consume a lot of computing resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data table connection method and device
  • Data table connection method and device
  • Data table connection method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0022] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

[0023] In the query process of a distributed data warehouse, it is often necessary to perform join calculations between data tables. In the prior art, when processing the Join operation between data tables, since the data tables to be joined are relatively large, generally all the data tables to be joined are shuffle-sorted by MapReduce first, and the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The application provides a data table connection method and device. The method includes: receiving a data table connection task, the data table connection task indicates to perform a connection operation on the first data table and the second data table according to the connection condition; according to the connection condition, loading the data records in the second data table into the distributed system On at least two nodes; read the data record in the first data table as the current data record, determine the target node from at least two nodes according to the connection condition corresponding to the current data record, and read the second data stored on the target node The data record in the data table is used as the target data record; the connection operation is performed on the current data record and the target data record. The application can reduce the computing resources consumed by data table connection operations.

Description

【Technical field】 [0001] The present application relates to the technical field of databases, in particular to a data table connection method and device. 【Background technique】 [0002] With the development of the Internet, data has shown explosive growth, and data structures have begun to diversify, and data contains more and more information. Data warehouses play a huge role in this context. Due to the advent of the big data era, data warehouses have been transformed into distributed architectures to meet the explosive growth of computing and storage needs. Distributed data warehouses generally use columnar storage and store data in the form of files. Therefore, using distributed data warehouses can improve the storage and computing performance of big data. [0003] In the query process of a distributed data warehouse, it is often necessary to perform join calculations between data tables. In the prior art, when processing the Join calculation between data tables, all th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22
CPCG06F16/2282G06F16/00
Inventor 吴炜
Owner ALIBABA GRP HLDG LTD