Connection query system and method for distributed data warehouse

Abstract

The invention discloses a connection query system and a connection query method for a distributed data warehouse. The system comprises a master node, mapping work nodes and reduction work nodes, wherein the master node calculates a fragment size according to the size of a data table and system performance, allocates a data block to the mapping work node based on the calculated fragment size, and formulates fragmentation mapping rules and summarization rules in the mapping work node; the mapping work node maps a query keyword in the data block to a corresponding fragment number according to the fragmentation mapping rules, and transmits data with the same fragment number to a specified reduction work node according to the summarization rules; and the reduction work node receives the data from the mapping work node, combines the data with the same fragment number and establishes connection according to the query keyword to obtain a final connection query result. By the system and the method, data transmission in a distributed system is reduced, the data volume and program complexity of the reduction work node are decreased, and the performance of the distributed data warehouse is improved.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to view more

AI Technical Summary

Benefits of technology

This patented technology helps optimize use of resources by dividing up large amounts of data across multiple servers or storage devices instead of sending it all over one network. It uses specialized software that allows users to connect with specific files quickly without having any unnecessary connections being sent through other parts of their computer networks. Additionally, this technique increases the number of connected computers while reducing redundancy levels. Overall, these improvements help make more efficient and effective data management systems run better than traditional methods like storing them locally at each server individually.

Problems solved by technology

Technological Problem: Current solutions involve distributing computational resources across multiple servers or machines, leading to slower communication times and lower throughput rates compared to more efficiently distributed computation models. Additionally, current methods require significant amounts of space allocation and may lead to poor overall performance if too much connections occur simultanously.

Images