Data mining method and node

Abstract

The invention provides a data mining method and node; the method comprises the following steps: obtaining an estimated execution time of each calculating sub-node in a present round iteration task, and assigning corresponding task data bulk for the calculating sub-nodes according to the estimated execution time; collecting execution condition information of each calculating sub-node of the present round iteration task after the present round iteration task is executed, and determining whether to adjust the task data bulk or not for each calculating sub-node of next round iteration task; carrying out the next round iteration task according to the adjusted task data bulk, thus assigning corresponding task data bulk according to the ability of each calculating sub-node, and adjusting the task data bulk for each calculating sub-node of the present round iteration task according to the execution conditions of the previous round; the method and node can prevent unnecessary load balancing process, reduce network expenditure, and can improve system data mining performance.

Description

technical field [0001] The embodiments of the present invention relate to communication technologies, and in particular to a data mining method and node. Background technique [0002] Big data, or huge amount of data, refers to a collection of data that involves such a large amount of data that it cannot be captured, managed, processed, and organized within a reasonable time by conventional software tools. With the advent of the cloud era, big data (Bigdata) has attracted more and more attention, and how to obtain useful information and knowledge from big data has become the focus of the industry. Data mining (Data Mining) is a technology that uses algorithms to search for hidden information from a large amount of data. Data mining usually uses statistics, online analysis and processing, intelligence retrieval, machine learning, expert systems (relying on past rules of thumb) and Pattern recognition and many other methods to achieve the above goals. [0003] In the process...

AI Technical Summary

Problems solved by technology

However, in the above-mentioned prior art, since the amount of task data input in each round of iteration of each calculation child node is constant, the load balancing in each round of iteration tasks is independent for the next round of iteration That is t

Images