Large-scale distributed data collection system and method

Abstract

The invention provides a large-scale distributed data collection system and method, and belongs to the technical field of data collection. The system comprises a task distributed node module, a management node module, a task warehouse module and a collection node module. A management node is used for providing a distributed synergic service, receiving and managing registered information of a collection node and a task distributed node, and performing communication on the collection node and the task distributed node on the basis of the certain frequency; the task distributed node is used for setting, distributing and monitoring collection tasks; a task warehouse is used for receiving and storing a collection task sent by the task distribution node; the collection node is used for obtaining the collection task from a task input warehouse and completing collection of corresponding data in an on-site control station; the collection task executed by the collection node can dynamically change according to user settings, a certain or some faults in the task distribution node and the collection node will not affect the current collection period collection result, and the large-scale distributed data collection system has the high flexibility, high tolerance and high reliability.

Description

technical field [0001] The invention belongs to the technical field of data acquisition, and in particular relates to a large-scale distributed data acquisition system and method. Background technique [0002] With the development of industrial enterprises, the expansion of production scale and the great increase in the amount of data collection, enterprises have higher and higher requirements for the reliability of data collection, and it is more and more difficult to ensure the reliability of data collection. [0003] The current patent on large-scale distributed data acquisition is mainly "CN105527948A (a large-scale distributed data acquisition system and method based on industrial process)", which uses a data acquisition scheduling module to distribute data to multiple data acquisition clients. For the acquisition task, when the acquisition client fails, the next acquisition cycle will be rescheduled by the acquisition scheduling module to ensure the reliability of the ...

AI Technical Summary

Problems solved by technology

Although this patent can improve the reliability of collection to a certain extent, there are still certain limitations in the collection process:

[0004] 1. In this patent, the collection scheduling module assigns collection tasks to the collection client according to the operation status information of the on-site control station. The collection performed can only be performed in units of on-site control stations, and cannot be collected in units of data items. And all collection tasks are not prioritized, and the collection flexibility is not high enough

For occasions that need to collect one or several items of data in all on-site control stations for analysis, or occasions where the importance of each collection task is different, this patent cannot meet the requirements

[0005] 2. The system described in this patent lacks an independent collaborative management module to manage and automatically load balance the nodes in the distributed system. When multiple collection and scheduling modules are required, the system is difficult to achieve dynamic expansion and system flexibility. not high enough

[0006] 3. This patent can only guarantee the normal collection of the next collection cycle when the collection client is faulty. For the current collection cycle of the fault, it does not provide a good disaster recovery processing mechanism to ensure the reliability and fault tolerance of collection. Therefore, once a fault occurs , the data at the time of the failure is easy to lose. For production occasions that require high collection reliability, this loss will also cause certain economic losses to the enterprise.

When multiple collection and scheduling modules are used, the coupling between the collection and scheduling modules and the collection client is relatively large, and the communication between the two is complicated. When multiple collection and scheduling modules send tasks to the same collection client, each collection and scheduling module’s The time for the task to arrive at the collection client is different, and the time for the collection client to wait for the assignment of the task is uncertain, resulting in a longer waiting time for the collection client and reduced system efficiency

[0008] To sum up, the existing patents have obvious deficiencies in data acquisition function, flexibility, high efficiency and ensuring the reliability of data acquisition, and it is difficult to meet the demand for real-time data acquisition in the process of the current manufacturing enterprises' transformation to intelligence

Images