Distributed data acquisition system operation method

Abstract

The invention relates to an operation method of a distributed data acquisition system. The operation method comprises the following steps: arranging acquisition units in the same local area network; performing acquisition configuration on a single acquisition unit, and synchronizing configuration information and performing antique configuration backup on other acquisition units; each acquisition unit dynamically and circularly applies for a task according to own hardware configuration and residual acquisition performance, and executes a response acquisition task; each acquisition unit broadcasts the acquired data to a local area network, and other acquisition units synchronously update the data result of the acquisition task. According to the method, a decentralized chained technology is adopted, data acquisition task configuration information is broadcasted in real time and synchronously stored in multiple units, configuration loss is not prone to being caused, and the operation and maintenance complexity is reduced; the data acquisition unit dynamically applies for and executes an acquisition task according to the performance configuration and the operation condition of the data acquisition unit, so that autonomous load balancing is achieved; in addition, capacity reduction and capacity expansion can be flexibly carried out, operation of the rest of the whole system is not affected, and the system stability is improved.

Description

technical field [0001] The invention relates to the technical field of data analysis and processing, in particular to an operation method of a distributed data acquisition system. Background technique [0002] The 21st century is the era of big data. High-value sample data is the basis of many application technologies such as statistics and deep learning. A stable and efficient data collection system is the cornerstone of all big data platforms. [0003] In the face of huge data collection requirements, the current traditional data collection system adopts single-point collection, and technicians calculate the workload of the collection task to manually deploy the corresponding number of services to carry out the collection work. Finally, each single-point service will collect The data is uploaded to the central database or business unit through the corresponding network for summary processing. [0004] And the defective that the above method exists is: [0005] 1. The ove...

AI Technical Summary

Problems solved by technology

[0005] 1. The overall stability of the system is poor, and a single point of failure will affect the failure of the corresponding collection task, resulting in the loss of upper-level business system data;

[0006] 2. The complexity of collection tasks is different, and the performance configuration of data collection equipment is also different. It is difficult to balance the load of each data collection unit and maximize the utilization of data collection equipment resources;

[0007] 3. If the hardware of the data acquisition unit is damaged or the system crashes, if it is not backed up to a stable network location in time, the system configuration will be lost, making the task of operation, maintenance and repair difficult

Images