Data acquisition method and related equipment

Abstract

The invention provides a data acquisition method and related equipment. A first-level distribution message containing attribute information and a second-level distribution identifier is pre-constructed for different to-be-acquired data tables, before the data acquisition of each to-be-acquired data table, whether corresponding to-be-acquired data tables need to be subjected to sub-table acquisition or not is determined according to a first-level distribution message, corresponding split information stored in a message queue is called from the to-be-split data tables which need to be subjected to sub-table acquisition, a plurality of data acquisition devices are controlled to perform segmented data acquisition on the to-be-split data tables according to the corresponding split information, and the plurality of acquired data sub-files are combined into one target data file, so that segmented parallel detection on the to-be-acquired data table with a large data volume is realized, the data acquisition efficiency is improved, and the technical problem of resource waste of data acquisition equipment caused by performing data acquisition on the to-be-acquired data tables by single data acquisition equipment is solved.

Description

technical field [0001] The present application relates to the field of communication technology, and more specifically relates to a data collection method and related equipment. Background technique [0002] At present, financial institutions such as banks involve a wide variety of businesses. In order to meet the different business needs of different customers, the banking system often needs to collect customer business information every day, such as data sheets of various businesses associated with customers, and send them to Store in the corresponding business platform for reference by other systems. [0003] Faced with a large number of data tables to be collected, the distributed control system deployed by the banking system usually uses multiple devices to simultaneously perform data collection tasks to complete the collection of a large number of data tables. However, in this data collection method, business personnel usually determine the data collection tasks of mu...

AI Technical Summary

Problems solved by technology

However, in this data collection method, business personnel usually determine the data collection tasks of multiple devices based on the data volume of the data tables to be collected. It is easy to happen that after a certain device completes the assigned data collection task, other devices are still performing data collection tasks, resulting in the inability to fully utilize the resources of each device and reducing the data collection efficiency

Images