A data acquisition processing method, system, device and storage medium

Abstract

The application discloses a kind of data acquisition processing method, system, device and storage medium, obtain the data of data source, according to the data of data source determine corresponding message body, message body is sent to message queue, message body is obtained from message queue in turn, according to message body determine the state of data source, when data source is existence and the state of opening, according to the data source coding and keyword information of data source determine the target processor of processing data, then, data is distributed to target processor and is cleaned and handled, and according to the data after cleaning obtains several data sets, data set is sent to database and is stored;Data of data source is obtained and according to data obtains message body, message body is sent to message queue, and data is handled from message queue in turn and is read for data processing, data source and data processing are decoupled, improve the efficiency of data processing;The present application embodiment can be widely applied in data processing technical field.

Images