Multi-source service data real-time processing method and device, terminal and storage medium

Abstract

The invention relates to the technical field of big data, and provides a multi-source service data real-time processing method and device, a terminal and a storage medium, and the method comprises thesteps: configuring a data push interface for a corresponding service system according to the database types of a plurality of service systems; when the service data pushed by the corresponding service system is received through the data push interface, aggregating the service data to obtain standard data; writing the standard data into a data center according to a preset rule; acquiring target data from the data center by adopting a preset window based on a preset data processing framework; when a preset timer expires, triggering a timing task to verify the target data; and when a verification result obtained by verification is a target result, generating a data report according to the target data. According to the invention, the service data of multiple data sources can be processed in real time, and the processing efficiency of the service data is improved.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a method, device, terminal and storage medium for real-time processing of multi-source business data. Background technique [0002] With the development of the company's business, there are more and more business data, and the sources of business data are becoming more and more complicated, for example, from relational databases, non-relational databases, or files such as excel. For business development, real-time The business data of these different business systems are aggregated and analyzed. [0003] However, the inventor found in the process of implementing the present invention that the existing data summarization method is to synchronize business data in an offline manner, and then summarize and analyze the synchronized business data. The timeliness of the summarization analysis is usually T -1 day, the analysis results of business data are lagging behind, and the...

AI Technical Summary

Problems solved by technology

[0003] However, the inventor found in the process of implementing the present invention that the existing data summarization method is to synchronize business data in an offline manner, and then summarize and analyze the synchronized business data. The timeliness of the summarization analysis is usually T -1 day, the analysis results of business data are lagging behind, and the accuracy rate cannot be guaranteed; in addition, only the data of a single business system can be summarized in real time. It is very difficult to cross real-time summary of the business data of the system

Images