Data processing method, system and computer readable storage medium

Abstract

The invention discloses a data processing method. A business processing system sends generated first business data to a streaming data processing system through a first path, and stores the first business data in a first database; the streaming data The processing system reads the second service data corresponding to the first service data from the first database; the stream data processing system performs stream processing according to the first service data and the second service data. The invention also discloses a data processing system and a computer-readable storage medium. The invention can guarantee the timeliness, accuracy and completeness of data processing in the business processing system.

Description

technical field [0001] The present invention relates to the field of data processing, in particular to a data processing method, system and computer-readable storage medium. Background technique [0002] With the development of science and technology, people's life has entered the data age, because many people's lives involve network business, banking business, etc., and each business will store the data of each user, so that a large number of users, or each user's preferences , behavior, etc., and then can fully and correctly understand the operation of the system, as well as correctly understand and infer user preferences, behavior, etc. [0003] For some special scenarios, such as transaction data, there are differentiated requirements, including the need for real-time reports, obtaining the characteristics of the customer's transaction behavior on the day, and risk control and warning of the customer's transaction behavior. [0004] At present, reports are generally obt...

AI Technical Summary

Problems solved by technology

However, there are defects in the way of adding new data processing module functions in the trading system for data synchronization processing: first, the system functions are coupled, and the trading system functions are chaotic and unclear; second, the scalability is poor, and the data requires multiple latitudes. There are so many rules that need to be monitored. With the increasing complexity of the business, it is not suitable for expansion; third, it affects the performance of the transaction system. The transaction system needs to add new data processing functions in the transaction process, marketing performance and timeliness

Because the streaming data processing system and the transaction system are asynchronous, the integrity of the data cannot be effectively guaranteed. If there is a problem in the path between the transaction system and the streaming data processing system, the entire streaming data processing system will be affected.

Images