High-concurrency and high-decoupling data processing method and data middle station system

Abstract

The invention discloses a high-concurrency and high-decoupling data processing method and a data middle station system and relates to the field of data middle stations, and the method comprises the following steps that: an Ngnix server receives packaged data and distributes the packaged data to a Publisher instance, the Publisher instance distributes the packaged data to a message queue accordingto the service type; and the Consumer instance acquires the packaged data from the message queue and processes the packaged data according to the service type to obtain a processing result, the Consumer instance stores the processing result to a local database, and the local database provides a local port according to the service dimension and provides a corresponding mapping port in a public network. The method has the advantages that high concurrency, high decoupling and low cost of the data center station system are achieved, the architecture is clear, and expansion is easy.

Description

technical field [0001] The technical field of data middle platform of the present invention, in particular relates to a high concurrency and high decoupling data processing method and a data middle platform system. Background technique [0002] Zhongtai is the first concept extended by Ali in 2015 in the strategy of "big, medium and small front desk". The overall definition of data middle platform has also become clear with the implementation of data types by head enterprises. Data middle platform It refers to the collection, calculation, storage, and processing of massive data through data technology, while unifying standards and calibers. According to the relevant data report of the well-known data analysis organization Canalys in February 2019, the global cloud computing market in 2018 exceeded 80 billion US dollars, reaching 80.4 billion US dollars; in the next 10-15 years, the "data center" may surpass today's cloud computing market, Form a trillion-level market. [0...

AI Technical Summary

Problems solved by technology

[0005] (1) Data processing lacks unified standards, is coupled with specific businesses, and has poor flexibility. How to unify the caliber, standards and high timeliness

[0006] (2) The boundary of the architecture is not clearly defined, the data reporting and data processing are not separated, there is ambiguity, and it is difficult to expand

[0007] (3) Cloud server rental costs are high, data storage costs are high, and later migration is limited, and it is forcibly bound by data center solution providers

Images