Application data management system and processing method, device and server

Abstract

The embodiment of the invention discloses an application data management system, an application data processing method, an application data processing device and a server. The system comprises an asynchronous communication component, a message publishing and subscribing component, a column database and a data consumption process group, and the asynchronous communication component caches application data asynchronously reported by different service applications; wherein the asynchronous communication component periodically writes the application data into the publish-subscribe message component, and the data consumption process group periodically pulls the application data from the publish-subscribe message component in parallel and transfers the application data into the column database. According to the technical scheme provided by the embodiment of the invention, the application data can still be stored according to the specified rhythm under high-concurrency application data reporting, the peak clipping is carried out on the storage efficiency of the application data, the high efficiency and stability of application data processing are improved, and the load pressure of a columntype database is greatly reduced; meanwhile, excessive machines do not need to be deployed in the service cluster, so that the deployment complexity of application data management is reduced.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of data processing, and in particular to a management system, processing method, device and server of application data. Background technique [0002] At present, a large number of application programs have been developed for different business types in the Internet field. When each application program performs any business operation, it will call the corresponding interface to report the corresponding application data to the server. Class application data is stored in the corresponding database, and then a daemon process is used to monitor the abnormality of the application data in the database; however, when the server does not respond in time due to excessive data reporting, it will directly affect the user The business execution on the application program and the high-concurrency data reporting will cause the storage bottleneck of the database. [0003] In response to the above ...

AI Technical Summary

Problems solved by technology

[0002] At present, a large number of application programs have been developed for different business types in the Internet field. When each application program performs any business operation, it will call the corresponding interface to report the corresponding application data to the server. Class application data is stored in the corresponding database, and then a daemon process is used to monitor the abnormality of the application data in the database; however, when the server does not respond in time due to excessive data reporting, it will directly affect the user The business execution on the application program, and the high concurrent data reporting will cause the storage bottleneck of the database

[0003] In response to the above problems, the existing data management system usually increases the machine deployment of the server to use a high-concurrency service cluster to receive the application data reported by each application. However, if the data reported by the application exceeds tens of thousands, it will As a result, too many machines in the service cluster are deployed, which increases the complexity of the data management system and increases the load pressure on the database, which directly affects the efficiency of data management under high concurrent data reporting.

Images