Big data statistical collection algorithm

Abstract

The invention discloses a big data statistical collection algorithm. Data marking is carried out at the source of data acquisition; service logic definition is carried out on the data; after definition, the system performs real-time calculation in a memory according to the data; according to the mark type of data acquisition; defining real-time computational models, the input original data is calculated in real time to generate a data result required by the real-time calculation service; results storage in memory, the data can be displayed in a big data platform through a websocket technologyor http polling according to needs; the data can be stored in the database through regular updating according to needs, and batch service storage is carried out according to the data marks, so that the calculation pressure required by service result calculation required by database query is reduced, a data source does not need to be loaded for repeated calculation, the equipment starting time is saved, and the efficiency is higher.

Description

【Technical field】 [0001] The invention relates to big data statistics and collection algorithm technology, in particular to a big data statistics collection algorithm. 【Background technique】 [0002] Today, with the rapid development of the Internet, the statistics and collection of data related to the system have become more and more complex and the amount of data is extremely large. At present, a large amount of data collection processes are stored in the database, and the database is in real time when a large amount of data is queried. It will be very slow. This method cannot better reflect the timeliness and efficiency brought by data statistics and collection. At this stage, big data storage statistics and collection exist in various business scenarios. At this stage, various enterprises adopt cloud The third-party plug-in provided by the manufacturer provides cloud analysis and data analysis, so it is necessary to recruit relevant deployment and operation and maintenan...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of the Internet today, the statistics and collection of system-related data have become more and more complex and the amount of data is extremely large. At present, a large amount of data collection process is stored in the database, and the database is queried in real time when a large amount of data It will be very slow. This method cannot better reflect the timeliness and efficiency brought by data statistics and collection. At this stage, big data storage statistics and collection exist in various business scenarios. At this stage, various enterprises adopt cloud The third-party plug-in provided by the manufacturer provides cloud analysis and data analysis, so it is necessary to recruit relevant deployment and operation and maintenance personnel. The related implementation cost is relatively high, and the implementation method is relatively complicated. It requires the coordination and cooperation of product development and cloud plug-in operation and maintenance personnel. In order to avoid the situation where the business of the big data scene is not complicated, how to realize the data statistics and collection of big data becomes extremely important. Set is extremely important to the cost of the enterprise and the complexity of operation and maintenance

Images