Cloud platform log storage system capacity expansion method and device

Abstract

The invention provides a cloud platform log storage system capacity expansion method and a device. The method comprises the steps of obtaining cluster resource use condition monitoring data of a log storage layer of a cloud platform log storage system; generating a capacity expansion request according to the use condition monitoring data and a preset cluster storage capacity warning value; and expanding the capacity of the cloud platform log storage system according to the capacity expansion request and a preset capacity expansion algorithm. The invention provides a capacity adaptive type transverse expansion log balanced storage method. On the premise that log query convenience of a user is not affected, three collective rapid transverse expansion is achieved according to alarm information of the log storage layer, meanwhile, after transverse expansion, an application log balanced distribution suggestion scheme of the log storage layer is provided, operation and maintenance personnelare suggested for adjustment, and therefore the problems that idle cluster resources are wasted, and the performance of a full-load cluster is reduced are solved.

Description

technical field [0001] The invention relates to data processing technology, in particular to a method and device for expanding the capacity of a cloud platform log storage system. Background technique [0002] With the increase of connected applications and the large-scale promotion of service work, the amount of container log data on the cloud is increasing. In order to facilitate data analysis and mining, centralized collection, analysis and storage of log data has become the only choice. [0003] In the existing technology, the centralized container log collection, analysis and storage architecture commonly used in the industry mainly includes four parts: the log collection layer, the log buffer layer, the log consumption layer and the log storage layer. Log data is usually divided by application or type. After the log collection layer collects the application logs, it puts them into the log buffer layer cache, and after temporary storage and analysis by the log consumpt...

AI Technical Summary

Problems solved by technology

[0004] As the amount of log data grows, the log buffer layer and log storage layer will face insufficient cluster resources, and the log consumption layer may face a situation where consumption capacity cannot keep up. The common method is to horizontally expand the layer according to the alarm of a certain layer , but in this way, the capabilities of the buffer layer, consumption layer, and storage layer will not match, so that the performance of the log centralized analysis storage system will be greatly reduced

Images