一种分片数动态调整方法、装置、设备及介质
By periodically monitoring historical data volume in Elasticsearch daily and dynamically adjusting the number of shards, the cluster downtime issue caused by improper default shard settings was resolved, achieving reasonable sharding and ensuring cluster stability and effective data ingestion.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU DBAPPSECURITY CO LTD
- Filing Date
- 2022-10-21
- Publication Date
- 2026-07-17
AI Technical Summary
In a big data environment, improper default sharding settings for Elasticsearch indexes can lead to disk overload or excessive JVM usage, causing cluster downtime, affecting data querying and aggregation, and even causing cluster crashes.
By regularly checking the historical data volume every day, the target index and target number of days are determined, the number of shards is dynamically adjusted, the expected number of shards is calculated based on the target number of days and data volume, the target number of shards is set in combination with the number of nodes, and the number of shards is adjusted in real time to achieve reasonable sharding.
This effectively avoids wasting sharding resources, prevents excessive index creation during rolling, achieves reasonable sharding, and ensures stable operation of the Elasticsearch cluster and normal data ingestion.
Smart Images
Figure CN115658689B_ABST