一种用于并发场景的多维度实时数据分析方法

By monitoring data skewness and access frequency deviations, and optimizing node granularity and the number of virtual nodes, the problems of uneven data distribution and node load imbalance in high-concurrency scenarios were solved, thereby improving request processing efficiency and data analysis stability.

CN121387551BActive Publication Date: 2026-07-17BEIJING ZHISHENGJIAO TECHNOLOGY CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-10-29
Publication Date
2026-07-17

AI Technical Summary

Technical Problem

Existing technologies lack the ability to dynamically monitor and optimize data distribution balance, node load status, and adjustment effects in high-concurrency scenarios, leading to increased data request processing latency and decreased data analysis stability.

Method used

By monitoring data skewness and access frequency deviations, physical node types are determined, and node optimization is performed. This includes adjusting node granularity and the number of virtual nodes to optimize data distribution and resource utilization, including dynamic adjustments based on SQL execution time and load saturation.

Benefits of technology

It improves request processing efficiency and real-time analysis stability in high-concurrency scenarios, avoids resource waste caused by indiscriminate adjustments, and ensures efficient and stable operation and resource utilization of distributed clusters.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121387551B_ABST
    Figure CN121387551B_ABST
Patent Text Reader

Abstract

本发明涉及数据分析技术领域,尤其涉及一种用于并发场景的多维度实时数据分析方法,包括:根据数据倾斜度以及访问热度偏差确定物理节点类型,并根据一类物理节点占比以及性能衰减率确定是否进行节点优化;节点优化中,根据各一类物理节点的物理节点状态确定基于数据增量值针对节点粒度上限值进行调节或基于评估比对值针对节点粒度下限值进行调节;基于调节异常度确定是否进行二次优化,二次优化中,基于倾斜参考值以及物理节点数量与虚拟节点数量的数量比值确定针对虚拟节点数量进行调节或针对传输间隔进行调节。本发明能够提高并发场景下的请求处理效率与实时分析稳定性。
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Data request processing method and device applied to high-concurrency scenario

    CN109150929A

  • Data skew optimization factor automatic adjustment method and device, equipment and medium

    CN114490160A

  • Rapid message processing method based on high-concurrency scene

    CN118377635A