The invention discloses a pre-aggregation storage table creation method, which relates to the technical field of
data processing, and comprises the following steps: S1, dynamic rule generation: firstly, through a
big data analysis
system, carrying out deep analysis on a large amount of historical data in an original table, comprehensively mastering
data type classification, and analyzing daily average data volume and data time window characteristics; the method comprises the following steps of S1, performing multi-level aggregation window division, S3, performing real-time feedback optimization, S4, performing distributed cooperative
processing, S5, performing a self-
adaptive compression mechanism, S6, performing a fault-tolerant
recovery mechanism, S7, performing
data quality monitoring, S8, performing
metadata management and S9, performing external
system butt joint. Through a dynamic rule generation real-time feedback optimization step and a self-
adaptive compression mechanism step, the effects of
data processing accuracy, high efficiency and storage cost reduction are achieved, and through a multi-stage aggregation window division distributed cooperative
processing step and a fault-tolerant
recovery mechanism step, the reliability of
data processing is improved. And the effects of enhancing the complex data
processing capability of the
system, improving the reliability of the system and guaranteeing the service continuity are achieved.