Index data pre-aggregation method

Abstract

The invention discloses an index data pre-aggregation method, which comprises the following steps of: index data is acquired in real time, and acquired burying point information is reported to a cat monitoring server; the cat monitoring server stores the burying point information details into a click home database and stores the aggregated index data into an InfluxDB database, and the index data is generated by aggregating the burying point information according to a preset rule; the burying point information is stored in a click database in the form of a burying point detail table, data query and analysis are carried out according to the burying point detail table, and according to the burying point detail table data, an index is generated through re-aggregation according to needs and is stored in the InfluxDB.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a method for pre-aggregating index data. Background technique [0002] With the advancement of the digital wave of enterprises, data has become the most important asset of most enterprises, and top companies all over the world rely on data to support their business decisions. Statistics show that most of the business decisions of the top 40% of companies are based on data-driven, that is, organizations that drive business decisions through data are more likely to reach the top of the industry than those driven by experience and instinct. [0003] Traditional enterprises often use tools such as Excel to organize and analyze data and generate corresponding data charts. However, with the exponential growth of business data and the rapid changes in the market environment, traditional data analysis models often It can only reflect expired data and support fixed-dimensional data ...

AI Technical Summary

Problems solved by technology

[0007] 1) The construction of big data system is complex and expensive:

[0008] If it is built according to the big data system, one is the high investment cost of technical personnel, and the other is the high investment cost of machine resources. For medium-sized enterprises with data scale and personnel scale, there will be a huge cost burden;

[0009] 2) The collected indicator data has a high threshold for use:

[0010] After the collected indicator data flows into the analytical data system, it is difficult for partners who do not understand technology such as product managers and operations to make offers and use it directly, because they need to understand the table structure and fields, and need to write SQL query statements to aggregate indicator data Queries often require the support of BI personnel to complete;

[0011] 3) Indicator data cannot be pre-aggregated on demand:

[0012] The collected and stored index data can be fully supported by employees in the enterprise when performing real-time query and analysis, but once it is extended to the client side (B-end, C-end) or applied in real-time quasi-real-time transaction calculations, it must be pre-prepared. Aggregation, otherwise big data technology is difficult to support (unless resources are used to heap)

However, pre-aggregation often fails to achieve the aggregation of past historical data, thus entering the dilemma of whether this indicator should be pre-aggregated or not.

Images