A kind of iot big data real-time time series flow analysis application technology method

Abstract

The invention discloses an application technology method for real-time time series flow analysis of IOT big data, which is characterized by comprising the following steps: Step 1: building a database of massive databases: S1: analyzing original data; S2: data classification and structured storage; S3: Data objects are mapped to the database structure to establish massive databases; S4: Establish massive database indexes; S5: Low real-time analysis service; Step 2: Real-time data analysis: S6: Massive data generated by caching; S7: Real-time statistical analysis service; S8: Cache Analyze the results. The invention has a reasonable design, introduces data flow storage and real-time analysis of flow data means, and combines the two organically with complementary advantages, solves massive real-time time series data storage, accelerates data query and analysis, and provides real-time data analysis and resource scheduling technology. Provide support.

Description

technical field [0001] The invention is based on the application field of big data real-time sequence flow analysis technology, and specifically relates to an application technology method for IOT big data real-time sequence flow analysis. Background technique [0002] The concept of Real-time Series Flow Analysis Of Big Data is to model and analyze the collected massive time series data according to the data characteristics, combined with relevant knowledge in the field or industry, and mine the value existing in the big data. , so that decision makers can make better decisions. In terms of technical logic, big data real-time time series flow analysis can be divided into "real-time data collection and storage + data real-time analysis". The real-time data collection and storage technology is to perform ETS (Extract-Transform-Save) operation on the data directly at the data source. A distributed stream storage. The data real-time data analysis technology analyzes the coll...

AI Technical Summary

Problems solved by technology

[0005] Efficient query statistical analysis: due to the massive data problems faced by various platforms, there are bottlenecks and difficulties in dynamic data query on the stored raw data, and various computing performances required for data analysis are also affected. Traditional data storage methods cannot Efficient and fast query and analysis data, the technical indicators can not reach 20 million records per second (single core, 16 bytes per record, full memory);

[0007] Real-time analysis of data: Big data analysis of many systems extracts and integrates data from required data sources, extracts data entities and relationships from them, stores data in a uniformly defined structure after association and aggregation, and then All data features are analyzed and processed, so that the analyzed results will lack real-time performance, which will not meet the analysis with high real-time requirements;

Images