Heterogeneous database oriented massive multicast data storage implementation method

Abstract

The invention provides a heterogeneous database oriented massive multicast data storage implementation method. The heterogeneous database oriented massive multicast data storage implementation method comprises receiving multicast massages according to a certain limit cycle and quantity restrictions; storing the required messages into a first level cache data file and triggering the analysis on the first level cache data file through signals; storing the massages into corresponding models of second level cache data files and establishing second level cache data file storage catalogues and related data files and control files corresponding to the models; performing synchronous triggering of storage operation on models of storage threads according to the signals, performing analysis on the second level cache data files to be stored and inputting the analyzed data into corresponding databases through calling of an SQL * Loader and storage control file data. According to the heterogeneous database oriented massive multicast data storage implementation method, the real-time stable receiving processing of massive multicast data can be achieved, the synchronous efficient storage of multi-objective databases can be achieved, the processing capacity on the massive multicast data is improved in practical application, the integrity of the data is ensured, and the efficiency of the application system is improved.

Description

technical field [0001] The invention belongs to the field of computer data collection, and in particular relates to a method for implementing storage of massive multicast data facing heterogeneous databases. Background technique [0002] In the parallel testing of multiple models of large-scale systems, in the face of heavy tasks, multiple models, and a large number of repeated tests, the massive test data generated by the test needs to be received in real time and stored in the database of each model. The processing of this massive data has real-time It has the characteristics of strong adaptability, large amount of data, complex data format and heavy storage pressure. [0003] Generally, multicast data storage is based on different data cache methods, and there are two storage methods based on shared buffers and data files. The storage based on the shared buffer generally adopts the "producer-consumer" model or its variants. The "producer" stores the received multicast da...

AI Technical Summary

Problems solved by technology

Store and classify multicast data through two-level cache data files, and ensure that the analysis and storage of each model's data is completed by a dedicated storage thread through multi-thread storage, which improves data processing efficiency and at the same time, Solved the problem of multiple heterogeneous database storage

Images