High-reliability and lossless data acquisition and storage method

Abstract

The invention discloses a high-reliability and lossless data acquisition and storage method. After receiving network data sent by a client, a server inserts the data into the tail of a message sendingqueue, informs a database storage thread and then stores the network data according to a massive storage overhead caching algorithm; the database storage thread reads the data from the head of the message sending queue and inserts the data into a background database; and a heartbeat message is sent between a main acquisition and transmission server and a slave acquisition and transmission serverregularly to detect the survivability of the main server. According to the method, the data can be sent to a server module in a high stability and high reliability mode in the high traffic and outburst huge traffic environment, and meanwhile it is guaranteed that the acquired data is losslessly forwarded to the background database on the abnormal condition generated in the switching process of a main server module and a slave server module.

Description

technical field [0001] The invention relates to the technical field of information collection and analysis, in particular to a highly reliable and lossless data collection and storage method. Background technique [0002] During the operation of the data acquisition system, it is necessary to record the parameters, status and other information of the system operation for future analysis of the system status. The military data acquisition system operates on airborne equipment in the field. It mainly operates in the field, and the external environment is harsh. situation. [0003] At present, the dual-machine backup technology is used to ensure the high reliability of data collection. The dual-machine backup includes the backup of the collection process and the backup of background data. Engine and database storage backup purpose; and the collection process backup is due to the different application purposes of each system. Currently, there are relatively few architectures d...

AI Technical Summary

Problems solved by technology

[0004] Chinese patent CN201610959821.7 reveals the design of an integrated high-reliability and high-precision long-term monitoring system, including two commutation acquisition and editing units, two integrated controllers, switches, and two monitoring computers, which solves the complex composition of system equipment and low precision , The problem of insufficient long-term reliability has realized the function of high reliability, high precision and long-term monitoring of the system. This invention adopts dual-machine backup on hardware to achieve high reliability. However, the system mainly uses hardware backup to achieve high reliability. There is no Solve the problem of data loss in the process of dual-machine switching

Images