System and method for collecting automatic weather station data

Abstract

The present invention relates to a system and a method for collecting automatic weather station data. The system comprises a message production module, a message queue module and a message consumption module; the message production module and the message consumption module are independent from each other; the message production module is provided with a compatible interface connected with different automatic weather stations; and the message consumption module is provided with a data analysis class and a data warehouse class that are matched with different automatic weather stations. According to the system for collecting automatic weather station data disclosed by the present invention, a proprietary system is constructed for the collection work of the automatic weather station data, more automatic station data formats are supported, the compatibility with vendors of different databases is increased, persistence processing is carried out on the collected data, the risk of data loss is reduced, human intervention in the data collection process is reduced, and the hourly peak pressure is buffered, so that even by using the database with lower configuration, the hourly peak can be processed stably.

Description

technical field [0001] The invention relates to the fields of meteorological science and big data technology application, in particular to a system and method for collecting data from automatic weather stations. Background technique [0002] At present, the meteorological department mainly requests data from the data interface of the automatic weather station and writes it into the database by writing programs in various projects in the aspect of automatic weather station data collection. Due to the large number of automatic weather stations, all station data are updated at the same time on the hour, and data write and read requests peak at the hour, resulting in an increased burden on the database and slow access at the hour. The current solution is to deal with the peak by allocating more CPU and memory on the database hardware resources; in addition, since the data request and storage are completed by the same program, for the frequent abnormal stop of the program, it can...

AI Technical Summary

Problems solved by technology

Due to the large number of automatic weather stations, all station data are updated at the same time on the hour, and the data write and read requests peak at the hour, resulting in an increased burden on the database and slow access

The current solution is to deal with the peak by allocating more CPU and memory on the database hardware resources; in addition, since the data request and storage are completed by the same program, for the frequent abnormal stop of the program, it can only be done manually Analyze the log collection of the collection program to understand the time when the interruption occurred, and run the data supplementary recording program at an appropriate time to fill in the missing data when the program is abnormal; at the same time, there is no automatic weather station data collection in China that can complete a large amount of automatic station data The proprietary system of collection and the data collection of automatic weather stations are only a subsidiary task of independent project development. During the project development process, different manufacturers use different technica...

Images