Meteorological grid point file application method based on resource set and big data technology

Abstract

The present invention discloses a meteorological grid point file application method based on a resource set and a big data technology. The method comprises an HDFS storage cluster, a resource indexingservice, an instruction service, a computing cluster and a file service. The present invention provides a brand-new grid point file application scheme, and through the cloud computing technology, thestandardized data processing in the cloud is realized, the data movement link is removed, the usage threshold of the grid point file is reduced, the use efficiency is improved, and the repetitive work is reduced; the REST specification is used, so that different system interconnections can be implemented by a user in a facilitated manner, and construction of a standardized system can be facilitated; and the present invention creates a grid point file resource set construction method, a grid point file resource standardized instruction set, and a grid point file application architecture basedon the resource set and the instruction set, and the grid point file resource set is composed of design of a resource index table, an operation on the resource index table, a file name parser, and a set of query methods.

Description

technical field [0001] The invention relates to the field of data application, in particular to a meteorological grid point file application method based on resource set and big data technology. Background technique [0002] Meteorological forecasting is developing in the direction of refinement. Whether it is a weather forecasting model or a climate forecasting model, the temporal resolution, spatial resolution, and forecast duration are constantly improving. The direct result of this development is that the data of weather forecasting is increasing. faster, [0003] With the development of fine forecasting technology, more and more large numerical forecast files have been produced, and standard data formats such as NC, Grib, etc. have been formed in the use of data, collectively referred to as grid file, which is a standardized self-describing file , this storage method stores the description information of the data and the data in the same file, and the variable name, di...

AI Technical Summary

Problems solved by technology

Although this kind of self-describing file brings great convenience to practical applications, with the generation of massive files, it brings great challenges to storage and processing. The current commonly used method is still very traditional. In business systems It is stored on the server like ordinary files, and the data is downloaded or exchanged through the FTP server. The use of files is realized separately based on different systems and technologies. A great bottleneck, need to design new methods to better use these grid files

Images