A regional multi-source meteorological information storage and management system

Abstract

The present invention proposes a regional multi-source meteorological information storage and management system, including a meteorological information storage and management system and a meteorological information comprehensive analysis and processing system. The meteorological information storage and management system includes a data collection and distribution subsystem, a data storage management subsystem, and a data monitoring subsystem. The data collection and distribution subsystem provides data support for the data storage management subsystem and the data monitoring subsystem. The data monitoring subsystem obtains corresponding data from the data collection and distribution subsystem and submits abnormal analysis results to the data storage management subsystem for storage; the meteorological information comprehensive analysis and processing system includes a data processing subsystem and a comprehensive analysis and display subsystem. This application takes the meteorological information storage and management system as the core and the meteorological information comprehensive analysis and processing system as the auxiliary to collect and distribute, store and manage, monitor data, analyze and process data from automatic stations, weather radars, manual observations, numerical forecasts, etc.

Description

Technical Field

[0001] The present invention belongs to the technical field of meteorological information storage and management, and in particular relates to a regional multi-source meteorological information storage and management system. Background Art

[0002] In recent years, with the continuous expansion of unit functions, a large number of new meteorological equipment have been put into use. These devices generate a wide variety of data and a large amount of information. These devices also suffer from practical problems such as fragmented storage, varying formats, data redundancy, and low overall utilization. This has hindered the full utilization of these devices and hindered the timely acquisition and efficient use of meteorological information. To address these issues, the present invention proposes a regional multi-source meteorological information storage and management system that meets the requirements for efficient and accurate storage and utilization of meteorological information. Summary of the Invention

[0003] The purpose of the present invention is to solve the above problems and establish a multi-source meteorological information storage and management system. The system takes the meteorological information storage and management system as the core and the meteorological information comprehensive analysis and processing system as the auxiliary, and performs operations such as collection and distribution, storage management, data monitoring, analysis and processing of data from automatic stations, weather radars, manual observations, numerical forecasts, etc.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] A regional multi-source meteorological information storage and management system, characterized by comprising a meteorological information storage and management system and a meteorological information comprehensive analysis and processing system,

[0006] The meteorological information storage and management system includes a data collection and distribution subsystem, a data storage management subsystem, and a data monitoring subsystem.

[0007] The data collection and distribution subsystem includes a data synchronization acquisition module, a data preprocessing module, a data distribution module and a configuration management module.

[0008] The data storage management subsystem includes a data directory management module, a data storage design module, a data security control module, a data clearing module, a backup and recovery module, and a proprietary data storage management module.

[0009] The data monitoring subsystem includes a data acquisition monitoring and statistics module, an arrival rate monitoring and statistics module, a resource environment monitoring and statistics module, a transmission volume monitoring and statistics module, and a transmission time monitoring and statistics module.

[0010] The data collection and distribution subsystem provides data support for the data storage management subsystem and the data monitoring subsystem.

[0011] The data monitoring subsystem obtains the corresponding data from the data collection and distribution subsystem, and submits the abnormal analysis situation to the data storage management subsystem for storage.

[0012] The data storage management subsystem stores abnormal analysis information and classifies, stores, clears and manages the data;

[0013] The meteorological information comprehensive analysis and processing system includes a data processing subsystem and a comprehensive analysis and display subsystem.

[0014] The data processing subsystem includes a data reading and aggregation module, a data integration and processing module, and a data storage and management module.

[0015] The comprehensive analysis and display subsystem includes a data retrieval and query module, a live data display module, a ground data statistical analysis module, an altitude data statistical analysis module, a route element profile analysis module, and a forecast production and scoring module.

[0016] The data processing subsystem processes the meteorological data on the server side.

[0017] The comprehensive analysis and display subsystem is used to implement interactive analysis and comprehensive display of meteorological data products based on a browser.

[0018] As the preferred technical solution of the present invention: the data monitoring subsystem is data-connected with the data collection and distribution subsystem, and the data monitoring subsystem sends the data source for monitoring transmission timeliness, data volume, and arrival rate to the data collection and distribution subsystem; the data collection and distribution subsystem is data-connected with the data storage management subsystem, and the data collection and distribution subsystem sends the data source for transmitting and storing meteorological data to the data storage management subsystem.

[0019] As the preferred technical solution of the present invention: the meteorological information comprehensive analysis and display subsystem is data-connected with the data processing subsystem, the meteorological information comprehensive analysis and display subsystem sends a data call application to the data processing subsystem, the data processing subsystem performs secondary processing of the data, and when it receives the data call application, it transmits the secondary processed data to the meteorological information comprehensive analysis and display subsystem.

[0020] As a preferred technical solution of the present invention: the data sources of the data synchronization acquisition module include local basic data, CMACast system data, CCTV2 data and FY-4 data.

[0021] As the preferred technical solution of the present invention: the collection methods of the data synchronization collection module include: Access database collection, ground meteorological observation and supervision platform data collection, lightning monitoring and early warning data collection, Excel data collection, text data collection, Oracle data collection, SQL Server data collection, wind tower data collection, numerical forecast product collection, and Fengyun-4 meteorological satellite data collection.

[0022] As a preferred technical solution of the present invention: it also includes a meteorological data integrated display platform and a two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological information storage and management system and the meteorological information comprehensive analysis and processing system are respectively connected to the meteorological data integrated display platform and the two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological data integrated display platform is used for meteorological data display software, which supports multi-source data display, multiple statistical plots, real-time data analysis, and online application updates. The raster meteorological data display WEBGIS platform is a platform developed based on the WEBGIS core and combines the display characteristics of spatiotemporal meteorological data to achieve two-dimensional visualization of such data.

[0023] As the preferred technical solution of the present invention: the meteorological data storage and management system also includes a large-scale parallel processing database and a distributed file storage system. The large-scale parallel processing database distributes tasks in parallel to multiple servers and nodes, and each node summarizes the results after completing the calculation. The distributed file storage system stores data in a dispersed manner on each independent device.

[0024] As a preferred technical solution of the present invention: the large-scale parallel processing database includes a database cluster, in which management nodes and instance nodes are set. The instance nodes are provided with independent disk storage systems and memory systems, which are used to divide data into each instance node through the management node according to the database model and application characteristics.

[0025] As a preferred technical solution of the present invention: the distributed file storage system includes a data writing device and a data reading device, the data writing device is used to receive the files to be written and store them in a decentralized manner through management nodes and data nodes, and the data reading device reads the data of the files to be written through management nodes and data points, and creates new files on each independent device.

[0026] As a preferred technical solution of the present invention: it also includes a data fusion and repair system, the data fusion and repair system includes a redundant backup module and a main device data module,

[0027] The data fusion correction of the main device data module and the redundant backup module is realized based on the Markov estimation algorithm, as follows:

[0028] Research subjects and hypotheses:

[0029] The research object of this algorithm can be expressed by formula (1):

[0030]

[0031] Where x is the state quantity, y is the observation quantity, v represents the random error, H is the observation matrix, m and n are the data dimensions of the observation quantity and state quantity respectively;

[0032] And assuming E(v)=0, define R=E(v·v T ) is the observation error covariance matrix;

[0033] Weighted least squares estimation:

[0034] The research goal is to solve for x. However, due to the unknowability of the error v, the true value of the state variable x cannot be solved. We can only solve an estimated value as close to the true value as possible.

[0035] Therefore, the objective function for solving this problem is:

[0036]

[0037] Where W is the weight and is a positive definite symmetric matrix,

[0038] To solve the objective function (2), we need to make:

[0039]

[0040] You can solve it and The error covariance matrix P = (H T WH) -1 H T WRWH(H T WH) -1 ,

[0041] To make the estimated value The error of is minimized and the objective function is constructed as shown in formula (3):

[0042] P(W)=(H T WH) -1 H T WRWH(H T WH) -1 =min! (4)

[0043] Using the Schwarz matrix inequality, we can get P(W) min =(H T R -1H) -1 , at this time the weight W=R -1 ,

[0044] So the Markov estimate is Its error covariance matrix is ​​P = (H T R - 1 H) -1 ,

[0045] Superiority of Markov estimation:

[0046] Take the wind speed of two automatic weather stations as an example, in formula (1) Then the error of y1 is σ which is smaller than the error of y2, 4σ.

[0047] If conventional least squares estimation is used, the estimated value The error is If Markov estimation is used, the estimated value Its error It is not only better than the error of the least squares estimate, but also better than the error σ of y1.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention has formulated meteorological data storage specifications, designed a real-time incremental synchronization storage process for multi-source data and a distributed meteorological data storage architecture, solved the problem of standardization of multi-source meteorological data storage, enhanced the automated storage and management capabilities of meteorological data, improved the robustness and scalability of the database, and increased data query efficiency by about 40 times compared to when the system was not used, and the efficiency of aggregate analysis by about 8 times.

[0050] 2. The present invention applies a homologous data fusion correction algorithm to reduce random errors in atmospheric detection data, taps into the efficiency of redundant meteorological equipment, and increases equipment data utilization to 100%; develops meteorological data application software such as an integrated meteorological data display platform, a meteorological data display WEBGIS platform, and a series of experimental meteorological support software, enriching data application forms and improving data interaction capabilities, providing strong support for daily meteorological business operations and scientific research and experimental task command and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a schematic diagram of the principle framework of the meteorological information storage and management system;

[0052] Figure 2 It is a schematic diagram of the principle framework of the meteorological information comprehensive analysis and processing system;

[0053] Figure 3 It is a diagram of data storage relationship;

[0054] Figure 4 It is the principle framework diagram of the meteorological information storage and management system;

[0055] Figure 5 It is a principle framework diagram for comprehensive analysis and processing of meteorological information;

[0056] Figure 6 It is a schematic diagram of the meteorological information storage and management system and external data sources;

[0057] Figure 7 It is a schematic diagram of the system's external interface;

[0058] Figure 8 It is a schematic diagram of the number determination process;

[0059] Figure 9 It is a diagram of the architecture of a massively parallel processing database;

[0060] Figure 10 This is a diagram of the distributed file storage system architecture;

[0061] Figure 11 This is the flowchart of incremental synchronization of meteorological data in the database;

[0062] Figure 12 This is a flowchart for real-time decoding and storage of text-format meteorological data;

[0063] Figure 13 This is the functional framework diagram of the observation and detection data uploading software;

[0064] Figure 14 This is the wind speed correction effect diagram;. DETAILED DESCRIPTION

[0065] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0066] like Figure 1-7 As shown, the present invention proposes a regional multi-source meteorological information storage and management system, including a meteorological information storage and management system and a meteorological information comprehensive analysis and processing system.

[0067] The meteorological information storage and management system includes a data collection and distribution subsystem, a data storage management subsystem, and a data monitoring subsystem.

[0068] The data collection and distribution subsystem includes a data synchronization acquisition module, a data preprocessing module, a data distribution module and a configuration management module.

[0069] The data storage management subsystem includes a data directory management module, a data storage design module, a data security control module, a data clearing module, a backup and recovery module, and a proprietary data storage management module.

[0070] The data monitoring subsystem includes a data acquisition monitoring and statistics module, an arrival rate monitoring and statistics module, a resource environment monitoring and statistics module, a transmission volume monitoring and statistics module, and a transmission time monitoring and statistics module.

[0071] The data collection and distribution subsystem provides data support for the data storage management subsystem and the data monitoring subsystem.

[0072] The data monitoring subsystem obtains the corresponding data from the data collection and distribution subsystem, and submits the abnormal analysis situation to the data storage management subsystem for storage.

[0073] The data storage management subsystem stores abnormal analysis information and classifies, stores, clears and manages the data;

[0074] The meteorological information comprehensive analysis and processing system includes a data processing subsystem and a comprehensive analysis and display subsystem.

[0075] The data processing subsystem includes a data reading and aggregation module, a data integration and processing module, and a data storage and management module.

[0076] The comprehensive analysis and display subsystem includes a data retrieval and query module, a live data display module, a ground data statistical analysis module, an altitude data statistical analysis module, a route element profile analysis module, and a forecast production and scoring module.

[0077] The data processing subsystem processes the meteorological data on the server side.

[0078] The comprehensive analysis and display subsystem is used to implement interactive analysis and comprehensive display of meteorological data products based on a browser.

[0079] The data monitoring subsystem is data-connected to the data collection and distribution subsystem, and the data monitoring subsystem sends the data source for monitoring transmission timeliness, data volume, and arrival rate to the data collection and distribution subsystem. The data collection and distribution subsystem is data-connected to the data storage management subsystem, and the data collection and distribution subsystem sends the data source for transmitting and storing meteorological data to the data storage management subsystem.

[0080] The meteorological information comprehensive analysis and display subsystem is data-connected to the data processing subsystem. The meteorological information comprehensive analysis and display subsystem sends a data call application to the data processing subsystem. The data processing subsystem performs secondary processing of the data. After receiving the data call application, it transmits the secondary processed data to the meteorological information comprehensive analysis and display subsystem.

[0081] The data sources collected by the data synchronization acquisition module include local basic data, CMACast system data, CCTV2 data and FY-4 data.

[0082] The data synchronization acquisition module's acquisition methods include: Access database acquisition, ground meteorological observation and supervision platform data acquisition, lightning monitoring and early warning data acquisition, Excel data acquisition, text data acquisition, Oracle data acquisition, SQL Server data acquisition, wind tower data acquisition, numerical forecast product acquisition, and Fengyun-4 meteorological satellite data acquisition.

[0083] The data security control module controls and manages access rights to data and users. Different database users have different security levels and access control capabilities for data.

[0084] Since meteorological information storage standards and technological development in different regions are not synchronized, they cannot adapt to the reality that the meteorological information storage in this application is scattered, with different contents and formats. To solve this problem, professional terms and definitions were clarified based on the actual business of the unit, data classification and coding were designed, and finally a multi-source meteorological data storage specification document dedicated to this application was formulated.

[0085] like Figure 8 The figure shows the process for generating unique numbers for each type of data generated by meteorological equipment. Based on meteorological operational practices, meteorological data is categorized by test area, equipment type, and data attributes, and each category is coded. Support areas are divided into seven major categories and ten minor categories; equipment types are divided into four major categories and 17 minor categories; and meteorological data are divided into 14 major categories.

[0086] Database table naming, file directory design, and file naming standards were implemented for structured and unstructured data. The table names and file directory structure were designed using the meteorological data categories and codes and common attribute codes in "QX / T 102-2009 Meteorological Data Classification and Coding." The secondary classification standards for equipment classification were also adopted, with the following naming rules:

[0087] {Time attribute}_{Data category}_{Data content}_{Device type}

[0088] For example, when naming the database table name for lightning locator detection data, the data table is named: FTM_UPAR_LIL_LL, which means: regular observation value_high-altitude meteorological data_lightning locator detection data_lightning locator.

[0089] The files generated by the observation and detection equipment are named according to the "QX-T 129-2011 Meteorological Data Transmission File Naming" and the naming format is as follows:

[0090] Z_{productidentifier}_I_{originator}_{yyyyMMddhhmmss}_{ftype}

[0091] [_{freeformat}].{tpye}[.{compression}].

[0092] like Figure 12 The figure shows the real-time decoding and storage process for text-formatted meteorological data. This process is similar to the incremental synchronization process for database meteorological data. Data is imported into the database after decoding using a code library written to read meteorological data in various text formats. This code library includes code for reading text data generated by various existing meteorological equipment, including optical wind theodolites, fully automatic high-altitude meteorological detection systems, Beidou high-altitude meteorological detection systems, high-altitude meteorological detection radars, wind towers, automatic weather stations, and wind profiler radars.

[0093] Figure 13 This is a diagram of the functional framework of the observation and sounding data upload software. The data required for this type of software is non-continuous observation and sounding data, which does not require high real-time performance. Currently, it is mainly used for uploading sounding data, which is implemented by meteorological stations to enable observation and upload at any time.

[0094] It also includes a meteorological data integrated display platform and a two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological information storage and management system and the meteorological information comprehensive analysis and processing system are respectively connected to the meteorological data integrated display platform and the two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological data integrated display platform is used for meteorological data display software, which supports multi-source data display, multiple statistical drawings, real-time data analysis, and online application updates. The raster meteorological data display WEBGIS platform is a platform developed based on the WEBGIS core, combined with the display characteristics of time-space meteorological data, to achieve two-dimensional visualization of such data.

[0095] In order to improve data utilization efficiency and serve weather forecasting and meteorological support, we have developed a number of data application software, including an integrated meteorological data display platform, a two-dimensional vector and raster meteorological data display WEBGIS platform, and a meteorological support series software.

[0096] The Meteorological Data Integration Platform (MDIP) is software for displaying meteorological data. It supports multi-source data display, various statistical plotting techniques, real-time data analysis, and online application updates. Since its launch, MDIP has continuously incorporated new equipment and data, and has undergone over 10 iterations. It is a software that forecasters rely heavily on when conducting weather forecasting.

[0097] The two-dimensional vector and raster meteorological data display WEBGIS platform is a platform developed based on the WEBGIS core and combines the display characteristics of time-space meteorological data to achieve two-dimensional visualization of such data.

[0098] Meteorological support software series are often used for image display and data link transmission in meteorological support, and are an indispensable part of meteorological support.

[0099] The meteorological data storage and management system also includes a large-scale parallel processing database and a distributed file storage system. The large-scale parallel processing database distributes tasks in parallel to multiple servers and nodes, and summarizes the results after each node completes the calculation. The distributed file storage system stores data in a dispersed manner on each independent device.

[0100] The large-scale parallel processing database includes a database cluster, in which management nodes and instance nodes are set. The instance nodes are provided with independent disk storage systems and memory systems for dividing data into each instance node through the management node according to the database model and application characteristics.

[0101] The distributed file storage system includes a data writing device and a data reading device. The data writing device is used to receive files to be written and store them in a decentralized manner through management nodes and data nodes. The data reading device reads the data of the files to be written through management nodes and data points and creates new files on each independent device.

[0102] like Figure 9 The following diagram shows the architecture of a massively parallel processing database (MPP database). This database supports large-scale clusters, with performance linearly proportional to the performance of the nodes being expanded. It also supports common SQL standards, allows for data backup, and theoretically allows for unlimited scalability.

[0103] In a database cluster, first, each node has an independent disk storage system and memory system. Second, business data is divided into each node according to the database model and application characteristics. Large-scale parallel processing databases distribute tasks in parallel to multiple servers and nodes. After each node completes the calculation, the results are summarized.

[0104] like Figure 10 The figure below shows a schematic diagram of the distributed file storage system architecture. This storage system stores data across multiple independent devices. This distributed network storage system utilizes a scalable architecture, sharing the storage load across multiple storage servers and locating stored information using management nodes, making it easy to expand. Currently, this storage system is primarily used to store large files such as Fengyun-4 satellite remote sensing data, numerical forecast products, reanalysis data, cloud maps, and weather radar.

[0105] like Figure 11 The figure below shows a schematic diagram of the incremental synchronization process for meteorological data in the database. This process is a one-time synchronization process that achieves real-time synchronization through periodic data increment detection. Timestamps and device names are used as specific attributes to determine the uniqueness of observation data. To avoid memory explosion caused by repeated execution of the process when synchronizing large amounts of data, a synchronization conflict detection mechanism has been designed.

[0106] When analyzing and applying multi-source meteorological data within the system, in order to ensure the continuity and reliability of the data, redundant backup is implemented for some meteorological equipment, and the data of the main equipment and redundant equipment are fused and corrected using a Markov estimation algorithm.

[0107] Specifically, it also includes a data fusion and repair system, which includes a redundant backup module and a main device data module.

[0108] The data fusion correction of the main device data module and the redundant backup module is realized based on the Markov estimation algorithm, as follows:

[0109] Research subjects and hypotheses:

[0110] The research object of this algorithm can be expressed by formula (1):

[0111]

[0112] Where x is the state variable (the actual atmospheric state of wind speed), y is the observation variable (the wind speed value detected by the automatic weather station), v represents the random error (including the random error caused by the inherent equipment and atmospheric turbulence), H is the observation matrix (in this case, it is a column vector with all values ​​1), m and n are the data dimensions of the observation variable and state variable respectively;

[0113] And assuming E(v)=0, define R=E(v·v T ) is the observation error covariance matrix;

[0114] Weighted least squares estimation:

[0115] The research goal is to solve for x. However, due to the unknowability of the error v, the true value of the state variable x cannot be solved. We can only solve an estimated value as close to the true value as possible.

[0116] Therefore, the objective function for solving this problem is:

[0117]

[0118] Where W is the weight and is a positive definite symmetric matrix,

[0119] To solve the objective function (2), we need to make:

[0120]

[0121] You can solve it and The error covariance matrix P = (H T WH) -1 H T WRWH(H T WH) -1 ,

[0122] To make the estimated value The error of is minimized and the objective function is constructed as shown in formula (3):

[0123] P(W)=(H T WH) -1 H T WRWH(H T WH) -1 =min! (4)

[0124] Using the Schwarz matrix inequality, we can get P(W) min =(H T R -1 H) -1 , at this time the weight W=R -1 ,

[0125] So the Markov estimate is Its error covariance matrix is ​​P = (H T R - 1 H) -1 ,

[0126] Superiority of Markov estimation:

[0127] Take the wind speed of two automatic weather stations as an example, in formula (1) Then the error of y1 is σ which is smaller than the error of y2, 4σ.

[0128] If conventional least squares estimation is used, the estimated value The error is If Markov estimation is used, the estimated value Its error It is not only better than the error of the least squares estimate, but also better than the error σ of y1.

[0129] like Figure 14 As shown in Figure 2, the wind speed is corrected by fusing two sets of automatic weather station data using a correction algorithm. The advantage of Markov estimation is that it can make full use of redundant equipment to obtain data with higher accuracy than a single device.

[0130] Based on the above structure, the invention of the present invention is:

[0131] 1. Exception handling mechanism:

[0132] To ensure the stability and reliability of the backend data support of the meteorological information storage and management system, the entire process of database data is monitored, including data synchronization monitoring, data transmission monitoring, data processing monitoring, and data storage monitoring, so as to promptly detect and handle various anomalies. A complete exception handling mechanism is established. For anomalies in various processing processes, the current system time is used as the data reception time and logged. For example, when the parsing of structured data fails, the specific cause of failure is recorded: file opening failure, unrecognizable encoding method, data length verification failure, etc. When recording into the database, if the database connection is abnormal, the message will not be confirmed and it will be processed again. When the parsed data time or station number is missing, it is regarded as abnormal data and logged. When the storage fails, for example: primary key conflict, abnormal characters in the structured query language, field length is too long, etc., log information is recorded to identify the cause of the failure.

[0133] 2. Multi-threaded / multi-process concurrent processing technology:

[0134] Utilizing multi-threaded / multi-process concurrent processing technology, database data synchronization and data decoding and storage processes are performed. The decoded data body is then decoded and the storage module is called to perform structured query language spelling on the decoded data body, batch storage, and log and exception information are sent based on the storage results. The decoding processes form a load-balancing and mutual backup relationship, and multi-threaded processing is used within the process to improve processing speed.

[0135] To improve data processing efficiency, the main process uses a multi-threaded processing architecture. The main process determines the size of the data processing thread pool based on the configuration file for each data type. The data processing thread pool includes a thread pool for index information storage and a thread pool for log information sending.

[0136] 3. Distributed computing and storage technology:

[0137] Based on a distributed file storage system, the Meteorological Data Center provides capabilities such as multi-source and diverse data collection, unified data storage, data management, distributed data processing and analysis, and unified task scheduling and management. This distributed file storage system fully leverages the performance of computer clusters for high-speed computing and storage. Furthermore, it provides high throughput for accessing application data, making it suitable for the meteorological industry's use of large data sets.

[0138] 4. Redundant equipment data fusion correction technology based on Markov estimation algorithm:

[0139] To ensure the reliability of meteorological detection data, redundant backup is implemented for some meteorological equipment. That is, multiple sets of the same type of meteorological detection equipment are set up at the same location. Data fusion and correction of the main equipment and redundant equipment are realized through the Markov estimation algorithm, thus obtaining atmospheric detection data with higher accuracy and smaller errors.

[0140] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any modification or equivalent variation based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A regional multi-source meteorological information storage and management system, characterized by: Including meteorological information storage and management system and meteorological information comprehensive analysis and processing system, The meteorological information storage and management system includes a data collection and distribution subsystem, a data storage management subsystem, and a data monitoring subsystem. The data collection and distribution subsystem includes a data synchronization acquisition module, a data preprocessing module, a data distribution module and a configuration management module. The data storage management subsystem includes a data directory management module, a data storage design module, a data security control module, a data clearing module, a backup and recovery module, and a proprietary data storage management module. The data monitoring subsystem includes a data acquisition monitoring and statistics module, an arrival rate monitoring and statistics module, a resource environment monitoring and statistics module, a transmission volume monitoring and statistics module, and a transmission time monitoring and statistics module. The data collection and distribution subsystem provides data support for the data storage management subsystem and the data monitoring subsystem. The data monitoring subsystem obtains the corresponding data from the data collection and distribution subsystem, and submits the abnormal analysis situation to the data storage management subsystem for storage. The data storage management subsystem stores abnormal analysis information and classifies, stores, clears and manages the data; The meteorological information comprehensive analysis and processing system includes a data processing subsystem and a comprehensive analysis and display subsystem. The data processing subsystem includes a data reading and aggregation module, a data integration and processing module, and a data storage and management module. The comprehensive analysis and display subsystem includes a data retrieval and query module, a live data display module, a ground data statistical analysis module, an altitude data statistical analysis module, a route element profile analysis module, and a forecast production and scoring module. The data processing subsystem processes the meteorological data on the server side. The comprehensive analysis and display subsystem is used to realize interactive analysis and comprehensive display of meteorological data products based on the browser; It also includes a data fusion and repair system, which includes a redundant backup module and a main device data module. The data fusion correction of the main device data module and the redundant backup module is realized based on the Markov estimation algorithm, as follows: Research subjects and hypotheses: The research object of this algorithm can be expressed by formula (1): (1) in is the state quantity, is the observed quantity, represents random error, is the observation matrix, 、 They are the data dimensions of observation quantity and state quantity respectively; And assume ,definition is the observation error covariance matrix; Weighted least squares estimation: Therefore, the objective function is: (2) in is the weight, and is a positive definite symmetric matrix, To solve the objective function (2), we need to make: (3) You can solve it ,and The error covariance matrix , To make the estimated value The error of is minimized and the objective function is constructed as shown in formula (4): (4) Using the Schwarz matrix inequality, we can get , at this time the weight , So the Markov estimate is , and its error covariance matrix is .

2. A regional multi-source meteorological information storage and management system according to claim 1, characterized in that: The data monitoring subsystem is data-connected to the data collection and distribution subsystem, and the data monitoring subsystem sends the data source for monitoring transmission timeliness, data volume, and arrival rate to the data collection and distribution subsystem. The data collection and distribution subsystem is data-connected to the data storage management subsystem, and the data collection and distribution subsystem sends the data source for transmitting and storing meteorological data to the data storage management subsystem.

3. The regional multi-source meteorological information storage and management system according to claim 1, characterized in that: The meteorological information comprehensive analysis and display subsystem is data-connected to the data processing subsystem. The meteorological information comprehensive analysis and display subsystem sends a data call application to the data processing subsystem. The data processing subsystem performs secondary processing of the data. After receiving the data call application, it transmits the secondary processed data to the meteorological information comprehensive analysis and display subsystem.

4. A regional multi-source meteorological information storage and management system according to claim 1, characterized in that: The data sources collected by the data synchronization acquisition module include local basic data, CMACast system data, CCTV2 data and FY-4 data.

5. The regional multi-source meteorological information storage and management system according to claim 1, characterized in that: The data synchronization acquisition module's acquisition methods include: Access database acquisition, ground meteorological observation and supervision platform data acquisition, lightning monitoring and early warning data acquisition, Excel data acquisition, text data acquisition, Oracle data acquisition, SQL Server data acquisition, wind tower data acquisition, numerical forecast product acquisition, and Fengyun-4 meteorological satellite data acquisition.

6. The regional multi-source meteorological information storage and management system according to claim 1, characterized in that: It also includes a meteorological data integrated display platform and a two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological information storage and management system and the meteorological information comprehensive analysis and processing system are respectively connected to the meteorological data integrated display platform and the two-dimensional vector and raster meteorological data display WEBGIS platform. The meteorological data integrated display platform is used for meteorological data display software, which supports multi-source data display, multiple statistical drawings, real-time data analysis, and online application updates. The raster meteorological data display WEBGIS platform is a platform developed based on the WEBGIS core, combined with the display characteristics of time-space meteorological data, to achieve two-dimensional visualization of such data.

7. The regional multi-source meteorological information storage and management system according to claim 1, characterized in that: The meteorological data storage and management system also includes a large-scale parallel processing database and a distributed file storage system. The large-scale parallel processing database distributes tasks in parallel to multiple servers and nodes, and summarizes the results after each node completes the calculation. The distributed file storage system stores data in a dispersed manner on each independent device.

8. A regional multi-source meteorological information storage and management system according to claim 7, characterized in that: The large-scale parallel processing database includes a database cluster, in which management nodes and instance nodes are set. The instance nodes are provided with independent disk storage systems and memory systems for dividing data into each instance node through the management node according to the database model and application characteristics.

9. The regional multi-source meteorological information storage and management system according to claim 7, characterized in that: The distributed file storage system includes a data writing device and a data reading device. The data writing device is used to receive files to be written and store them in a decentralized manner through management nodes and data nodes. The data reading device reads the data of the files to be written through management nodes and data nodes and creates new files on each independent device.

Images