A meteorological service cloud platform based on virtualization and containerization

Abstract

This invention discloses a meteorological service cloud platform based on virtualization and containerization. The platform comprises multiple virtual servers deployed on a Kubernetes cloud platform and managed using Docker containers. The multiple virtual servers include an Nginx reverse proxy server, a file sharing server, a data acquisition server, a database storage server, and multiple application servers. This invention integrates and dynamically schedules physical server resources through virtualization technology, and builds an integrated Kubernetes cloud platform based on Docker container technology, enabling flexible expansion and efficient management of meteorological resources.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological services, and in particular to a meteorological service cloud platform based on virtualization and containerization. Background Art

[0002] With the accelerated advancement of meteorological service informatization, meteorological operational systems are facing the dual challenges of surging data volumes and diversified service demands. Meteorological service centers operate multiple heterogeneous business systems, each deployed independently. This leads to significant barriers to data sharing and inefficient resource allocation. Statistics show that the traditional "one machine, one application" deployment model results in average server CPU utilization of less than 15%, while operational and maintenance costs are increasing at an average annual rate of 23%, severely hindering the improvement of meteorological service efficiency. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a meteorological service cloud platform based on virtualization and containerization, integrate and dynamically schedule physical server resources through virtualization technology, and build an intensive Kubernetes cloud platform based on Docker container technology to achieve elastic expansion and efficient management of meteorological resources.

[0004] The technical solution of the present invention is:

[0005] A meteorological service cloud platform based on virtualization and containerization, including multiple virtual servers deployed on the Kubernetes cloud platform and managed by Docker containers. The multiple virtual servers include an Nginx reverse proxy server, a file sharing server, a data collection server, a database storage server, and multiple application servers;

[0006] The nginx reverse proxy server is used as a proxy server to accept connection requests sent by clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the client;

[0007] The file sharing server is used to provide NFS file sharing services and store shared files;

[0008] The data acquisition server serves as a data acquisition tool, establishes a connection with the meteorological data platform, and collects various meteorological data;

[0009] The database storage server is used to provide data storage services and realize virtual storage of meteorological data;

[0010] The multiple application servers are used for the management and analysis of meteorological data, and the production, publication and display of meteorological graphic products.

[0011] The multiple virtual servers include the following ten subsystems:

[0012] (1) Data service: collect various meteorological data, process and store the collected meteorological data according to business needs, and provide data interface services to the client;

[0013] (2) Comprehensive analysis: used for real-time meteorological monitoring, meteorological data analysis, radar and satellite data analysis, configuration of elements, layers and sites, and refined meteorological forecast analysis;

[0014] (3) Industry meteorological services: Generate weather conditions, forecasts, and meteorological index information for different industry service needs, set corresponding meteorological thresholds according to different industries, compare the real-time monitored meteorological data with the set meteorological thresholds, and issue over-limit meteorological warnings;

[0015] (4) Production of meteorological products: Based on the analysis results of meteorological data, meteorological graphic products, meteorological index products and meteorological data query and statistical products are produced, and meteorological product templates are produced;

[0016] (5) Meteorological product sharing: providing visualization sharing services for meteorological products;

[0017] (6) Management of all-media meteorological materials: management of the addition, modification and deletion of all-media meteorological materials used in the production of meteorological products, as well as the production of meteorological audio and video products;

[0018] (7) Meteorological product release: access different release channels to achieve manual and automatic release of meteorological products, as well as management of the release process and feedback of product release status information;

[0019] (8) System management: including user management, process management and meteorological product library management;

[0020] (9) System monitoring: database status monitoring, server operation status monitoring and data access monitoring;

[0021] (10) Large-screen display: Provide large-screen interface display function, conduct large-screen visual display of meteorological products, large-screen visual release of meteorological warning information, large-screen visual publicity of meteorological science, large-screen visual monitoring and large-screen dispatching and command based on integrated media.

[0022] The multiple virtual servers are connected to the virtual local area network using the address exchange function of the virtual switch, so that the multiple virtual servers are deployed on the Kubernetes cloud platform.

[0023] The database storage server provides virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

[0024] The Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Multiple virtual servers are divided into multiple Docker containers through Docker container technology, each running ten subsystems. The Kubernetes cloud platform builds Pods based on multiple Docker containers and selects appropriate working nodes according to the Pod configuration file to complete the maintenance of the Pod throughout its life cycle. Among them, Pod is the smallest unit of Kubernetes deployment and scheduling, representing a running instance. A Pod contains one or more related but independent Docker containers.

[0025] The meteorological data platform includes the Public Meteorological Service Center, the National Integrated Meteorological Information Sharing Platform CIMISS, the Meteorological Information Processing and Control System MICAPS, the National Meteorological Information Center database and various industry databases.

[0026] The various meteorological data include meteorological monitoring data, land hydrological data, weather radar data, satellite cloud image data, meteorological analysis data, ocean monitoring data, meteorological fax data and map geographic data.

[0027] The publishing channels include FTP file transfer protocol, text messages, emails and shared directories.

[0028] The industry meteorological services include forestry meteorology, transportation meteorology and tourism meteorology services.

[0029] The method for producing meteorological products includes a spatial interpolation method, a weather identification method, a climate statistics method, an indicator model method, a spatial analysis method, a meteorological data rendering method and a meteorological drawing method.

[0030] Advantages of the present invention:

[0031] The virtualization of the present invention includes three parts: server virtualization, network virtualization, and storage virtualization. By virtualizing the server, multiple virtual servers can be built on the physical server, which can effectively improve the resource utilization of the server, accelerate the deployment of various subsystems of the meteorological service cloud platform, improve the convenience of management and maintenance, and use dynamic migration to improve service responsiveness, meeting the needs of meteorological big data, high density and high performance computing.

[0032] The present invention integrates and dynamically schedules physical server resources through virtualization technology, and builds an intensive Kubernetes cloud platform based on Docker container technology to achieve elastic expansion and efficient management of resources. Practical applications have shown that the constructed meteorological service cloud platform not only opens up the docking channel between the meteorological service business platform and the multi-source heterogeneous data environment of the provincial meteorological bureau, but also supports the automated processing and one-click multi-channel publishing functions of meteorological products. The constructed meteorological service cloud platform has also significantly improved the integration efficiency of meteorological service resources, optimized business response capabilities, and provided accurate and efficient information service support for the public and industry users. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a functional distribution diagram of the ten subsystems of the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0035] A provincial meteorological service cloud platform based on virtualization and containerization, including fourteen virtual servers deployed on the Kubernetes cloud platform and managed by Docker containers. The fourteen virtual servers are connected to the virtual local area network using the address switching function of the virtual switch, so that the fourteen virtual servers are deployed on the Kubernetes cloud platform;

[0036] Fourteen virtual servers are deployed on three physical servers (model Cisco B200). The configuration of Cisco B200 is as follows: a quad-core Inter(R) e5-2620 CPU, 24 DIMM slots for DDR4 memory, 28 cores per slot, a speed of 2666MHz, 3TB of memory when using 128GB DIMMs, two hot-swappable hard disk drives (HDDs) and solid-state drives (SDDs), 2GB of cache, a SAS RAID controller, and two sets of 4×10Gbps unified I / O ports, providing a connection speed of 10Gbps.

[0037] The fourteen virtual servers include one nginx reverse proxy server, one file sharing server, one data acquisition server, two database storage servers and nine application servers. The application deployment of the fourteen virtual servers is shown in Table 1 below.

[0038] Table 1

[0039]

[0040] An nginx reverse proxy server is used as a proxy server to accept connection requests from clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the client;

[0041] A file sharing server is used to provide NFS file sharing services and store shared files;

[0042] A data collection server serves as a data collection tool, establishing connections with meteorological data platforms (public meteorological service centers, the national integrated meteorological information sharing platform CIMISS, meteorological information processing and control system MICAPS, the National Meteorological Information Center database, and various industry databases) to collect various meteorological data, including meteorological monitoring data, land hydrological data, weather radar data, satellite cloud image data, meteorological analysis data, ocean monitoring data, meteorological fax data, and map geographic data;

[0043] Two database storage servers are used to provide data storage services and realize virtual storage of meteorological data. The database storage servers provide virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

[0044] The Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Fourteen virtual servers are divided into 72 Docker containers through Docker container technology, which run ten subsystems and 31 functional modules respectively. The Kubernetes cloud platform builds 880 Pods based on 72 Docker containers. According to the Pod configuration file, the appropriate working node is selected to complete the maintenance of the Pod throughout its life cycle. Among them, Pod is the smallest unit of Kubernetes deployment and scheduling, representing a running instance. A Pod contains one or more related but independent Docker containers.

[0045] See Figure 1 , the fourteen virtual servers include the following ten subsystems and thirty-one functional modules:

[0046] (1) Data service: collect various meteorological data, process and store the collected meteorological data according to business needs, and provide data interface service (API) to the client;

[0047] (2) Comprehensive analysis: used for meteorological data analysis, radar and satellite data analysis, configuration of elements, layers and sites, and refined weather forecast analysis;

[0048] (3) Industry meteorological services: Industry meteorology includes forestry meteorology, transportation meteorology and tourism meteorology services. It generates weather conditions, forecasts and meteorological index information for the service needs of different industries (forestry, transportation and tourism). It also sets corresponding meteorological thresholds according to different industries, compares the real-time monitored meteorological data with the set meteorological thresholds, and issues over-limit meteorological warnings.

[0049] (4) Production of meteorological products: Based on the analysis results of meteorological data, use spatial interpolation methods, weather identification methods, climate statistics methods, indicator model methods, spatial analysis methods, meteorological data rendering methods and meteorological mapping methods to produce meteorological graphic products, meteorological index products and meteorological data query statistics products, and produce meteorological product templates;

[0050] (5) Meteorological product sharing: providing visualization sharing services for meteorological products;

[0051] (6) Management of all-media meteorological materials: management of the addition, modification and deletion of all-media meteorological materials used in the production of meteorological products, as well as the production of meteorological audio and video products;

[0052] (7) Meteorological product release: access to different release channels (FTP, SMS, email, and shared directories) to achieve manual and automatic release of meteorological products (release mode selection), as well as management of the release process and feedback of product release status information;

[0053] (8) System management: including user management, process management and meteorological product library management;

[0054] (9) System monitoring: database status monitoring, server operation status monitoring, data access monitoring and task process monitoring;

[0055] (10) Large-screen display: Provide large-screen interface display function, conduct large-screen visual display of meteorological products, large-screen visual release of meteorological warning information, large-screen visual publicity of meteorological science, large-screen visual monitoring and large-screen dispatching and command based on integrated media.

[0056] Example verification:

[0057] The present invention has been deployed and implemented in 16 prefecture-level meteorological units in Anhui Province, and has been running stably for more than nine months, with a service continuity of 99.98%. During this period, a total of 3.2PB of meteorological observation data was processed, and 1.27 million meteorological products were successfully generated. No data loss or business interruption due to platform failure occurred. The transmission delay from provincial meteorological products to county-level nodes (clients) was compressed from 15 minutes to 8 seconds, and the data flow efficiency was increased by 112 times; the deployment density of single physical node containers was increased to 24 instances / server, and the resource utilization efficiency was 7 times that of the traditional architecture; in terms of functional performance, the multi-source data access delay was ≤800ms, the short-term forecast product generation timeliness was increased by 41%, the container startup delay was <3s, and the anomaly detection accuracy rate reached 98.7%.

[0058] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A meteorological service cloud platform based on virtualization and containerization, characterized by: It includes multiple virtual servers deployed on the Kubernetes cloud platform and managed by Docker containers. The multiple virtual servers include nginx reverse proxy servers, file sharing servers, data collection servers, database storage servers, and multiple application servers. The nginx reverse proxy server is used as a proxy server to accept connection requests sent by clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the client; The file sharing server is used to provide NFS file sharing services and store shared files; The data acquisition server serves as a data acquisition tool, establishes a connection with the meteorological data platform, and collects various meteorological data; The database storage server is used to provide data storage services and realize virtual storage of meteorological data; The multiple application servers are used for the management and analysis of meteorological data, and the production, publication and display of meteorological graphic products; The multiple virtual servers include the following ten subsystems: (1) Data service: collect various meteorological data, process and store the collected meteorological data according to business needs, and provide data interface services to the client; (2) Comprehensive analysis: used for real-time meteorological monitoring, meteorological data analysis, radar and satellite data analysis, configuration of elements, layers and sites, and refined meteorological forecast analysis; (3) Industry meteorological services: Generate weather conditions, forecasts, and meteorological index information for different industry service needs, set corresponding meteorological thresholds according to different industries, compare the real-time monitored meteorological data with the set meteorological thresholds, and issue over-limit meteorological warnings; (4) Production of meteorological products: Based on the analysis results of meteorological data, meteorological graphic products, meteorological index products and meteorological data query and statistical products are produced, and meteorological product templates are produced; (5) Meteorological product sharing: providing visualization sharing services for meteorological products; (6) Management of all-media meteorological materials: management of the addition, modification and deletion of all-media meteorological materials used in the production of meteorological products, as well as the production of meteorological audio and video products; (7) Meteorological product release: access different release channels to achieve manual and automatic release of meteorological products, as well as management of the release process and feedback of product release status information; (8) System management: including user management, process management and meteorological product library management; (9) System monitoring: database status monitoring, server operation status monitoring and data access monitoring; (10) Large screen display: Provide large screen interface display function, conduct large screen visual display of meteorological products, large screen visual release of meteorological warning information, large screen visual publicity of meteorological science, large screen visual monitoring and large screen dispatching and command based on integrated media; The Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Fourteen virtual servers are divided into seventy-two Docker containers through Docker container technology, which run ten subsystems and thirty-one functional modules respectively. The Kubernetes cloud platform builds 880 Pods based on the seventy-two Docker containers. According to the Pod configuration file, the appropriate working node is selected to complete the maintenance of the entire Pod life cycle. Among them, Pod is the smallest unit of Kubernetes deployment and scheduling, representing a running instance. A Pod contains one or more related but independent Docker containers.

2. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The multiple virtual servers are connected to the virtual local area network using the address exchange function of the virtual switch, so that the multiple virtual servers are deployed on the Kubernetes cloud platform.

3. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The database storage server provides virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

4. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The meteorological data platform includes the Public Meteorological Service Center, the National Integrated Meteorological Information Sharing Platform CIMISS, the Meteorological Information Processing and Control System MICAPS, the National Meteorological Information Center database and various industry databases.

5. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The various meteorological data include meteorological monitoring data, land hydrological data, weather radar data, satellite cloud image data, meteorological analysis data, ocean monitoring data, meteorological fax data and map geographic data.

6. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The publishing channels include FTP file transfer protocol, text messages, emails and shared directories.

7. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The industry meteorological services include forestry meteorology, transportation meteorology and tourism meteorology services.

8. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The method for producing meteorological products includes a spatial interpolation method, a weather identification method, a climate statistics method, an indicator model method, a spatial analysis method, a meteorological data rendering method and a meteorological drawing method.

Images