Meteorological service cloud platform based on virtualization and containerization

Abstract

The invention discloses a meteorological service cloud platform based on virtualization and containerization, which comprises a plurality of virtual servers which are distributed on a Kubernetes cloud platform and are managed by adopting a Docker container, and the plurality of virtual servers comprise an nginx reverse proxy server, a file sharing server, a data acquisition server, a database storage server and a plurality of application servers. According to the invention, physical server resources are integrated and dynamically scheduled through a virtualization technology, and an intensive Kubernetes cloud platform is constructed based on a Docker container technology, so that elastic expansion and efficient management of meteorological resources are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of meteorological services, and specifically to a meteorological service cloud platform based on virtualization and containerization. Background Art

[0002] With the accelerating progress of the informatization process of meteorological services, meteorological business systems are facing the dual challenges of a sharp increase in data scale and diversified service demands. The meteorological service center operates multiple heterogeneous business systems, and each system adopts an independent deployment mode, resulting in prominent data sharing barriers and low resource allocation efficiency. Statistics show that in the traditional "one machine, one application" deployment method, the average CPU utilization rate of the server is less than 15%, while the operation and maintenance cost increases at an annual rate of 23%, seriously restricting the improvement of meteorological service efficiency. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a meteorological service cloud platform based on virtualization and containerization, which integrates and dynamically schedules physical server resources through virtualization technology, and constructs an intensive Kubernetes cloud platform based on Docker container technology to achieve elastic expansion and efficient management of meteorological resources.

[0004] The technical solution of the present invention is as follows:

[0005] A meteorological service cloud platform based on virtualization and containerization includes multiple virtual servers deployed on the Kubernetes cloud platform and managed by Docker containers. The multiple virtual servers include an nginx reverse proxy server, a file sharing server, a data collection server, a database storage server, and multiple application servers;

[0006] The nginx reverse proxy server is used as a proxy server to accept connection requests sent by clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the clients;

[0007] The file sharing server is used to provide NFS file sharing services and store shared files;

[0008] The data collection server is used as a data collection tool to establish a connection with the meteorological data platform and collect various meteorological data;

[0009] The database storage server is used to provide data storage services and realize virtual storage of meteorological data;

[0010] The multiple application servers are used for the management and analysis of meteorological data, and the production, release, and display of meteorological graphic products.

[0011] The multiple virtual servers include the following ten subsystems:

[0012] (1) Data service: collect various meteorological data, process and store the collected meteorological data according to business needs, and provide data interface services to the client;

[0013] (2) Comprehensive analysis: used for real-time meteorological monitoring, meteorological data analysis, radar and satellite data analysis, configuration of elements, layers and sites, and refined meteorological forecast analysis;

[0014] (3) Industry meteorological services: Generate weather conditions, forecasts, and meteorological index information for different industry service needs, set corresponding meteorological thresholds according to different industries, compare the real-time monitored meteorological data with the set meteorological thresholds, and issue over-limit meteorological warnings;

[0015] (4) Production of meteorological products: Based on the analysis results of meteorological data, meteorological graphic products, meteorological index products and meteorological data query and statistical products are produced, and meteorological product templates are produced;

[0016] (5) Meteorological product sharing: providing visual sharing services for meteorological products;

[0017] (6) Management of all-media meteorological materials: management of the addition, modification and deletion of all-media meteorological materials used in the production of meteorological products, as well as the production of meteorological audio and video products;

[0018] (7) Meteorological product release: access different release channels to achieve manual and automatic release of meteorological products, as well as management of the release process and feedback of product release status information;

[0019] (8) System management: including user management, process management and meteorological product library management;

[0020] (9) System monitoring: database status monitoring, server operation status monitoring and data access monitoring;

[0021] (10) Large-screen display: Provide large-screen interface display function, conduct large-screen visual display of meteorological products, large-screen visual release of meteorological warning information, large-screen visual publicity of meteorological science, large-screen visual monitoring and large-screen dispatching and command based on integrated media.

[0022] The multiple virtual servers are connected to the virtual local area network using the address exchange function of the virtual switch, so that the multiple virtual servers are deployed on the Kubernetes cloud platform.

[0023] The database storage server provides virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

[0024] The described Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Multiple virtual servers are divided into multiple Docker containers through Docker container technology, and ten subsystems are respectively run. The Kubernetes cloud platform constructs Pods based on multiple Docker containers and selects appropriate worker nodes to complete the maintenance of the entire life cycle of the Pods according to the Pod configuration files. Among them, a Pod is the smallest unit of Kubernetes deployment and scheduling, representing a running instance, and a Pod can accommodate one or more related but independent Docker containers.

[0025] The described meteorological data platform includes a public meteorological service center, the national integrated meteorological information sharing platform CIMISS, the meteorological information processing and control system MICAPS, the national meteorological information center database, and various industry databases.

[0026] The described various meteorological data include meteorological monitoring data, land hydrological data, weather radar data, satellite cloud map data, meteorological analysis data, ocean monitoring data, meteorological facsimile data, and map geographical data.

[0027] The described publishing channels include the FTP file transfer protocol, SMS, email, and shared directories.

[0028] The described industry meteorological services include forestry meteorology, transportation meteorology, and tourism meteorology services.

[0029] The described method for meteorological product production includes spatial interpolation methods, weather recognition methods, climate statistical methods, index model methods, spatial analysis methods, meteorological data rendering methods, and meteorological mapping methods.

[0030] Advantages of the present invention:

[0031] The virtualization of the present invention includes three parts: server virtualization, network virtualization, and storage virtualization. By virtualizing the server, multiple virtual servers can be constructed on the physical server, which can effectively improve the resource utilization rate of the server, accelerate the deployment speed of various subsystems of the meteorological service cloud platform, enhance the convenience of management and maintenance, and can use dynamic migration to improve the service response ability, meeting the requirements of meteorological big data, high density, and high-performance computing.

[0032] The present invention integrates and dynamically schedules physical server resources through virtualization technology, and builds an intensive Kubernetes cloud platform based on Docker container technology to achieve elastic expansion and efficient management of resources. Practical applications show that the constructed meteorological service cloud platform not only opens up the docking channel between the meteorological service business platform and the multi-source heterogeneous data environment of the provincial meteorological bureau, but also supports the automated processing and one-click multi-channel publishing functions of meteorological products. The constructed meteorological service cloud platform also significantly improves the integration efficiency of meteorological service resources, optimizes business response capabilities, and provides accurate and efficient information service support for the public and industry users. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a functional distribution diagram of the ten subsystems of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] A provincial meteorological service cloud platform based on virtualization and containerization, including fourteen virtual servers deployed on the Kubernetes cloud platform and managed by Docker containers, the fourteen virtual servers are connected to the virtual local area network using the address exchange function of the virtual switch, so that the fourteen virtual servers are deployed on the Kubernetes cloud platform;

[0036] Fourteen virtual servers are deployed on three physical servers (model Cisco B200). The configuration of Cisco B200 is as follows: quad-core Inter(R) e5-2620 CPU, 24 DIMM slots of DDR4 memory, 28 cores per slot, speed 2666MHz, 3TB memory when using 128GBDIMM, two hot-swappable hard disk drives HDD, solid-state drive SDD, 2GB cache, SAS RAID controller, two sets of 4×10Gbps unified I / O ports, providing a connection speed of 10Gbps;

[0037] The fourteen virtual servers include one nginx reverse proxy server, one file sharing server, one data collection server, two database storage servers and nine application servers. The application deployment of the fourteen virtual servers is shown in Table 1 below.

[0038] Table 1

[0039]

[0040] An Nginx reverse proxy server is used as a proxy server to accept connection requests sent by clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the clients.

[0041] A file sharing server is used to provide NFS file sharing services and store shared files.

[0042] A data acquisition server serves as a data acquisition tool, establishes connections with meteorological data platforms (public meteorological service centers, the National Comprehensive Meteorological Information Sharing Platform CIMISS, the Meteorological Information Processing and Control System MICAPS, the National Meteorological Information Center database, and various industry databases), and acquires various meteorological data, including meteorological monitoring data, land hydrological data, weather radar data, satellite cloud image data, meteorological analysis data, ocean monitoring data, meteorological facsimile data, and map geographic data.

[0043] Two database storage servers are used to provide data storage services and achieve virtual storage of meteorological data. The database storage servers provide virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

[0044] The Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Fourteen virtual servers are segmented into seventy-two Docker containers through Docker container technology, running ten subsystems and thirty-one functional modules respectively. The Kubernetes cloud platform builds 880 Pods based on the seventy-two Docker containers, and selects appropriate worker nodes to complete the maintenance of the entire lifecycle of the Pod according to the Pod configuration file. Among them, a Pod is the smallest unit of Kubernetes deployment and scheduling, representing a running instance, and a Pod contains one or more related but independent Docker containers.

[0045] See Figure 1 , the fourteen virtual servers include the following ten subsystems and thirty-one functional modules:

[0046] (1). Data service: Collect various meteorological data, process and store the collected meteorological data according to business requirements, and provide data interface services (APIs) to clients.

[0047] (2). Comprehensive analysis: Used for meteorological data analysis, radar and satellite data analysis, configuration of elements, layers, and stations, and refined meteorological forecast analysis.

[0048] (3) Industry meteorological services: Industry meteorology includes forestry meteorology, transportation meteorology, and tourism meteorology services. Weather conditions, forecasts, and meteorological index information are generated for the service needs of different industries (forestry, transportation, and tourism). Corresponding meteorological thresholds are set according to different industries, and the real-time monitored meteorological data is compared with the set meteorological thresholds to issue over-limit meteorological warnings;

[0049] (4) Meteorological product production: Based on the analysis results of meteorological data, meteorological graphic and text products, meteorological index products, and meteorological data query and statistical products are produced using spatial interpolation methods, weather recognition methods, climate statistical methods, index model methods, spatial analysis methods, meteorological data rendering methods, and meteorological mapping methods, and the production of meteorological product templates is carried out;

[0050] (5) Meteorological product sharing: Provide visual sharing services for meteorological products;

[0051] (6) All-media meteorological material management: Manage the addition, modification, and deletion of all-media meteorological materials used for meteorological product production, as well as the production of meteorological audio-visual products;

[0052] (7) Meteorological product release: Connect to different release channels (FTP file transfer protocol, SMS, email, and shared directory) to achieve manual and automatic release of meteorological products (release mode selection), as well as the management of the release process and the feedback of product release status information;

[0053] (8) System management: Includes user management, process management, and meteorological product library management;

[0054] (9) System monitoring: Database status monitoring, server operation status monitoring, data access monitoring, and task process monitoring;

[0055] (10) Large screen display: Provide large screen interface display functions for large screen visual display of meteorological products, large screen visual release of meteorological warning information, large screen visual publicity of meteorological science popularization, large screen visual monitoring, and large screen scheduling and command based on the media convergence;

[0056] Example verification:

[0057] The present invention has been deployed and implemented in 16 municipal meteorological units in Anhui Province, and has been running stably for more than nine months, with a service continuity of 99.98%. During this period, the total amount of meteorological observation data processed reached 3.2 PB, 1.27 million meteorological products were successfully generated, and no data loss or service interruption events caused by platform failures occurred. The transmission delay of provincial meteorological products to county-level nodes (clients) was compressed from 15 minutes to 8 seconds, and the data flow efficiency was increased by 112 times; the deployment density of single physical node containers was increased to 24 instances / server, and the resource utilization efficiency was 7 times that of the traditional architecture; in terms of functional efficiency, the access delay of multi-source data ≤ 800 ms, the generation time of short-term and imminent forecast products was increased by 41%, the container startup delay < 3 s, and the abnormal detection accuracy rate reached 98.7%.

[0058] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A meteorological service cloud platform based on virtualization and containerization, characterized in that: It includes multiple virtual servers deployed on the Kubernetes cloud platform and managed using Docker containers. The multiple virtual servers include an nginx reverse proxy server, a file sharing server, a data collection server, a database storage server, and multiple application servers; The nginx reverse proxy server is used as a proxy server to accept connection requests sent by clients on the Kubernetes cloud platform, forward the requests to other virtual servers, and return the results obtained from other application servers to the clients; The file sharing server is used to provide NFS file sharing services and store shared files; The data collection server, as a data collection tool, establishes a connection with the meteorological data platform to collect various meteorological data; The database storage server is used to provide data storage services and realize the virtual storage of meteorological data; The multiple application servers are used for the management and analysis of meteorological data, and the production, release, and display of meteorological graphic products; 2. The meteorological service cloud platform based on virtualization and containerization according to claim 1, wherein: The multiple virtual servers include the following ten subsystems: (1), Data service: Collect various meteorological data, process and store the collected meteorological data according to business requirements, and provide data interface services to clients; (2), Comprehensive analysis: Used for real-time meteorological monitoring, meteorological data analysis, radar and satellite data analysis, configuration of elements, layers, and stations, and refined meteorological forecast analysis; (3), Industry meteorological service: Generate weather conditions, forecasts, and meteorological index information for different industry service requirements, set corresponding meteorological thresholds according to different industries, compare the real-time monitored meteorological data with the set meteorological thresholds, and issue over-limit meteorological warnings; (4), Meteorological product production: Produce meteorological graphic products, meteorological index products, and meteorological data query and statistics products based on the analysis results of meteorological data, and produce meteorological product templates; (5), Meteorological product sharing: Provide visual sharing services for meteorological products; (6), All-media meteorological material management: Manage the addition, modification, and deletion of all-media meteorological materials used for meteorological product production, and produce meteorological audio and video products; (7), Meteorological product release: Connect to different release channels to realize the manual and automatic release of meteorological products, as well as the management of the release process and the feedback of product release status information; (8), System management: Includes user management, process management, and meteorological product library management; (9), System monitoring: Database status monitoring, server running status monitoring, and data access monitoring; (10), Large screen display: Provide large screen interface display functions for the large screen visual display of meteorological products, the large screen visual release of meteorological warning information, the large screen visual publicity of meteorological science popularization, large screen visual monitoring, and large screen scheduling and command based on the media integration; 3. A meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The multiple virtual servers are connected to the virtual local area network by using the address exchange function of the virtual switch, so that the multiple virtual servers are deployed on the Kubernetes cloud platform.

4. A meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The described database storage server provides virtual volumes through the SAN architecture, integrating the data storage system into a complete resource pool.

5. A meteorological service cloud platform based on virtualization and containerization according to claim 2, characterized in that: The described Kubernetes cloud platform uses Docker container virtualization technology to run multiple subsystems independently in isolated spaces. Multiple virtual servers are partitioned into multiple Docker containers through Docker container technology, and ten subsystems are respectively run. The Kubernetes cloud platform constructs Pods based on multiple Docker containers and selects appropriate working nodes to complete the maintenance of the entire lifecycle of the Pods according to the Pod configuration files. Among them, a Pod is the smallest unit of deployment and scheduling in Kubernetes, representing a running instance, and a Pod contains one or more related but independent Docker containers.

6. The meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The described meteorological data platform includes a public meteorological service center, the national integrated meteorological information sharing platform CIMISS, the meteorological information processing and control system MICAPS, the national meteorological information center database, and various industry databases.

7. A meteorological service cloud platform based on virtualization and containerization according to claim 1, characterized in that: The described various meteorological data include meteorological monitoring data, land hydrological data, weather radar data, satellite cloud image data, meteorological analysis data, ocean monitoring data, meteorological facsimile data, and map geographic data.

8. The meteorological service cloud platform based on virtualization and containerization according to claim 2, wherein: The described distribution channels include the FTP file transfer protocol, SMS, email, and shared directories.

9. The meteorological service cloud platform based on virtualization and containerization according to claim 2, wherein: The described industry meteorological services include forestry meteorology, traffic meteorology, and tourism meteorology services.

10. A meteorological service cloud platform based on virtualization and containerization according to claim 2, characterized in that: The described method for meteorological product production includes spatial interpolation methods, weather recognition methods, climate statistical methods, index model methods, spatial analysis methods, meteorological data rendering methods, and meteorological mapping methods.

Images