A method for constructing a cross-cloud PAAS configuration data service in near real-time

Abstract

The present invention provides a method for constructing a cross-cloud PAAS configuration data service in quasi-real time. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time includes: a service layer, a data layer, a collection layer and a storage layer; the service layer is divided into the following two modules according to data processing logic and function: (1). Interface service module, the interface service module has the following requirements: 1). Different data models can be designed according to different operation and maintenance scenarios to provide upper-layer applications. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time provided by the present invention has a highly available and scalable architecture, is suitable for metadata management of various PAAS components, monitors and collects target components in real time and accurately, and can provide configuration data services for various operation and maintenance scenarios for applications, meeting the basic demands of multi-cloud, agile, automatic and intelligent operation and maintenance management.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud IT resource management, and in particular relates to a method for constructing a cross-cloud PAAS configuration data service in quasi-real time. Background Art

[0002] As the convenience of cloud services is increasingly accepted by the market, corporate customers are increasingly inclined to adopt public cloud services to supplement the deficiencies of their existing on-premises data centers and private cloud infrastructure. Therefore, hybrid cloud has become an important development trend in the cloud service market. Currently, the hybrid cloud form that enterprises use more frequently is a combination of public cloud + private cloud.

[0003] After searching, related technologies have disclosed a system and method for tenants to quickly access the financial cloud agency accounting PaaS service, which solves the problems of cumbersome configuration, high data redundancy, and slow tenant access speed of traditional financial cloud technology. In the present invention, in the definition center, the business bills, business rules, and business accounting of each business type are designed according to the granularity of bills, rules, and accounting lines respectively; in the construction center, a common bill set, rule set, and accounting line set are given to each business type, so that each business type has the ability to automatically process accounting; after the tenant's master data is imported into the business center, a business plan is specified for each tenant's company, and a user account and permissions are assigned to the tenant or company to log in to the financial cloud platform and use the PaaS service provided by the financial cloud platform.

[0004] After searching, it was found that in the related technologies, an information integration system and method based on the electric power PaaS cloud platform were disclosed, which involves the electric power PaaS cloud platform service field. The system includes: an application stack module, which is used to subdivide the stack structure of complex application systems by components, manage, configure and deploy them by levels, and maximize the decoupling relationship between levels; an environment orchestration module, which is used to handle the relationship between different components in the application environment; a microservice architecture module, which is used to gradually transform the existing application system or build a new micro-application system based on the integration capability of the PaaS platform and the microservice architecture model; an application interaction and integration module, which is used to open up the interface interaction of the application system based on the service-oriented architecture; a data interaction and integration module, which is used to open up the data interaction mode of the application system based on the service-oriented architecture. By adopting application stack technology, the present invention provides an application environment for microservice architecture type systems in a more convenient way.

[0005] Hybrid cloud helps enterprises extend their existing IT environments to the public cloud. This allows them to leverage the private cloud to ensure the security of their core business and private data while leveraging the public cloud's on-demand delivery and flexible resource scheduling advantages. However, as business expands and cluster scale evolves, the siloed O&M architecture and fragmented management create numerous challenges for O&M personnel:

[0006] 1. In terms of quality, both hardware and architecture suffer from scattered resources, making it difficult for both online and offline platforms to coordinate platform instance monitoring and daily operations. This leads to missed reports and false alarms, and errors in operations and maintenance.

[0007] 2. In terms of efficiency, the resources being operated are located on different platforms or multiple tenants on the same platform. These resources are relatively isolated and dispersed, making it difficult to integrate processes and automation. Frequent manual context switching results in low efficiency.

[0008] 3. In terms of cost, hardware resource utilization is difficult to analyze and calculate centrally, making asset statistics and resource recovery inconvenient.

[0009] Faced with these problems, the existing operation and maintenance methods and processes are difficult to meet the operation and maintenance needs under the huge business volume. Therefore, it is necessary to introduce a PAAS configuration data service construction solution to deepen the integration of the operation and maintenance system, improve the agility of IT operation and maintenance, and expand the scope and value of operation and maintenance services.

[0010] Therefore, it is necessary to provide a new method for building cross-cloud PAAS configuration data services in quasi-real time to solve the above technical problems. Summary of the Invention

[0011] The technical problem solved by the present invention is to provide a highly available and scalable architecture that is suitable for metadata management of various PAAS components, can monitor and collect target components in real time and accurately, can provide configuration data services for various operation and maintenance scenarios for applications, and meet the basic demands of multi-cloud, agile, automatic and intelligent operation and maintenance management. The method for building cross-cloud PAAS configuration data services in quasi-real time.

[0012] To solve the above technical problems, the present invention provides a method for constructing a cross-cloud PAAS configuration data service in quasi-real time, comprising: a service layer, a data layer, a collection layer, and a storage layer;

[0013] The service layer is divided into the following two modules according to data processing logic and functions:

[0014] (1) Interface service module, the interface service module requirements are as follows:

[0015] 1) Design different data models and provide upper-layer applications based on different operation and maintenance scenarios;

[0016] 2) To ensure seamless integration with upper-layer applications, interface standardization must be implemented;

[0017] (2) Resource data visualization module, the requirements of the resource data visualization module are as follows:

[0018] 1) Resource overview, providing a unified view and asset inventory statistics;

[0019] 2) Resource search, providing search capabilities by IP, component, and module, and displaying the permission status, subordination, and dependency relationships of the current resource;

[0020] 3) Change events, resource online and offline, and landing logs are provided for audit review.

[0021] As a further solution of the present invention, the data layer: the processing layer associates the collected data with resources, including sorting out the calling relationship, subordinate relationship, and dependency relationship, labels them, and integrates them into a complete module link.

[0022] As a further solution of the present invention, the collection layer includes collection objects, collection methods, row collection storage and collection indicators.

[0023] As a further solution of the present invention, the acquisition object: cloud components obtain component information, including the following three methods:

[0024] (1). Zookeeper: provides cluster coordination services;

[0025] (2).apiserver: provided through api;

[0026] (3) There is no independent operation instance of the bypass module.

[0027] As a further solution of the present invention, the collection method is specifically as follows:

[0028] (1) Zookeeper: The collection program registers the Watcher to the node path of the component cluster status in Zookeeper. When there is a data change, the server notifies the collection program to call back and capture the data;

[0029] (2) apiserver: The collection program obtains the interface cluster component information and collects and saves the version changes based on the version field in the information returned by the interface.

[0030] As a further solution of the present invention, the row collection and storage: for those that do not provide coordination services or API interfaces, the collection program needs to periodically remotely detect instance port status information.

[0031] As a further solution of the present invention, the collected indicators are cluster component status, which include health status, master-slave information, flow indicators, load indicators and capacity indicators.

[0032] As a further solution of the present invention, the storage layer is implemented by integrating physical data storage with Zookeeper and MongoDB, as follows:

[0033] (1) Zookeeper: System configuration center service, used for system initial configuration file management and storage of cloud resource pool configuration required by the system;

[0034] (2) MongoDB: Builds different data models based on various upper-layer application scenarios. Each piece of data has a version_id number, which is used to record resource changes.

[0035] Compared with related technologies, the method for building a cross-cloud PAAS configuration data service in quasi-real time provided by the present invention has the following beneficial effects:

[0036] 1. The architecture of the present invention is highly available and scalable, and is suitable for a variety of PAAS component metadata management solutions. This solution can be applied to scenarios where PAAS resources are automatically discovered in multiple clouds, monitor and collect target components in real time and accurately, manage multiple versions, automatically perceive the component's online and offline linkage operation and maintenance processes, and provide configuration data services for various operation and maintenance scenarios for applications, meeting the basic requirements of multi-cloud, agile, automatic, and intelligent operation and maintenance management, and realizing the transformation of enterprise IT operation and maintenance systems from silos to platforms, and from functional islands to integrated collaboration;

[0037] 2. The present invention supports the configuration data service requirements of various operation and maintenance scenarios of PAAS components under multi-cloud. It obtains the component role status information changes of the cluster by registering and monitoring the cloud cluster configuration center (zookeeper coordination management system) or through the open API interface on the cloud. After JSON serialization, the model is defined according to the difference in operation and maintenance scenarios, and the data is labeled and associated. Multiple versions are stored in MongoDB. The version difference comparison is used to perceive the online and offline of cluster instances, and then the zookeeper node information is monitored. The standardized API interface provides accurate data sources for various operation and maintenance scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0039] Figure 1 It is a schematic diagram of the technical framework of the present invention;

[0040] Figure 2 It is a boundary schematic diagram of the present invention;

[0041] Figure 3 Schematic diagram of the functional framework of the present invention. DETAILED DESCRIPTION

[0042] Please refer to Figure 1 、 Figure 2 and Figure 3 ,in, Figure 1 It is a schematic diagram of the technical framework of the present invention; Figure 2It is a boundary schematic diagram of the present invention; Figure 3 The framework diagram of the present invention is as follows. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time includes: a service layer, a data layer, a collection layer, and a storage layer;

[0043] The service layer is divided into the following two modules according to data processing logic and functions:

[0044] (1) Interface service module, the interface service module requirements are as follows:

[0045] 1) Design different data models and provide upper-layer applications based on different operation and maintenance scenarios;

[0046] 2) To ensure seamless integration with upper-layer applications, interface standardization must be implemented;

[0047] (2) Resource data visualization module, the requirements of the resource data visualization module are as follows:

[0048] 1) Resource overview, providing a unified view and asset inventory statistics;

[0049] 2) Resource search, providing search capabilities by IP, component, and module, and displaying the permission status, subordination, and dependency relationships of the current resource;

[0050] 3) Change events, resource online and offline, and landing logs provide audit review.

[0051] The data layer: The processing layer associates the collected data with resources, including sorting out the calling relationship, subordinate relationship, and dependency relationship, labeling it, and integrating it into a complete module link.

[0052] The collection layer includes collection objects, collection methods, collection storage and collection indicators.

[0053] The collection object: cloud components obtain component information, including the following three methods:

[0054] (1). Zookeeper: provides cluster coordination services;

[0055] (2).apiserver: provided through api;

[0056] (3) There is no independent operation instance of the bypass module.

[0057] The collection method is as follows:

[0058] (1) Zookeeper: The collection program registers the Watcher to the node path of the component cluster status in Zookeeper. When there is a data change, the server notifies the collection program to call back and capture the data;

[0059] (2) apiserver: The collection program obtains the interface cluster component information and collects and saves the version changes based on the version field in the information returned by the interface.

[0060] The line collection and storage: For those that do not provide coordination services or API interfaces, the collection program needs to periodically remotely detect the instance port status information.

[0061] The collected indicators are cluster component status, including health status, master-slave information, flow indicators, load indicators and capacity indicators.

[0062] The storage layer is implemented by integrating ZooKeeper and MongoDB into the physical data storage, as follows:

[0063] (1) Zookeeper: System configuration center service, used for system initial configuration file management and storage of cloud resource pool configuration required by the system;

[0064] (2) MongoDB: Builds different data models based on various upper-layer application scenarios. Each piece of data has a version_id number, which is used to record resource changes.

[0065] To ensure the scalability of the system, the platform aims at modularization, with module and component clustering capabilities as the boundaries. The acquisition and output interfaces of each module and component are independent of each other, without direct coupling relationships, and the storage is also independent of each other, only ensuring the standardized output data of the upper-level application.

[0066] The present invention provides application use through the json intermediate format, which is flexible to use and has a low threshold for use. It supports cross-cloud management, unified collection and storage of metadata of multi-cloud components, dynamic query of the subordination and dependency of PAAS IP resources, and the IT operation and maintenance system is transformed from chimneys to platforms, and from functional islands to integrated collaboration.

[0067] The present invention supports multiple PAAS components within the cloud and cross-cloud horizontal modules to ensure the scalability of the data platform. It uniformly collects, stores, and analyzes PAAS resources on the cloud, and provides standardized API interface services for application scenarios, realizing the IT operation and maintenance system from chimney to platform, functional island to integrated collaboration. It adopts the zookeeper node watch mechanism to ensure the accuracy and real-time performance of data resource services, and adopts zookeeper as the configuration center service to ensure the high availability and ease of use of the system.

[0068] The present invention provides data services in multiple resource pools of the PAAS system on two clouds of a certain operator, manages more than 10 types of PAAS, and manages more than 3,000 node instances, realizing the IT operation and maintenance system from chimney to platform, and functional island to integrated collaboration, meeting the multi-cloud, agile, automatic and intelligent operation and maintenance management requirements.

[0069] The present invention adopts the Zookeeper node watch mechanism and version comparison mechanism to realize automatic discovery of PAAS resources and ensure the accuracy and real-time performance of data resource services;

[0070] After capturing PAAS components in real time, through data labeling processing, link integration, and standardized interface services, it seamlessly connects to multiple upper-level operation and maintenance scenarios, realizing the transformation of operation and maintenance capabilities from atomization to scenario-based and process-based.

[0071] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention within the technical scope disclosed by the present invention, and they should be covered by the scope of protection of the present invention.

Claims

1. A method for building a cross-cloud PAAS configuration data service in quasi-real time, characterized in that: include: Service layer, data layer, collection layer and storage layer; The service layer is divided into the following two modules according to data processing logic and functions: (1) Interface service module, the interface service module requirements are as follows: 1) Design different data models and provide upper-layer applications based on different operation and maintenance scenarios; 2) To ensure seamless integration with upper-layer applications, interface standardization must be implemented; (2) Resource data visualization module, the requirements of the resource data visualization module are as follows: 1) Resource overview, providing a unified view and asset inventory statistics; 2) Resource search, providing search capabilities by IP, component, and module, and displaying the permission status, subordination, and dependency relationships of the current resource; 3) Change events, resource online and offline, and landing logs provide audit review; The collection layer includes collection objects, collection methods, row collection and storage, and collection indicators; wherein, the collection methods are specifically as follows: (1) Zookeeper: The collection program registers the Watcher to the node path of the component cluster status in Zookeeper. When there is a data change, the server notifies the collection program to call back and capture the data; (2) API server: The collection program obtains the information of the interface cluster components and collects and saves the information with version changes based on the version field in the information returned by the interface; The storage layer is implemented by integrating ZooKeeper and MongoDB into the physical data storage, as follows: (1) Zookeeper: System configuration center service, used for system initial configuration file management and storage of cloud resource pool configuration required by the system; (2) MongoDB: Builds different data models based on various upper-layer application scenarios. Each piece of data has a version_id number, which is used to record resource changes.

2. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time according to claim 1, characterized in that: The data layer associates the collected data with resources, including sorting out the calling relationship, subordinate relationship, and dependency relationship, labels them, and integrates them into a complete module link.

3. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time according to claim 1, characterized in that: The collection object: cloud components obtain component information, including the following three methods: (1). Zookeeper: provides cluster coordination services; (2).apiserver: provided through api; (3) There is no independent operation instance of the bypass module.

4. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time according to claim 1, characterized in that: The line collection and storage: For those that do not provide coordination services or API interfaces, the collection program needs to periodically remotely detect the instance port status information.

5. The method for constructing a cross-cloud PAAS configuration data service in quasi-real time according to claim 1, characterized in that: The collected indicators are cluster component status, including health status, master-slave information, flow indicators, load indicators and capacity indicators.

Images