Heterogeneous computing power pool unified scheduling and management system and system based on OpenAPI gateway

Abstract

The invention relates to a heterogeneous computing power pool unified scheduling and management system based on an OpenAPI gateway. The heterogeneous computing power pool unified scheduling and management system comprises a cloud management platform, a system layer, a service layer and a resource pool layer. The cloud management platform is used for taking over each platform, realizing heterogeneous compatibility and providing infrastructure as a service and container arrangement functions, and the system layer is used for realizing business support, operation management and cloud monitoring; the service layer is used for unified authentication management, unified resource operation and unified protocol detection; and the resource pool layer is different platform resource pools of each manufacturer. According to the method, the requirements of unified scheduling management of the enterprise on the multi-heterogeneous resource pool for improving the resource utilization rate, reducing the cost and enhancing the system flexibility and reliability are met.

Description

Technical Field

[0001] The present invention relates to the field of cloud management technology, and in particular to a unified scheduling and management system for heterogeneous computing power pools based on an OpenAPI gateway. Background Art

[0002] The office network computing resource pool is divided into two resource pools: external service and internal management. The external service resource pool includes multi-platform resource pools. When the remaining resources in the resource pool will not be able to meet the subsequent information system construction requirements of various business departments, the virtualization resource pool has improved the resource utilization rate of physical equipment to a certain extent, but it still has a large number of manual operations and low automation in terms of virtual machine security protection and platform operation and maintenance. At the same time, considering the application needs of new technologies such as big data and artificial intelligence, it is necessary to establish a management cloud platform. On the basis of the existing office network expansion construction and use planning needs, the unified resource scheduling and management needs that may be involved in the overall construction in the future should be considered. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention proposes a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway and a system thereof, aiming to meet the needs of enterprises for unified scheduling and management of multiple heterogeneous resource pools to improve resource utilization, reduce costs, and enhance system flexibility and reliability.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway, including a cloud management platform, a system layer, a service layer and a resource pool layer; the cloud management platform is used to take over various platforms, realize heterogeneous compatibility, and provide infrastructure as a service and container orchestration functions; the system layer is used to realize business support, operation management and cloud monitoring; the service layer is used for unified authentication management, unified resource operation and unified detection protocol; the resource pool layer is the resource pool of different platforms of each manufacturer.

[0006] As an optimization, the resource pool includes but is not limited to Huawei resource pool, Green Alliance security resource pool, and newly built heterogeneous resource pool.

[0007] As an optimization, after the cloud management platform manages Huawei resource pools and creates new heterogeneous resource pools, it pools resources and maps resources in a cloud platform to one or more logical resource pools by defining resource pools, so that resource pools can be freely divided and quota managed. The resource pool can be specified to use specified computing resources, storage resources and network resources in a cloud platform, and the available resource usage can be specified; the resource pool must be completely decoupled from departments and projects, and usage authorization must be performed according to actual needs.

[0008] As an optimization, the cloud management platform technology route chooses to combine OpenStack and Kubernetes to achieve heterogeneous compatibility of the cloud platform; and provides Infrastructure as a Service IaaS and container orchestration functions.

[0009] Specifically, for the unified management of infrastructure resources, OpenStack provides a unified cloud resource management platform that can manage multiple types of resources, such as virtual machines, storage, networks, and identity authentication, while Kubernetes can manage and orchestrate containerized applications. The combination of these two platforms can achieve unified management and scheduling of infrastructure resources, allowing different hardware platforms to run the same virtual machines and containerized applications.

[0010] Specifically, for cross-platform container orchestration: Kubernetes is a container orchestration platform that can manage and orchestrate containerized applications; it can automatically create, scale out, and scale down containers, as well as schedule and run applications in a cluster; by combining with OpenStack, Kubernetes can achieve cross-platform container orchestration and management in different cloud environments.

[0011] Specifically, for flexible resource scheduling: OpenStack provides powerful resource scheduling functions, which can allocate and schedule resources according to different strategies and requirements; Kubernetes can also schedule and manage containers according to different rules and strategies; the combination of the two can achieve more flexible and efficient resource scheduling to meet the needs of different users.

[0012] Specifically, regarding interoperability: OpenStack and Kubernetes are both open source cloud computing platforms with broad interoperability; they can be integrated with other cloud platforms, container technologies, and tools to provide a more complete solution; this interoperability allows different users to choose appropriate components and technologies to build a cloud platform according to their needs.

[0013] As an optimization, the cloud management platform supports data encryption in backup storage to ensure the security of disaster recovery backup data transmission and storage. After the calculation is completed, the data will be transmitted to the disaster recovery backup server. The data will always remain encrypted during transmission and storage. When the data is restored, the stored data will be sent directly to the target client, and then the client will decrypt the data. Backup storage supports AES256 and SM4 encryption algorithms.

[0014] Specifically, to ensure the security of backup set data, the data backup subsystem itself should have disaster recovery storage read and write protection capabilities. This function can protect the physical path of the backup set data. Only programs that are registered and authenticated with the disaster recovery storage can read, write, and modify the protected directory or stored data. Even if hackers log in with the highest management account authority, they cannot read or tamper with the data, which can effectively prevent data leakage or ransomware.

[0015] As an optimization, the unified data backup platform achieves user management security through diversified management; diversified separation: a four-member power-sharing system is established, including system administrator, security administrator, and audit administrator. Each administrator role is independent of each other but also restricts each other.

[0016] Specifically, the system administrator is responsible for daily maintenance and management work, including global monitoring, site management (site configuration, site grouping, site deletion), task monitoring, viewing and exporting task execution reports, editing alarm rules, viewing alarms (including disaster recovery sites, tasks, systems, quota alarm information), system settings, system security policy settings, license settings, SMTP settings, and custom configuration; the security administrator is responsible for system security and user management, including user log viewing and user management; the audit administrator: is responsible for the audit of task execution, alarm information and administrator operations, and does not have operation permissions, including viewing and exporting task execution reports, viewing alarms, and viewing user logs.

[0017] Specifically, the time required to achieve RPO includes the backup task data query traversal time and data transmission time. The unified data backup platform uses permanent incremental technology to ensure that the full amount of data needs to be transmitted for the first time, but only the changed data is transmitted for each subsequent backup. Generally, the data transmission time is related to the actual data type and data volume.

[0018] The time required to achieve RTO includes leadership decision-making time, IT switching time (data recovery and application verification) and business verification time. The data recovery time depends on the data query traversal time and data transmission time. Generally, the data transmission time is related to the actual data type and data volume.

[0019] As an optimization, the requirements for heterogeneity include: the cloud management platform and the equipment brands to be used in the project are heterogeneous; the storage servers with data redundancy based on erasure coding technology and the storage servers with data redundancy based on multi-copy technology are heterogeneous in brand and chip type; the chip types of computing servers, management servers, and network servers must be heterogeneous.

[0020] Specifically, when selecting heterogeneous models: the cloud management software manufacturer is Inspur; the cloud management server manufacturer is Lenovo, and the chip is Hygon; the network server manufacturer is Lenovo, and the chip is Hygon.

[0021] As an optimization, for storage needs, the storage resource pool can support distributed storage architecture and flexible storage cluster deployment methods; the storage resource pool supports multiple disk deployments, including independent deployment and hybrid deployment of multiple types of disks such as SAS, SATA and SSD, supports storage SLA capabilities, and maximizes hardware performance; the storage resource pool must support unified resource management and allocation of the storage resource pool on the cloud management platform through docking with the upper-level cloud management platform; for storage characteristics, a distributed software-defined storage architecture is adopted to provide distributed storage services, and support the provision of multiple storage services, including distributed block, object, and NAS storage services; for storage data redundancy technology, a dual-stack technology route is adopted, and data redundancy adopts two technologies: erasure coding and three copies.

[0022] As an optimization, the overall architecture of the cloud management platform is designed as a modular and microservice design, following the principle of openness. The platform's functions can be easily expanded by adding modules based on customized development.

[0023] As an optimization, the cloud management platform defines a standard SDK interface, shields the differences in function implementation among various cloud vendors, and realizes the decoupling of multiple platforms, multiple virtualizations, and multiple versions.

[0024] Specifically, the cloud management platform provides a unified adaptation interface and has flexible cloud management expansion capabilities, and can be expanded and adapted to other cloud platforms in the future.

[0025] The present invention has the following advantages:

[0026] (1) Hardware compatibility: The system supports various types of hardware devices, including servers, storage devices, and network devices.

[0027] (2) Network compatibility: The system supports various types of network protocols and network equipment; it has scalable network management and security mechanisms to ensure user data security and business continuity.

[0028] (3) Storage compatibility: The system supports various types of data storage and management, including file storage, object storage, block storage, etc., and has a scalable data storage and management mechanism to ensure user data reliability and consistency.

[0029] (4) Software compatibility: The system supports various types of software, including operating systems, databases, middleware, etc. It has a rich software stack and tools to facilitate users to quickly deploy and run various applications in the environment. At the same time, the system supports different versions of software to meet the needs of different users. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1This is a framework diagram of a unified scheduling and management system for heterogeneous computing power pools based on an OpenAPI gateway as described in the present invention.

[0031] Figure 2 This is a diagram of the logical architecture of the cloud management platform described in the present invention.

[0032] Figure 3 This is a physical architecture topology diagram. DETAILED DESCRIPTION

[0033] The present invention is further described in detail below in conjunction with the accompanying drawings. In the description of the present invention, it should be understood that the directions or positional relationships indicated by directional words such as "upper, lower" and "top, bottom" are usually based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description. Unless otherwise stated, these directional words do not indicate or imply that the devices or components referred to must have a specific direction or be constructed and operated in a specific direction, and therefore cannot be understood as limiting the scope of protection of the present invention; the directional words "inside and outside" refer to the inside and outside relative to the contours of each component itself.

[0034] like Figure 1-3 As shown, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway includes a cloud management platform, a system layer, a service layer and a resource pool layer; the cloud management platform is used to take over each platform, realize heterogeneous compatibility, and provide infrastructure as a service and container orchestration functions; the system layer is used to realize business support, operation management and cloud monitoring; the service layer is used for unified authentication management, unified resource operation and unified detection protocol; the resource pool layer is the resource pool of different platforms of each manufacturer.

[0035] In this embodiment, the resource pool includes but is not limited to a Huawei resource pool, a NSFOCUS security resource pool, and a newly-built heterogeneous resource pool.

[0036] In this embodiment, after the cloud management platform manages Huawei resource pools and creates new heterogeneous resource pools, the resources are pooled. By defining resource pools, the resources in a cloud platform are mapped to one or more logical resource pools, so that the resource pools can be freely divided and quota managed. The resource pool can be specified to use specified computing resources, storage resources and network resources in a cloud platform, and the available resource usage can be specified; the resource pool must be completely decoupled from departments and projects, and usage authorization is performed according to actual needs.

[0037] In this embodiment, the cloud management platform technology route selects a combination of OpenStack and Kubernetes to achieve heterogeneous compatibility of the cloud platform; and provides Infrastructure as a Service IaaS and container orchestration functions.

[0038] Specifically, for the unified management of infrastructure resources, OpenStack provides a unified cloud resource management platform that can manage multiple types of resources, such as virtual machines, storage, networks, and identity authentication, while Kubernetes can manage and orchestrate containerized applications. The combination of these two platforms can achieve unified management and scheduling of infrastructure resources, allowing different hardware platforms to run the same virtual machines and containerized applications.

[0039] Specifically, for cross-platform container orchestration: Kubernetes is a container orchestration platform that can manage and orchestrate containerized applications; it can automatically create, scale out, and scale down containers, as well as schedule and run applications in a cluster; by combining with OpenStack, Kubernetes can achieve cross-platform container orchestration and management in different cloud environments.

[0040] Specifically, for flexible resource scheduling: OpenStack provides powerful resource scheduling functions, which can allocate and schedule resources according to different strategies and requirements; Kubernetes can also schedule and manage containers according to different rules and strategies; the combination of the two can achieve more flexible and efficient resource scheduling to meet the needs of different users.

[0041] Specifically, regarding interoperability: OpenStack and Kubernetes are both open source cloud computing platforms with broad interoperability; they can be integrated with other cloud platforms, container technologies, and tools to provide a more complete solution; this interoperability allows different users to choose appropriate components and technologies to build a cloud platform according to their needs.

[0042] In this embodiment, in order to ensure the security of the disaster recovery backup data transmission process and storage, the cloud management platform supports data encryption in the backup storage; after the calculation is completed, the data is transmitted to the disaster recovery backup server, and the data always remains encrypted during transmission and storage. When the data is restored, the stored data will be sent directly to the target client, and then the client will decrypt the data. The backup storage supports AES256 and SM4 encryption algorithms.

[0043] Specifically, to ensure the security of backup set data, the data backup subsystem itself should have disaster recovery storage read and write protection capabilities. This function can protect the physical path of the backup set data. Only programs that are registered and authenticated with the disaster recovery storage can read, write, and modify the protected directory or stored data. Even if hackers log in with the highest management account authority, they cannot read or tamper with the data, which can effectively prevent data leakage or ransomware.

[0044] In this embodiment, the unified data backup platform achieves user management security through diversified management; diversified separation: a four-member power division system is established, including system administrator, security administrator, and audit administrator, and each administrator role is independent of each other but restricts each other.

[0045] Specifically, the system administrator is responsible for daily maintenance and management work, including global monitoring, site management (site configuration, site grouping, site deletion), task monitoring, viewing and exporting task execution reports, editing alarm rules, viewing alarms (including disaster recovery sites, tasks, systems, quota alarm information), system settings, system security policy settings, license settings, SMTP settings, and custom configuration; the security administrator is responsible for system security and user management, including user log viewing and user management; the audit administrator: is responsible for the audit of task execution, alarm information and administrator operations, and does not have operation permissions, including viewing and exporting task execution reports, viewing alarms, and viewing user logs.

[0046] Specifically, the time required to achieve RPO includes the backup task data query traversal time and data transmission time. The unified data backup platform uses permanent incremental technology to ensure that the full amount of data needs to be transmitted for the first time, but only the changed data is transmitted for each subsequent backup. Generally, the data transmission time is related to the actual data type and data volume.

[0047] The time required to achieve RTO includes leadership decision-making time, IT switching time (data recovery and application verification) and business verification time. The data recovery time depends on the data query traversal time and data transmission time. Generally, the data transmission time is related to the actual data type and data volume.

[0048] In this embodiment, the requirements for heterogeneity include: the cloud management platform and the equipment brands to be used in the project are heterogeneous; the storage servers with data redundancy based on erasure coding technology and the storage servers with data redundancy based on multiple copies technology are heterogeneous in brand and chip type; the chip types of the computing servers, management servers, and network servers must be heterogeneous.

[0049] Specifically, when selecting heterogeneous models: the cloud management software manufacturer is Inspur; the cloud management server manufacturer is Lenovo, and the chip is Hygon; the network server manufacturer is Lenovo, and the chip is Hygon.

[0050] In this embodiment, for storage requirements, the storage resource pool can support distributed storage architecture and flexible storage cluster deployment methods; the storage resource pool supports multiple disk deployments, including independent deployment and hybrid deployment of multiple types of disks such as SAS, SATA and SSD, supports storage SLA capabilities, and maximizes hardware performance; the storage resource pool must support unified resource management and allocation of the storage resource pool on the cloud management platform through docking with the upper-level cloud management platform; for storage characteristics, a distributed software-defined storage architecture is adopted to provide distributed storage services, and support the provision of multiple storage services, including distributed blocks, objects, and NAS storage services; for storage data redundancy technology, a dual-stack technology route is adopted, and data redundancy adopts two technologies: erasure coding and three copies.

[0051] In this embodiment, the overall architecture of the cloud management platform is designed as a modular, microservice design that follows the principle of openness. The functionality of the platform can be easily expanded by adding modules based on customized development.

[0052] As an optimization, the cloud management platform defines a standard SDK interface, shields the differences in function implementation among various cloud vendors, and realizes the decoupling of multiple platforms, multiple virtualizations, and multiple versions.

[0053] Specifically, the cloud management platform provides a unified adaptation interface and has flexible cloud management expansion capabilities, and can be expanded and adapted to other cloud platforms in the future.

[0054] The present invention has the following advantages:

[0055] (1) Hardware compatibility: The system supports various types of hardware devices, including servers, storage devices, and network devices.

[0056] (2) Network compatibility: The system supports various types of network protocols and network equipment; it has scalable network management and security mechanisms to ensure user data security and business continuity.

[0057] (3) Storage compatibility: The system supports various types of data storage and management, including file storage, object storage, block storage, etc., and has a scalable data storage and management mechanism to ensure user data reliability and consistency.

[0058] (4) Software compatibility: The system supports various types of software, including operating systems, databases, middleware, etc. It has a rich software stack and tools to facilitate users to quickly deploy and run various applications in the environment. At the same time, the system supports different versions of software to meet the needs of different users.

[0059] Finally, it should be noted that those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway, characterized in that: It includes a cloud management platform, a system layer, a service layer and a resource pool layer; the cloud management platform is used to take over various platforms, achieve heterogeneous compatibility, and provide infrastructure as a service and container orchestration functions; the system layer is used to achieve business support, operation management and cloud monitoring; the service layer is used for unified authentication management, unified resource operation and unified detection protocols; the resource pool layer is the resource pool of different platforms of each manufacturer.

2. According to claim 1, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway, characterized in that: The resource pools include but are not limited to Huawei resource pools, NSFOCUS security resource pools, and newly built heterogeneous resource pools.

3. According to claim 2, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: After the cloud management platform manages Huawei resource pools and creates new heterogeneous resource pools, it pools resources and maps resources in a cloud platform to one or more logical resource pools by defining resource pools, thereby freely dividing and quota-managing resource pools. It can specify that a resource pool uses specified computing resources, storage resources, and network resources in a cloud platform, and specifies the amount of resources that can be used. The resource pool must be completely decoupled from departments and projects, and authorization for use must be granted based on actual needs.

4. According to claim 3, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: The cloud management platform technology route chooses to combine OpenStack and Kubernetes to achieve heterogeneous compatibility of the cloud platform; it provides Infrastructure as a Service IaaS and container orchestration functions.

5. According to claim 4, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: In order to ensure the security of the disaster recovery backup data transmission process and storage, the cloud management platform supports data encryption in backup storage; after the calculation is completed, the data is transmitted to the disaster recovery backup server, and the data always remains encrypted during transmission and storage. When the data is restored, the stored data will be sent directly to the target client, and then the client will decrypt the data. The backup storage supports AES256 and SM4 encryption algorithms.

6. According to claim 5, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: The unified data backup platform achieves user management security through diversified management; diversified separation: a four-member power-sharing system is established, including system administrator, security administrator, and audit administrator. Each administrator role is independent of each other but also restricts each other.

7. According to claim 6, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: Requirements for heterogeneity include: the cloud management platform and the equipment brands to be used in the project are heterogeneous; the storage servers with data redundancy based on erasure coding technology and the storage servers with data redundancy based on multiple copies technology are heterogeneous in brand and chip type; the chip types of computing servers, management servers, and network servers must be heterogeneous.

8. According to claim 7, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway is characterized in that: For storage requirements, the storage resource pool can support distributed storage architecture and flexible storage cluster deployment methods; the storage resource pool supports multiple disk deployments, including independent deployment and hybrid deployment of multiple types of disks such as SAS, SATA and SSD, supports storage SLA capabilities, and maximizes hardware performance; the storage resource pool must support unified resource management and allocation of the storage resource pool on the cloud management platform through docking with the upper-level cloud management platform; for storage characteristics, a distributed software-defined storage architecture is adopted to provide distributed storage services, and support the provision of multiple storage services, including distributed block, object, and NAS storage services; for storage data redundancy technology, a dual-stack technology route is adopted, and data redundancy adopts two technologies: erasure coding and three copies.

9. According to claim 8, a unified scheduling and management system for heterogeneous computing power pools based on OpenAPI gateway, characterized in that: The overall architecture of the cloud management platform is designed as a modular, microservice design that follows the principle of openness. The functionality of the platform can be easily expanded by adding modules based on customized development.

10. The system for unified scheduling and management of heterogeneous computing power pools based on OpenAPI gateway according to claim 9 is characterized in that: The cloud management platform defines a standard SDK interface, shields the differences in function implementation among various cloud vendors, and realizes the decoupling of multiple platforms, multiple virtualizations, and multiple versions.

Images