High-availability cloud security resource pool building method based on edge network environment

Abstract

A high-availability cloud security resource pool building method based on an edge network environment adopts a mode of two servers and external services to deploy a high-availability cloud security resource pool, a third etcd and ceph monitor service is built in other edge cloud stock equipment communicated with a platform management network, the equipment can be shared with other services for use, and the edge network environment can be shared with other services. And additional equipment is not added. Compared with a traditional deployment scheme, the method not only can save hardware resources such as a server and a network, but also provides a high availability characteristic, and aiming at the characteristics that edge network scenes are widely distributed and numerous, the deployment scheme can remarkably reduce the deployment cost of a cloud security resource pool product.

Description

Technical Field

[0001] The present invention relates to the technical field of secure resource pools for mobile clouds, and particularly to a method for building a highly available cloud security resource pool based on an edge network environment. Background Art

[0002] A cloud security resource pool is a secure SAAS service platform built based on cloud computing technology, which uses kubernetes and openstack to provide basic resources for creating security network elements and provides life cycle management of security network elements and security service functions.

[0003] There are mainly two mainstream solutions for building a cloud security resource pool, namely a single-node deployment solution and a cluster deployment solution:

[0004] ① Single-node deployment solution: At least 1 server is required to build a cloud security resource pool;

[0005] ② Cluster deployment solution: At least 3 servers are required to build a cloud security resource pool, and middleware (such as keepalive, Pacemaker, etc.) and load balancing software (such as haproxy, lvs, etc.) are used to build a highly available cloud security resource pool.

[0006] The disadvantages of these two solutions are as follows:

[0007] ① Single-node solution: It cannot provide high availability function. After the server fails, it cannot provide normal business access, and the impact of node failure is very serious.

[0008] ② Cluster solution: At least 3 servers are required to support the deployment, which occupies more physical resources and network resources. Summary of the Invention

[0009] The present disclosure provides a method for building a highly available cloud security resource pool based on an edge network environment, which realizes the construction of a highly available cloud security resource pool by using 2 servers plus an external service, not only realizes the high availability function, but also saves physical server resources and network resources, and solves the problems that the single-node deployment solution lacks high availability function and the traditional cluster deployment occupies more physical resources.

[0010] The method for building a highly available cloud security resource pool based on an edge network environment provided by the present disclosure mainly includes the following steps:

[0011] S1, constructing a kubernetes cluster based on two servers, and etcd on the two servers forms a two-node cluster;

[0012] S2, generating etcd deployment resources for the external service according to the current etcd cluster information and combining with the external server information;

[0013] S3. Create a Ceph cluster using the Helm method on the current two servers;

[0014] S4. Generate the Ceph monitor deployment resources for the external service based on the current Ceph cluster information;

[0015] S5. Build the cloud security resource pool business platform using the Helm method on the current two servers;

[0016] S6. On the underlay workload carrier in the external edge cloud environment, generate the external etcd and Ceph monitor services based on the external resources generated in steps S2 and S4; among them, the external etcd service and the etcd service already built in this cluster form a three-node etcd cluster, and the external Ceph monitor service and the Ceph monitor service already built in this cluster form a three-node Ceph monitor cluster;

[0017] S7. Deploy the external service in an existing Kubernetes cluster, virtualization platform, or other servers with existing services.

[0018] Further, the step S1 specifically includes:

[0019] S11. Install and configure the basic middleware keepalive and the cluster software haproxy on the two servers. Among them, keepalive provides the vip function and the failover function, and Haproxy provides the load balancing function for the kubernetes api communication port 6443;

[0020] S12. Install the first node of the Kubernetes cluster on the first server using the kubeadminit command. The system automatically starts the kubelet service and starts the kubernetes-apiserver, kubernetes-controller-manager, kubernetes-scheduler, kube-porxy, and etcd services in a containerized form;

[0021] S13. Use the kubeadminit command on the second server to join the Kubernetes cluster created on the first server. The system automatically starts the kubelet service and automatically starts the kubernetes-apiserver, kubernetes-controller-manager, kubernetes-scheduler, kube-proxy, and etcd services in a containerized form;

[0022] The etcd on the 2 servers forms a two-node etcd cluster.

[0023] Further, in the step S2:

[0024] The generated etcd deployment resources for the external service include: etcd container image, etcd certificate file, external etcd container yaml file;

[0025] Among them, the etcd container image and the external etcd container yaml file resources are automatically generated in combination with the environment usage resources and environment information.

[0026] Further, in the step S4:

[0027] The generated ceph monitor deployment resources for the external service include: ceph monitor container image, ceph certificate file, external cephmonitor container yaml file, ceph configuration file; among them, the image, yaml, and configuration file resources are automatically generated in combination with the environment usage resources and environment information.

[0028] Further, in the step S6, the steps of generating the external etcd and cephmonitor containers specifically include:

[0029] Import the images, including: etcd container image, ceph monitor container image;

[0030] Import the certificates, including: external etcd container yaml file, external cephmonitor container yaml file;

[0031] Create the services, including: etcd service and ceph monitor service.

[0032] Further, in the step S7, the external service shares the server with other business services and does not require a separate server for deployment.

[0033] Compared with the prior art, the beneficial effects of the present disclosure are as follows: ① Compared with the traditional deployment scheme, it can not only save hardware resources such as servers and networks, but also ensure high availability, solving the problems of the lack of high availability in the single-node deployment scheme and the large physical resource occupation in the traditional cluster deployment; ② For the characteristics of the wide distribution and large number of edge network scenarios, this deployment scheme can significantly reduce the deployment cost of the cloud security resource pool product and comply with the national energy conservation and emission reduction policy; ③ The product adopting this deployment method has a broader market prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] By describing the exemplary embodiments of the present disclosure in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of the present disclosure will become more apparent. Among them, in the exemplary embodiment mode of the present disclosure, the same reference numerals generally represent the same components.

[0035] Figure 1 It is a schematic diagram of the structure of the security resource pool according to the exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to convey the scope of the present disclosure to those skilled in the art completely.

[0037] In view of the problem of frequent lack of hardware resources in the edge network environment, the present disclosure provides a method for building a highly available cloud security resource pool based on the edge network, that is, deploying a highly available cloud security resource pool by using two servers plus an external service.

[0038] In an exemplary embodiment according to the present disclosure, the constructed security resource pool is as shown in the accompanying Figure 1 figures. The cloud security resource pool needs to use etcd and ceph monitor services. Based on the cluster mechanism of etcd and ceph monitor, three services need to be deployed on different nodes to form a cluster to achieve the highly available cluster function. For the service characteristics, in this embodiment, the third etcd and ceph monitor services are built in other edge cloud inventory devices that are connected to the platform management network. This device can be shared with other services and does not add additional devices.

[0039] The specific building process is as follows:

[0040] 1) Install and configure the basic middleware (keepalive) and cluster software (haproxy) on two servers. Keepalive provides VIP function and failover function. Haproxy provides load balancing function for the API communication port 6443 of Kubernetes.

[0041] 2) Use the kubeadminit command to install the first node of the Kubernetes cluster on the first server. The system will automatically start the kubelet service and start the Kubernetes-apiserver, Kubernetes-controller-manager, Kubernetes-scheduler, kube-porxy, and etcd services in a containerized form. The command is as follows:

[0042] kubeadminit

[0043] --control-plane-endpoint k8svip:16443

[0044] --kubernetes-version=v1.21.4

[0045] --pod-network-cidr=10.244.0.0 / 16,2001:db8:42:0:: / 56

[0046] --service-cidr=10.96.0.0 / 16,2001:db8:42:1:: / 112--token"xxxxxxxxx"

[0047] --token-ttl 0

[0048] --ignore-preflight-errors=Swap

[0049] --feature-gates IPv6DualStack=true

[0050] --upload-certs

[0051] 3) Use the kubeadminit command on the second server to join the Kubernetes cluster created on the first server. The system will automatically start the kubelet service, and in a containerized form, it will automatically start the kubernetes-apiserver, kubernetes-controller-manager, kubernetes-scheduler, kube-proxy, and etcd services. The etcd on the two servers will form a two-node cluster. However, due to the working mechanism of the etcd cluster, the two-node etcd cluster at this time cannot provide normal high-availability functions. When one of the nodes fails, the cluster will not be able to achieve majority consistency, resulting in data inconsistency or service unavailability. The command is as follows:

[0052] kubeadm join k8svip:16443

[0053] --token 82nygr.04ujazebkddtkecr

[0054] --discovery-token-ca-cert-hash xxxxxxxxx

[0055] --control-plane

[0056] --certificate-key xxxxxxxxx

[0057] 4) Based on the current etcd cluster information (IP, port, hostname, container image, etc.) combined with the external server information (IP, port, hostname, etc.), automatically generate the etcd deployment resources for the external service, including (etcd container image, etcd certificate file, external etcd container yaml file).

[0058] Command to obtain the etcd certificate file:

[0059] kubectl create secret generic

[0060] --from-file=server.crt= / etc / kubernetes / pki / etcd / server.crt

[0061] --from-file=server.key= / etc / kubernetes / pki / etcd / server.key

[0062] --from-file=peer.crt= / etc / kubernetes / pki / etcd / peer.crt

[0063] --from-file=peer.key= / etc / kubernetes / pki / etcd / peer.key

[0064] --from-file=ca.crt= / etc / kubernetes / pki / etcd / ca.crt

[0065] --namespace=venus-thcloudetcd-cmss-tls

[0066] --dry-run=client -o yaml>etcd-tls.yaml

[0067] The images and yaml resources are combined with the environment to call the written tool to automatically generate the resources and environment information.

[0068] 5) Deploy the kubernetes cluster network plugin, and the command is as follows:

[0069] kubectl apply -f calico.yaml

[0070] 6) Deploy ceph in the helm way. Based on the information of the current 2 servers, network planning information, and external server information, generate the helm configuration file, create a ceph cluster on the current 2 servers. After installation, there are 2 ceph monitor services. The ceph monitor nodes use the Paxos consensus algorithm to determine the cluster status and configuration changes. The Paxos algorithm requires at least more than half of the nodes to reach an agreement to make a status change. Therefore, at least 3 nodes can tolerate the failure of one node and still be able to perform status updates and cluster configuration changes. At this time, there are only 2 ceph monitor nodes. Although they can provide services normally, they do not have the high-availability function. The deployment commands are as follows:

[0071] helm upgrade --install ceph-monceph-mon --namespace=ceph --values= / tmp / ceph.yaml

[0072] helm upgrade --install ceph-osdceph-osd --namespace=ceph --values= / tmp / ceph.yaml

[0073] helm upgrade --install ceph-client ceph-client --namespace=ceph --values= / tmp / ceph.yaml

[0074] helm upgrade --install ceph-provisioners ceph-provisioners --namespace=ceph --values= / tmp / ceph.yaml

[0075] 7) Automatically generate the Ceph monitor deployment resources for external services based on the current Ceph cluster information (Ceph certificates, IPs, ports, hostnames, etc.), including (Ceph monitor container images, Ceph certificate files, external Ceph monitor container yaml files, Ceph configuration files).

[0076] The commands to obtain Ceph certificate files are as follows:

[0077] kubectl get secret -n ceph -o yaml ceph1-bootstrap-mds-keyring >> cephmon-base.yaml

[0078] kubectl get secret -n ceph -o yaml ceph1-bootstrap-mgr-keyring >> cephmon-base.yaml

[0079] kubectl get secret -n ceph -o yaml ceph1-bootstrap-osd-keyring >> cephmon-base.yaml

[0080] kubectl get secret -n ceph -o yaml ceph1-client-admin-keyring >> cephmon-base.yaml

[0081] kubectl get secret -n ceph -o yaml ceph1-mon-keyring >> cephmon-base.yaml

[0082] kubectl get secret -nceph -o yaml pvc -ceph1 -conf -combined -storageclass >> cephmon -base.yaml

[0083] kubectl get secret -nceph -o yaml csi -cephfs -secret >> cephmon -base.yaml

[0084] Images, YAML, and configuration file resources are combined with the environment to use resources and environment information to call the written tools to generate automatically.

[0085] 8) Deploy the cloud security resource pool business platform in the helm way. According to the information of the current 2 servers and network planning information (disks, network cards, IPs, etc.), generate the helm configuration file, and use the helm way to build the cloud security resource pool business platform on the current 2 servers. The business pods of the cloud security resource pool use a single replica. After a server fails, rely on the kubernetes cluster characteristics for pod migration to achieve the high - availability characteristics of the cloud security resource pool.

[0086] 9) On the underlay workload carrier in the external edge cloud environment, generate external resources according to steps 4 and 6, and generate external etcd and ceph monitor containers. The external etcd container and the etcd service already built in this cluster form a three - node etcd cluster, and the external ceph monitor container and the ceph monitor service already built in this cluster form a three - node ceph monitor cluster. Overall, provide high - availability functions. The specific operation process is as follows:

[0087] Import the image

[0088] docker load < etcd.tar

[0089] docker load < ceph - mon.tar

[0090] Import the certificate

[0091] kubectl apply -f etcd -tls.yaml

[0092] kubectl apply -f cephmon -base.yaml

[0093] Create the service

[0094] kubectl apply -f etcd -pod.yaml

[0095] kubectl apply -f cephmon-pod.yaml

[0096] 10) External services (etcd, ceph monitor) can be deployed in an existing Kubernetes cluster, virtualization platform, or other servers with existing services. Servers can be shared with other business services, and there is no need to provide an independent server for deployment. In the above case, the external service deployment adopts the solution of being deployed in a Kubernetes cluster.

[0097] In this embodiment, a combination of software and hardware is adopted to deploy external software services on existing edge devices and form a 2-node highly available cloud security resource pool cluster in combination with the cloud security resource pool installed on 2 nodes; the high availability of the cloud security resource pool business service is realized based on the functions of the 2-node Kubernetes cluster formed by this method; an external service resource package is automatically generated to facilitate the rapid deployment of external resources.

[0098] Compared with the single-node deployment solution of the cloud security resource pool, this solution provides a highly available function, which plays a crucial role in platform stability, data security, user experience, and operation and maintenance efficiency.

[0099] Compared with the cluster deployment solution of the cloud security resource pool, one server and some network resources are saved. Considering the characteristics of wide distribution and large quantity in edge network scenarios, a large amount of physical resources can be saved.

[0100] The above technical solutions are only exemplary embodiments of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is very easy to make various types of improvements or deformations, not limited to the methods described in the above specific embodiments of the present invention. Therefore, the above-described manner is only preferred and does not have a restrictive meaning.

Claims

1. A method for building a highly available cloud security resource pool based on an edge network environment, comprising the following steps: S1. Build a kubernetes cluster based on two servers, and etcd on the two servers forms a two-node cluster; S2. Generate etcd deployment resources for external services according to the current etcd cluster information and in combination with external server information; S3. Create a ceph cluster on the current two servers using the helm method; S4. Generate ceph monitor deployment resources for external services according to the current ceph cluster information; S5. Build a cloud security resource pool business platform on the current two servers using the helm method; S6. On the underlay workload carrier in the external edge cloud environment, generate external etcd and ceph monitor services according to the external resources generated in steps S2 and S4; wherein, the external etcd service and the etcd service already built in this cluster form a three-node etcd cluster, and the external ceph monitor service and the ceph monitor service already built in this cluster form a three-node ceph monitor cluster; S7. Deploy the external services in an existing kubernetes cluster, virtualization platform or other servers with existing services.

2. The method according to claim 1, wherein The specific steps of step S1 include: S11. Install and configure the basic middleware keepalive and the cluster software haproxy on the two servers. Among them, keepalive provides the vip function and the failover function, and haproxy provides the load balancing function for the api communication port 6443 of kubernetes; S12. Use the kubeadminit command on the first server to install the first node of the kubernetes cluster. The system automatically starts the kubelet service and starts the kubernetes-apiserver, kubernetes-controller-manager, kubernetes-scheduler, kube-porxy and etcd services in a containerized form; S13. Use the kubeadminit command on the second server to join the kubernetes cluster created on the first server. The system automatically starts the kubelet service and automatically starts the kubernetes-apiserver, kubernetes-controller-manager, kubernetes-scheduler, kube-porxy and etcd services in a containerized form; etcd on the 2 servers forms a two-node etcd cluster.

3. The method according to claim 1, wherein In step S2: The generated etcd deployment resources for external services include: etcd container images, etcd certificate files, and external etcd container yaml files; Among them, the etcd container image, the external etcd container yaml file resource are automatically generated in combination with the environment usage resources and environment information.

4. The method according to claim 1, wherein In the step S4: The generated ceph monitor deployment resources for the external service include: ceph monitor container image, ceph certificate file, external ceph monitor container yaml file, ceph configuration file; among them, the image, yaml, and configuration file resources are automatically generated in combination with the environment usage resources and environment information.

5. The method according to claim 1, wherein In the step S6, the steps for generating the external etcd and ceph monitor containers specifically include: Importing images, including: etcd container image, ceph monitor container image; Importing certificates, including: external etcd container yaml file, external ceph monitor container yaml file; Creating services, including: etcd service and ceph monitor service.

6. The method according to claim 1, wherein In the step S7, the external service shares the server with other business services and does not require a dedicated server for deployment.

Images