Convenient deployment method for multi-choice component based on DevOps

Abstract

The invention discloses a DevOps-based multi-choice component convenient deployment method, which comprises the following steps: interactively acquiring cluster configuration items from a user, including basic parameter configuration of a cluster and starting parameter configuration of cluster selectable components, to generate a main configuration file, analyzing the main configuration file to obtain an installation file for cluster deployment, executing the installation file, and executing the installation file for cluster deployment. A cluster environment and optional components are integrally and conveniently deployed; the method comprises the following steps of: configuring a trigger mechanism for a code warehouse, pushing an update code to the code warehouse by the developer, triggering an assembly line task of an automatic tool, performing automatic code compiling and mirror image construction on a selectable component code, and automatically pushing a mirror image to a mirror image warehouse by the developer; according to the method, mirror image updating of the selectable components is executed through an automatic tool to complete automatic deployment of the selectable components, so that the efficiency of cluster management and component updating deployment is improved, and the delivery efficiency and quality of a software system are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of cluster automated deployment, and in particular relates to a convenient deployment method for multiple selectable components based on DevOps. Background Art

[0002] With the development of cloud computing and containerization technology, Kubernetes (K8s for short, an open source container orchestration platform, mainly used for automated deployment, expansion and management of containerized applications) has become a standard platform for managing containerized applications. However, the deployment and management process of Kubernetes is usually complicated and requires precise configuration and operation steps. The traditional manual deployment method has many disadvantages, such as the need to manually write a large number of configuration scripts, which is time-consuming and error-prone. After installing the basic Kubernetes cluster, users also need to install other cluster-related components again according to specific needs. The steps are cumbersome and inefficient, affecting the stability and reliability of the system.

[0003] The invention patent with publication number CN115981669A discloses a multi-architecture-based DevOps (a culture and practice method that emphasizes the collaboration between development and operation, and aims to improve the efficiency and quality of software development and delivery through automated processes and continuous integration and continuous delivery (CI / CD)) platform deployment method, which implements offline automated deployment of the basic Kubernetes cluster based on the K8s architecture and Ansible architecture. However, it is still necessary to manually create directories and manually write the host list information of the cluster host, and no unified installation and deployment is implemented for other cluster-related components required by users. Summary of the invention

[0004] In order to solve the shortcomings of the prior art and achieve the purpose of interactive parameter configuration and convenient deployment of optional components, the present invention adopts the following technical solutions:

[0005] A convenient deployment method for multiple optional components based on DevOps is as follows:

[0006] Interactively obtain cluster configuration items from users, including basic cluster parameter configuration and enabled parameter configuration of optional cluster components, to generate a main configuration file, parse the main configuration file to obtain the installation file for cluster deployment, execute the installation file, and conveniently deploy the cluster environment and optional components in an integrated manner. The selection of interactive cluster configuration items can lower the user's usage threshold, greatly reduce working time, and reduce the error rate of manual input;

[0007] For developers' modifications to optional components, a trigger mechanism is configured for the code repository. Developers push updated code to the code repository, triggering the pipeline task of the automation tool to automatically compile and build the optional component code, and automatically push the image to the image repository. The image update of the optional component is executed by the automation tool to complete the automated deployment of the optional component. By automatically executing the entire code compilation, image building, and component deployment process, when upgrading or updating the optional components, operational errors caused by manual execution are eliminated, efficiency is improved to improve development efficiency and delivery frequency. Developers can obtain deployment feedback information in a short time, discover and fix problems in a timely manner, thereby shortening the development cycle and achieving continuous delivery and rapid iteration of applications.

[0008] Furthermore, the basic parameter configuration includes host information and cluster configuration information. The host information is used to confirm the cluster type, the role of the cluster node, and the image warehouse node information. The master node and the working node information are confirmed based on the cluster type and the node role. The configuration of address information, password, and host name are used to configure these configurations. These configurations are crucial to the normal operation and management of the cluster. Failure to perform these configurations will result in the cluster being unable to be deployed or run correctly. Based on the image warehouse node information, the cluster is confirmed to obtain the corresponding container image, thereby ensuring that the K8s cluster can correctly pull the required image; the cluster configuration information is used to confirm the network mode of the cluster, which is used for the network settings of the cluster to avoid network conflicts and correctly set up the network;

[0009] Parse the main configuration file, extract the configuration information related to the container, parse and configure it into the container's daemon process, and write the read configuration items into the daemon.json file for the installation of the container Docker; generate the automated operation and maintenance Ansible configuration file based on the host information, which is used to set up load balancing and proxy services to improve the performance and reliability of applications and services, as well as the Keepalived file for configuring and managing virtual address allocation to set up virtual IP addresses and network interfaces to achieve high availability, so as to automate the cluster configuration process and adapt to different cluster setting requirements by dynamically modifying the configuration file; based on the cluster configuration information, process the K8s cluster network storage configuration, and correctly configure them to Expand the functions of the K8s cluster, otherwise, the cluster may lack necessary functions or services, affecting the integrity and functionality of the system; extract the host IP address of the image warehouse of the container, and update the host name Hostname in the configuration dictionary to the specified host IP address, so as to ensure that Harbor is configured with the correct IP address and generate the Harbor.yaml configuration file for the image warehouse settings, otherwise the storage and management of the container image will be affected, which may cause the image to be unavailable or difficult to manage; extract the cluster type and the role of the cluster node to configure each node of the cluster, ensure that each node of the cluster can be correctly configured and work normally, and provide a stable control plane and business load support for the cluster.

[0010] Furthermore, the installation file extracts the Kubeadm configuration file for cluster deployment management based on the parsed main configuration file and the default configuration file, initialization configuration, cluster configuration, proxy configuration, and node management and operation Kubelet configuration. The node is the most basic scheduling unit of the cluster, representing a group of one or more containers. These containers share storage, network and operating environment, and generate the corresponding cluster deployment management configuration YAML file for K8s cluster initialization;

[0011] In the cluster installation phase, based on the cluster deployment management configuration YAML file, cluster initialization, installation of the image repository and the container, creation of a cluster network and cluster storage are performed respectively, and corresponding components are installed and configured on each node according to the cluster type and the role of the cluster node.

[0012] Furthermore, the initialization configuration is used to set the basic parameters of the K8s cluster, extract the initialization configuration from the main configuration file and the default configuration file, and generate the corresponding Kubeadm initialization configuration file for cluster deployment management, including the control plane settings, authentication and encryption configuration of the cluster, etc., to ensure that the cluster can be started normally and preliminarily configured;

[0013] The cluster configuration is used to define the network and storage settings of the cluster, extract the cluster configuration and the default cluster configuration to generate a Kubeadm cluster configuration file for cluster deployment management;

[0014] The proxy configuration is to configure the proxy service to process the network traffic routing within the cluster to ensure the correct forwarding of network traffic, the correct operation of services and load balancing, extract the proxy configuration and the default proxy configuration, and generate the Kubeadm proxy configuration file for cluster deployment management;

[0015] The Kubelet configuration for node management and operation is used to manage the container operation of the node, and the Kubelet configuration for node management and operation is extracted to generate a Kubelet configuration file for node management and operation.

[0016] Furthermore, the automated operation and maintenance Ansible configuration file includes configuring global variables, obtaining the IP address of the image warehouse Harbor, reading and checking whether the master node is also set as a working node, updating the global variable configuration file, including the image warehouse Harbor IP address and the working status of the master node, and regenerating the global variable configuration file;

[0017] During the cluster installation phase, check the network connection of the IP address of the specified image repository Harbor and the IP address of the master node to ensure that there are no problems with network communication. This is an important step to ensure cluster stability and normal operation of services.

[0018] Furthermore, the cluster type includes a three-master control cluster, in which there are three master control nodes, and the three-master control cluster includes a control initialization node, a master control joining node, and a working node. The obtained user interaction information includes: a master control initialization node, a working node, and a master control joining node, wherein the master control initialization node is used for cluster initialization, and the IP addresses and SSH root user passwords of the three master control nodes are input; the virtual IP address and the network card name bound thereto are configured for Keepalived configuration; the working node is used to input the IP address and host name of the working node; the master control joining node is used to join an existing cluster, and the IP address of the master control initialization node is input;

[0019] The Ansible configuration file for automated operation and maintenance includes configuring the Ansible host file. For the three master nodes, the IP addresses and status of the three master nodes are updated to correctly identify and manage the master nodes in the cluster, including the IP addresses and roles of the nodes, so that Ansible knows the location and role of each master node.

[0020] The Ansible configuration file for automated operation and maintenance includes configuring the load balancing and proxy service HAProxy files, adding the new master node IP address and port information to ensure that the HAProxy load balancer can correctly distribute traffic to the three master nodes;

[0021] The master initialization node performs cluster initialization operations, including configuring a highly available control plane, starting the cluster, configuring load balancing, etc. The information obtained includes: the IP addresses of the three master nodes, the root user passwords of the servers where the three master nodes are located, and the VIP address of Keepalived and the name of the network card bound to the VIP need to be configured to achieve high availability of the master node; in subsequent operations, the initial node will be set as the first master node in the cluster, and the cluster will be initialized, and other master nodes will join this initialized cluster;

[0022] The master joins the node to the configured three-master cluster and becomes a new master node of the cluster; the information to be obtained: the IP address of the master initialization node, which is used to connect to the existing cluster; in subsequent operations, the new master node is connected to the existing cluster through the cluster join command. During the joining process, the node is configured to collaborate with other master nodes and synchronize the configuration and status of the cluster;

[0023] Worker nodes are applicable to single-master clusters and three-master clusters. They do not assume the responsibility of the control plane and are only used to run application containers. In a single-master cluster, the master node and the worker node are usually the same machine. In a three-master cluster, the node node acts as a worker node and is responsible for running business loads. Information to be obtained: the IP address of the worker node and / or the host name of the worker node (optional). Subsequent operations: the worker node joins the cluster and runs the service as a worker node. In a single-master cluster, configure the role of the master node and run it as a worker node. In a three-master cluster, configure the ordinary worker node to join the cluster and ensure that it can accept the workload.

[0024] During the cluster installation phase, for the three-master node mode, the master initialization node installs the automated management Ansible tool to deploy HAProxy load balancing and Keepalived according to the operating system type for load balancing and high availability configuration, reducing the complexity and error risk of manual operations. Through automated scripts, it can be ensured that each node is configured as expected and easy to manage and maintain; the automated management tool Ansible needs to be able to execute commands or scripts between different nodes. If you are required to enter a password every time, it will be very troublesome and not conducive to automated operations. By configuring SSH mutual trust, these operations can be implemented without a password. In order to ensure that SSH connections can be made between different hosts in a cluster environment without a password, SSH mutual trust needs to be configured; establish SSH mutual trust with the image warehouse Harbor server, and send the local SSH public key to the image warehouse H arbor server, establish SSH mutual trust between the local machine and the image repository Harbor server; among the three master nodes, establish SSH mutual trust between all master nodes. If the node working node is included, the node working node also needs to establish SSH mutual trust with all master nodes; for the master initialization node, install and configure it by installing the automated management Ansible tool. For the master joining node, obtain the certificate key for cluster authentication from the master node to ensure that the new master node can perform correct security verification, and generate a command to join the cluster on the new master node to join the cluster. Through this process, it can be ensured that the new master node is correctly added to the K8s cluster and configured to work normally. For the working node, obtain the command for the node to join the cluster from the master node, and generate a command to join the cluster on the new working node to join the cluster.

[0025] Furthermore, the cluster type includes a single-master control cluster, in which there is only one master control node for all control and management tasks of the cluster. The single-master control cluster includes a master control node and a working node. The acquired user interaction information includes:

[0026] Master node: responsible for managing the control plane of the entire cluster. Information that needs to be obtained: IP address of the master node and SSH root user password of the master node, choose whether to use the master node as a working node; in subsequent operations, configure a single master node and ensure that it can handle all control plane tasks in the cluster; configure whether the master node is used as a working node to run business loads;

[0027] Worker nodes are applicable to single-master clusters and three-master clusters. They do not assume the responsibility of the control plane and are only used to run application containers. In a single-master cluster, the master node and the worker node are usually the same machine. In a three-master cluster, the node node acts as a worker node and is responsible for running business loads. Information to be obtained: the IP address of the worker node and / or the host name of the worker node (optional). Subsequent operations: the worker node joins the cluster and runs the service as a worker node. In a single-master cluster, configure the role of the master node and run it as a worker node. In a three-master cluster, configure the ordinary worker node to join the cluster and ensure that it can accept the workload.

[0028] During the cluster installation phase, the automated management tool Ansible needs to be able to execute commands or scripts between different nodes. If you are required to enter a password each time, it will be very troublesome and not conducive to automated operations. By configuring SSH mutual trust, these operations can be implemented without a password. In order to ensure that SSH connections can be made between different hosts in a cluster environment without a password, it is necessary to configure SSH mutual trust; establish SSH mutual trust with the image warehouse Harbor server, send the local SSH public key to the image warehouse Harbor server, and establish SSH mutual trust between the local machine and the image warehouse Harbor server; for the single master node mode, it is only necessary to establish SSH mutual trust between the node working node and a single master node; for the master node, the cluster is initialized and the master node is set up based on the cluster deployment management configuration YAML file; for the working node, the command for the node to join the cluster is obtained from the master node, and a command to join the cluster is generated on the new working node to join the cluster.

[0029] Furthermore, NTP node information is obtained through host information to confirm the time source of time synchronization, because the time difference between different nodes may cause various problems. In a multi-node K8s cluster, the time of all nodes needs to be consistent. If the time difference between nodes is too large, it may cause problems in tasks scheduling, logging, data synchronization and other operations in the cluster.

[0030] The Ansible configuration file for automated operation and maintenance includes configuring the Chrony file and updating the NTP server information in the Chrony configuration file to ensure system time synchronization;

[0031] During the cluster installation phase, NTP nodes are installed to synchronize the time between all working nodes and the master node, which is crucial for the normal operation and scheduling of distributed systems. The master node is usually configured with NTP service in the early stage of cluster deployment, so it does not need to be processed separately in subsequent scripts. This ensures that the time of the entire cluster is consistent, thus avoiding various potential problems caused by time asynchrony.

[0032] Furthermore, the enabling parameter configuration is the configuration of relevant information of the optional components, including configuration information of the cluster management visualization component, the cluster monitoring visualization component, and the cluster log visualization component;

[0033] Generate corresponding configuration files through the enabling parameters of optional components (cluster management platform, monitoring visualization platform, log visualization platform), extract the IP address of the open component service and the address of the harbor image warehouse from the host configuration file, pass the IP address and the image warehouse address as parameters to the installation processing function of the optional component to generate the monitoring.yaml file for installing the cluster monitoring visualization component, the cluster_manager.yaml file for the cluster management visualization component, and the logcollect.yaml file for the cluster log visualization interface, which are used for the subsequent installation and deployment of optional components;

[0034] During the cluster installation phase, the optional-components_install.sh script is executed to install the corresponding optional components according to the user's configuration parameters. The switch for installing optional components is read from the main configuration file to decide whether to execute the installation script of the optional components. If it is turned on, after verifying the health status of the cluster and the installation check conditions of the optional components, the installation script optional-components_install.sh is executed to read the installation yaml files of the cluster management visualization component, cluster monitoring visualization component, and cluster log visualization component to complete the installation of the optional components, and finally output the installation result information.

[0035] Furthermore, the automated deployment of the optional component is to point to the code repository plug-in of the automation tool through the location identifier in the code repository, build a pipeline in the automation tool to set a series of tasks at different stages in the workflow of continuous integration and continuous delivery / deployment; the different stages include pulling the code from the code repository, building the npm project, making the image through the container docker, pushing the image to the image repository harbor, and executing the yaml file of the optional component to complete the deployment of the component;

[0036] When the automation tool detects that the code in the code repository has changed, it automatically triggers and executes the build process to configure the locator of the code repository, specify the code branch to be built, configure the build trigger (triggered when the code changes), configure the build environment, configure the build, the username and password of the image repository Harbor, configure the K8s cluster credentials, add the K8s cluster credentials in the automation tool, and execute the Kubectl command for the files of the optional components in the pipeline of the automation tool;

[0037] When changes in the code repository are monitored, a request is sent to the location identifier, and the code repository plug-in of the automation tool receives and processes the request to trigger the execution of the set task. The automation tool will pull the latest code for compilation to build a container image, and push the container image to the image repository. After the build is completed, the local container image is cleaned up to free up space, the newly built container image is used in the cluster and the corresponding image configuration is updated, and the corresponding scheduling unit of the cluster is replaced to automatically update the status of the deployment optional components.

[0038] Through the above process, the automation tool Jenkins can automatically build the Docker image of the component after detecting code changes, push it to the image repository, and update the Deployment in the cluster K8s to use the new image, thereby achieving automated deployment. The entire process ensures that when the functional code of the component changes, the Pod will automatically update the status of the deployed optional component to keep the system up to date.

[0039] The advantages and beneficial effects of the present invention are:

[0040] The DevOps-based convenient deployment method for multiple selectable components proposed in the present invention aims to realize the K8s cluster basic environment by interactively inputting user configuration parameters, and integrates the convenient deployment of cluster-related visualization components. In addition, it also adopts an automated DevOps process to realize automatic compilation, automatic image construction, and automatic deployment of optional components for user-selectable cluster visualization components (cluster management visualization platform components, cluster monitoring visualization platform components, and cluster log visualization platform components), thereby improving the efficiency of cluster management and component update deployment, and improving the delivery efficiency and quality of the software system. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a method in an embodiment of the present invention.

[0042] Figure 2 It is a convenient deployment flow chart of K8s cluster, an optional component for interactively inputting parameters in an embodiment of the present invention.

[0043] Figure 3a It is a structural diagram of a cluster node role category of a single master control cluster in an embodiment of the present invention.

[0044] Figure 3b It is a schematic diagram of the structure of a three-master cluster in which the cluster node role category is an embodiment of the present invention.

[0045] Figure 4 It is a flowchart of configuring and enabling optional component parameters in an embodiment of the present invention.

[0046] Figure 5It is a schematic diagram of the composition of the Kubeadm.yaml file in an embodiment of the present invention.

[0047] Figure 6 It is a flow chart of the installation file for updating the optional components in the embodiment of the present invention.

[0048] Figure 7 It is a flow chart of installing optional components in an embodiment of the present invention.

[0049] Figure 8 It is a flowchart for realizing optional component automation based on DevOps in an embodiment of the present invention.

[0050] Fig. 9 The embodiment of the present invention implements the automatic deployment of optional components based on DevOps.

[0051] Fig.10 It is a Jenkins pipeline task structure diagram in an embodiment of the present invention.

[0052] Fig.11 This is the automated deployment process of the cluster monitoring component in the embodiment of the present invention. DETAILED DESCRIPTION

[0053] The specific implementation of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the present invention, and is not used to limit the present invention.

[0054] A convenient deployment method for multiple optional components based on DevOps, including convenient K8s cluster deployment of optional components with interactive parameter input and automated deployment of optional components based on DevOps.

[0055] 1. Overall design Figure 1 As shown:

[0056] When installing the basic K8s environment, users can interactively input cluster configuration items, including the basic parameter configuration of the K8s cluster and the enabling parameter configuration of user-optional components related to the cluster. The convenient deployment parameter configuration and parsing function generates the yaml (a markup language for data serialization) configuration file required for cluster installation, and executes the above-generated yaml file through the kubectl apply command (Kubectl is a command-line tool for K8s that is used to interact and manage K8s clusters) to install the basic K8s environment and optional components, and realize a consistent process for convenient cluster deployment.

[0057] At the same time, it also supports developers to implement automated deployment processes for installed optional components. When the functional code of an optional component changes and the webhook (a user-defined HTTP callback mechanism that allows an application to send real-time data or notifications to another application when a specific event occurs) configured to listen for code changes will trigger the Jenkins (an automation tool for continuously building software projects, making it easier for developers to integrate changes into projects and for users to obtain new builds) pipeline task, which in turn executes a series of actions including code compilation, image building, image push, and component update deployment of visual components to achieve the function of updating and upgrading optional components. The above steps are all automated processes and do not require manual participation, which greatly frees up human resources. At the same time, it improves the efficiency of component upgrade deployment and realizes the integration of DevOps and convenient deployment.

[0058] 2. Optional components that can interactively input parameters for easy deployment of K8s clusters:

[0059] The convenient deployment based on K8s architecture and Ansible in the existing technology can realize one-click deployment of the basic cluster environment, but it still needs to manually write the host list. The K8s architecture receives the host list containing architecture parameters and sends the architecture parameters in the host list to the ansible (an automated operation and maintenance tool for configuration management, application deployment, task execution and multi-node coordination) architecture. The ansible architecture sends the shell script with K8s component information to the target host architecture. The target host architecture installs and deploys the K8s component according to the K8s component information in the shell script; and it cannot integrate the relevant visualization components of the cluster, such as the cluster management visualization interface, the cluster monitoring visualization interface, and the cluster log visualization interface.

[0060] In order to solve the above problems, the present invention provides a method that supports users to interactively input configuration parameters for cluster deployment, so as to solve the problem of manually writing cluster parameter files, and in this process, third-party components that users can choose to install (cluster management visualization platform component, cluster monitoring visualization platform component, cluster log visualization platform component) are added. The function of the third-party components is mainly parameter visualization. Otherwise, users need to manually enter cumbersome command lines on the server to view data changes, which reduces the user experience. The present invention allows users to deploy third-party components on demand during cluster installation and deployment, without the need for users to manually deploy them, thereby improving the user experience. The specific process is as follows: Figure 2 As shown:

[0061] The user inputs (the user is prompted on the page to enter the required information), and the information is written into the main configuration file cloud.yaml through the ConfigHost parameter configuration function. This file is mainly used to collect host configuration information, K8s configuration information, and optional component configuration information; the main configuration file is then parsed through the ParseConfFile parameter parsing function to write the user input information into the harbor.yam file, ansible host file, kubeadm file, K8s network and storage files, and optional component configuration files; finally, the above installation files are installed through the install.sh file through the kubectl apply command, and finally the basic K8s cluster environment and cluster management visualization components, cluster monitoring visualization components, and cluster log visualization components are integrated and conveniently deployed. Specifically, the code design process includes the following steps:

[0062] Step S1.1: Environmental pre-check

[0063] Check the system environment, including the operating system version, kernel version, configured DNS server address, etc., to ensure that they meet the installation requirements. The purpose of the pre-check is to ensure that the system environment and configuration meet the requirements for software installation and operation to reduce the risk of problems and failures during the subsequent installation process.

[0064] Step S1.2: Interactively input configuration items

[0065] The interactive input step allows users to interactively input parameter configurations according to specific needs, avoiding users from manually writing yaml files. This step can improve flexibility. Users can choose to install optional components, set network modes, enter server addresses, etc. It can also reduce the error rate of manual input. It also integrates cluster-related optional component parameter selection (cluster management visualization interface, cluster monitoring visualization interface, cluster log visualization interface). The above components can be selected and installed by users during the interactive parameter input process, providing convenient deployment of cluster integration, lowering the user's usage threshold, and greatly reducing work time.

[0066] At this stage, run the input_config.sh script, which prompts the user to enter the required information on the page in turn and writes the information to the main configuration file cloud.yaml through the ConfigHost function. The information that users need to enter includes host information IP (Internet Protocol, responsible for transmitting data packets from the source address to the destination address, and is the core protocol in the Internet and local area network), K8s related configuration information, and related information configuration of optional components, which are specifically divided into: confirm the cluster type (single master, three masters), confirm that it is master-init, node, master-join mode, enter the IP of each master node, the root user password, Keepalived (a Linux software tool for high availability and load balancing, mainly used to manage VIPs and provide high availability of services) VIP (Virtual IP, a virtual IP address, an IP address not bound to any single physical network interface) address, the name of the network card bound to the Keepalived VIP address, the IP / private_key of the harbor host, and the IP address of the NTP (Network Time Protocol, a protocol for synchronizing time in computer networks) server; K8s cluster network mode, network mode subnet segment (optional); select the components to be installed (cluster management visualization component, cluster monitoring visualization component, cluster log visualization component), IP address, and port number. Among them, Harbor is an enterprise-level Docker image repository that provides storage, management and distribution functions for Docker images, while Docker is a platform for developing, transmitting and running applications, allowing developers to package applications and all their dependencies (including libraries and configuration files) into a lightweight, portable virtualization environment called a "container".

[0067] The interactive input configuration items include host information configuration, K8s related information configuration, and enabling optional component configuration:

[0068] 1. Host information configuration

[0069] When interactively inputting configuration items, first enter the host-related configuration. The main purpose of this step is to set the network and identity information of each host node in the cluster. Use the ConfigHost function to interactively configure host-related information for use in cluster deployment. The implementation process includes:

[0070] 1) Configure cluster type: Determine the cluster type based on the user's choice. signal-master indicates a single-master cluster, and 3-masters indicates a three-master cluster. A single-master cluster means that there is only one master node in the cluster. This master node is responsible for all control and management tasks of the cluster, including scheduling, cluster status management, API services, etc. A three-master cluster means that there are three master nodes in the cluster. In this architecture, the master node is part of the control plane (Control Plane, which is the core component of K8s and is responsible for managing and scheduling all activities in the cluster) to provide high availability and fault tolerance.

[0071] 2) Configure node roles: Select the role of the host according to the cluster type (such as master-init, node, master-join), and perform corresponding configuration based on the selection.

[0072] 3) According to the selection of cluster type and node role, you can configure the detailed information of different nodes. The ConfigHost function will write this information into the corresponding configuration file. Figure 3a , Figure 3b shown.

[0073] For a single-master cluster, including a master node and a worker node, the information that the user needs to enter is:

[0074] Master: Enter the IP address of the master and the SSH root password of the master. Select the master as a worker node (optional). SSH is commonly used for tasks such as system management, file transfer, and remote command execution.

[0075] Worker node: Enter the IP address and host name of the worker node.

[0076] For a three-master cluster, including the master initialization node, master joining node, and working node, the information that users need to enter is:

[0077] Master initialization node (master-init, for cluster initialization): Enter three master nodes (master-1, master-2,

[0078] master-3) IP address and SSH root user password. Configure the virtual IP and VIP bound network card name for

[0079] Keepalived configuration.

[0080] Worker node: Enter the IP address and host name of the worker node.

[0081] Master-join node (master-join, used to join an existing cluster): Enter the IP address of the master initialization node.

[0082] 4) Configure NTP nodes: NTP ensures that all computer systems and servers are synchronized to an accurate time source. This is especially important in a cluster environment, as time differences between different nodes can cause various problems. In a multi-node K8s cluster, the time of all nodes needs to be consistent. If the time difference between nodes is too large, it may cause problems in operations such as task scheduling, logging, and data synchronization in the cluster.

[0083] By default, the NTP server is deployed on the node. If the node needs to deploy the NTP client, you need to enter the IP address of the NTP server so that the node can obtain time from the server.

[0084] Configure host-related information through interactive prompts to ensure that all necessary parameters are set correctly before cluster deployment. The function selects the cluster type and node role based on user input, and configures the corresponding IP address, SSH password, host name, and NTP settings. These configurations are critical to the normal operation and management of the cluster. Failure to perform these configurations will result in the cluster being unable to be deployed or run correctly.

[0085] 5) Deploy Harbor nodes: Users are required to enter the address of the image repository required for cluster deployment. Components involved in K8s deployment and management (such as kube-apiserver, kube-controller-manager, kube-scheduler, etc.) usually require the use of Docker images. These images are obtained from private image repositories. Here, you need to correctly configure the image repository address to ensure that the K8s cluster can correctly pull the required images, and enter the Harbor host IP and ssh root password.

[0086] 2. K8s related information configuration

[0087] Use the ConfigK8sNetWork() function to configure the network mode of the K8s cluster, including selecting a network plugin (such as Flannel or Multus) and its related settings. Allow users to select the network mode of the K8s cluster. The default selection is Multus, requiring users to enter the subnet segment under the Multus network mode. This configuration is used to avoid network conflicts and set up the network correctly.

[0088] Correct network configuration is an important part of cluster deployment. Ensuring that the network mode selection and related parameter configuration are correct is the key to ensuring the stable operation of the cluster. Failure to perform these configurations may lead to network conflicts, performance issues, or the cluster may not work properly.

[0089] 3. Configure and enable optional components

[0090] like Figure 4 As shown in the figure, the optional component parameter configuration section for users to choose to install is added to the convenient deployment process of optional components with interactive parameter input. In the interactive parameter input interface, a component selection list is provided to allow users to select optional components to be installed. These components include: cluster management platform (used to simplify cluster management), monitoring visualization platform (used to display various monitoring data of the cluster), and log visualization platform (used to process and display cluster log data). The configuration parameter processing script will write the components selected by the user and their related configuration information into the main configuration file cloud.yaml for subsequent configuration file parsing and cluster installation. This information includes the component's installation path, port number, dependencies, etc., so that subsequent deployment scripts or management tools can correctly install and configure components based on these configurations.

[0091] This approach can significantly improve the deployment efficiency, flexibility, and maintainability of third-party components in the K8s cluster, including the following advantages:

[0092] 1) Simplify the deployment process

[0093] Users can enter all necessary parameters through a unified interface and select the components to be installed from the component selection list. The parameter parsing script will automatically complete the configuration and installation of the components based on the user's selection, greatly reducing the number of manual operation steps. Through the interactive input interface, users only need to focus on the necessary parameters without having to delve into the underlying details, reducing the risk of configuration errors.

[0094] 2) Enhance user experience

[0095] An interactive parameter input interface is provided, and users can easily select the components to be installed through a graphical or command line interface, which reduces the complexity of command line input and improves the user experience.

[0096] 3) Improve deployment efficiency

[0097] Automated installation and configuration: The system automatically calls the corresponding configuration function according to the user's selection to complete the installation and configuration of the components, avoiding tedious manual operations. There is no need to repeatedly configure the same component parameters. The system will automatically optimize the configuration based on the input parameters, reducing the cost of repeated labor.

[0098] 4) Flexibility and scalability

[0099] Users can flexibly choose whether to install components such as the cluster management platform, monitoring visualization platform, and log visualization platform based on actual needs, without having to install all components by default to save resources.

[0100] 5) Lowering the technical threshold

[0101] This method lowers the technical threshold. Even for users who are not familiar with K8s, they can complete complex cluster deployment tasks through a simple interactive interface, reducing the reliance on high-level technical personnel. By automatically configuring functions to handle all underlying operations, users can complete deployment without understanding the complexity of K8s, simplifying the learning and use process.

[0102] Step S1.3: Parsing the configuration file

[0103] Based on the configuration parameters entered by the user through the interactive interface, the ParseConfFile function parses the configuration file and performs various configuration and generation tasks, and records logs and related error information based on the results. The process of parsing the configuration file includes the following steps:

[0104] Step S1.3.1: Read the default Kubeadm configuration file from the specified path and parse its contents into a list to provide basic data for subsequent processing steps;

[0105] Kubeadm is a tool provided by K8s for cluster initialization, node addition, and upgrade operations. It aims to simplify the deployment and management of K8s clusters. The Kubeadm configuration file usually contains the initialization parameters of the K8s cluster.

[0106] Step S1.3.2: Configure Kubeadm file

[0107] like Figure 5 As shown, the Kubeadm file contains initialization configuration, cluster configuration, proxy configuration, and Kubelet (a key component in K8s, responsible for managing and running Pods on each node) configuration. Pod is the most basic scheduling unit in K8s, representing a group of one or more containers that share storage, network, and operating environment.

[0108] Initialization configuration InitConfiguration is used to set the basic parameters of the K8s cluster. Extract the initialization configuration from the main configuration file and the default configuration file, and generate the corresponding Kubeadm initialization configuration file, including the cluster's control plane settings, authentication and encryption configuration, etc. These parameters are required for cluster initialization to ensure that the cluster can start normally and perform preliminary configuration.

[0109] ClusterConfiguration is used to define the network, storage and other settings of the cluster. Extract the cluster configuration and the default cluster configuration to generate the Kubeadm cluster configuration file.

[0110] Configure the proxy service KubeProxyConfiguration to ensure the correct forwarding of network traffic by configuring the Kube-proxy proxy service. Kube-proxy is responsible for handling network traffic routing within the cluster to ensure the correct operation of services and load balancing. Extract the proxy configuration and the default proxy configuration to generate the Kubeadm proxy configuration file.

[0111] Kubelet configuration KubeletConfiguration, that is, configure the Kubelet service and manage the container operation of the node. Extract the Kubelet configuration to generate the Kubelet configuration file.

[0112] Combine all the above configurations into a complete Kubeadm configuration YAML file for use when the K8s cluster is initialized.

[0113] Step S1.3.3: Parsing the main configuration file

[0114] Extract host information from the configuration file and parse its content into a list. This list contains different types of configurations, usually including the host's IP address, network settings, Docker, NTP configuration, etc. Write the above information into Docker, NTP, and Ansible files respectively.

[0115] Step S1.3.4: Process Docker configuration files

[0116] In order not to affect the management and deployment of containers, extract the Docker configuration information from the main configuration file, parse the extracted Docker configuration information, and apply these configurations to the Docker daemon process. Write the read configuration items into the daemon.json file for Docker installation.

[0117] Step S1.3.5: Configure Ansible files

[0118] In order to achieve load balancing and high availability and avoid single points of failure and performance bottlenecks, the processed host configuration information is used to generate Ansible configuration files for setting up HAProxy (High Availability Proxy, a high-availability load balancer and proxy server, mainly used to improve the performance and reliability of applications and services) and Keepalived. The cluster configuration process is automated to adapt to different cluster setting requirements by dynamically modifying the configuration files.

[0119] Configure multiple Ansible-related files to prepare for cluster management, load balancing, VIP configuration, time synchronization, and global variable settings. Specifically, the following operations are performed based on the code:

[0120] 1. Configure Ansible hosts file

[0121] Update the IP addresses and status of the three master nodes in the Ansible host file to ensure that Ansible can correctly identify and manage the master nodes in the cluster, including the IP addresses and roles of the nodes. This allows Ansible to understand the location and role of each master node.

[0122] 2. Configure HAProxy file

[0123] Remove the existing server entries from the HAProxy configuration file and add the new master node IP address and port information. This ensures that HAProxy's load balancer can correctly distribute traffic to the three master nodes.

[0124] 3. Configure Keepalived file

[0125] Update the Keepalived configuration file variables to set the virtual IP address and network interface. This ensures that Keepalived can correctly manage the allocation of virtual IP addresses for high availability.

[0126] 4. Configure Chrony files

[0127] Update the NTP server information in the Chrony configuration file to ensure system time synchronization.

[0128] 5. Configure global variables

[0129] Get the IP address of Harbor. Read and check whether the Master node is also set as a working node. Update the global variable configuration file, including the HarborIP and the working status of the Master node, and regenerate the global variable configuration file.

[0130] Step S1.3.6: Handle K8s network storage configuration

[0131] The configuration parameters of the K8s network and the default storage method are written to the cluster-config yaml file. Such as Flannel network plug-in, Rook-Ceph storage. These components provide additional features, such as network plug-ins and storage solutions. Proper configuration of them can extend the functionality of the K8s cluster. Otherwise, the cluster may lack necessary functions or services, affecting the integrity and functionality of the system.

[0132] Step S1.3.7: Process Harbor configuration

[0133] Extract the host IP address of the Harbor container image repository from the main configuration file, and update the Hostname in the configuration dictionary to the specified host IP address, so that Harbor is configured with the correct IP address. Generate the Harbor.yaml configuration file for the image repository settings. Otherwise, the storage and management of container images will be affected, which may make the images unavailable or difficult to manage.

[0134] Step S1.3.8: Get node type and cluster mode

[0135] Extract configuration information: Extract K8s node type, whether it is a working node, and cluster mode from the main configuration file cloud.yaml. The obtained node types can be divided into: master initialization node (master-init), ordinary node (node), master joining node (master-join), and master node (master).

[0136] 1. Master initialization node, only applicable to three-master cluster:

[0137] This node will perform cluster initialization operations, including configuring a highly available control plane, starting the cluster, configuring load balancing, etc. Information that needs to be obtained: the IP addresses of the three master nodes, the root user passwords of the servers where the three master nodes are located, and the VIP address of Keepalived and the name of the network card bound to the VIP need to be configured to achieve high availability of the master node. Subsequent operations: The initial node will be set as the first master node in the cluster, and the cluster will be initialized. Other master nodes will join this initialized cluster.

[0138] 2. The master joins the node, which is only applicable to a three-master cluster:

[0139] The node will be added to the configured three-master cluster and become a new master node in the cluster. Information to be obtained: IP address of the master initialization node, used to connect to the existing cluster. Subsequent operations: Connect the new master node to the existing cluster through the cluster join command. During the joining process, configure the node to collaborate with other master nodes and synchronize the cluster configuration and status.

[0140] 3. The master node is the only master node in a single master cluster:

[0141] This node is responsible for managing the control plane of the entire cluster. Information to be obtained: IP address of the master node, root user password of the server where the master node is located, and whether to use the master node as a working node. Subsequent operations: Configure a single master node and ensure that it can handle all control plane tasks in the cluster; configure whether the master node is used as a working node to run business loads.

[0142] 4. Worker nodes, applicable to single-master clusters and three-master clusters:

[0143] Node is an ordinary worker node that does not assume the responsibilities of the control plane and is only used to run the application's containers. In a single-master cluster, the master node and the worker node are usually the same machine. In a three-master cluster, the node node serves as a worker node and is responsible for running business workloads. Information to be obtained: the IP address of the worker node and the host name of the worker node (optional). Subsequent operations: The worker node joins the cluster and runs the service as a worker node. In a single-master cluster, the role of the master node is usually configured and it runs as a worker node. In a three-master cluster, configure the ordinary worker node to join the cluster and ensure that it can accept the workload.

[0144] The above steps are to ensure that each node in the cluster can be correctly configured and work normally, and provide a stable control plane and business load support for the cluster.

[0145] Step S1.4: Process optional component configuration

[0146] Generate a configuration file based on the enabling parameters of the multiple-selectable components (cluster management platform, monitoring visualization platform, log visualization platform) entered by the user, such as Figure 6 As shown, the process includes the following:

[0147] First, according to the user's selection, the requirements of the components to be installed are determined, and the installation components are confirmed;

[0148] Then, extract the IP address of the open component service and the address of the harbor warehouse from the host configuration file cloud.yaml entered by the user;

[0149] Finally, the IP and image repository address are passed as parameters to the installation processing function of the optional component to generate monitoring.yaml for installing the cluster monitoring visualization component, cluster_manager.yaml for the cluster management visualization component, and logcollect.yaml for the cluster log visualization interface, which are used for the subsequent installation and deployment of optional components.

[0150] Step S1.5: Install the basic environment, that is, install the K8s environment, including the following steps:

[0151] Step S1.5.1: Network connection check

[0152] Check the network connection: Call the CheckNetworkConnect function to check the network connection of the specified Harbor IP address and the IP of the master node to ensure that there are no problems with network communication. This is an important step to ensure cluster stability and normal service operation.

[0153] Step S1.5.2: Configure SSH mutual trust

[0154] In a cluster environment, the automated management tool Ansible needs to be able to execute commands or scripts between different nodes. If you are required to enter a password every time, it will be very troublesome and not conducive to automated operations. By configuring SSH mutual trust, these operations can be implemented without a password. In order to ensure that SSH connections can be made between different hosts in a cluster environment without a password, you need to configure SSH mutual trust.

[0155] Establish SSH mutual trust with the Harbor server: Send the local SSH public key to the Harbor server to establish SSH mutual trust between the local machine and the Harbor server.

[0156] Three master nodes mode: SSH mutual trust is established between all master nodes. If node nodes are included, the node nodes also need to establish SSH mutual trust with all master nodes.

[0157] Single master node mode: You only need to establish SSH mutual trust between the node and a single master node.

[0158] Step S1.5.3: Install NTP

[0159] To ensure the time synchronization between all working nodes and the master node is essential for the normal operation and scheduling of the distributed system. The master node is usually configured with NTP service in the early stage of cluster deployment, so it does not need to be processed separately in subsequent scripts. This ensures that the time of the entire cluster is consistent, thus avoiding various potential problems caused by time asynchrony.

[0160] Step S1.6: Cluster and component installation

[0161] Step S1.6.1: Install Harbor

[0162] Both the master-init node in the three-master mode and the master node in the single-master mode need to install Harbor, which is an enterprise-level image repository for managing Docker images to support the storage and management of container images.

[0163] Step S1.6.2: Install Docker

[0164] For the node, in order to provide the container runtime environment and ensure that the container can start and run normally, you need to install Docker. Execute the Docker installation script and command to install the Docker engine and related components.

[0165] Install Ansible and deploy HAProxy / Keepalived (three-master mode only)

[0166] In the three-master mode and master-init node, install the Ansible tool according to the operating system type; use Ansible to deploy HAProxy and Keepalived, which are used for load balancing and high availability configuration. Using Ansible in a cluster environment can automate the deployment and configuration of various components (such as HAProxy and Keepalived), reducing the complexity and error risks of manual operations. Through automated scripts, you can ensure that each node is configured as expected and is easy to manage and maintain.

[0167] In a K8s cluster, HAProxy is usually used to load balance requests from different master nodes to ensure even distribution of requests and improve the overall performance and reliability of the system. Keepalived is usually used to manage the high availability of master nodes to ensure that when a master node fails, the system can automatically switch to other healthy nodes to ensure the continuous operation of the cluster and high availability of services.

[0168] Step S1.6.3: Install K8s

[0169] The K8s cluster needs to be installed on each node. Here, the corresponding components need to be installed and configured in the K8s cluster according to different node types (master-init, master-join, node).

[0170] 1. Three-master mode master-init node initialization K8s

[0171] The three-master mode cluster initialization mainly uses Ansible to perform installation and configuration.

[0172] Prepare the environment: Execute the prepare.yaml Ansible script to prepare the installation environment for the Kubernetes cluster. This step usually includes installing necessary software packages, configuring system parameters, and closing the swap area.

[0173] Establish a cluster: Execute the Ansible script build_cluster.yaml and use Kubeadm to initialize the K8s cluster. This step will call the Kubeadminit command to initialize the k8s cluster and set the cluster parameters according to the content in the configuration file.

[0174] Complete the subsequent configuration: Execute the Ansible script post_cluster.yaml to complete the subsequent configuration of the Kubernetes cluster, including configuring Kubectl permissions, auto-completion, and using the master node as a working node.

[0175] 2. Master-join nodes in three-master mode join the cluster

[0176] After confirming that the cluster environment is normal, the node obtains the certificate key for cluster authentication from the master node to ensure that the new master node can perform correct security verification, and generates a command to join the cluster on the new master node to join the cluster. Through this process, it can be ensured that the new master node is correctly added to the K8s cluster and configured to work normally.

[0177] 3. Initialize K8s on the master node in single master control mode

[0178] Preparation before installing k8s: Prepare the packages required to install K8s according to the operating system type and architecture, disable the swap partition, and ensure that the key component Kubeadm is available.

[0179] Use Kubeadm to initialize the K8s cluster: Execute the kubeaminit command according to the previously configured k8s configuration file to initialize the cluster. Finally, configure the relevant settings of the master node.

[0180] 4. Worker nodes join the cluster

[0181] After confirming that the cluster environment is normal, the node obtains the command for the node to join the cluster from the master node, generates a command to join the cluster on the new working node, and then joins the cluster.

[0182] Step 1.7: Create a network in the K8s cluster

[0183] The network needs to be created only for the initialized K8s cluster. Get the network type parameter (flannel or multus) from the main configuration file entered by the user to determine the type of K8s network to be created.

[0184] Step 1.8: Install storage

[0185] Storage services are usually used to provide persistent storage solutions to support the storage needs of applications and services in the K8s cluster. By automatically executing the storage_install.sh script, the process does not require manual intervention, which simplifies the complexity of cluster deployment and configuration and improves efficiency and consistency.

[0186] Step 1.9: Install optional components

[0187] like Figure 7 As shown in the figure, according to the user's configuration parameters, the optional-components_install.sh script is executed to install the corresponding optional components. The switch for installing optional components is read from the main configuration file to determine whether to execute the installation script of the optional components. If the switch is true, after verifying the health status of the cluster and the installation check conditions of the optional components, the installation script optional-components_install.sh is executed to read the installation yaml files of the cluster management visualization component, cluster monitoring visualization component, and cluster log visualization component to complete the installation of the optional components, and finally output the installation result information.

[0188] On the other hand, based on the third-party components that users choose to deploy as mentioned above, automated deployment based on DevOps optional components is performed.

[0189] Currently, when developers or operation and maintenance personnel need to upgrade or update deployed optional components, they often need to manually compile code, build images, push and pull images, and manually update the YAML files of optional components. The above process relies on manual execution and is prone to operational errors. In addition, since each step needs to be performed manually, the entire update process takes a long time and lacks the support of automated tools, making it impossible to achieve continuous integration and delivery, which limits development efficiency and delivery frequency.

[0190] Therefore, in order to solve the above problems, the present invention uses Jenkins to automatically compile the components selected by the user for deployment and deploy them to the K8s cluster to solve the automation problem. The specific process is as follows: Figure 8 As shown. This method can automatically execute the entire code compilation, image building, and component deployment process, thereby eliminating manual intervention and reducing human errors. Developers only need to submit the code, and the system will automatically trigger subsequent build and deployment operations to achieve continuous delivery and ensure that the application can be quickly iterated. Through automated deployment, CICD can quickly feedback the build results after the code is submitted. Developers can obtain deployment feedback information in a short time, discover and fix problems in time, thereby shortening the development cycle. CI stands for continuous integration and CD stands for continuous delivery / deployment. The CICD process ensures that each deployment goes through the same automated steps, and there will be no inconsistencies in the environment configuration or deployment steps due to manual operations, reducing the probability of errors.

[0191] The main steps of the above process are as follows: users complete the installation and deployment of optional components through interactive parameter input. When developers need to modify the functional code of optional components to upgrade and update optional components, they first push the updated code to the code repository Gitlab (a web-based Git repository management tool). The pre-configured Gitlab webhook will automatically trigger the pipeline task (pre-written) of the automation tool to complete the automatic compilation and image building process of the optional component code, and automatically push the image to the harbor image repository. Finally, the pod image update of the optional component is executed through the automation tool to complete the automatic deployment of the optional component. The detailed process for realizing automatic deployment of optional components is as follows: Fig. 9 As shown, the following steps are included:

[0192] Step 2.1: Environment preparation

[0193] 1. Configure GitLab Webhook

[0194] The automation of code building is mainly achieved by Jenkins, so you need to add a Webhook URL to point to the GitLab plug-in of Jenkins in the settings of the GitLab project.

[0195] 2. Configure Jenkins to build the pipeline

[0196] Create a new Pipeline Job in Jenkins and configure the build and deployment process. The pipeline is used to define and automate a series of tasks or steps in the continuous integration (CI) and continuous delivery / deployment (CD) workflows to ensure a smooth transition of code from development to production environments. The pipeline stage can be divided into: pulling git repository code, building npm projects, making images through docker, pushing images to harbor, and executing component yaml files to complete component deployment. Fig.10 shown.

[0197] When Jenkins detects changes in GitLab code, it will automatically trigger and execute the build process, including configuring the URL of the GitLab code repository, specifying the build branch, configuring the build trigger (triggered when the code changes), configuring the build environment, configuring the build, Harbor username and password, configuring K8s credentials, and adding K8s credentials in Jenkins to execute Kubectl commands in Jenkins Pipeline.

[0198] Step 2.2: Trigger the build

[0199] GitLab listens to changes in the code repository (code submission) and sends a POST request to the configured Webhook URL. The GitLab plugin of Jenkins receives and processes this Webhook request, and then starts executing the corresponding build task according to the configured trigger.

[0200] After receiving the notification, Jenkins will start executing the build task according to the configured Pipeline script to complete the build project, pull the latest code and Dockfile from GitLab, and run the build script (compile code and generate binary files).

[0201] Step 2.3: Push the image

[0202] Jenkins builds the Docker image and adds the build number as the image tag. It uses the Docker push command to push the image to the Harbor image repository. After the build is completed, it cleans up the local Docker image to free up space.

[0203] Step 2.4: Update optional components

[0204] After the build is completed and the new image is pushed, Jenkins updates the image information in the Deployment configuration in K8s (an object in K8s used to manage applications, mainly responsible for application deployment, upgrades, rollbacks, and expansions), uses the newly built Docker image in the above process, and updates the Deployment image configuration through the kubectl set image command to replace the old Pod with the new Pod.

[0205] Through the above process, Jenkins can automatically build the Docker image of the component after detecting the code change, push it to the image repository, and update the Deployment in K8s to use the new image, thereby achieving automated deployment. The entire process ensures that when the functional code of the component changes, the Pod will automatically update the status of the deployment optional component to keep the system up to date.

[0206] In one embodiment of the present invention:

[0207] Take the function code change of the cluster monitoring visualization component as an example to realize the automation process of the component. After the user interactively enters the parameters, he confirms that the optional component for cluster monitoring has been installed. This component can facilitate the user to observe the changes of various cluster resources very intuitively on the client. At present, the component developers need to modify its function code and add a resource monitoring page for rail transit business. However, the component has been running normally in the cluster environment. Now the component is automatically compiled, built, and deployed. The specific process is as follows: Fig.11 shown.

[0208] 1. Code submission (GitLab)

[0209] Developers modify code: The functional code developer of the cluster monitoring visualization component completes code modification locally and adds resource monitoring page functions for rail transit services.

[0210] Code submission to GitLab: Developers use the git push command to push the modified component function code to the branch in the GitLab repository.

[0211] 2. Continuous integration and automatic build (Jenkins)

[0212] Trigger Jenkins build: When GitLab receives code submission, it triggers Jenkins to automatically build through webhook. Jenkins will perform the following steps according to the pipeline configuration:

[0213] Pull code: Use the git pull command to pull the latest monitoring component code from the GitLab repository;

[0214] Compile code: Use the npm build command to compile the front-end code of the monitoring component;

[0215] Build image: Use the docker build command to create a new image;

[0216] Push the image to Harbor: Use the docker push command to push the new image to the Harbor image repository. Usually, different image versions are distinguished by version numbers or tags. At this time, the new image of the component containing the rail transit business monitoring page has been stored in the Harbor repository.

[0217] 3. Deploy to K8s cluster

[0218] Jenkins uses kubectl apply yaml files to monitor cluster deployment of image components. K8s will perform rolling updates, gradually replacing running old version components with components containing new functions to ensure uninterrupted services.

[0219] 4. Functional Verification

[0220] After deployment is complete, developers can access the service address of the monitoring component to ensure that the new function has been successfully deployed and running. By accessing the rail transit business resource monitoring page, verify whether it is working properly and test its functions.

[0221] Based on all the above, the present invention proposes a convenient deployment method for multiple optional components based on DevOps. In order to simplify the deployment process of K8s and its components, a method for interactive input parameter configuration is provided, which includes parameter configuration of user-optional components that meet different user needs, and for the problem of manually compiling code for user-optional components to update and upgrade, a DevOps process of automated code compilation, image building, and deployment is adopted to provide an efficient and reliable method. The main advantages of this method include:

[0222] 1. Simplify configuration and operation

[0223] By interactively inputting configuration parameters and automatically generating installation YAML files, the complexity of manual modification and configuration is significantly reduced, deployment errors are reduced, and deployment efficiency and accuracy are improved.

[0224] 2. Integrated human-computer interaction interface is an optional component

[0225] In the process of interactive parameter input configuration, the cluster management platform (used to simplify cluster management), monitoring visualization platform (used to display various monitoring data of the cluster), and log visualization platform (used to process and display cluster log data) are integrated as optional components for users to choose. Users only need to enter the corresponding parameters for installing optional components, and the parameter configuration parsing script can automatically parse and install the optional components selected by the user, reducing the user's self-installation, environment configuration, image pulling and other processes, greatly improving the user experience.

[0226] It allows users to flexibly select and deploy K8s components according to actual needs, avoiding resource waste, meeting personalized needs, and improving the scalability and flexibility of the system.

[0227] 3. Integrate DevOps to automate optional components

[0228] For component upgrades, the components that users choose to install need to be updated. Developers need to manually update the component images, and then manually install and deploy new components. This process not only consumes a lot of manpower and material resources, but may also lead to problems such as data loss. Combined with Jenkins automated code compilation, image building and deployment processes, continuous integration and continuous delivery are achieved, manual operations and configurations are reduced, and system delivery efficiency and quality are improved.

[0229] 4. Improve user experience

[0230] It provides a unified interface for interactive parameter input, allowing users to configure and operate more intuitively, lowering the technical threshold for users and optimizing the user experience.

[0231] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A convenient deployment method based on DevOps with multiple optional components, characterized by The details are as follows: Interactively obtain cluster configuration items from users, including basic cluster parameter configuration and enabled parameter configuration of optional cluster components, to generate a main configuration file, parse the main configuration file to obtain the installation file for cluster deployment, execute the installation file, and conveniently deploy the cluster environment and optional components in an integrated manner; For developers' modifications to optional components, a trigger mechanism is configured for the code repository. Developers push updated code to the code repository, triggering the pipeline tasks of the automation tool to automatically compile and build images of the optional component codes, and automatically push the images to the image repository. The image updates of the optional components are executed through the automation tool to complete the automated deployment of the optional components.

2. The DevOps-based convenient deployment method with multiple selectable components according to claim 1, characterized in that: The basic parameter configuration includes host information and cluster configuration information. The host information is used to confirm the cluster type, the role of the cluster node, and the image warehouse node information. The master node and the working node information are confirmed based on the cluster type and the node role. Based on the image warehouse node information, it is confirmed that the cluster obtains the corresponding container image. The cluster configuration information is used to confirm the cluster's network mode for cluster network settings; Parse the main configuration file, extract configuration information related to the container, and use the read configuration items for installing the container; Generate an automated operation and maintenance configuration file based on the host information, which is used to set up load balancing and proxy services, as well as the Keepalived file for configuring and managing virtual address allocation; process the cluster network storage configuration based on the cluster configuration information; extract the host address of the container's image repository and update it to the specified host address; extract the cluster type and the role of the cluster node to configure each cluster node.

3. The DevOps-based convenient deployment method with multiple selectable components according to claim 2, characterized in that: The installation file extracts the initialization configuration, cluster configuration, proxy configuration, node management and operation configuration of the cluster deployment management configuration file based on the parsed main configuration file and the default configuration file. The node is the most basic scheduling unit of the cluster, representing a group of one or more containers, and generates a corresponding cluster deployment management configuration file for cluster initialization; In the cluster installation phase, based on the cluster deployment management configuration file, cluster initialization, installation of the image repository and the container, creation of a cluster network and cluster storage are performed respectively, and corresponding components are installed and configured on each node according to the cluster type and the role of the cluster node.

4. The DevOps-based convenient deployment method with multiple selectable components according to claim 3, characterized in that: The initialization configuration is used to set the basic parameters of the cluster, extract the initialization configuration from the main configuration file and the default configuration file, and generate the corresponding initialization configuration file for cluster deployment management to ensure that the cluster can be started normally and perform preliminary configuration; The cluster configuration is used to define the network and storage settings of the cluster, extract the cluster configuration and the default cluster configuration to generate a cluster configuration file for cluster deployment management; The proxy configuration is to configure the proxy service to process the network traffic routing within the cluster to ensure the correct forwarding of network traffic, the correct operation of services and load balancing, extract the proxy configuration and the default proxy configuration, and generate a proxy configuration file for cluster deployment management; The node management and operation configuration is used to manage the container operation of the node, and extract the node management and operation configuration to generate a node management and operation configuration file.

5. The DevOps-based convenient deployment method with multiple selectable components according to claim 2, characterized in that: The automated operation and maintenance configuration file includes configuring global variables, obtaining the address of the image repository, reading and checking whether the master node is also set as a working node, updating the global variable configuration file, including the image repository address and the working status of the master node, and regenerating the global variable configuration file; During the cluster installation phase, the network connection between the specified address and the address of the master node is checked.

6. The DevOps-based convenient deployment method with multiple selectable components according to claim 2, characterized in that: The cluster type includes a three-master control cluster, in which there are three master control nodes. The three-master control cluster includes a control initialization node, a master control joining node, and a working node. The obtained user interaction information includes: a master control initialization node, a working node, and a master control joining node. The master control initialization node is used for cluster initialization, and the addresses and user passwords of the three master control nodes are input; the virtual address and the network card name bound thereto are configured for Keepalived configuration; the working node is used to input the address and host name of the working node; the master control joining node is used to join an existing cluster, and the address of the master control initialization node is input; The automated operation and maintenance configuration file includes configuring the host file. For the three master nodes, the IP addresses and status of the three master nodes are updated to correctly identify and manage the master nodes in the cluster, including the IP addresses and roles of the nodes. The automated operation and maintenance configuration file includes configuring the load balancing and proxy service files, adding the new master node IP address and port information to ensure that the load balancer can correctly distribute traffic to the three master nodes; The master initialization node performs cluster initialization operations and obtains information including: the IP addresses of the three master nodes, the root user passwords of the servers where the three master nodes are located, and the VIP address of Keepalived and the name of the network card bound to the VIP need to be configured to achieve high availability of the master node; in subsequent operations, the initial node will be set as the first master node in the cluster, and the cluster will be initialized, and other master nodes will join this initialized cluster; The master joins the node to the configured three-master cluster and becomes a new master node of the cluster; the information to be obtained: the IP address of the master initialization node, which is used to connect to the existing cluster; in subsequent operations, the new master node is connected to the existing cluster through the cluster join command. During the joining process, the node is configured to collaborate with other master nodes and synchronize the configuration and status of the cluster; Worker nodes are only used to run application containers; Information to be obtained: Worker node address and / or worker node host name; Subsequent operations: Worker nodes join the cluster and run services as worker nodes; During the cluster installation phase, for the three-master node mode, the master initialization node installs the automated management tool to deploy load balancing and Keepalived according to the operating system type for load balancing and high availability configuration; the automated management tool executes commands or scripts between different nodes, establishes mutual trust with the image warehouse server, sends the local public key to the image warehouse server, and establishes mutual trust between the local machine and the image warehouse server; among the three master nodes, mutual trust is established between all master nodes. If working nodes are included, the working nodes also need to establish mutual trust with all master nodes; for the master initialization node, it is installed and configured by installing the automated management tool. For the master joining node, the certificate key for cluster authentication is obtained from the master node to ensure that the new master node can perform correct security verification, and a command to join the cluster is generated on the new master node to join the cluster. For the working node, the command for node joining the cluster is obtained from the master node, and a command to join the cluster is generated on the new working node to join the cluster.

7. The DevOps-based convenient deployment method with multiple selectable components according to claim 2, characterized in that: The cluster type includes a single-master control cluster, in which there is only one master control node for all control and management tasks of the cluster. The single-master control cluster includes a master control node and a working node. The acquired user interaction information includes: Master node: responsible for managing the control plane of the entire cluster. Information that needs to be obtained: IP address of the master node and SSH root user password of the master node, choose whether to use the master node as a working node; in subsequent operations, configure a single master node and ensure that it can handle all control plane tasks in the cluster; configure whether the master node is used as a working node to run business loads; Worker nodes are only used to run application containers; Information to be obtained: IP address of the worker node and / or host name of the worker node; Subsequent operations: The worker node joins the cluster and runs the service as a worker node; During the cluster installation phase, the automated management tool executes commands or scripts between different nodes, establishes mutual trust with the image warehouse server, sends the local machine's public key to the image warehouse server, and establishes mutual trust between the local machine and the image warehouse server; for the single master node mode, it is only necessary to establish mutual trust between the working node and the single master node; for the master node, the cluster is initialized and the master node is set up based on the cluster deployment management configuration file; for the working node, the command for the node to join the cluster is obtained from the master node, and a command to join the cluster is generated on the new working node to join the cluster.

8. The DevOps-based convenient deployment method with multiple selectable components according to claim 2, characterized in that: Obtain NTP node information through host information to confirm the time source for time synchronization; The automated operation and maintenance configuration file includes configuring the Chrony file and updating the NTP server information in the Chrony configuration file; During the cluster installation phase, install the NTP node for time synchronization between all working nodes and the master node.

9. The DevOps-based convenient deployment method with multiple selectable components according to claim 1, characterized in that: The enabling parameter configuration is the configuration of the relevant information of the optional components, including the configuration information of the cluster management visualization component, the cluster monitoring visualization component, and the cluster log visualization component; Generate the corresponding configuration file through the enabling parameters of the optional component, extract the address of the open component service and the address of the image repository from the host configuration file, and pass them to the installation processing function of the optional component to generate files for installing the cluster monitoring visualization component, the cluster management visualization component, and the cluster log visualization interface, which are used for the subsequent installation and deployment of the optional component; During the cluster installation phase, the switch for installing optional components is read from the main configuration file. If it is turned on, the installation is performed after verifying the cluster status and the installation check conditions of the optional components, and the files of the cluster management visualization component, cluster monitoring visualization component, and cluster log visualization component are read.

10. The DevOps-based convenient deployment method with multiple selectable components according to claim 1, characterized in that: The automated deployment of the optional components is to point to the code repository plug-in of the automation tool through the location identifier in the code repository, and to build a pipeline in the automation tool to set a series of tasks at different stages; The different stages include pulling code from the code repository, building the project, creating an image through a container, pushing the image to the image repository, and executing files of optional components; When changes in the code repository are monitored, a request is sent to the location identifier, and the code repository plug-in of the automation tool receives and processes the request to trigger the execution of the set task. The automation tool will pull the latest code for compilation to build a container image, and push the container image to the image repository. The newly built container image is used in the cluster and the corresponding image configuration is updated, and the corresponding scheduling unit of the cluster is replaced to automatically update the status of the deployment of optional components.

Images