Continuous integration and deployment method, system and equipment based on multiple regions and multiple clusters

By deploying Kubernetes clusters in each computer room and performing pipeline tasks in container groups, the poor scalability of traditional Jenkins components and the bottleneck of cross-computer room networks are solved, efficient multi-region continuous integration and deployment are achieved, and resource utilization is improved. Network security.

CN118945064APending Publication Date: 2024-11-12CHINA ASEAN INFORMATION PORT CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411065333.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional Jenkins component continuous integration and continuous deployment systems have problems such as poor scalability, cross-computer room network bottlenecks and low resource utilization efficiency, especially when dealing with multilingual and multi-version code compilation tasks and cross-computer room deployment.

Method used

Adopting a continuous integration and deployment method based on multi-region and multi-clusters, by deploying Kubernetes clusters in each computer room, editing pipeline configurations to select Kubernetes clusters that perform tasks, and performing pipeline tasks in container groups, including code pulling and static code. Scan, compile, mirror build and deploy, launch and destroy container groups on demand to optimize resource utilization.

Benefits of technology

It improves the scalability and resource utilization of the system, reduces cross-computer room traffic, improves network security and overall performance, and ensures efficient deployment and stability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945064A_ABST
    Figure CN118945064A_ABST
Patent Text Reader

Abstract

The invention discloses a continuous integration and deployment method based on multiple regions and multiple clusters, and relates to the Internet technology, and the method comprises the steps: deploying a Kubernetes cluster in each machine room; the flow line configuration is edited, so that the Kubernetes cluster for executing the task is selected from all the Kubernetes clusters; the method comprises the following steps: establishing a connection between a Jenkins component and a selected Kubernetes cluster, and starting a container group in the selected Kubernetes cluster; and executing the defined assembly line task in the container group. The invention further discloses a continuous integration and deployment system based on multiple regions and multiple clusters and continuous integration and deployment equipment based on multiple regions and multiple clusters. According to the invention, the problems of poor expansibility, large cross-machine-room network overhead, insufficient security, low resource utilization rate and the like in the traditional CI / CD system are solved, and the flexibility and efficiency of the system are also remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to Internet technology, and more specifically, to a method, system and device for continuous integration and deployment based on multiple regions and multiple clusters. Background Art

[0002] With the continuous development of information technology and the rapid changes in business needs, enterprises have higher and higher requirements for business agility and market response speed, which has promoted the transformation of technical architecture from traditional monolithic architecture to cloud native architecture. Cloud native fully utilizes the advantages of cloud computing and realizes high availability, scalability and elasticity of software systems through technical means such as containerization, microservice architecture, orchestration and automation.

[0003] Continuous Integration (CI) and Continuous Deployment (CD), as the core of cloud-native technology, automate a series of software development, testing and deployment processes, and greatly improve software development efficiency and delivery quality by seamlessly connecting various links. Continuous integration means frequently integrating code into the trunk and verifying the correctness of the integration through automated testing. Continuous deployment is to automatically deploy code to the production environment based on continuous integration, so that new features or fixes can be quickly delivered to users. This automated process not only improves development efficiency, but also reduces the risk of human operation and improves the reliability and stability of the software.

[0004] Jenkins is a Java-based, open source automation server that has been widely used in the field of continuous integration and continuous deployment. Its main advantages include: rich software ecology, easy configuration and use, extensive integration capabilities, flexible architecture, and automated management. Although Jenkins has many advantages, with the increasing complexity of Internet business, the current traditional Jenkins continuous integration and continuous deployment system has also exposed some shortcomings, mainly reflected in:

[0005] 1. Poor scalability, especially high difficulty in vertical expansion: In the traditional Jenkins component environment, code compilation tasks that support multiple programming languages ​​and versions require the installation of multiple development tools and environments on the working node at the same time. This operation is not only complex and error-prone, but also requires dealing with compatibility issues between different programming languages ​​and versions. Supporting multiple languages ​​and versions requires the development team to invest a lot of time and energy to manage and maintain the development environment on the working node.

[0006] 2. Network bottleneck of centralized CI / CD system: When deploying applications, centralized CI / CD systems often need to transfer the built application software across computer rooms to different target hosts. This approach will face huge network overhead problems. When the host and CI / CD system are in different computer rooms, in order to deploy the application software to multiple hosts, it is usually necessary to transfer the same application software across computer rooms multiple times. This approach not only increases the time and cost of network transmission, but may also cause network congestion and transmission failure, affecting deployment efficiency.

[0007] These problems put forward new requirements for CI / CD systems, which need to further improve the scalability, security and resource utilization efficiency of the system while ensuring efficient work. In order to solve these problems, we proposed a continuous integration and deployment method, system and equipment based on multi-region and multi-cluster. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a continuous integration and deployment method, system and device based on multi-region and multi-cluster in view of the shortcomings of the existing technology, which effectively increases the scalability of the system, reduces the cross-computer room traffic and improves resource utilization.

[0009] The present invention provides a multi-region, multi-cluster based continuous integration and deployment method, which comprises:

[0010] A Kubernetes cluster is deployed in each computer room;

[0011] Edit the pipeline configuration to select the Kubernetes cluster to execute the task from among all the mentioned Kubernetes clusters;

[0012] Establishing a connection between the Jenkins component and the selected Kubernetes cluster and starting a container group in the selected Kubernetes cluster;

[0013] The defined pipeline tasks are executed in the container group.

[0014] As a further improvement, the step of executing the defined pipeline task in the container group includes:

[0015] Step 1: Execute the git command inside the container group to pull the latest code from the defined code repository into the container group;

[0016] Step 2: Performing static code scanning on the code to detect potential coding errors or security vulnerabilities;

[0017] Step 3: Execute a code compilation command in the compilation container of the container group according to the code language of the code to obtain a task and a task path; upload the task to the Jenkins component for archiving according to the task path;

[0018] Step 4: Build an image for the task according to the Dockerfile file in the code repository, and upload the built Docker image to the image repository;

[0019] Step 5: Execute the image publishing instruction to replace the image address of the selected Kubernetes cluster with the image address of the Docker image in the image warehouse;

[0020] Step 6: The execution host issues instructions to send the task to the target host, and restart the service after executing the task on the target host.

[0021] As a further improvement, after the pipeline task is executed, the Jenkins component destroys the used container group.

[0022] Furthermore, a deployment pipeline editing interface is initialized, and the pipeline configuration is edited on the pipeline editing interface.

[0023] Furthermore, the pipeline editing interface displays the task status according to the callback information of the target host.

[0024] As a further improvement, during the process of starting the container group, the container group is placed in a namespace set in the Kubernetes cluster, and the Kubernetes cluster allocates a set network segment to the container group through a network plug-in.

[0025] Furthermore, the network segment is set to not perform network address translation, and the network segment is restricted by the network settings of the Kubernetes cluster and the upper switch.

[0026] For further improvement, the deployed Kubernetes cluster is a newly created Kubernetes cluster or an existing Kubernetes cluster.

[0027] A system using the multi-region, multi-cluster based continuous integration and deployment method comprises:

[0028] The pipeline editing section is used to provide an interface for configuring the parameters of the pipeline task;

[0029] A configuration manager, configured to receive a request for editing a section of the pipeline, convert the parameters into a configuration file that can be recognized by a Jenkins component, and then send the configuration file to the Jenkins component;

[0030] A Jenkins component for responding to the triggering action of the pipeline according to the configuration file;

[0031] A Kubernetes cluster, used to start a container group therein according to the instructions of the Jenkins component in response to the trigger action to execute the pipeline task;

[0032] The upper-layer switches and distributed storage infrastructure are used to provide network connections and data storage services for the system.

[0033] A multi-region, multi-cluster continuous integration and deployment device, including:

[0034] A memory for storing a computer program for executing the multi-region, multi-cluster based continuous integration and deployment method;

[0035] A processor, configured to retrieve and execute the computer program;

[0036] Network devices are used to provide network connections for the processor to communicate with external hosts.

[0037] Beneficial Effects

[0038] The advantages of the present invention are:

[0039] 1. Scalability:

[0040] (1) Vertical expansion: Using image containers, and using container technologies such as Docker, build environments of different programming languages ​​and versions can be packaged into independent images. Building tasks in containers avoids compatibility issues between different languages ​​and versions. Each task runs in an independent environment to ensure consistency and reliability. The containerized build environment can be updated and replaced at any time without affecting other tasks, which improves the flexibility and maintainability of the system.

[0041] (2) Horizontal expansion: Freely add and delete Kubernetes clusters: The system can easily add or delete Kubernetes clusters when needed. New clusters can be quickly added to the CI / CD process, and existing clusters can be easily exited without complicated configuration processes; Dynamic expansion of Kubernetes nodes: Each Kubernetes cluster can dynamically add or delete nodes based on resource requirements. The automatic expansion mechanism ensures that working nodes are added during high loads and idle resources are released during low loads, thereby optimizing resource utilization.

[0042] 2. Reduce traffic across computer rooms:

[0043] A Kubernetes cluster is deployed in each computer room, and the build tasks are assigned to the Kubernetes cluster in the computer room for execution. In this way, the build tasks are completed in the same computer room, avoiding cross-computer room transmission, improving transmission efficiency, and reducing network traffic overhead. Localized task execution not only speeds up transmission, but also reduces the pressure of cross-regional network transmission, improving the overall performance and stability of the system.

[0044] 3. Improve resource utilization:

[0045] When executing pipeline tasks, the corresponding container groups are started on demand to build. After the build task is completed, these container groups are destroyed in time to release resources. This on-demand allocation and dynamic management mechanism avoids long-term idle resources and improves resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flowchart of the continuous integration and deployment method based on multiple regions and multiple clusters of the present invention;

[0047] Figure 2 It is a schematic diagram of the execution pipeline task flow of the present invention;

[0048] Figure 3 It is a schematic diagram of the structure of the continuous integration and deployment system based on multiple regions and multiple clusters of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the embodiments, but it does not constitute any limitation to the present invention. Any limited number of modifications made by anyone within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0050] In view of the problems existing in the current continuous integration and deployment system, such as poor scalability, network bottlenecks of centralized CI / CD systems, and waste of idle resources, the present invention combines Jenkins with cloud native technology to propose a continuous integration and deployment method, system, and device based on multi-region and multi-cluster. Different Kubernetes clusters in multiple regions are connected through the Jenkins tool, and containers are started in different Kubernetes clusters to execute pipeline tasks. This method can bring the following significant advantages: effectively increase the scalability of the system, reduce cross-computer room traffic, improve network security, and improve resource utilization. The following will introduce a continuous integration and deployment method, system, and device based on multi-region and multi-cluster of the present invention through corresponding implementation methods.

[0051] Embodiment 1

[0052] like Figure 1 As shown, the present invention provides a multi-region, multi-cluster based continuous integration and deployment method, the method comprising the following steps:

[0053] Step S1, initialize and deploy Pipeline-UI, Pipeline and Jenkins components. Pipeline-UI is the pipeline editing section, Pipeline is the configuration manager, and Pipeline-UI and Pipeline are mainly used to realize the visual configuration function of Jenkins components.

[0054] Step S2: Set up multiple Kubernetes clusters that need to be connected. Specifically, deploy a Kubernetes cluster in each computer room.

[0055] Furthermore, the Kubernetes cluster in step S2 can be a newly created cluster or an existing cluster. In the subsequent process, the user can still adjust the scale and configuration of the connected Kubernetes cluster, that is, freely add and delete Kubernetes clusters.

[0056] In the traditional Jenkins component environment, the system still has the problem of cumbersome horizontal expansion. Horizontal expansion means adding worker nodes to share the build tasks. However, the traditional Jenkins component requires complex environment configuration when adding new worker nodes, including installing necessary development tools and setting related configurations. This process is not only cumbersome, but also requires the node to be in the system for a long time. Only manual exit can release resources, and the worker node also needs to clear data before it can be used. This embodiment provides a Kubernetes cluster for each computer room, and in the subsequent process, the user can still adjust the scale and configuration of the connected Kubernetes cluster, freely add and delete Kubernetes clusters, that is, the system can easily add or delete Kubernetes clusters when needed, new clusters can be quickly added to the CI / CD process, and the original clusters can also be easily exited without the need for a complicated configuration process, which greatly improves the system's horizontal expansion capabilities.

[0057] Preferably, multiple Kubernetes clusters distributed in multiple computer rooms can be set up at the same time for pipeline configuration selection.

[0058] Step S3: The user edits the pipeline configuration on the Pipeline-UI.

[0059] Furthermore, in step S3, specific configuration items include: selecting a Kubernetes cluster to execute tasks, setting code repository pull parameters, configuring static code scanning parameters, setting code compilation parameters, defining image build parameters, specifying image publishing parameters, and specifying host publishing parameters.

[0060] Preferably, selecting a Kubernetes cluster located in the same computer room as the target release environment to execute tasks can effectively reduce cross-computer room traffic.

[0061] Step S4: The user triggers the pipeline build. The build can be triggered manually on the Pipeline-UI, by code repository callback, or by custom callback. By triggering the build, the system starts the corresponding container group on demand to build when executing the pipeline task.

[0062] Step S5. After the Jenkins component establishes a connection with the selected Kubernetes cluster, it starts the container group.

[0063] Further, in step S5, during the process of starting the container group, the container group will be placed in the namespace specified in the Kubernetes cluster. The Kubernetes cluster allocates a specific network segment to the container group through the network plug-in. The network segment does not perform network address translation (NAT), and the network segment is restricted by the network settings of Kubernetes and the upper switch to ensure the security of the network. That is, in this embodiment, the Kubernetes security policy is adopted and the upper switch security policy is coordinated to jointly ensure network security, thereby improving network security.

[0064] Furthermore, in the above step S5, the image used to compile the container in the container group is determined by the selected code language and its version. The strategy of compiling in the container can fully utilize the advantages of containerization and avoid the need to deal with compatibility issues between different language versions.

[0065] Step S6. Once the container group has been successfully started on the Kubernetes cluster, the Jenkins component will connect to these containers and start executing the defined pipeline tasks.

[0066] Further, such as Figure 2 As shown, in step S6, the pipeline task consists of multiple sub-steps, specifically:

[0067] Step S6.1: Pull the code. Execute the git command inside the container group to pull the latest code from the defined code repository into the container group.

[0068] Step S6.2: Perform static code scanning to detect potential coding errors or security vulnerabilities.

[0069] Step S6.3: Execute the code compilation command in the compilation container of the container group according to the code language to obtain the task and task path; upload the task to the Jenkins component for archiving according to the task path.

[0070] Step S6.4: Build a mirror for the task according to the Dockerfile file in the code repository, and upload the built Docker image to the image repository.

[0071] Step S6.5, execute the image publishing instruction to replace the image address of the selected Kubernetes cluster with the image address of the Docker image in the image warehouse to achieve the upgrade of the container service.

[0072] Step 6.6: Execute the host release command, send the task to the target host, and restart the service after executing the task on the host to complete the upgrade of the host service.

[0073] Step S7: Once the pipeline task is completed, the Jenkins component will destroy the used container group. If these resources are not used by new tasks for a long period of time, then these resources can be reallocated to other business containers.

[0074] After being added to the system, the worker node needs to continue waiting for build tasks. When there is no build task, the CPU and memory resources of the worker node are idle and cannot be released for use. This static resource management mode causes the system to be unable to dynamically adjust resource configuration according to the actual load, reducing the overall resource utilization efficiency. In this embodiment, when the pipeline task is completed, the Jenkins component will destroy the used container group. This operation will release the system resources occupied by the container group, and implement an automatic expansion mechanism that allows each Kubernetes cluster to dynamically add or delete nodes according to resource requirements. The automatic expansion mechanism ensures that worker nodes are added under high load and idle resources are released under low load to optimize resource utilization.

[0075] Step S8, Pipeline-UI displays the task status according to the callback information, which can be viewed and archived by the user in real time.

[0076] Embodiment 2

[0077] like Figure 3 As shown, the present invention provides a multi-region, multi-cluster based continuous integration and deployment system, which is used to implement the above-mentioned multi-region, multi-cluster based continuous integration and deployment method, and is mainly composed of the following parts:

[0078] 1. Pipeline-UI: It provides users with a simple configuration interface, which can easily edit the parameters of pipeline tasks, thus avoiding the need to directly write Jenkins pipeline configuration files and reducing the user's learning cost.

[0079] 2. Configuration Manager (Pipeline): It receives requests from Pipeline-UI and converts the user's configuration into a configuration file that can be recognized by the Jenkins component, and then sends it to the Jenkins component. Pipeline-UI and Pipeline are key components for implementing the visual configuration function of Jenkins components, and are mainly used to implement the visual configuration function of Jenkins.

[0080] 3. Jenkins component: This component is responsible for receiving configuration information from Pipeline and responding to user trigger actions. It interacts with the Kubernetes cluster to implement functions such as pulling up container groups, executing pipeline tasks, and destroying container groups.

[0081] Kubernetes cluster: The Jenkins component connects to the Kubernetes cluster, starts the container group and executes the pipeline tasks.

[0082] 5. Upper-layer switches and distributed storage infrastructure: provide network connection and data storage services for the system to ensure the stability of system operation and the reliability of data.

[0083] Embodiment 3

[0084] The present invention provides a multi-region, multi-cluster based continuous integration and deployment device, comprising a processor, a memory connected to the processor, and a network device.

[0085] The memory is used to store a computer program. The computer program is used to execute the above method;

[0086] The processor is used to call and execute the computer program in the memory.

[0087] Network devices are used to provide network connections and set security policy rules.

[0088] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A continuous integration and deployment method based on multiple regions and multiple clusters, characterized in that: The method is: A Kubernetes cluster is deployed in each computer room; Edit the pipeline configuration to select the Kubernetes cluster where the task will be executed. Establishing a connection between the Jenkins component and the selected Kubernetes cluster and starting a container group in the selected Kubernetes cluster; The defined pipeline tasks are executed in the container group.

2. According to claim 1, a multi-region, multi-cluster based continuous integration and deployment method is characterized in that: The steps of executing the defined pipeline task in the container group include: Step 1: Execute the git command inside the container group to pull the latest code from the defined code repository into the container group; Step 2: Performing static code scanning on the code to detect potential coding errors or security vulnerabilities; Step 3: Execute a code compilation command in the compilation container of the container group according to the code language of the code to obtain a task and a task path; upload the task to the Jenkins component for archiving according to the task path; Step 4: Build an image for the task according to the Dockerfile file in the code repository, and upload the built Docker image to the image repository; Step 5: Execute the image publishing instruction to replace the image address of the selected Kubernetes cluster with the image address of the Docker image in the image warehouse; Step 6: The execution host issues instructions to send the task to the target host, and restart the service after executing the task on the target host.

3. The method for continuous integration and deployment based on multiple regions and multiple clusters according to claim 1, characterized in that: After the pipeline task is executed, the Jenkins component destroys the used container group.

4. The method for continuous integration and deployment based on multiple regions and multiple clusters according to claim 2, characterized in that: Initialize the deployment pipeline editing interface and edit the pipeline configuration on the pipeline editing interface.

5. The method for continuous integration and deployment based on multiple regions and multiple clusters according to claim 4, characterized in that: The pipeline editing interface displays the task status according to the callback information of the target host.

6. The method for continuous integration and deployment based on multiple regions and multiple clusters according to claim 1, characterized in that: In the process of starting the container group, the container group is placed in a namespace set in the Kubernetes cluster, and the Kubernetes cluster allocates a set network segment to the container group through a network plug-in.

7. The method for continuous integration and deployment based on multiple regions and multiple clusters according to claim 6, characterized in that: The network segment is set to not perform network address translation, and the network segment is restricted by the network settings of the Kubernetes cluster and the upper-layer switch.

8. The method of continuous integration and deployment based on multiple regions and multiple clusters according to claim 1, characterized in that: The deployed Kubernetes cluster is a newly created Kubernetes cluster or an existing Kubernetes cluster.

9. A system using the multi-region, multi-cluster based continuous integration and deployment method according to any one of claims 1 to 8, characterized in that: include: The pipeline editing section is used to provide an interface for configuring the parameters of the pipeline task; A configuration manager, configured to receive a request for editing a section of the pipeline, convert the parameters into a configuration file that can be recognized by a Jenkins component, and then send the configuration file to the Jenkins component; A Jenkins component for responding to the triggering action of the pipeline according to the configuration file; A Kubernetes cluster, used to start a container group therein according to the instructions of the Jenkins component in response to the trigger action to execute the pipeline task; The upper-layer switches and distributed storage infrastructure are used to provide network connections and data storage services for the system.

10. A multi-region, multi-cluster continuous integration and deployment device, characterized in that: include: A memory for storing a computer program for executing the multi-region, multi-cluster based continuous integration and deployment method according to any one of claims 1 to 8; A processor, configured to retrieve and execute the computer program; Network devices are used to provide network connections for the processor to communicate with external hosts.

Citation Information

Cited By

  • Cloud platform system supporting multiple construction clusters and construction method thereof

    CN121691419A