Jenkins-based build task processing method and device, electronic equipment and medium

Abstract

The present disclosure provides a Jenkins-based build task processing method and device, electronic equipment and medium, which can be applied to the field of computer technology or the field of financial technology. The method comprises the following steps: in response to a target build task from a build task distribution platform, querying the resource consumption state of a plurality of worker nodes in a container cluster to determine a target worker node from the plurality of worker nodes; sending the target build task to the target worker node to determine a target deployment unit in the target worker node; sending a Jenkins image to the target deployment unit to load the Jenkins image in the target deployment unit to obtain a Jenkins container; and controlling the target worker node to send the target build task to the target deployment unit to control the Jenkins container to process the target build task.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer or financial technology, and more particularly, to a Jenkins-based build task processing method and device, electronic equipment and medium. BACKGROUND

[0002] With the practical development of cloud computing technology in the industry, cloud native architecture has completed the evolution and upgrading of IT architecture in the cloud computing era. When building projects in the cloud, the most commonly used build scheduling tool at the present stage is open source software Jenkins. Jenkins is used to automate various tasks, including building, testing and software deployment. Therefore, the stability of Jenkins not only determines the smooth running of the entire cloud platform construction, but also determines the construction efficiency of the entire platform. Therefore, the high availability of Jenkins has become the top priority of cloud computing construction technology.

[0003] In the process of implementing the present disclosure, the inventors found that at least the following problems exist in the related art: Jenkins itself includes a Master node and a Slave node. The Master node generally serves as a control node, and the Slave generally serves as a task execution node. Generally, the Slave node is mounted under the Master node. The Master node only performs task scheduling work, and the build task is performed on the Slave node. When performing a build task job, if the number of mounted Slave nodes is small, the performance requirement of large concurrency cannot be met, and the job needs to be in the queue for a long time. If a large number of Slave nodes are mounted, a large amount of resources will be wasted during idle time, and the maintenance cost is also very high. SUMMARY

[0004] Therefore, the present disclosure provides a Jenkins-based build task processing method and device, electronic equipment, readable storage medium and computer program product.

[0005] One aspect of the present disclosure provides a Jenkins-based build task processing method, comprising: in response to a target build task from a build task distribution platform, querying the resource consumption state of a plurality of worker nodes in a container cluster to determine a target worker node from the plurality of worker nodes; sending the target build task to the target worker node to determine a target deployment unit in the target worker node; sending a Jenkins image to the target deployment unit to load the Jenkins image in the target deployment unit to obtain a Jenkins container; and controlling the target worker node to send the target build task to the target deployment unit to control the Jenkins container to process the target build task.

[0006] According to an embodiment of the present disclosure, the Jenkins-based build task processing method further includes: constructing the Jenkins image based on the Jenkins master application; and storing the Jenkins image in an image warehouse, wherein the control node of the container cluster pulls the Jenkins image from the image warehouse.

[0007] According to an embodiment of the present disclosure, the Jenkins image is constructed based on the Jenkins master application, including: constructing the Jenkins image on a base image by using a component related to task construction in the Jenkins master application, wherein a preset operating system kernel is configured in the base image.

[0008] According to an embodiment of the present disclosure, the method of querying the resource consumption states of the plurality of worker nodes in the container cluster to determine the target worker node from the plurality of worker nodes includes: obtaining running information of the plurality of worker nodes, wherein the running information is periodically sent by an agent component of the worker node to a control node of the container cluster; determining the resource consumption states of the plurality of worker nodes based on the running information of the plurality of worker nodes; and determining the target worker node from the plurality of worker nodes by using a preset load balancing strategy.

[0009] According to an embodiment of the present disclosure, a preset number of first deployment units in a long connection state are configured in the worker node; wherein the target deployment unit in the target worker node is determined by: querying the working state of the first deployment unit in the target worker node; in the case where it is determined that there is a target first deployment unit in the target worker node in an idle state, determining the target first deployment unit as the target deployment unit; and in the case where it is determined that none of the first deployment units in the target worker node is in the idle state, creating a second deployment unit in the target worker node, and determining the second deployment unit as the target deployment unit.

[0010] According to an embodiment of the present disclosure, a Jenkins container in a long connection state is running in the first deployment unit; the method further includes: after determining that the target first deployment unit is the target deployment unit, controlling the target worker node to send the target build task to the target first deployment unit, so as to control the Jenkins container in the long connection state in the target first deployment unit to process the target build task.

[0011] According to an embodiment of the present disclosure, the Jenkins-based build task processing method further includes: in response to the Jenkins container completing processing of the target build task, controlling the Jenkins container to stop running and recycling resources occupied by the Jenkins container.

[0012] Another aspect of the present disclosure also provides a Jenkins-based build task processing apparatus, comprising: a query module configured to query resource consumption states of a plurality of worker nodes in a container cluster in response to a target build task from a build task distribution platform, to determine a target worker node from the plurality of worker nodes; a first sending module configured to send the target build task to the target worker node, to determine a target deployment unit in the target worker node; a second sending module configured to send a Jenkins image to the target deployment unit, to load the Jenkins image in the target deployment unit to obtain a Jenkins container; and a control module configured to control the target worker node to send the target build task to the target deployment unit, to control the Jenkins container to process the target build task.

[0013] Another aspect of the present disclosure also provides an electronic device, comprising: one or more processors; and a memory configured to store one or more instructions, wherein the one or more instructions, when executed by the one or more processors, cause the one or more processors to implement the Jenkins-based build task processing method.

[0014] Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the Jenkins-based build task processing method.

[0015] Another aspect of the present disclosure also provides a computer program product, comprising computer executable instructions, which when executed by a processor, are configured to implement the Jenkins-based build task processing method.

[0016] According to an embodiment of the present disclosure, by building Jenkins as a container and controlling the Jenkins container to process a target build task. When the target build task is received, because the container cluster is utilized to manage the Jenkins container, the control node in the container cluster can control the target deployment unit of the container cluster to run the Jenkins container, and further control the Jenkins container to process the target build task, so as to realize dynamic creation and use of the Jenkins container, and further realize pool high availability of Jenkins, reduce the maintenance cost of Jenkins, at least partially overcome the technical problems of high maintenance cost and high resource waste rate in the related art, and further achieve the technical effects of reducing the maintenance cost of Jenkins and improving the utilization rate of build resources. BRIEF DESCRIPTION OF DRAWINGS

[0017] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0018] Figure 1 An exemplary system architecture diagram of a Jenkins-based build task processing method and device according to an embodiment of the present disclosure is schematically shown;

[0019] Figure 2 A flowchart of a Jenkins-based build task processing method according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A flowchart of a Jenkins-based build task processing method according to another embodiment of the present disclosure is schematically shown;

[0021] Figure 4 A block diagram of a Jenkins-based build task processing device according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 5 A block diagram of an electronic device suitable for implementing a Jenkins-based build task processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description is merely exemplary and is not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. However, it would be apparent that one or more embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessary obscuring the concept of the present disclosure.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the term "includes" and tautological equivalents thereof, means that the named feature, step, operation, and / or component is present, but not that one or more other features, steps, operations, and / or components are excluded.

[0025] All terms used herein (including technical and scientific terms) have the meaning commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein are defined as consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0026] In situations where similar terminology is used for similar items, but not all items are in common, one of ordinary skill in the art would understand the descriptions to refer to the specific situation based on the context of the specific situation, but the specific situation would not impose a narrow interpretation on the generic term. Where the expression similar to "at least one of A, B and C, etc." is used, it should generally be interpreted to include the possibilities of no A, or no B, or no C, or no A and no B, or no A and no C, or no B and no C, or no A, no B, and no C, or A and B and C, etc.

[0027] In the past decade, cloud computing technology has been developing rapidly, and the form of cloud has been evolving. With the further practice and development of cloud native technology concept in the industry, cloud native architecture has completed the evolution and upgrade of IT architecture in the cloud computing era. Cloud native technologies represented by CI / CD, DevOps, and microservices architecture drive and lead the business development of enterprises with their efficient and stable, fast response characteristics, helping enterprises build application services more suitable for the cloud. When building projects in the cloud, the most commonly used build scheduling tool at this stage is open source software Jenkins. Jenkins itself supports the Master node / Slave node scheme, which is to mount Slave nodes under the Master node. The Master node only performs task scheduling, and the build task is performed on the Slave node. Compared with simply using the Master node to build, this solves the problem of high resource occupation of the Master node service memory and CPU, which leads to the crash of the Master node and low construction efficiency. However, each application version deployment job is usually performed at a specific time point, such as once a week or once a month, and the number of concurrent applications may be very high at that specific time. When performing build task jobs, if the number of mounted Slave nodes is small, the performance requirement of large concurrency cannot be met, and the job needs to wait in the queue for a long time; if a large number of Slave nodes are mounted, a large amount of resources will be wasted during idle time, and the maintenance cost will also be high.

[0028] Therefore, the present application proposes a Jenkins pooling high availability scheme, which no longer uses the Master node for task scheduling and the Slave node for compiling and building tasks. All Jenkins are containerized, and Kubernetes is used for unified management. Jenkins images are started, mounted, and recycled through Kubernetes. Each Jenkins is only run when there is a task to be executed, unlike the Master node in the Master node / Slave node scheme, which is always in a running state. This can better realize the maximization of resource utilization and save unnecessary waste, and reduce the time consumption caused by the Master node scheduling Slave nodes.

[0029] Specifically, this disclosure provides a Jenkins-based build task processing method, a Jenkins-based build task processing apparatus, an electronic device, a readable storage medium, and a computer program product. These are used to reduce maintenance costs and improve resource utilization. The method includes: responding to a target build task from a build task distribution platform, querying the resource consumption status of multiple worker nodes in a container cluster to determine a target worker node from the multiple worker nodes; sending a target build task to the target worker node to determine a target deployment unit within the target worker node; sending a Jenkins image to the target deployment unit to load the Jenkins image into the target deployment unit, obtaining a Jenkins container; and controlling the target worker node to send the target build task to the target deployment unit to control the Jenkins container to process the target build task.

[0030] It should be noted that the Jenkins-based build task processing method and apparatus defined in this disclosure can be used in the fields of computer technology or fintech. The Jenkins-based build task processing method and apparatus defined in this disclosure can also be used in any field other than computer technology and fintech. This disclosure does not limit the application areas of the defined Jenkins-based build task processing method and apparatus.

[0031] In the technical solution disclosed herein, the acquisition, storage, and application of user personal information comply with relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals. In the technical solution disclosed herein, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0032] Figure 1 The diagram schematically illustrates an exemplary system architecture diagram to which a Jenkins-based build task processing method and apparatus can be applied according to embodiments of the present disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0033] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a task distribution platform 104, and a server 105.

[0034] Terminal devices 101, 102, and 103 can be various electronic devices in public or independent environments, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0035] Various communication client applications can be installed on the terminal device, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0036] The user can perform operations in the client application of the terminal device 101, 102, 103, such as code compilation, etc., to execute a business function, and submit to the build task distribution platform 104 to execute a build target build task.

[0037] The build task distribution platform 104 can be linked between the terminal device 101, 102, 103 and the server 105 through wired and / or wireless communication links, etc.

[0038] The server 105 can be a server that provides various services, such as a cloud server, which maintains a container cluster that can provide analysis processing for the target build task submitted by the build task distribution platform. Specifically, the control node in the container cluster can determine a target work node from the plurality of work nodes in the query container cluster according to the target build task, determine a current deployment unit in the target work node, and obtain a Jenkins container in the target deployment unit, so as to process the target build task in the Jenkins container.

[0039] It should be noted that the Jenkins-based build task processing method provided by the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the Jenkins-based build task processing apparatus provided by the embodiments of the present disclosure can generally be provided in the server 105. The Jenkins-based build task processing method provided by the embodiments of the present disclosure can also be executed by a server or server cluster different from the server 105 and capable of communicating with the build task distribution platform 104 and / or the server 105. Accordingly, the Jenkins-based build task processing apparatus provided by the embodiments of the present disclosure can also be provided in a server or server cluster different from the server 105 and capable of communicating with the build task distribution platform 104 and / or the server 105. Alternatively, the Jenkins-based build task processing method provided by the embodiments of the present disclosure can also be executed by the build task distribution platform 104, or by another build task distribution platform different from the build task distribution platform 104. Accordingly, the Jenkins-based build task processing apparatus provided by the embodiments of the present disclosure can also be provided in the build task distribution platform 104, or in another build task distribution platform device different from the build task distribution platform 104.

[0040] It should be understood that, Figure 1The number of terminal devices, networks and servers in the figure is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0041] Figure 2 A flowchart of a Jenkins-based build task processing method according to an embodiment of the present disclosure is schematically shown.

[0042] As Figure 2 The method includes operations S201-S204.

[0043] In operation S201, in response to a target build task from a build task distribution platform, the resource consumption state of a plurality of worker nodes in a container cluster is queried to determine a target worker node from the plurality of worker nodes.

[0044] In operation S202, the target build task is sent to the target worker node to determine a target deployment unit in the target worker node.

[0045] In operation S203, the Jenkins image is sent to the target deployment unit to load the Jenkins image in the target deployment unit to obtain a Jenkins container.

[0046] In operation S204, the target worker node is controlled to send the target build task to the target deployment unit to control the Jenkins container to process the target build task.

[0047] According to an embodiment of the present disclosure, the target build task can be a build task independently created by the build task distribution platform, or can be obtained by the build task distribution platform in response to a build task request from a terminal device. Specifically, in a container cluster (Kubernetes), there are two types of nodes: Master nodes and worker nodes, wherein the Master nodes can serve as control nodes and at least one (and multiple for improving system availability) can be deployed. The worker nodes serve as load nodes and multiple can be deployed according to system requirements to support dynamic addition and deletion. There are four key processes in the Master nodes: Kube-controller-manager, Kube-apiserver, Kube-scheduler, and etcd. The Kube-apiserver interacts with the Kube-controller-manager and the Kube-scheduler. A user can interact with the Kube-apiserver through a kubectl command. The Kube-apiserver provides a key service process of an HTTP Rest interface, is the only entrance for all resource addition, deletion, modification, and query operations in the Kubernetes, and is an entrance process for controlling the Kubernetes cluster; the Kube-controller-manager is an automatic control center for all resource objects in the Kubernetes, and can be understood as a "manager" of the resource objects; the Kube-scheduler is responsible for a resource scheduling (deployment unit scheduling) process, and is equivalent to a "dispatching room" of a bus company. The build task distribution platform can send a task processing request about the target build task through the Kube-apiserver of the Master node.

[0048] According to an embodiment of the present disclosure, the target worker node can be an idle worker node in the plurality of worker nodes of the Kubernetes cluster that still has resources that can be utilized, and the idle worker node can be used to process the target build task. Specifically, the worker node can be a virtual machine or a physical machine, which can be adapted according to the type of the Kubernetes cluster. The services on the worker node can include a Docker Engine, a kubelet, and a kube-proxy. The worker node can be dynamically added to the Kubernetes cluster during runtime, provided that the above-mentioned key processes have been correctly installed, configured, and started on the node. By default, the kubelet registers itself with the Master node, which is also the recommended way of managing worker nodes by the Kubernetes cluster. Once the worker node is brought under the management of the Kubernetes cluster, the kubelet process periodically reports its status to the Master node, such as the operating system, the Docker version, the CPU and memory status of the machine, and which deployment units are currently running, so that the Master node can know the resource usage of each worker node and implement an efficient and balanced resource scheduling strategy.

[0049] According to an embodiment of the present disclosure, the worker node can include a plurality of deployment units. The deployment unit is a basic component of the Kubernetes cluster and is the smallest object that can be created or deployed in the Kubernetes cluster. A deployment unit represents a running process in the cluster. A deployment unit is equivalent to a configuration group sharing a context, and in the same context, the application can also have an independent cgroup isolation mechanism. A deployment unit is a "logical host" in a container environment, which can include one or more closely connected applications, which can be on the same physical host or virtual machine. Each worker node can have the necessary services for running the deployment unit and be managed by the components of the Master node. The target deployment unit can be an idle deployment unit in the plurality of deployment units of the target worker node, or a new deployment unit created in the case where all existing deployment units are in a non-idle state.

[0050] According to embodiments of this disclosure, a deployment unit encapsulates an application container (in some cases, multiple containers, defined in the deployment unit's YAML file), storage resources, a unique network IP address, and configurations controlling how the container should run. A single application instance in a Kubernetes cluster may contain a single container or a small number of tightly coupled, resource-sharing containers. The Jenkins container runs within the deployment unit; when a task is available, a Jenkins container can be run within the deployment unit, and when the build task ends, the running container can be stopped using the Kubernetes container recycling mechanism, thereby reclaiming resources.

[0051] According to embodiments of this disclosure, a Jenkins container can be used to execute build tasks. When a target build task arises, the build task distribution platform runs a Jenkins container on a deployment unit within a Kubernetes cluster. This Jenkins container can perform build tasks such as pulling code, compiling and packaging, and code quality scanning.

[0052] According to embodiments of this disclosure, Jenkins is built as a container, and the Jenkins container is controlled to process the target build task. When the target build task is received, because the Jenkins container is managed using a container cluster, the control node in the container cluster can control the target deployment unit of the container cluster to run the Jenkins container, thereby controlling the Jenkins container to process the target build task. This enables dynamic creation and use of the Jenkins container, thereby achieving pooled high availability of Jenkins, reducing the maintenance cost of Jenkins, and at least partially overcoming the technical problems of high maintenance cost and high resource waste rate in related technologies. This achieves the technical effect of reducing the maintenance cost of Jenkins and improving the utilization rate of build resources.

[0053] According to embodiments of this disclosure, a Jenkins image is built based on the Jenkins main application; and the Jenkins image is stored in an image repository, wherein the control node of the container cluster pulls the Jenkins image from the image repository.

[0054] According to embodiments of this disclosure, building a Jenkins image based on the Jenkins master application includes: using task-building related components in the Jenkins master application to build a Jenkins image on a base image, wherein the base image is configured with a preset operating system kernel.

[0055] According to the embodiments of the present disclosure, since the Jenkins container mainly functions to perform a build task, the Jenkins image can include components related to the task build. The components related to the task build can include Maven, JDK, SonarScanner, etc. The preset operating system kernel can include a linux kernel of a CentOS operating system, a linux kernel of an ubuntu operating system. The base image can be an image including the preset operating system kernel.

[0056] According to the embodiments of the present disclosure, the built Jenkins container is stored in a unified image warehouse, such as a Harbor image warehouse, and the image warehouse can uniformly manage and store the image. In an embodiment, an etcd service can also be deployed on the Master node, because all resource object data in the Kubernetes is saved in the etcd, so when the Jenkins container is used, the Jenkins image can be pulled from the Harbor image warehouse through the etcd service on the Master node of the Kubernetes cluster. The Jenkins image is running in the deployment unit of the Kubernetes when performing the build task, and each deployment unit is a closed container environment, which provides all resources for the independent running of the Jenkins image.

[0057] According to the embodiments of the present disclosure, the operation S201 can further include the following operations: obtaining running information of the plurality of worker nodes, wherein the running information is periodically sent by an agent component of the worker node to a control node of the container cluster; determining resource consumption states of the plurality of worker nodes based on the running information of the plurality of worker nodes; and determining a target worker node from the plurality of worker nodes by using a preset load balancing strategy.

[0058] According to the embodiments of the present disclosure, the agent component can include a Docker Engine, a kubelet, a kube-proxy, etc. For example, once the worker node is included in the management range of the Kubernetes cluster, the kubelet process will report its own situation to the Master node in a timely manner, such as an operating system, a Docker version, a CPU and a memory condition of the machine, and which deployment units are currently running, etc., so that the Master node can know the resource usage of each worker node, and implement an efficient and balanced resource scheduling strategy.

[0059] According to the embodiments of the present disclosure, the preset load balancing strategy is used to distribute the request for processing the build task to each worker node corresponding to the build task by the Master node in the Kubernetes cluster, so that the load of each worker node in the Kubernetes cluster can reach a balanced state.

[0060] According to an embodiment of the present disclosure, when there is a build task requirement, the Master node of the Kubernets cluster queries the resource usage of the worker nodes, then determines the worker nodes that can still be invoked according to a preset load balancing strategy, and creates a deployment unit in the worker node, and runs the Jenkins image by using the deployment unit, so that the Jenkins image performs a specific build task. When there are multiple tasks, the Kubernetes can start multiple containers, and the starting of the multiple containers can be distinguished by different ports.

[0061] According to an embodiment of the present disclosure, the worker node can also be configured with a preset number of first deployment units in a long connection state. Operation S202 can further include the following operations: querying the working state of the first deployment units in the target worker node; in a case where it is determined that there is a target first deployment unit in an idle state in the target worker node, determining the target first deployment unit as the target deployment unit; and in a case where it is determined that none of the first deployment units in the target worker node is in an idle state, creating a second deployment unit in the target worker node, and determining the second deployment unit as the target deployment unit.

[0062] According to an embodiment of the present disclosure, the deployment unit can be a deployment unit in the Kubernets cluster. To ensure the timeliness of the build task, 10-20 deployment units in the worker node can be kept in a long connection state. The Kubernets cluster can dynamically increase the number of deployment units according to the number of build tasks, and automatically scale the node resources. The preset number can also be adaptively adjusted according to actual needs.

[0063] According to an embodiment of the present disclosure, when the current deployment unit is in an idle state, the deployment unit can be used to run the Jenkins container; when the current deployment unit is in a non-idle state, the Kubernets cluster creates a new deployment unit, and uses the newly created deployment unit to run the Jenkins container. Specifically, when the build task is scheduled, the Kubernets cluster creates a deployment unit in the worker node, and runs the Jenkins container in the Pod to perform the actual build task, so that the Jenkins node is scaled, started and stopped, and the container is recycled.

[0064] According to an embodiment of the present disclosure, in the conventional mode of deploying the Jenkins Master master node and the Slave node on a virtual machine or a physical machine, when the number of build tasks is greater than the number of Jenkins Slave nodes, the build will be blocked, thereby reducing the build efficiency, which can only be solved by increasing the number of virtual machines or physical machines. According to an embodiment of the present disclosure, only the number of deployment units of Kubernetes needs to be increased, and a Jenkins container can be run in each deployment unit, and theoretically, up to 130 containers can be run in each deployment unit. Compared with the mode of running one build task on each physical machine or virtual machine in the conventional scheme, the resource utilization has been significantly improved.

[0065] According to an embodiment of the present disclosure, the Jenkins container in the first deployment unit is in a long connection state; after determining that the target first deployment unit is the target deployment unit, the target worker node is controlled to send the target build task to the target first deployment unit, so that the Jenkins container in the target first deployment unit in the long connection state processes the target build task.

[0066] According to an embodiment of the present disclosure, in order to ensure the timeliness of the build task, 10-20 Jenkins containers in the deployment unit can be kept in a long connection state, and the Kubernetes cluster can dynamically increase the number of Jenkins containers according to the number of build tasks and automatically scale the container resources. The preset number can also be adaptively adjusted according to actual needs.

[0067] According to an embodiment of the present disclosure, the operation of the Jenkins container in the long connection state to process the target build task can include functions such as code pulling, compiling and packaging, and code quality scanning.

[0068] According to an embodiment of the present disclosure, when the Jenkins container completes the processing of the target build task, the Jenkins container is controlled to stop running and recycle the resources occupied by the Jenkins container.

[0069] According to an embodiment of the present disclosure, when the build task ends, Kubernetes can be directly called to stop the container and recycle the resources occupied by the container. Specifically, the Jenkins container is started using the Kubectl command, and when the task on the Jenkins is executed, the command kubectl stop rc jenkins can be used to stop the container and recycle the resources occupied by the container.

[0070] Figure 3A flowchart of a Jenkins-based build task processing method according to another embodiment of the present disclosure is schematically shown.

[0071] As shown in Figure 3 The Jenkins-based build task processing method according to another embodiment of the present disclosure can include the following operations.

[0072] A Kubernetes cluster is built to control task scheduling of the Jenkins container. Since the Jenkins container is mainly used to execute build tasks, the Jenkins image only needs to include task build related components, such as Maven, JDK, SonarScanner, etc. The base image for building the Jenkins image can only be a Linux kernel base image, such as operating systems like Centos, ubuntu, etc. To ensure the timeliness of the build task, 10-20 Jenkins containers can be kept in a long link state. The Kubernetes cluster can dynamically increase the number of Jenkins containers according to the number of build tasks, and automatically scale the container resources.

[0073] A Jenkins container is built, and the containerized image is stored in a unified image repository, such as a Harbor image repository. The Jenkins image is run in a Pod node of the Kubernetes when executing a build task. Each Pod node is a closed container environment that provides all resources for the independent running of the Jenkins image.

[0074] Build task scheduling: the Kubernetes cluster creates a Pod node in the Node worker node, and the Jenkins container in the Pod node is used to execute the actual build task. The Jenkins node is scaled, started and stopped, and the container is recycled, etc. The specific steps are as follows.

[0075] When there is a build task requirement, the Master node of the Kubernetes cluster queries the resource usage of the Node worker node, and then determines the Node worker node to create a Pod node to run the Jenkins image, so as to execute the specific build task. When there are multiple tasks, the Kubernetes starts multiple containers, and the multiple containers are distinguished by different ports.

[0076] When the construction task ends, Kubernetes is directly called to stop the container and recycle the resources occupied by the container. The command for starting the Jenkins container using the Kubectl command is as follows: kubectl run jenkins --image=jenkins-image --replicas=2 --port=80. This command is used to create two instances of the jenkins-image pod using the kubectl command, and two running instances are created to ensure that there are always two Pod nodes running, thereby realizing the high availability of the Jenkins-based construction task processing method.

[0077] Once the Pod nodes are successfully created, the starting and running states of all Pod nodes can be viewed using the Kubectl get pods command.

[0078] When the task on Jenkins ends, the container can be stopped using the command: kubectl stop rc jenkins, and the occupied resources are also recycled.

[0079] According to an embodiment of the present disclosure, the traditional Jenkins high availability mainly adopts a master-slave scheme in a virtual machine or physical machine mode, and the Jenkins Master node and the Jenkins Slave node are both deployed on a virtual machine or a physical machine. When there is a lack of Jenkins high availability resources, the number of Slave nodes needs to be continuously increased to improve the running efficiency, which will result in an increasing number of Slave nodes and a higher maintenance cost. According to the fully containerized mode of Jenkins according to the embodiment of the present disclosure, the Master node and the Slave node are not distinguished, and all containers are used to execute construction tasks. Only when there is a construction task, Jenkins containers are created through a Kubernetes cluster; when the construction task ends, the Jenkins containers can be recycled, so that the maintenance of the execution nodes is completely managed by the Kubernetes cluster, greatly reducing the maintenance cost.

[0080] According to an embodiment of the present disclosure, in the task construction process, when the Jenkins Slave node in the traditional high availability scheme is down, the construction task cannot proceed, and the operation and maintenance personnel need to manually start the Slave node to perform construction again. According to the Kubernetes management execution node mode according to the embodiment of the present disclosure, when the container is down, Kubernetes will automatically restart the container to execute the construction task again, which can free up manpower and reduce the operation and maintenance cost.

[0081] According to an embodiment of the present disclosure, dynamic creation, stopping and recycling of resources of the Jenkins container can be achieved by combining Kubernetes, thereby reducing the maintenance cost of Jenkins and improving the utilization rate of the build resources.

[0082] It should be noted that, unless it is explicitly stated that there is an execution sequence between different operations or there is an execution sequence in the technical implementation of different operations, the execution sequence of multiple operations can not be distinguished, and multiple operations can be executed simultaneously.

[0083] Figure 4 A block diagram of a Jenkins-based build task processing apparatus according to an embodiment of the present disclosure is schematically shown.

[0084] As shown in Figure 4 The Jenkins-based build task processing apparatus 400 includes a query module 410, a first sending module 420, a second sending module 430, and a control module 440.

[0085] The query module 410 is configured to query resource consumption states of a plurality of worker nodes in a container cluster in response to a target build task from a build task distribution platform, to determine a target worker node from the plurality of worker nodes.

[0086] The first sending module 420 is configured to send the target build task to the target worker node, to determine a target deployment unit in the target worker node.

[0087] The second sending module 430 is configured to send a Jenkins image to the target deployment unit, to load the Jenkins image in the target deployment unit, and obtain a Jenkins container.

[0088] The control module 440 is configured to control the target worker node to send the target build task to the target deployment unit, to control the Jenkins container to process the target build task.

[0089] According to an embodiment of the present disclosure, the Jenkins is built into a container, and the Jenkins container is controlled to process a target build task. When the target build task is received, because the Jenkins container is managed by the container cluster, a control node in the container cluster can control a target deployment unit of the container cluster to run the Jenkins container, and further control the Jenkins container to process the target build task, so that dynamic creation and use of the Jenkins container are achieved, and further Jenkins pooling high availability is achieved, the maintenance cost of the Jenkins is reduced, and at least part of the technical problems of high maintenance cost and high resource waste rate in the related art are overcome, and further the technical effects of reducing the maintenance cost of the Jenkins and improving the utilization rate of the build resources are achieved.

[0090] According to an embodiment of the present disclosure, the Jenkins-based build task processing apparatus further comprises a build module and a storage module.

[0091] The build module is configured to build a Jenkins image based on a Jenkins master application.

[0092] The storage module is configured to store the Jenkins image in an image warehouse, and a control node of the container cluster pulls the Jenkins image from the image warehouse.

[0093] According to an embodiment of the present disclosure, the build module further comprises a build unit.

[0094] The build unit is configured to build the Jenkins image on a base image by using a task build-related component in the Jenkins master application, and the base image is configured with a preset operating system kernel.

[0095] According to an embodiment of the present disclosure, the query module further comprises an acquisition unit, a first determination unit, and a second determination unit.

[0096] The acquisition unit is configured to acquire running information of the plurality of worker nodes, and the running information is periodically sent by an agent component of the worker node to the control node of the container cluster.

[0097] The first determination unit is configured to determine a resource consumption state of the plurality of worker nodes based on the running information of the plurality of worker nodes.

[0098] The second determination unit is configured to determine a target worker node from the plurality of worker nodes by using a preset load balancing strategy.

[0099] According to an embodiment of the present disclosure, the first sending module can further comprise a query unit, a first creation unit, and a second creation unit.

[0100] The query unit is configured to query a working state of the first deployment unit in the target working node.

[0101] The first creation unit is configured to determine the target first deployment unit as the target deployment unit in a case where it is determined that the target first deployment unit in the target working node is in an idle state.

[0102] The second creation unit is configured to create a second deployment unit in the target working node and determine the second deployment unit as the target deployment unit in a case where it is determined that none of the first deployment units in the target working node is in an idle state.

[0103] According to an embodiment of the present disclosure, the Jenkins-based build task processing apparatus further comprises a control unit.

[0104] The control unit is configured to control the target working node to send the target build task to the target first deployment unit after determining that the target first deployment unit is the target deployment unit, so that the Jenkins container in the target first deployment unit in a long connection state processes the target build task.

[0105] According to an embodiment of the present disclosure, the Jenkins-based build task processing apparatus further comprises a recycling module.

[0106] The recycling module is configured to control the Jenkins container to stop running and recycle resources occupied by the Jenkins container in a case where the Jenkins container completes processing of the target build task.

[0107] Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of hardware or firmware through integration or packaging of circuits, or in any one of software, hardware and firmware or in an appropriate combination of any of them. Alternatively, one or more of the modules, sub-modules, units, sub-units according to the embodiments of the present disclosure can be at least partially implemented as computer program modules that can perform corresponding functions when executed.

[0108] For example, any of the query module 410, the first sending module 420, the second sending module 430, and the control module 440 can be combined in one module / unit / sub-unit for implementation, or any of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the function of one or more of the modules / units / sub-units can be combined with at least part of the function of other modules / units / sub-units, and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the query module 410, the first sending module 420, the second sending module 430, and the control module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of hardware or firmware that can be integrated or packaged with a circuit, or implemented in any one of software, hardware, and firmware or in a proper combination of any of them. Alternatively, at least one of the query module 410, the first sending module 420, the second sending module 430, and the control module 440 can be at least partially implemented as a computer program module that can perform the corresponding function when the computer program module is run.

[0109] It should be noted that the Jenkins-based build task processing apparatus part in the embodiments of the present disclosure corresponds to the Jenkins-based build task processing method part in the embodiments of the present disclosure, and the description of the Jenkins-based build task processing apparatus part is specifically referred to the Jenkins-based build task processing method part, which will not be repeated here.

[0110] Figure 5 A block diagram of an electronic device suitable for implementing the Jenkins-based build task processing method according to an embodiment of the present disclosure is schematically shown. Figure 5 The electronic device shown is merely an example, and should not bring any limitation to the function and use range of the embodiments of the present disclosure.

[0111] As Figure 5As shown, the computer electronic device 500 according to embodiments of the present disclosure includes a processor 501 that can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 502 or loaded into a random access memory (RAM) 503 from a storage section 508. The processor 501 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), and so on. The processor 501 can also include an on-board memory for cache use. The processor 501 can include a single processing unit or multiple processing units for executing different actions of the method processes according to embodiments of the present disclosure.

[0112] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method processes according to embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. Note that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method processes according to embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0113] According to embodiments of the present disclosure, the electronic device 500 can also include an input / output (I / O) interface, which is also connected to the bus 504. The electronic device 500 can also include one or more of the following components connected to the I / O interface: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read out therefrom is installed in the storage section 508 as necessary.

[0114] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product comprising a computer program carrying computer program code embodied in a computer readable storage medium, the computer program code comprising program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0115] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments, or exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, which when executed, implement the methods according to the embodiments of the present disclosure.

[0116] According to an embodiment of the present disclosure, the computer readable storage medium can be a non-volatile computer readable storage medium. For example, it can include but is not limited to a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0117] For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories of the ROM 502 and / or the RAM 503 described above and / or one or more memories other than the ROM 502 and the RAM 503.

[0118] The embodiments of the present disclosure also include a computer program product comprising a computer program containing program code for executing the method provided by the embodiments of the present disclosure, which program code is used to make the electronic device implement the Jenkins-based build task processing method provided by the embodiments of the present disclosure when the computer program product is running on the electronic device.

[0119] When the computer program is executed by the processor 501, the above-described functions defined in the system / apparatus of the embodiments of the present disclosure are performed. According to the embodiments of the present disclosure, the system, apparatus, module, unit, etc. described above can be implemented through a computer program module.

[0120] In an embodiment, the computer program can be stored in a tangible storage medium, such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be programmed as computer program code, transmitted over network medium, distributed, and installed by the communication part 509, and / or installed from the detachable medium 511. The computer program code contained in the computer program can be transmitted using any suitable network medium, including, but not limited to, wireless, wired, etc., or any suitable combination of the above.

[0121] According to the embodiments of the present disclosure, the program code for the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented using a high-level procedural and / or object-oriented programming language, and / or an assembly / machine language. The programming language includes, but is not limited to, such as Java, C++, python, "C" language, or similar programming languages. The program code can be executed entirely on a user computing device, partially on a user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected through the Internet by using an Internet service provider).

[0122] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0123] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A Jenkins-based build task processing method, comprising: in response to a target build task from a build task distribution platform, querying resource consumption states of a plurality of worker nodes in a container cluster to determine a target worker node from the plurality of worker nodes; sending the target build task to the target worker node to determine a target deployment unit in the target worker node; sending a Jenkins image to the target deployment unit to load the Jenkins image in the target deployment unit to obtain a Jenkins container; and controlling the target worker node to send the target build task to the target deployment unit to control the Jenkins container to process the target build task, wherein the Jenkins container does not distinguish between Master nodes and Slave nodes, all the Jenkins containers are used to process the target build task, and in a case where the Jenkins container completes processing of the target build task, the Jenkins container is controlled to stop running and resources occupied by the Jenkins container are recycled. 2.The method of claim 1, further comprising: building the Jenkins image based on a Jenkins master application; and storing the Jenkins image in an image repository, wherein a control node of the container cluster pulls the Jenkins image in the image repository. The building of the Jenkins image based on the Jenkins master application comprises: building the Jenkins image on a base image by using components related to task building in the Jenkins master application, wherein a preset operating system kernel is configured in the base image.

3. The method of claim 2, wherein, The querying of the resource consumption states of the plurality of worker nodes in the container cluster to determine the target worker node from the plurality of worker nodes comprises: obtaining running information of the plurality of worker nodes, wherein the running information is periodically sent by an agent component of the worker node to a control node of the container cluster; 4. The method of claim 1, wherein, determining resource consumption states of the plurality of worker nodes based on the running information of the plurality of worker nodes; and determining the target worker node from the plurality of worker nodes by using a preset load balancing strategy. A preset number of first deployment units in a long connection state are configured in the worker node. The determining of the target deployment unit in the target worker node comprises:

5. The method of claim 1, wherein, querying working states of the first deployment units in the target worker node; in a case where it is determined that a target first deployment unit in an idle state exists in the target worker node, determining the target first deployment unit as the target deployment unit; and in a case where it is determined that none of the first deployment units in the target worker node is in the idle state, creating a second deployment unit in the target worker node and determining the second deployment unit as the target deployment unit. The first deployment unit runs a Jenkins container in a long connection state. ​ 6. The method of claim 5, wherein, ​ The method further includes: After determining the target first deployment unit as the target deployment unit, the target worker node is controlled to send the target build task to the target first deployment unit, so that the Jenkins container in the long link state in the target first deployment unit processes the target build task.

7. A Jenkins-based build task processing apparatus, comprising: a querying module configured to query resource consumption states of a plurality of worker nodes in a container cluster in response to a target build task from a build task distribution platform, to determine a target worker node from the plurality of worker nodes; a first sending module configured to send the target build task to the target worker node, to determine a target deployment unit in the target worker node; a second sending module configured to send a Jenkins image to the target deployment unit, to load the Jenkins image in the target deployment unit to obtain a Jenkins container; and a control module configured to control the target worker node to send the target build task to the target deployment unit, to control the Jenkins container to process the target build task, wherein the Jenkins container does not distinguish between Master nodes and Slave nodes, and all the Jenkins containers are used to process the target build task, and the Jenkins-based build task processing apparatus further comprises: a recycling module configured to control the Jenkins container to stop running and recycle resources occupied by the Jenkins container in a case where the Jenkins container completes processing of the target build task.

8. An electronic device, comprising: one or more processors; a memory configured to store one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors are caused to implement the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to implement the method of any one of claims 1 to 6.

10. A computer program product, comprising computer-executable instructions that, when executed, implement the method of any one of claims 1 to 6.

Images