Code compiling method and system based on Jenkins assembly line
By using containerized compilation clusters and Kubernetes technology in the Jenkins pipeline, containerized compilation node pods with multiple compilation environments are provided, conflicts and resource occupation problems in different locales are solved, low-cost code compilation is achieved, and security is improved sex.
Patent Information
- Application Number
- CN202411208644.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-12-20
AI Technical Summary
The existing code compilation scheme based on Jenkins pipelines has caused environmental conflicts and excessive resource usage due to the mixed use of compilation environments in different languages, resulting in high code compilation costs and security risks caused by business code leakage and abuse of permissions.
The containerized compilation cluster based on Kubernetes is adopted to schedule the compilation node pods through the containerized compilation node, providing different compilation environments, and releasing the pods after compilation is completed, realizing an on-use and freeing compilation environment to avoid the retention of source code and sensitive information.
It reduces the server resources required for code compilation, reduces costs, and effectively avoids the security risks brought about by business code leakage and permission abuse.
Smart Images
Figure CN119166153A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of software development technology, and in particular to a code compilation method and system based on a Jenkins pipeline. Background Art
[0002] Jenkins is an open source, scalable web-based software development and operation platform for continuous integration, delivery, and deployment. The Jenkins pipeline is a method of automating software development workflows that can be used for tasks such as building, compiling, and deploying.
[0003] In the prior art, the solution of implementing code compilation based on Jenkins pipeline is generally used for Figure 1 In the architecture shown, the Jenkins server controls N slave nodes (Slave nodes) as the master node (Master node). Different businesses are developed in different languages. Mixing different language compilation environments will cause environmental conflicts or unexpected errors. Therefore, only one independent compilation environment is generally deployed on a Slave node, such as: Slave1 node provides Java compilation environment, Slave2 node provides mobile terminal compilation environment, Slave3 node provides Golang compilation environment, Slave4 node provides front-end compilation environment, Slave5 node provides embedded compilation environment, etc. Therefore, the Jenkins server needs to connect to a large number of Slave node servers with different compilation environments. A set of continuous integration and continuous deployment CICD system needs to invest and occupy a large amount of server resources, and the code compilation cost is high. Summary of the invention
[0004] In view of this, the present application provides a code compilation method and system based on Jenkins pipeline to reduce the cost of business code compilation.
[0005] The technical solution is as follows:
[0006] In a first aspect, an embodiment of the present application provides a code compilation system based on a Jenkins pipeline, the system being scheduled and executed by a Jenkins pipeline, the system comprising: a Jenkins server, a containerized compilation cluster connected to the Jenkins server, the containerized compilation cluster comprising a plurality of containerized compilation nodes, each of the plurality of containerized compilation nodes being used to schedule a plurality of compilation node Pods; wherein each of the plurality of compilation node Pods being used to provide a different compilation environment.
[0007] In an optional implementation, the containerized compilation cluster is constructed based on Kubernetes virtual container technology.
[0008] In an optional implementation, each of the multiple compilation node Pods is constructed based on a different container image construction instruction Dockerfile, and the container image construction instruction Dockerfile is written based on code compilation environment requirements.
[0009] In an optional implementation, the containerized compiling node is used to build a target compiling node Pod to compile the target business code, and the target compiling node Pod is used to provide a compiling environment for the target business code.
[0010] In an optional implementation, the containerized compiling node is further used to release the target compiling node Pod after completing the compilation of the target business code.
[0011] In a second aspect, an embodiment of the present application provides a code compilation method based on a Jenkins pipeline, the method is applied to any system provided in the first aspect, the method is executed by a Jenkins pipeline scheduling, and the method includes:
[0012] The containerized compiling node obtains a target service code, and the containerized compiling node belongs to the containerized compiling cluster;
[0013] The containerized compiling node constructs a target compiling node Pod based on the target business code, and the target compiling node Pod is used to provide a compiling environment for the target business code;
[0014] The containerized compile node compiles the target business code based on the target compile node Pod.
[0015] In an optional implementation, the containerized compilation cluster is constructed based on Kubernetes virtual container technology.
[0016] In an optional implementation, the containerized compiling node creates a target compiling node Pod based on the target business code, including:
[0017] The containerized compilation node determines a target compilation environment for the target business code;
[0018] The containerized compilation node obtains a target container image corresponding to the target compilation environment, and creates a target compilation node Pod based on the target container image.
[0019] In an optional implementation, the containerized compilation node obtains the target container image corresponding to the target compilation environment, including:
[0020] The containerized compilation node obtains the target container image from the image repository based on the target compilation environment; wherein the image repository includes different container images, and different container images are constructed based on different container image construction instructions Dockerfile, and the container image construction instructions Dockerfile are written based on code compilation environment requirements. In an optional implementation, the method further includes:
[0021] After completing the compilation of the target business code, the containerized compilation node releases the target compilation node Pod.
[0022] The above technical solution has the following beneficial effects:
[0023] A code compilation system based on a Jenkins pipeline is provided in an embodiment of the present application, and the system includes: a Jenkins server, a containerized compilation cluster connected to the Jenkins server, each of the containerized compilation nodes in the containerized compilation cluster is used to schedule multiple compilation node Pods; wherein each of the multiple compilation node Pods is used to provide different compilation environments. In an embodiment of the present application, a containerized compilation form is provided by constructing a containerized compilation cluster, and the compilation of different business codes is realized by scheduling compilation node Pods that support different compilation environments, so that the Jenkins server does not need to connect to a large number of slave node servers, reducing the server resources required for code compilation, thereby reducing the cost of code compilation.
[0024] The embodiment of the present application also provides a method applied to the above system, which has the same beneficial effects as the above system. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] In order to more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0026] Figure 1 A schematic diagram of the architecture of an existing Jenkins pipeline-based code compilation system provided in an embodiment of the present application;
[0027] Figure 2 A schematic diagram of the architecture of a code compilation system based on a Jenkins pipeline provided in an embodiment of the present application;
[0028] Figure 3 A flowchart of making a container image file for a compiling node provided in an embodiment of the present application;
[0029] Figure 4 A flowchart of a code compilation method based on a Jenkins pipeline provided in an embodiment of the present application;
[0030] Figure 5 A flowchart of another code compilation method based on a Jenkins pipeline is provided in an embodiment of the present application. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0032] In order to reduce the code compilation cost in the code compilation scenario of the Jenkins pipeline, the embodiment of the present application provides a code compilation system based on the Jenkins pipeline, referring to Figure 2 The system architecture diagram shown is a system executed by Jenkins pipeline scheduling, and the system includes: a Jenkins server, a containerized compilation cluster connected to the Jenkins server, and an exemplary, Figure 2 The containerized compilation cluster includes: K8s1 containerized compilation node, K8s2 containerized compilation node and K8s3 containerized compilation node. Each containerized compilation node in the multiple containerized compilation nodes is used to schedule multiple compilation node Pods.
[0033] Among them, the containerized compilation cluster is built based on Kubernetes virtual container technology.
[0034] Kubernetes is an open source containerized application for managing multiple hosts in a cloud platform. Build a containerized compilation cluster based on Kubernetes virtual container technology to facilitate the scheduling and use of compilation node Pods in subsequent containerized compilation nodes.
[0035] It should be noted that the embodiment of the present application does not limit the number of containerized compilation nodes contained in the containerized compilation cluster, and the technicians can set it according to the actual business needs. Exemplarily, the number of containerized compilation nodes in the containerized compilation cluster can be set to be greater than or equal to 2, and optionally, the number of containerized compilation nodes in the containerized compilation cluster is 3. It can be seen that the embodiment of the present application adopts a containerized compilation environment, and the original multiple servers used to support different compilation environments can be reduced to at least 3, saving the server resources required for compilation from the cost aspect.
[0036] Among them, each compilation node Pod in the multiple compilation node Pods is used to provide a different compilation environment. Exemplarily, each containerized compilation node can be scheduled to be used as compilation node Pod1 to compilation node PodN, wherein compilation node Pod1 is used to provide a Java compilation environment, compilation node Pod2 is used to provide a mobile terminal compilation environment, compilation node Pod3 is used to provide a Golang compilation environment, compilation node Pod4 is used to provide a front-end compilation environment, compilation node Pod5 is used to provide an embedded compilation environment, and so on.
[0037] Among them, the compilation node Pod can be created, used and released by the containerized compilation node. Specifically, the image repository pre-stores the container image files corresponding to different compilation node Pods. Under the scheduling of the containerized compilation node, the required container image can be directly obtained from the image repository, and the compilation node Pod can be created based on the container image. The compilation node Pod is a minimum deployment unit that Kubernetes combines application containers through abstraction, making the application easier to discover and manage. The compilation node Pod can be one or more containers.
[0038] Reference Figure 3 The flowchart of making a container image file of a compilation node shown includes: after the container development environment is prepared, the corresponding container image building instruction Dockerfile is written for different compilation environments; then, the container images of different compilation environments are packaged based on different Dockerfiles, and container images for different compilation environments are constructed, and finally each container image is stored in the image warehouse.
[0039] Among them, Dockerfile is a text file that contains all the instructions for building a Docker image. Dockerfile defines the image building process through a series of instructions, which describe how to start from the base image, add, modify or install necessary software, configuration, etc., and finally generate a new Docker image. These instructions can be about setting up the working environment, installing software, defining environment variables, etc., to ensure that the image has all the required functions and configurations.
[0040] After the container image file is produced, the image repository stores container images that support various types of compilation environments. The containerized compilation node can subsequently pull these container images to create the required compilation node Pod.
[0041] It should be noted that the image repository provided in the embodiment of the present application can adopt a Docker image repository or a Harbor image repository, and the embodiment of the present application does not limit the type of the image repository.
[0042] Furthermore, in another aspect of the embodiment of the present application, in order to overcome security issues caused by business code leakage or abuse of authority, in the code compilation system based on the Jenkins pipeline provided in the embodiment of the present application, the containerized compilation node releases the target compilation node Pod after completing the compilation of the target business code, and the target compilation node Pod is used to provide a compilation environment for the target business code.
[0043] Exemplarily, the target compile node Pod can be destroyed by the kubectl delete command to release the target compile node Pod, and the status of the target compile node Pod can be checked by the kubectl get pods command to confirm whether the target compile node Pod is correctly released.
[0044] In the prior art, in order to improve the calling efficiency of each compilation node, when configuring the credentials of each slave node, any Jenkins pipeline is usually set to call any slave node, and in order to prevent errors caused by permissions during the compilation process, the execution permissions within the slave node are usually given to the maximum, or even to the root level. However, this leads to the risk of abuse of the slave node credentials. For example, user A who executes a certain Jenkins pipeline can call other slave compilation nodes at will and obtain the business code or output built by user B on the compilation node, which may lead to business code leakage or unauthorized operations that bring security risks.
[0045] Therefore, the containerized compilation form provided in the embodiment of the present application adopts scheduling compilation node Pods that support different compilation environments to realize the compilation of different business codes, and releases the corresponding target compilation node Pod after completing the code compilation, that is, a ready-to-use and ready-to-release code compilation environment is provided, without leaving any source code and sensitive information on the compilation server, which can effectively avoid business code leakage and solve security-related issues caused by credential abuse.
[0046] In summary, the embodiment of the present application provides a code compilation system based on the Jenkins pipeline, the system comprising: a Jenkins server, a containerized compilation cluster connected to the Jenkins server, the containerized compilation cluster comprising multiple containerized compilation nodes, each of the multiple containerized compilation nodes being used to schedule multiple compilation node Pods; wherein each compilation node Pod in the multiple compilation node Pods is used to provide different compilation environments. It can be seen that in the embodiment of the present application, by constructing a containerized compilation cluster, a containerized compilation form is provided, and the compilation of different business codes is realized by scheduling compilation node Pods that support different compilation environments, so that the Jenkins server does not need to connect to a large number of Slave node servers, reducing the server resources required for code compilation, thereby reducing the cost of code compilation, and the embodiment of the present application provides a ready-to-use and release code compilation environment, which effectively avoids security issues caused by business code leakage or abuse of authority.
[0047] Applied to the above system, the embodiment of the present application also provides a code compilation method based on the Jenkins pipeline, which is executed by the Jenkins pipeline scheduling, referring to Figure 4 A flow chart of the method is shown, and the method may include:
[0048] Step S401: a containerized compilation node obtains a target business code, and the containerized compilation node belongs to a containerized compilation cluster.
[0049] Specifically, the Jenkins pipeline schedules the Jenkins server to obtain the business code to be compiled, that is, the target business code, from the business code repository, and pushes the target business code to the containerized compilation node. It should be noted that the embodiment of the present application does not limit the way in which the Jenkins server pushes the code to the containerized compilation cluster. When the Jenkins server pushes the target business code to the containerized compilation node, it can push according to the load balancing rules set in the containerized compilation cluster, and push the target business code to the containerized compilation node in the containerized compilation cluster that complies with the load balancing rules; it can also push according to preset rules, for example, the Jenkins server pushes the target business code to the containerized compilation node that is closest.
[0050] Step S402: The containerized compile node creates a target compile node Pod based on the target business code, and the target compile node Pod is used to provide a compile environment for the target business code.
[0051] Next, the Jenkins pipeline schedules the containerized compilation node to create a target compilation node Pod based on the compilation environment of the target business code. The target compilation node Pod can provide the compilation environment of the target business code.
[0052] In an optional implementation, the containerized compilation cluster is built based on the Kubernetes virtual container technology.
[0053] In an optional implementation, the above step S402 specifically includes: the containerized compilation node determines the target compilation environment of the target business code; the containerized compilation node obtains the target container image corresponding to the target compilation environment, and creates a target compilation node Pod based on the target container image.
[0054] Specifically, the containerized compilation node first determines the target compilation environment of the target business code, and obtains the target container image from the image repository based on the target compilation environment; wherein the image repository includes different container images, and different container images are constructed based on different container image construction instructions Dockerfile, and the container image construction instructions Dockerfile are written based on the code compilation environment requirements.
[0055] Step S403: The containerized compile node compiles the target business code based on the target compile node Pod.
[0056] The target compilation node Pod can provide a compilation environment for the target business code, and thus the containerized compilation node can implement the compilation of the target business code based on the created target compilation node Pod.
[0057] In an optional implementation, the Jenkins pipeline-based code compilation method provided in the embodiment of the present application may also include: after the containerized compilation node completes the compilation of the target business code, releasing the target compilation node Pod.
[0058] Specifically, after the containerized compilation node completes the compilation of the target business code, it releases the target compilation node Pod. Therefore, the containerized compilation form provided in the embodiment of the present application uses the compilation node Pod that supports different compilation environments to implement the compilation of different business codes, and releases the corresponding target compilation node Pod after the code compilation is completed, that is, a ready-to-use and ready-to-release code compilation environment is provided, without leaving any source code and sensitive information on the compilation server, which can effectively avoid the leakage of business codes and solve the security-related problems caused by the abuse of credentials.
[0059] In an optional implementation, the code compilation method based on the Jenkins pipeline provided in the embodiment of the present application may also include: after the containerized compilation node completes the compilation of the target business code, it obtains the target business package and pushes the target business package to the business package warehouse for storage.
[0060] It should be noted that the steps and related technical features in the code compilation based on the Jenkins pipeline provided in the embodiment of the present application correspond to the system provided in the embodiment of the application. The description of the method part can be found in the embodiment of the aforementioned system part and will not be repeated here.
[0061] Combine the following Figure 5 A flow chart of another code compilation method based on Jenkins pipeline is shown, and the method is executed by Jenkins pipeline scheduling, and the method includes:
[0062] First, the Jenkins pipeline schedules the Jenkins server to pull the business code from the code repository. The business code is the code to be compiled; then, the Jenkins server pushes the business code to the containerized compilation node, and the containerized compilation node creates a containerized compilation node Pod based on the compilation environment of the business code; then, the containerized compilation node compiles the business code based on the created containerized compilation node Pod, obtains the business package, and pushes the business package to the business package warehouse for storage; finally, the containerized compilation node Pod is released, thereby completing the compilation of the business code.
[0063] As can be seen from the above, in the embodiment of the present application, a containerized compilation cluster is constructed to provide a containerized compilation form, and the compilation of different business codes is realized by scheduling compilation node Pods that support different compilation environments. Therefore, the Jenkins server does not need to connect to a large number of Slave node servers, reducing the server resources required for code compilation, thereby reducing the code compilation cost. In addition, the embodiment of the present application provides a ready-to-use and release code compilation environment, which effectively avoids security issues caused by business code leakage or abuse of authority.
[0064] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0065] Those skilled in the art can understand that the flowchart shown in the figure is only an example in which the embodiments of the present application can be implemented, and the scope of application of the embodiments of the present application is not limited by any aspect of the flowchart.
[0066] In the several embodiments provided in the present application, it should be understood that the disclosed methods, devices and equipment can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.
[0067] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0068] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0069] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A code compilation system based on Jenkins pipeline, characterized by: The system is scheduled and executed by the Jenkins pipeline. The system includes: a Jenkins server, a containerized compilation cluster connected to the Jenkins server, the containerized compilation cluster including multiple containerized compilation nodes, each of the multiple containerized compilation nodes is used to schedule multiple compilation node Pods; wherein each of the multiple compilation node Pods is used to provide a different compilation environment.
2. The system according to claim 1, wherein: The containerized compilation cluster is built based on Kubernetes virtual container technology.
3. The system according to claim 1 or 2, characterized in that Each of the multiple compilation node Pods is constructed based on a different container image construction instruction Dockerfile, and the container image construction instruction Dockerfile is written based on code compilation environment requirements.
4. The system according to claim 3, characterized in that The containerized compilation node is used to build a target compilation node Pod to compile the target business code, and the target compilation node Pod is used to provide a compilation environment for the target business code.
5. The system according to claim 4, characterized in that The containerized compilation node is further configured to release the target compilation node Pod after completing compilation of the target business code.
6. A code compilation method based on Jenkins pipeline, the method being applied to the system according to any one of claims 1 to 5, characterized in that: The method is executed by Jenkins pipeline scheduling, and the method includes: The containerized compilation node obtains the target business code, and the containerized compilation node belongs to the containerized compilation cluster; The containerized compilation node creates a target compilation node Pod based on the target business code, and the target compilation node Pod is used to provide a compilation environment for the target business code; The containerized compile node compiles the target business code based on the target compile node Pod.
7. The method according to claim 6, characterized in that The containerized compilation cluster is built based on Kubernetes virtual container technology.
8. The method according to claim 6 or 7, characterized in that The containerized compilation node creates a target compilation node Pod based on the target business code, including: The containerized compilation node determines a target compilation environment for the target business code; The containerized compilation node obtains a target container image corresponding to the target compilation environment, and creates a target compilation node Pod based on the target container image.
9. The method according to claim 8, characterized in that The containerized compilation node obtains a target container image corresponding to the target compilation environment, including: The containerized compilation node obtains the target container image from the image repository based on the target compilation environment; wherein, the image repository includes different container images, and different container images are constructed based on different container image construction instructions Dockerfile, and the container image construction instructions Dockerfile is written based on the code compilation environment requirements.
10. The method according to claim 8, characterized in that The method further comprises: After completing the compilation of the target business code, the containerized compilation node releases the target compilation node Pod.