A container-based continuous integration method and system

Abstract

The application provides a container-based continuous integration method and system, and a target image is generated by automatically executing docker commit through a shell script, the preparation of a base image of the device realizes an automatic generation process from source code to a docker image, is suitable for all application programs written in go, has high reusability, and provides a basic guarantee for continuous integration and continuous deployment (CI / CD). The application adopts a continuous integration mode, verifies through script automatic construction, so that integration errors can be found as soon as possible, integration problems are greatly reduced, and a team can develop cohesive software more quickly.

Description

Technical Field

[0001] This application relates to the fields of development and continuous integration technology, and specifically to a container-based continuous integration method and system. Background Technology

[0002] With the rapid development of cloud computing and blockchain technologies, the application of the Go programming language is becoming increasingly widespread. Go, with its advantages of high concurrency, simple deployment, excellent language design, and high execution performance, has been rapidly adopted in many fields, especially server programming, distributed systems, databases, network programming, in-memory databases, and cloud platforms. To further improve developer efficiency, I researched and implemented this containerized continuous integration (CI) device. This device automatically triggers compilation and deployment functions after code is committed to a GitHub repository, ultimately generating a Docker image usable in the production environment. Using this CI device allows developers to focus more on the development of specific business functions, reducing errors from manual operations and environmental impacts during compilation and packaging, and ensuring the quality and efficiency of service submissions.

[0003] As software projects become more complex, more demands are placed on continuous integration. Early, frequent, and regular integration helps projects identify risks and quality issues early on. If these problems are only discovered later, the cost of resolving them is very high, and it is very likely to lead to project delays or project failure. Summary of the Invention

[0004] To address the aforementioned technical issues, this application proposes a container-based continuous integration method and system, focusing on how a Go language program can implement a continuous integration mechanism. This application employs continuous integration, using script-automated build processes (including compilation, deployment, and automated testing) for verification, thereby quickly identifying integration errors, significantly reducing integration problems, and enabling teams to develop cohesive software more rapidly. The technical solution adopted in this application is as follows:

[0005] A container-based continuous integration method includes the following steps:

[0006] Step 1: Obtain the source code of the Go application and include the dependency packages using the vendor tool;

[0007] Step 2: Generate the build image "golang builder" using the assemble script in the generalized build script and the base image;

[0008] Step 3: Run the compilation image golang builder, inject the golang application source code into the container, and use the s2i script to install the dependency packages and complete the compilation to generate a runtime image of the golang application source code;

[0009] Step 4: Run the runtime image directly on Docker or a container cloud platform.

[0010] Furthermore, in step 1, the vendor tool is used to prepare the dependency packages of the Go application and download them to the local machine. The vendor tool automatically pulls the source code of the Go application from the code repository through the built-in git program and archives it to the specified directory for later compilation.

[0011] Furthermore, in step 2, the generalized compilation script is placed into the compilation image, and the generalized compilation script is searched and executed in the order of absolute path within the image, relative path on the host, and file on the network.

[0012] Furthermore, the generalized compilation script includes necessary scripts and optional scripts. The necessary scripts include the assemble script and the run script, and the optional scripts include the save-artifacts script, the usage script, and the test / run script.

[0013] Furthermore, in step 3, a runtime image of the Go application source code is generated, including the following steps:

[0014] Step 301: Download the s2i script;

[0015] Step 302: Determine whether to perform incremental compilation. If yes, save intermediate dependencies; otherwise, proceed to step 303.

[0016] Step 303: Download the Go application source code and create a tarball;

[0017] Step 304: Run the compilation and build process to generate a runtime image;

[0018] Step 305: Determine if a runtime error has occurred. If yes, import the Go application source code and the s2i script, and return to step 304; otherwise, proceed to step 306.

[0019] Step 306: Submit the generated runtime image.

[0020] A container-based continuous integration system for implementing the above method, characterized in that the system includes a source code acquisition module, a compilation image module, and an automation script module;

[0021] The source code retrieval module is used to retrieve the source code of Golang applications;

[0022] The compilation image module is used to generate the compilation image "golang builder" from the assemble script in the generalized compilation script and the base image;

[0023] The automation script module is used to prepare various control scripts for the compilation process and provide these control scripts to the compilation image module.

[0024] Furthermore, the source code acquisition module uses a vendor tool to prepare the dependency packages of the Go application and download them to the local machine. The vendor tool automatically pulls the source code of the Go application from the code repository through the built-in Git program and archives it to a specified directory for later compilation.

[0025] Furthermore, the compilation image module places the generalized compilation script into the compilation image, and searches for and executes the generalized compilation script in the order of absolute path within the image, relative path on the host, and file on the network.

[0026] Furthermore, the various control scripts include: generalized compilation scripts, security processing scripts, assembly scripts, image build scripts, and image push scripts.

[0027] Furthermore, the generalized compilation script includes necessary scripts and optional scripts. The necessary scripts include the assemble script and the run script, and the optional scripts include the save-artifacts script, the usage script, and the test / run script.

[0028] The following technical effects can be achieved through the embodiments of this application:

[0029] (1) Standardization: Establish a standard Go language compilation environment, standardize the development process, and use best practices to complete the compilation and release process of Go language;

[0030] (2) Fast speed: This builder image eliminates the need to compile code, write Dockerfile, and build Docker image, directly generating an image containing a runnable program;

[0031] (3) High efficiency: It can prevent developers from performing arbitrary yum installation type operations during the building of their applications, which would slow down the development iteration. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A schematic diagram of the continuous integration process;

[0034] Figure 2 A flowchart illustrating the process of working with mirroring. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0036] Currently, common continuous integration tools and methods include Jenkins and GitLab CI. Jenkins is the most widely used continuous integration solution, but it cannot have tasks requiring agent switching in the build plan. GitLab CI, on the other hand, supports configuring triggers on the project for scheduling and can be run via shell, SSH, Docker, etc. However, GitLab CI has a significant disadvantage: configuring stages and jobs via YAML requires learning additional file syntax, making it less user-friendly.

[0037] To address the aforementioned issues in existing technologies, this application provides a container-based continuous integration method. Programmers submit code to a Git repository, which automatically triggers the compilation process and ultimately generates a Docker image containing an executable program. This achieves a structured and standardized software development process, allowing programmers to focus on code logic and functionality implementation.

[0038] Before describing the technical solution of this application in detail, let's look at the process of generating a Docker image. Currently, there are two main ways to create application images:

[0039] (1) Create using Docker file: The traditional method of creating an image is to write your own Docker file, use a base image, copy the source code, use the RUN command to install the dependency package, use the shell command to compile the program, specify the executable program in the CMD name, and finally use docker build to generate the image;

[0040] (2) Create using docker run and docker commit: Run the base image with docker run, and then use the interactive shell to commit the container to generate the image.

[0041] The purpose of this application is to automate the building of Docker images. Therefore, this application adopts the second method mentioned above, that is, to create images using `docker run` and `docker commit`, and to automatically execute `docker commit` via a shell script to generate new images. In this continuous integration process, there are three types of images:

[0042] (1) Base image: The base image is the official Go language image downloaded from Docker Hub. The base image contains the language's compilation environment but has no other functions.

[0043] (2) Compiled image: The compiled image is generated by Docker file on the basis of the base image. It inherits the base image and can automatically compile code and automatically complete deployment and operation.

[0044] (3) Runtime image: The runtime image is built by compiling and executing scripts after the compilation work has been completed. It contains the compiled executable program and can be started by running the container directly.

[0045] Continuous integration process such as Figure 1 As shown, this continuous integration process includes four steps. The first step corresponds to the base image among the three types of images mentioned above, the second step corresponds to the compilation image, the fourth step corresponds to the runtime image, and the third step is the runtime image production process. This production process will... Figure 2 A detailed description will be provided. This includes the following steps:

[0046] Step 1: Obtain the source code of the Go application and include the dependency packages using the vendor tool;

[0047] In step 1, the base image information is recorded in the Dockerfile file within the obtained Go application source code. This base image is usually provided by the official Docker repository.

[0048] Step 2: Generate the build image "golang builder" using the assemble script in the generalized build script and the base image;

[0049] In step 2, the compilation image golang builder serves as the carrier for the compilation process of the golang application source code, containing the compilation and runtime environment of the application source code.

[0050] Step 3: Run the compilation image golang builder, inject the golang application source code into the container, and use the s2i script to install the dependency packages and complete the compilation to generate a runtime image of the golang application source code;

[0051] In step 3, the runtime image is a new image automatically generated by the compiled image Golang Builder, used to run the application.

[0052] The generalized build script is a part of the s2i script. In addition to compilation, the s2i script also has multiple function scripts such as assembly, packaging, and automatic uploading. The generalized build script is only used for program compilation and controls the build process. It has been generalized to adapt to different types of programs.

[0053] Step 4: Run the runtime image directly on Docker or a container cloud platform;

[0054] In the steps described above, the runtime image is the target image, i.e., the final delivered image, whose generation includes three elements: the compiled image, the application source code, and the automation scripts.

[0055] The continuous integration system includes a source code acquisition module, a compilation image module, and an automation script module, with each module having corresponding functional scripts;

[0056] The source code retrieval module is used to retrieve the source code of Golang applications;

[0057] The compilation image module is used to generate the compilation image "golang builder" from the assemble script in the generalized compilation script and the base image;

[0058] The automation script module is used to prepare various control scripts for the compilation process and provide these control scripts to the compilation image module.

[0059] The source code acquisition module uses a vendor tool to prepare and download the dependency packages of the Go application to the local machine. The vendor tool automatically pulls the Go application source code from the code repository using its built-in Git program and archives it to a specified directory for later compilation. This source code acquisition module acts as an initialization module for dependency packages and the Go application source code, ensuring that the source code can be compiled successfully.

[0060] The compilation image module is a process control module that completes various stages such as program building, security processing, image building, and image pushing according to pre-designed steps. The compilation image module places a generalized compilation script into the compilation image and searches for and executes the script according to the order of absolute path within the image, relative path on the host, and file on the network.

[0061] The automation script module prepares various control scripts for the compilation process, used to implement the functions of each build step. These control scripts include: a generalized compilation script, a security handling script, an assembly script, an image build script, and an image push script.

[0062] The generalized compilation script includes necessary scripts and optional scripts. The necessary scripts include the assemble script and the run script. The optional scripts include the save-artifacts script, the usage script, and the test / run script.

[0063] (1)assemble script

[0064] The `assemble` script is used to compile the source code of a Go application, generate a runtime image, and place the runtime image in the appropriate directory for the target image to call when it starts.

[0065] The workflow of the assemble script is as follows:

[0066] • Save the parts that need to be reused during compilation, so that these files can be used directly in the next compilation, thereby speeding up the compilation and installation process;

[0067] • Pull the source code of the Go application into the appropriate directory within the Docker container for use by the compiler;

[0068] • Generate a runtime image by compiling a program, install the runtime image, and use it when the target image starts.

[0069] The user used for installation and compilation in the assemble script is a different user from the user who will run the application. The user who will run the application is specified in the Dockerfile and is generally required to be a non-root user.

[0070] (2)run script

[0071] The run script defines the information of the executable program to be run. When the Docker container starts, the run script is automatically called and the commands in the run script are executed.

[0072] (3)save-artifacts script

[0073] The save-artifacts script is used to collect intermediate dependency packages or temporary dependency files, which can significantly improve the compilation speed of the main program.

[0074] (4)usage script

[0075] The usage script is used to display help information. By writing the output information during the compilation process or the image usage information into the usage script, the usage script will be automatically called to display help information after the target image is built.

[0076] (4) test / run script

[0077] The test / run script is used to start a process to check whether the target image is running normally.

[0078] The workflow of the test / run script is as follows:

[0079] • Create a compilation image;

[0080] • Run the compilation image to verify that the usage script and the assemble script are functioning correctly.

[0081] • Run the build image again to confirm that the save-artifacts script is working correctly;

[0082] Run the final runtime image to verify that the application is running correctly.

[0083] The mirroring workflow is as follows Figure 2 As shown. In step 3, a runtime image of the Go application source code is generated, including the following steps:

[0084] Step 301: Download the s2i script;

[0085] Step 302: Determine whether to perform incremental compilation. If yes, save intermediate dependencies; otherwise, proceed to step 303.

[0086] Step 303: Download the Go application source code and create a tarball;

[0087] Step 304: Run the compilation and build process to generate a runtime image;

[0088] Step 305: Determine if a runtime error has occurred. If yes, import the Go application source code and the s2i script, and return to step 304; otherwise, proceed to step 306.

[0089] Step 306: Submit the generated runtime image.

[0090] In summary, this application completes a continuous integration (CI) mechanism for the Go language, enabling continuous integration through the integration of GitHub and Docker images. It also completes the creation of the compiled image, employing different user roles during the build and runtime phases, particularly emphasizing the requirement to run as a non-root user, ensuring successful image building and subsequent security and reliability. The runtime script provides specialized project processing algorithms, including multiple methods for obtaining project variable names, guaranteeing the stable operation of the delivered product. The base image creation in this application automates the process from source code to Docker image generation. This technology is applicable to all applications written in Go, offering high reusability and providing a fundamental guarantee for continuous integration and continuous deployment (CI / CD). Furthermore, this application boasts excellent versatility, automatically recognizing project names and centrally managing dependencies through vendors. It fully supports various types of microservices or applications, completing continuous integration without configuring various parameters, significantly improving the software release process and increasing release efficiency.

[0091] While specific embodiments of this application have been described above, those skilled in the art should understand that these are merely illustrative examples, and the scope of protection of this application is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principles and essence of this application, but all such changes and modifications fall within the scope of protection of this application.

Claims

1. A container-based continuous integration method, characterized in that, Includes the following steps: Step 1: Obtain the source code of the Go application and include the dependency packages using the vendor tool; Step 2: Generate the build image golangbuilder using the assemble script in the generalized build script and the base image; Step 3: Run the compilation image golang builder, inject the golang application source code into the container, and use the s2i script to install the dependency packages and complete the compilation to generate a runtime image of the golang application source code; Step 4: Run the runtime image directly on Docker or a container cloud platform; In step 2, the generalized compilation script is placed into the compilation image, and the generalized compilation script is searched and executed in the order of absolute path in the image, relative path on the host, and file on the network. The generalized compilation script includes necessary scripts and optional scripts. The necessary scripts include the assemble script and the run script, and the optional scripts include the save-artifacts script, the usage script, and the test / run script.

2. The method according to claim 1, characterized in that, In step 1, the vendor tool is used to prepare the dependency packages of the Go application and download them to the local machine. The vendor tool automatically pulls the source code of the Go application from the code repository through the built-in git program and archives it to the specified directory for later compilation.

3. The method according to claim 1, characterized in that, In step 3, a runtime image of the Go application source code is generated, including the following steps: Step 301: Download the s2i script; Step 302: Determine whether to perform incremental compilation. If yes, save intermediate dependencies; otherwise, proceed to step 303. Step 303: Download the Go application source code and create a tarball; Step 304: Run the compilation and build process to generate a runtime image; Step 305: Determine if a runtime error has occurred. If yes, import the Go application source code and the s2i script, and return to step 304; otherwise, proceed to step 306. Step 306: Submit the generated runtime image.

4. A container-based continuous integration system for implementing the method as described in any one of claims 1 to 3, characterized in that, The system includes: The source code retrieval module is used to retrieve the source code of Golang applications; The build image module is used to generate the build image golang builder from the assemble script in the general build script and the base image; The automation script module is used to prepare control scripts for the compilation process and provide these control scripts to the compilation image module.

5. The system according to claim 4, characterized in that, The source code acquisition module uses the vendor tool to prepare the dependency packages of the Go application and download them to the local machine. The vendor tool automatically pulls the source code of the Go application from the code repository through the built-in Git program and archives it to the specified directory for later compilation.

6. The system according to claim 4, characterized in that, The compilation image module places the generalized compilation script into the compilation image and searches for and executes the generalized compilation script in the order of absolute path within the image, relative path on the host, and file on the network.

7. The system according to claim 4, characterized in that, The control scripts include: a generalized compilation script, a security processing script, an assembly script, an image build script, and an image push script.

8. The system according to claim 7, characterized in that, The generalized compilation script includes necessary scripts and optional scripts. The necessary scripts include the assemble script and the run script, and the optional scripts include the save-artifacts script, the usage script, and the test / run script.

Images