A Jenkins-based software compiling method, system, device and storage medium

Abstract

This invention relates to the field of software development technology, specifically to a software compilation method, system, device, and storage medium based on Jenkins. The method includes: pre-setting a manifest file describing the project structure and building a base build image encapsulating a complete compilation environment; pulling the full code of each project to a physical server; triggering an automated compilation task when a commit event occurs in the code repository; the task updating the manifest file according to the commit information and generating an updated manifest file with a sequence number; performing incremental pulls on the physical server for the changed projects based on the updated manifest file; synchronizing the code to a container environment started by the base build image via a local Git HTTP server; and finally performing automated compilation within the container. This invention can significantly improve the compilation efficiency of large projects, completely eliminate environmental differences, and enhance the stability and reliability of the automated process.

Description

Technical Field

[0001] This invention relates to the field of software development technology, and specifically to a software compilation method, system, device, and storage medium based on Jenkins. Background Technology

[0002] In the software development process, the compilation stage is crucial, as it is responsible for transforming human-readable high-level programming language source code into machine-executable target code. Traditional compilation work heavily relies on manual operations, including configuring the compilation environment, writing compilation scripts, and manually triggering the compilation process. This not only consumes a lot of time and effort but is also prone to compilation failures or the generation of defective products due to human negligence (such as improper environment configuration or script errors).

[0003] To address these issues, automated compilation technology has emerged. Automated compilation, leveraging specific tools and scripts (such as Jenkins and CMake), standardizes the "code submission → automatic compilation → artifact output" process, automating the compilation task. Typical existing automated compilation solutions usually consist of event triggering by a version control system (such as Git) and execution by build tools. However, existing automated compilation technologies, especially when handling large, multi-module projects, still have the following significant drawbacks:

[0004] (1) Inefficient and unstable project retrieval: Existing solutions typically require a full clone or retrieval of the entire project or its repository when triggering compilation. For large projects, this process is extremely time-consuming. At the same time, the retrieval operation often relies directly on accessing the version control server via the public network, and network fluctuations can easily cause retrieval failures, thereby interrupting the entire compilation process.

[0005] (2) Difficulty in ensuring consistency of the compilation environment: Existing solutions mostly rely on the compilation toolchain and dependency libraries that are "pre-installed on the host machine" or are temporarily installed before compilation using simple scripts. This approach makes it difficult to ensure the consistency of the compilation environment on different machines and at different times, which can easily lead to the problem of "it can be compiled on my machine" due to environmental differences, resulting in unreliable compilation results.

[0006] Therefore, there is an urgent need for an automated compilation method that can achieve high efficiency, stability, and environmental consistency to overcome the shortcomings of existing technologies. Summary of the Invention

[0007] One of the objectives of this invention is to provide a software compilation method based on Jenkins, which solves the problems of low project retrieval efficiency, compilation failure due to inconsistent environments, and unreliability of the process caused by network fluctuations in the existing automated compilation process.

[0008] To achieve the above objectives, a software compilation method based on Jenkins is provided, including the following steps:

[0009] S1. Based on the dependencies of the project to be compiled, prepare a manifest file describing the structure of the project to be compiled;

[0010] S2. Build a base build image that includes a pre-built build environment;

[0011] S3. Retrieve all items from the list file to a fixed directory on the physical server.

[0012] S4. Configure the triggering mechanism for submission events based on the items in the manifest file;

[0013] S5. When the triggering mechanism is activated, an automated compilation task is triggered, and the following process is executed:

[0014] S51. Based on the commit event, update the manifest file to generate an updated manifest file with serial numbers;

[0015] S52. Based on the updated manifest file, perform incremental fetch operations on the relevant items in the fixed directory;

[0016] S53. Synchronize the project code in the fixed directory to the container environment started by the base compilation image via a local file server;

[0017] S54. In the container environment, execute the automated compilation process.

[0018] Furthermore, the manifest file is an XML-formatted Manifest file used to record the repository addresses, branches, and commit hash information of multiple projects.

[0019] Furthermore, in step S2, building a base compilation image containing the compilation environment based on a Dockerfile specifically includes: defining the base image in the Dockerfile and pre-installing the software packages and third-party libraries required by the project, and generating the base compilation image through a build command.

[0020] Furthermore, the commit event is a response to a commit event in the code repository, specifically implemented by configuring a Webhook for the project in the manifest file in the version control system. When a code push event occurs, the Webhook triggers the automated compilation task.

[0021] Furthermore, the automated compilation task is a Jenkins task; the update of the manifest file in step S1 specifically includes: the Jenkins task traverses the commit information of all projects in the manifest file and compares it with the historical project information stored in a database to generate the updated manifest file.

[0022] Furthermore, the local file server is equipped with a Git HTTP server, which hosts code pulled from the code repository and provides code synchronization services based on the HTTP protocol.

[0023] Furthermore, in step S53, parallel compilation technology is used to perform compilation within the container environment.

[0024] A second objective of this invention is to provide a Jenkins-based software compilation system that utilizes the Jenkins-based software compilation method described above, comprising the following modules:

[0025] Pre-built modules: These are used to prepare manifest files describing the structure of the project to be compiled, based on the project's dependencies; they are also used to build a base build image that includes a pre-built build environment.

[0026] Code synchronization module: used to pull all projects to a fixed directory on the physical server based on the manifest file;

[0027] Trigger mechanism configuration module: used to configure the trigger mechanism for submission events based on the items in the manifest file;

[0028] Automated compilation module: Used to trigger an automated compilation task and execute the following process when the triggering mechanism is activated:

[0029] Based on the submission event, update the manifest file to generate an updated manifest file with serial numbers;

[0030] Based on the updated manifest file, perform incremental fetch operations on the relevant items in a fixed directory;

[0031] The project code in the fixed directory is synchronized to the container environment started by the base build image via a local file server;

[0032] S54. In the container environment, execute the automated compilation process.

[0033] A third objective of this invention is to provide an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a Jenkins-based software compilation method as described above.

[0034] The fourth objective of this invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a Jenkins-based software compilation method as described above.

[0035] Principles and advantages:

[0036] 1. Regarding compilation efficiency, this invention fundamentally changes the traditional fully fetched code model in automated compilation by introducing manifest file management and a local code repository synchronization mechanism. When a code commit event is triggered, the system only needs to perform incremental fetching and synchronization based on the updated manifest file, greatly reducing network data transmission. Combined with the complete compilation toolchain and dependency libraries pre-built into the container image, the time spent on configuring and installing the environment before each compilation is eliminated. Furthermore, implementing parallel compilation technology in a containerized, isolated compilation environment significantly shortens the overall compilation and build time for large projects or multi-module projects with complex dependencies, thereby accelerating the integration and delivery cycle of software development.

[0037] 2. Regarding environmental consistency and reliability, this invention achieves absolute consistency across development, testing, and production stages by completely encapsulating the compilation environment within a standardized container image. This "build once, run anywhere" approach completely eliminates the classic problem of "it compiles on my machine" caused by differences in host environments, ensuring the determinism and reproducibility of compiled outputs. Simultaneously, since the main code synchronization operations occur between fixed servers on the intranet or local machine, transmitted via an efficient local file server protocol, it effectively avoids the risks of fetch failures or timeouts caused by directly relying on unstable external network connections, greatly improving the robustness and success rate of the entire automated compilation process.

[0038] 3. Regarding process automation and maintainability, this invention deeply integrates the Webhook mechanism of continuous integration tools and version control systems, achieving seamless, fully automated connection from code submission to compilation triggering. This solution not only reduces the frequency and intensity of manual intervention in the compilation process, lowering the risk of human error, but also centrally manages project dependency versions through manifest files, making the maintenance and updates of complex project structures clearer and more convenient. The entire system architecture is clear, and the responsibilities of each component are well-defined, providing stable, efficient, and easily maintainable infrastructure support for the continuous integration and continuous delivery practices of large software development teams. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating a software compilation method based on Jenkins according to an embodiment of the present invention. Detailed Implementation

[0040] The following detailed description illustrates the specific implementation method:

[0041] Example

[0042] A software compilation method based on Jenkins, basically as follows: Figure 1 As shown, it includes the following steps:

[0043] S1. Based on the dependencies of the project to be compiled, prepare a manifest file describing the structure of the project to be compiled; the manifest file is an XML format Manifest file used to record the repository addresses, branches, and commit hash information of multiple projects.

[0044] Example Scenario: Suppose there is an e-commerce platform called "ShopPlatform" that adopts a microservice architecture, consisting of multiple independent sub-projects (services), including user-service, product-service, order-service, and a shared common-lib library. All project code is hosted on the company's internal GitLab server.

[0045] Create the manifest file (repo-manifest.xml):

[0046] Create a dedicated "configuration repository" on GitLab to store the manifest file that manages the entire platform. This file is an XML-formatted Manifest file that explicitly lists the repository addresses, branches, and commit hashes used to lock versions for all subprojects (initially empty or pointing to the latest).

[0047] S2. Construct a base compilation image containing a pre-configured compilation environment; In step S2, a base compilation image containing a compilation environment is constructed based on a Dockerfile, specifically including: defining a base image in the Dockerfile and pre-installing the software packages and third-party libraries required by the project, and generating the base compilation image through a build command.

[0048] In this embodiment, the base build image (Dockerfile.compile) is constructed:

[0049] The configuration repository also contains a Dockerfile that defines the build environment. This file specifies a lightweight Linux base image and pre-installs all necessary build toolchains (such as GCC, Maven, Go, etc.), system dependencies, and third-party libraries required by the entire project.

[0050] The operations and maintenance personnel used the command `docker build -t shop-platform-builder:latest -fDockerfile.compile .` to build a compiled image named `shop-platform-builder` and pushed it to the company's private Docker repository.

[0051] S3. Retrieve all items from the list file to a fixed directory on the physical server.

[0052] Here is an example of a full fetch:

[0053] On an internal network server (referred to as the "synchronization server") with large-capacity storage and a stable network, use the repo tool or write a script to clone all ShopPlatform sub-projects to a fixed directory, such as / data / git-mirrors / shop-platform / , based on the repo-manifest.xml file. This directory will serve as the local mirror source for subsequent incremental synchronizations.

[0054] S4. Configure a triggering mechanism for commit events based on the items in the manifest file; the commit event is a response to the commit event of the code repository, specifically implemented by configuring a Webhook for the items in the manifest file in the version control system. When a code push event occurs, the Webhook triggers the automated compilation task in step S5.

[0055] Configure GitLab Webhook:

[0056] On GitLab, set up Push Events Webhooks for the repositories of four projects: user-service, product-service, order-service, and common-lib. The webhook URL points to a specific address on the company's internal Jenkins server, which is used to trigger a Jenkins task named shop-platform-incr-build.

[0057] S5. When the triggering mechanism is activated, an automated compilation task is triggered (the automated compilation task is a Jenkins task).

[0058] The triggering mechanism is shown in the following example:

[0059] When a developer pushes a new commit to the main branch of product-service (assuming the commandash is abc123), GitLab immediately sends a POST request to the default Jenkins Webhook URL, carrying the details of the push.

[0060] After the automated compilation task is triggered, the following process is executed:

[0061] S51. Based on the commit event, update the manifest file to generate an updated manifest file with a serial number; after the Jenkins task shop-platform-incr-build is triggered, execute the following core logic:

[0062] Event parsing: Jenkins parses the Webhook load and finds that the changed project is product-service, the new commit hash is abc123, and the branch is main.

[0063] Information comparison: The Jenkins task reads the baseline repo-manifest.xml in the configuration repository and queries a project information database (which can be a relational database or a simple file record) to obtain the commit hash recorded when each project was last successfully compiled in the file.

[0064] The updated manifest file: The task identified a change in the product-service version, while the versions of other projects recorded in the database remained consistent with the baseline manifest and unchanged. Therefore, it generated a new manifest file with a unique serial number, such as repo-manifest-20231027-001.xml. This file, based on the baseline manifest, precisely identifies the product-service version as abc123 and generates a serial number 001 for this compilation.

[0065] S52. Based on the updated manifest file, perform incremental fetch operations on the relevant items in a fixed directory; an example of incremental fetching is as follows:

[0066] The Jenkins task connects to the "synchronization server (physical server)" via SSH. Within a fixed directory on this server ( / data / git-mirrors / shop-platform / ), it performs `git fetch` and `git checkout abc123` operations only on the `product-service` project, achieving incremental fetching. Other projects, whose versions remain unchanged, require no network fetching, significantly saving time and bandwidth.

[0067] S53. Synchronize the project code in the fixed directory to the container environment started by the base build image via a local file server; the local file server has a Git HTTP server deployed on it to host the code pulled from the code repository and provide code synchronization services based on the HTTP protocol. git-http-server is a specific software implementation of the "local GitHTTP server".

[0068] Synchronize to the container environment via git-http-server:

[0069] On the "sync server", start git-http-server or other equivalent static file server to expose the / data / git-mirrors / shop-platform / directory via HTTP (e.g., http: / / sync-server.internal.com / git-mirror / ).

[0070] The Jenkins task starts a new Docker container using the same image that was previously built: shop-platform-builder:latest.

[0071] Inside the container, the `repo` tool is used, guided by `repo-manifest-20231027-001.xml`, to synchronize code from `http: / / sync-server.internal.com / git-mirror / `. Because the source is a high-speed internal server, and only actually changed files are synchronized, this process is very fast and unaffected by external network fluctuations.

[0072] S54. In the container environment, execute the automated compilation process.

[0073] Once the code synchronization is complete, the / workspace directory inside the Docker container will contain the complete code for all projects in the specified version.

[0074] The build script (such as build.sh) is invoked. This script may first compile common-lib, and then use commands such as make-j4 or mvn -T 4 to compile user-service, product-service, and order-service in parallel.

[0075] The entire compilation process runs in a container that is isolated from the host machine and has a completely consistent environment (as guaranteed by the shop-platform-builder image), ensuring the reliability of the results.

[0076] After successful compilation, the generated binary package or artifact is archived in Jenkins or a dedicated artifact repository. Simultaneously, the manifest file repo-manifest-20231027-001.xml used in this compilation and the final commit hashes of each project are updated in the project information database as a benchmark for the next comparison.

[0077] A Jenkins-based software compilation system, employing the Jenkins-based software compilation method described above, includes the following modules:

[0078] Pre-built modules: These are used to prepare manifest files describing the structure of the project to be compiled, based on the project's dependencies; they are also used to build a base build image that includes a pre-built build environment.

[0079] Code synchronization module: used to pull all projects to a fixed directory on the physical server based on the manifest file;

[0080] Trigger mechanism configuration module: used to configure the trigger mechanism for submission events based on the items in the manifest file;

[0081] Automated compilation module: Used to trigger an automated compilation task and execute the following process when the triggering mechanism is activated:

[0082] Based on the submission event, update the manifest file to generate an updated manifest file with serial numbers;

[0083] Based on the updated manifest file, perform incremental fetch operations on the relevant items in a fixed directory;

[0084] The project code in the fixed directory is synchronized to the container environment started by the base build image via a local file server;

[0085] An automated compilation process is executed within the container environment.

[0086] A computer-readable storage medium having a computer program stored thereon, which, when executed, implements a Jenkins-based software compilation method as described above.

[0087] Those skilled in the art will understand that all or part of the processes in the above-described multimodal mechanical fault identification method for high-voltage circuit breakers can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes described in various embodiments of the Jenkins-based software compilation method. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0088] An electronic device includes a processor and a memory, the memory storing a computer program, and the processor executing the program to implement a Jenkins-based software compilation method as described above.

[0089] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A software compilation method based on Jenkins, characterized in that, Includes the following steps: S1. Based on the dependencies of the project to be compiled, prepare a manifest file describing the structure of the project to be compiled; S2. Build a base build image that includes a pre-built build environment; S3. Retrieve all items from the list file to a fixed directory on the physical server. S4. Configure the triggering mechanism for submission events based on the items in the manifest file; S5. When the triggering mechanism is activated, an automated compilation task is triggered, and the following process is executed: S51. Based on the commit event, update the manifest file to generate an updated manifest file with serial numbers; S52. Based on the updated manifest file, perform incremental fetch operations on the relevant items in the fixed directory; S53. Synchronize the project code in the fixed directory to the container environment started by the base compilation image via a local file server; S54. In the container environment, execute the automated compilation process.

2. The software compilation method based on Jenkins according to claim 1, characterized in that: The manifest file is an XML format manifest file used to record the repository addresses, branches, and commit hash information of multiple projects.

3. The software compilation method based on Jenkins according to claim 2, characterized in that: In step S2, a basic compilation image containing a compilation environment is built based on a Dockerfile. Specifically, this includes: defining the basic image in the Dockerfile and pre-installing the software packages and third-party libraries required by the project, and generating the basic compilation image through a build command.

4. The software compilation method based on Jenkins according to claim 3, characterized in that: The commit event is a response to a commit event in the code repository. Specifically, it is implemented by configuring a Webhook for the project in the manifest file in the version control system. When a code push event occurs, the Webhook triggers the automated compilation task.

5. The software compilation method based on Jenkins according to claim 4, characterized in that: The automated compilation task is a Jenkins task; The step S1 of updating the manifest file specifically includes: the Jenkins task traverses the commit information of all projects in the manifest file and compares it with the historical project information stored in a database to generate the updated manifest file.

6. The software compilation method based on Jenkins according to claim 1, characterized in that: The local file server is equipped with a Git HTTP server, which hosts code pulled from the code repository and provides code synchronization services based on the HTTP protocol.

7. The software compilation method based on Jenkins according to claim 1, characterized in that: In step S53, parallel compilation technology is used to perform compilation within the container environment.

8. A software compilation system based on Jenkins, characterized in that, The software compilation method based on Jenkins, as described in any one of claims 1 to 5, includes the following modules: Pre-built modules: These are used to prepare manifest files describing the structure of the project to be compiled, based on the project's dependencies; they are also used to build a base build image that includes a pre-built build environment. Code synchronization module: used to pull all projects to a fixed directory on the physical server based on the manifest file; Trigger mechanism configuration module: used to configure the trigger mechanism for submission events based on the items in the manifest file; Automated compilation module: Used to trigger an automated compilation task and execute the following process when the triggering mechanism is activated: Based on the submission event, update the manifest file to generate an updated manifest file with serial numbers; Based on the updated manifest file, perform incremental fetch operations on the relevant items in a fixed directory; The project code in the fixed directory is synchronized to the container environment started by the base build image via a local file server; S54. In the container environment, execute the automated compilation process.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a Jenkins-based software compilation method as described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a Jenkins-based software compilation method as described in any one of claims 1 to 5.

Images