Continuous integration job processing method, device, equipment and storage medium

Abstract

The present application provides a continuous integration job processing method, device, equipment and storage medium, which belongs to the field of computer technology. The method includes: obtaining a virtual machine configuration file, a system image file and a container template file; executing a pre-stored kernel traversal script to determine a standard image copy file and a standard virtual machine copy file; creating at least one compilation platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file and the container template file. Start the compilation container and obtain the source code of the software to be compiled; in the compilation container, compile and package the source code of the software to be compiled to generate a software package; delete the compilation container; delete the compilation virtual machine and the files associated with the compilation virtual machine. The present application avoids the situation where the occupied resources cannot be used for other purposes when the continuous integration job is not executed, resulting in resource waste, and can reuse the occupied resources to save resources.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a continuous integration job processing method, apparatus, device, and storage medium. Background Art

[0002] With the development of the computer industry, more and more developers are using continuous integration to shorten the development, integration, testing, and deployment process. Continuous integration is a software development practice that automatically compiles and tests code whenever it changes, providing feedback on the results.

[0003] Currently, existing continuous integration development technologies require configuring various kernel and operating system combinations to form a compilation or testing environment. When compiling a software package, the configured compilation environment is connected to the CI (Continuous Integration) system before the compilation is performed. When testing the software package, the configured test environment is connected to the CI system before the test is performed.

[0004] However, the inventors found that the existing technology has at least the following technical problems: for large and complex software, the configuration requirements of the continuous integration compilation environment and test environment are high, and they can only be used to perform operations related to compilation and testing. As a result, the continuous integration compilation environment and test environment need to occupy resources all the time, and the occupied resources cannot be used for other purposes when they are not performing continuous integration jobs, resulting in resource waste. Summary of the Invention

[0005] The present application provides a continuous integration job processing method, apparatus, device and storage medium, which avoids the waste of resources caused by the occupied resources being unable to be used for other purposes when the continuous integration job is not being executed, and enables the occupied resources to be reused to save resources.

[0006] In a first aspect, the present application provides a method for processing a continuous integration job, comprising:

[0007] Obtain virtual machine configuration files, system image files, and container template files;

[0008] Execute the pre-stored kernel traversal script to determine the standard image copy file and the standard virtual machine copy file;

[0009] Creating at least one compilation platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container;

[0010] Start the compilation container and obtain the source code of the software to be compiled;

[0011] In the compilation container, compiling and packaging the software source code to be compiled to generate a software package;

[0012] When it is detected that the compilation and packaging task in the compilation container is completed, deleting the compilation container;

[0013] When it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, the compilation virtual machine and files associated with the compilation virtual machine are deleted to complete the continuous integration job.

[0014] In one possible implementation, executing a pre-stored kernel traversal script to determine a standard image copy file and a standard virtual machine copy file includes: executing a pre-stored kernel traversal script to determine kernel version information and job-specified platform information; and determining a standard image copy file and a standard virtual machine copy file based on the kernel version information and the job-specified platform information.

[0015] In one possible implementation, creating at least one compilation platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file includes: creating at least one compilation virtual machine based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, wherein one compilation virtual machine corresponds to one operating platform; starting the compilation virtual machine, and creating at least one compilation container in the compilation virtual machine based on the standard image copy file and the container template file, so as to complete the creation of at least one compilation platform.

[0016] In a possible implementation, after obtaining the source code of the software to be compiled, the process further includes: when it is detected that the time for abnormal operation information of the compilation virtual machine and / or the compilation container exceeds a preset compilation timeout threshold, deleting the compilation container, the compilation virtual machine, and files associated with the compilation virtual machine, and re-executing the steps of obtaining the virtual machine configuration file, the system image file, and the container template file.

[0017] In one possible implementation, the software package includes a kernel-mode module and a user-mode module; accordingly, after executing a pre-stored kernel traversal script and determining a standard image copy file and a standard virtual machine copy file, the method further includes: creating at least one test platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each test platform includes at least one test virtual machine, and each test virtual machine includes at least one test container; installing the kernel-mode module in the test virtual machine, and installing the user-mode module in the test container; starting the test container; testing the software package in the test container to generate test results; deleting the test container when it is detected that the test task in the test container is completed; deleting the test virtual machine and associated files when it is detected that the test tasks of all test containers in the test virtual machine are completed to complete the continuous integration job.

[0018] In one possible implementation, the creating of at least one test platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file and the container template file includes: creating at least one test virtual machine based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file and the system image file, wherein one test virtual machine corresponds to one operating platform; starting the test virtual machine, and creating at least one test container in the test virtual machine based on the standard image copy file and the container template file, so as to complete the creation of at least one test platform.

[0019] In one possible implementation, after installing the kernel-mode module in the test virtual machine and installing the user-mode module in the test container, the method further includes: when it is detected that the time for abnormal job information to appear in the test virtual machine and / or the test container exceeds a preset test timeout threshold, deleting the test container, the test virtual machine, and files associated with the test virtual machine, and re-executing the steps of obtaining the virtual machine configuration file, the system image file, and the container template file.

[0020] In a possible implementation, after executing a pre-existing kernel traversal script to determine the standard image copy file and the standard virtual machine copy file, the method further includes: starting a pre-existing container orchestration management tool to manage all compiled containers in the compiled virtual machine.

[0021] In a second aspect, the present application provides a continuous integration job processing device, comprising:

[0022] The acquisition module is used to obtain virtual machine configuration files, system image files, and container template files;

[0023] A script execution module is used to execute a pre-stored kernel traversal script to determine a standard image copy file and a standard virtual machine copy file;

[0024] a compilation environment creation module, configured to create at least one compilation platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container;

[0025] A startup module, used to start the compilation container and obtain the source code of the software to be compiled;

[0026] A compiling and packaging module compiles and packages the source code of the software to be compiled in the compilation container to generate a software package;

[0027] a deletion module, configured to delete the compilation container when detecting that the compilation and packaging task in the compilation container is completed;

[0028] The deletion module is further configured to delete the compilation virtual machine and files associated with the compilation virtual machine when it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, so as to complete the continuous integration job. In a third aspect, the present application provides a computer device comprising: at least one processor and a memory;

[0029] The memory stores computer-executable instructions;

[0030] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect above.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the continuous integration job processing method described in the first aspect above is implemented.

[0032] The continuous integration job processing method, device, equipment and storage medium provided in the present application create at least one compilation platform by copying files and standard virtual machine copying files using virtual machine configuration files, system image files, container template files and the standards determined by executing the predicted kernel traversal script. Each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container. Then, the compilation container is started, and the source code of the software to be compiled is obtained. After the compilation and packaging tasks in the compilation container are completed, the compilation container can be deleted after the software package is generated, thereby releasing resource occupation. Similarly, when the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, all compilation virtual machines and files associated with the compilation virtual machine are deleted, and resource occupation is released while completing the continuous integration job. When the compilation task is not executed, the resources occupied by the compilation virtual machine and the compilation container can be used for other purposes, thereby saving more resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1 A schematic diagram of an application scenario of the continuous integration processing method provided in an embodiment of the present application;

[0035] Figure 2 A flowchart of a continuous integration job processing method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of the compilation environment provided in the embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of the test environment provided in the embodiment of the present application;

[0038] Figure 5 A schematic block diagram of the software package provided in an embodiment of the present application being installed on a test platform;

[0039] Figure 6 Schematic diagram of the structure of the continuous integration job processing device provided in the embodiment of the present application Figure 1 ;

[0040] Figure 7 Schematic diagram of the structure of the continuous integration job processing device provided in the embodiment of the present application Figure 2 ;

[0041] Figure 8A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] Continuous integration is a software development practice. In practice, it means that as long as the code is changed, compilation and testing are automatically performed and the operation results are fed back. Due to its high development efficiency, it is used by more and more project developers. In the prior art, it is necessary to first configure various kernel and operating system combinations to form a compilation environment or test environment. When compiling a software package, the configured compilation environment is connected to CI (Continuous Integration) and then compilation is performed. When testing a software package, the configured test environment is connected to CI and then testing is performed. When the inventor used the prior art to perform continuous integration operations, because the compilation environment needs to be configured with various kernels and operating systems, and the third-party software and versions that different software compilations rely on are not the same, for example, different C or C++ projects have different version requirements for the programming language compiler suite GCC. Therefore, after the configured compilation environment is connected to CI, it needs to be in standby mode at all times and is only used for the compilation of the corresponding project. When compilation is not performed, in order to avoid damaging the compilation environment, the environment configuration cannot be changed at will, resulting in a problem of waste of occupied resources. At the same time, when encountering a continuous integration test environment, when defects in the test software cause system or frequent problems during the test process, and the test environment needs to be manually restored, the steps for cleaning up the environment are complicated and time-consuming, resulting in inconvenient maintenance of the entire test environment.

[0044] To solve the above technical problems, the embodiments of the present application provide the following technical concepts for solving the problems: using pre-existing virtual machine configuration files, system image files, container template files, standard copy files and standard virtual machine copy files to create a virtual machine, and create a container in the virtual machine. Different versions of virtual machines and containers of different operating systems can form different compilation platforms. The software source code is compiled and packaged in the container. After the compilation is completed, the container and virtual machine can be deleted. When the compilation task is not executed, the resources occupied by the continuous basic processing job can be used for other purposes, saving resources.

[0045] refer to Figure 1 , Figure 1This is a schematic diagram of an application scenario of the continuous integration processing method provided in the embodiment of the present application, such as Figure 1 As shown, it includes: multiple developer hosts 101, a code storage terminal 102 and a host machine 103.

[0046] Developer host 101 is used by developers to write or update code based on project requirements and push the code to code storage 102. Code storage 102 receives all code pushed by developer host 101 and sends it to host 103. Host 103 can be a cloud server or a physical server. Host 103 compiles and tests the source code to be compiled and returns the results to developer host 101, completing a continuous integration process.

[0047] Figure 2 This is a flow chart of the continuous integration job processing method provided in the embodiment of the present application. The execution subject of this embodiment can be Figure 1 The host machine 103 in the embodiment shown may also be other computer-related devices, and this embodiment is not particularly limited. Figure 2 As shown, the method includes:

[0048] S201: Obtain a virtual machine configuration file, a system image file, and a container template file.

[0049] In this embodiment, the virtual machine configuration file can be a description file of virtual machines including various kernel versions, such as an extensible markup language XML configuration file, which can be used to define various description parameters of a virtual machine, such as the name, UUID (Universally Unique Identifier), CPU (Central Processing Unit), memory, virtual disk, network card, etc.

[0050] The system image file is a specific series of files made into a single file according to a certain format, which is convenient for users to download and use. In this embodiment, the system image file and the virtual CD-ROM drive constitute the operating system of the virtual machine, and one operating system corresponds to one system image file.

[0051] A container template file is the basic file for building a container image. It can contain multiple instructions, parameters, and descriptions required to build a container image, such as the command prompt CMD container startup instruction and the container creation instruction RUN.

[0052] S202: Execute a pre-stored kernel traversal script to determine a standard image copy file and a standard virtual machine copy file.

[0053] In this embodiment, the pre-stored kernel traversal script can be a manually pre-compiled job script for traversing kernel versions. Executing the pre-stored kernel traversal script can access all kernel versions, and then obtain the standard image copy file and standard virtual machine copy file of the kernel version virtual machine of the specified platform.

[0054] The standard image copy file can be a known or pre-existing virtual machine disk file, such as a qcow2 image file. The standard virtual machine copy file can be a known or pre-existing virtual machine startup configuration file, such as a known standard XML copy file.

[0055] Specifically, in an optional embodiment of the present application, step S202 includes:

[0056] S202a: Execute a pre-stored kernel traversal script to determine kernel version information and job-specified platform information.

[0057] S202b: Determine a standard image copy file and a standard virtual machine copy file according to the kernel version information and the job-specified platform information.

[0058] In this embodiment, the kernel version information can be the core of the virtual machine operating system, which is responsible for managing the system's processes, device drivers, files, and network systems. For example, the Linux virtual machine version is Fedora 5, and the kernel version is 2.6.15 or 3.10. Each kernel version corresponds to a standard image copy file, a standard virtual machine copy file, and a platform, utilizing known kernel version information and jobs. The job-specified platform information can be a pre-stored code segment used to automatically specify the corresponding platform and kernel version based on the requirements of the continuous integration job.

[0059] S203: Create at least one compilation platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container.

[0060] In this embodiment, the at least one may be one or more, and the number of specific compilation platforms may be determined according to compilation requirements. For example, if the compiled software only needs to run in Windows or Linux, then the number of compilation platforms may be two.

[0061] refer to Figure 3 , Figure 3 This is a schematic diagram of the structure of the compilation environment provided in the embodiment of this application. Figure 3As shown, the compilation environment includes multiple compilation platforms, such as compilation platform 1 and compilation platform 2. Compilation platform 1 includes multiple compilation virtual machines, such as compilation virtual machine 1 and compilation virtual machine 2. Each compilation virtual machine includes multiple compilation containers. The operating system versions of each two compilation containers are different or the operating systems are different. The kernel versions of compilation virtual machine 1 in compilation platform 1 and compilation virtual machine 2 in compilation platform 2 are different.

[0062] In this embodiment, a compilation platform can correspond to a combination of a kernel version and multiple operating systems. Random combinations of different kernel versions and operating systems can form multiple compilation platforms. A compilation virtual machine can be a complete computer system with complete hardware system functions, running in a completely isolated environment, simulated by software. This computer system has compilation capabilities. A compilation container can be used to bundle an application and all its necessary files into a runtime environment. Compilation can be performed in the compilation container as a completely isolated environment, and compilation tasks in the compilation container do not interfere with compilation tasks in other compilation containers.

[0063] Specifically, in an optional embodiment of the present application, step S203 includes:

[0064] S203a: Creating at least one compiled virtual machine according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, wherein one compiled virtual machine corresponds to one operating platform.

[0065] S203b: Start the compilation virtual machine, and create at least one compilation container in the compilation virtual machine according to the standard image copy file and the container template file, so as to complete the creation of at least one compilation platform.

[0066] Continue to refer Figure 3 In this embodiment, existing standard virtual machine copy files and standard image copy files are used to modify the parameters and configuration of the virtual machine configuration file and system image file to create a compilation virtual machine. In this embodiment, the operating platform refers to the compilation platform. Compilation containers can also use existing standard image copy files and modify the container template file to create a compilation container in the compilation virtual machine. Each compilation container corresponds to an operating system.

[0067] S204: Start the compilation container and obtain the source code of the software to be compiled.

[0068] In this embodiment, the compilation container can be started using a container startup command. For example, the container startup command can be a docker run command. The source code of the software to be compiled can be as follows: Figure 1 The code shown is the source code sent by the storage end.

[0069] S205: In the compilation container, the source code of the software to be compiled is compiled and packaged to generate a software package.

[0070] In this embodiment, the compilation script pre-stored in the container is automatically executed after the container is started to compile and package the source code of the software to be compiled to generate a software package. For example, the software package may include the source data yum source of the software package.

[0071] S206: When it is detected that the compilation and packaging task in the compilation container is completed, the compilation container is deleted.

[0072] In this embodiment, whether the compilation and packaging task is completed can be determined by detecting whether all processes in the compilation container are stopped. When it is detected that no process is running in the compilation container, it means that all compilation and packaging tasks in the compilation container are completed, and the compilation container can be deleted.

[0073] S207: When it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, the compilation virtual machine and files associated with the compilation virtual machine are deleted to complete the continuous integration job.

[0074] In this embodiment, after each compilation virtual machine completes all corresponding compilation and packaging tasks, it is necessary to delete the virtual machine and associated files. After all compilation virtual machines and their associated files are deleted, it indicates that the cleaning of the entire multi-platform compilation environment has been completed.

[0075] This embodiment corresponds to the compilation stage on the continuous integration job pipeline. When the developer triggers the pipeline, he only chooses to execute compilation and packaging. Then, the continuous integration job to step S207 forms a complete continuous integration job processing mechanism.

[0076] In summary, the continuous integration job processing method provided in the embodiment of the present application creates at least one compilation platform by copying files and standard virtual machine copy files using virtual machine configuration files, system image files, container template files, and the standards determined by executing the predicted kernel traversal script. Each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container. Then the compilation container is started, and the source code of the software to be compiled is obtained. After the compilation and packaging tasks in the compilation container are completed, the compilation container can be deleted after the software package is generated, thereby releasing resource occupation. Similarly, when the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, all compilation virtual machines and files associated with the compilation virtual machine are deleted, and resource occupation is released while completing the continuous integration job. When the compilation task is not executed, the resources occupied by the compilation virtual machine and the compilation container can also be used for other purposes, thereby saving more resources.

[0077] Based on the above embodiment, in an optional embodiment of the present application, after step S204, the following steps are further included:

[0078] Step A: When it is detected that the time for abnormal operation information of the compiled virtual machine and / or compiled container exceeds the preset compilation timeout threshold, the compiled container, compiled virtual machine and files associated with the compiled virtual machine are deleted, and the steps of obtaining the virtual machine configuration file, system image file and container template file are re-executed.

[0079] In this embodiment, the abnormal operation information detected may include abnormal operation information occurring in the compilation virtual machine or the compilation container, abnormal operation information occurring in the compilation container but not in the compilation virtual machine, abnormal operation information occurring in the compilation virtual machine but not in the compilation container, or abnormal operation information occurring in both the compilation container and the compilation virtual machine. Abnormal operation information may be information generated by an abnormal operation that causes any of steps S201 to S207 to stop. For example, abnormal operation information may include abnormal operation information generated by abnormal conditions such as an operating system crash in the compilation container, a virtual machine shutdown, or a process hang.

[0080] In this embodiment, the preset compilation timeout threshold is a manually set time value. When the time when the abnormal job information appears exceeds the preset compilation timeout threshold, it is necessary to delete the compilation container, the compilation virtual machine, and the files associated with the compilation virtual machine to complete the cleanup and re-execute step S201.

[0081] In summary, the continuous integration job processing method provided in the embodiment of the present application deletes the compilation container, the compilation virtual machine and the files associated with the compilation virtual machine when the time for detecting abnormal job information in the compilation virtual machine and / or the compilation container exceeds the preset compilation timeout threshold, quickly cleans up the compilation environment and restores the compilation environment, making the maintenance of the compilation environment more convenient and improving the efficiency of continuous integration jobs.

[0082] The above embodiment describes the compilation and packaging phase of a continuous integration operation. Based on the above embodiment, in an optional embodiment of the present application, the software package generated in step S205 includes a kernel-mode module and a user-mode module. Accordingly, after step S02, the following steps are also included:

[0083] Step B: Create at least one test platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file and the container template file, wherein each test platform includes at least one test virtual machine, and each test virtual machine includes at least one test container.

[0084] refer to Figure 4 , Figure 4This is a schematic diagram of the structure of the test environment provided in the embodiment of the present application. Figure 4 As shown, the test environment includes multiple test platforms, such as test platform 1 and test platform 2. Test platform 1 includes multiple test virtual machines, such as test virtual machine 1 and test virtual machine 2. Each test virtual machine includes multiple test containers, and the operating system versions of each two test containers are different or the operating systems are different. The kernel versions of test virtual machine 1 in test platform 1 and test virtual machine 2 in test platform 2 are different.

[0085] Specifically, in an optional embodiment of the present application, step B includes:

[0086] Step b1: creating at least one test virtual machine according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file and the system image file, wherein one test virtual machine corresponds to one operating platform.

[0087] Step B2: Start the test virtual machine, and create at least one test container in the test virtual machine according to the standard image copy file and the container template file, so as to complete the creation of at least one test platform.

[0088] In this embodiment, the process and principles for creating a test virtual machine and test container in the test platform differ from step S203 in the above embodiment in that the configuration parameters of the compilation container and the test container may differ, and the various environment configurations and dependent software packages of the compilation virtual machine and the test virtual machine may differ. For example, the compilation virtual machine may only require one virtual disk, while the test virtual machine may require three virtual disks.

[0089] Step C: Install the kernel-mode module in the test virtual machine and the user-mode module in the test container.

[0090] refer to Figure 5 , Figure 5 This is a schematic diagram of the software package provided in the embodiment of the present application being installed on a test platform. Figure 5 As shown, since the test container lacks a dedicated kernel, the kernel-mode modules of the software package cannot be installed in the test container. Instead, the kernel of the test virtual machine must be used. Furthermore, since the test virtual machine contains at least one test container, kernel-mode modules can be installed only once. However, when installing the user-mode modules of the software package, they must be installed in each test container until all test containers have installed them, completing the software package deployment process in the entire test environment.

[0091] Step D: In the test container, the software package is tested and processed to generate test results.

[0092] In this embodiment, the test results can be test reports generated by automated testing of the software package on various test platforms, used to verify the accuracy of the code. For example, the test results can be log files. In this embodiment, the test process can be an operation in which a tester pre-sets test cases and scenarios and executes tests to check for deviations from expected outputs. For example, the test process can be a sanity test, an integration test, or a stress test.

[0093] Step E: When it is detected that the test task in the test container is completed, the test container is deleted.

[0094] In this embodiment, whether the test task is completed can be determined by detecting whether all processes in the test container are stopped. When it is detected that no process is running in the test container, it means that all test tasks in the test container are completed, and the test container can be deleted.

[0095] Step F: When it is detected that the test tasks of all test containers in the test virtual machine are completed, the test virtual machine and associated files are deleted to complete the continuous integration job.

[0096] In this embodiment, after each test virtual machine completes the test task, the test virtual machine needs to be deleted to clean up the test environment.

[0097] In summary, the continuous integration job processing method provided in this embodiment creates at least one test platform by using virtual machine configuration files, system image files, container template files, and standards determined by executing a predicted kernel traversal script to copy files and standard virtual machine copy files. Each test platform includes at least one test virtual machine, and each test virtual machine includes at least one test container. The kernel state module of the software package is then installed in the test virtual machine, and the user state module of the software package is installed in the test container. The test container is then started, the test task in the test container is completed, and the test container can be deleted after the test results are generated, thereby releasing resource occupation. Similarly, when the test tasks of all test containers in the test virtual machine are completed, all test virtual machines and files associated with the test virtual machine are deleted, and resource occupation is released while completing the continuous integration job. When the test task is not executed, the resources occupied by the test virtual machine and the test container can be used for other purposes, thereby saving more resources.

[0098] At the same time, when there are a large number of test platforms, a multi-platform test environment can be formed on a single host machine, so that the testing phase of the continuous integration job occupies fewer resources.

[0099] Based on the above embodiment, in an optional embodiment of the present application, after step C, the following steps are further included:

[0100] Step G: When it is detected that the time for abnormal operation information to appear in the test virtual machine and / or the test container exceeds the preset test timeout threshold, delete the test container, the test virtual machine and the files associated with the test virtual machine, and re-execute the steps of obtaining the virtual machine configuration file, the system image file and the container template file.

[0101] In this embodiment, the implementation principle and technical effect of step G are similar to those of step A, and will not be repeated here in this embodiment.

[0102] Based on the above embodiment, as an optional embodiment of the present application, after step S202, the following is further included:

[0103] Step I: Start a pre-existing container orchestration management tool to manage all compiled containers in the compiled virtual machine.

[0104] In this embodiment, the pre-existing container orchestration management tool can be a basic mechanism for scheduling and managing clusters. For example, the pre-existing container orchestration management tool can be a Kubernetes cluster. In this embodiment, when compiling large and complex software, the number of compilation containers increases, and in this case, the pre-existing container orchestration management tool can be used to manage the compilation containers.

[0105] In summary, the continuous integration job processing method provided in this embodiment can improve the convenience of deployment management after the creation of the compilation container and the convenience of subsequent maintenance of continuous integration jobs by using a pre-existing container orchestration management tool to manage the compilation container in the compilation virtual machine.

[0106] In an optional embodiment of the present application, before step B, the method described in step I can be used to start a pre-existing container orchestration management tool to manage all test containers in the test virtual machine. This is to improve the convenience of deployment management after the test container is created and the convenience of subsequent maintenance of continuous integration jobs.

[0107] refer to Figure 6 , Figure 6 Schematic diagram of the structure of the continuous integration job processing device provided in the embodiment of the present application Figure 1 .like Figure 6 As shown, the device includes: an acquisition module 61, a script execution module 62, a compilation environment creation module 63, a startup module 64, a compilation and packaging module 65 and a deletion module 66.

[0108] The acquisition module 61 is used to acquire a virtual machine configuration file, a system image file, and a container template file.

[0109] The script execution module 62 is used to execute a pre-stored kernel traversal script to determine a standard image copy file and a standard virtual machine copy file.

[0110] The compilation environment creation module 63 is used to create at least one compilation platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container.

[0111] The startup module 64 is used to start the compilation container and obtain the source code of the software to be compiled.

[0112] The compiling and packaging module 65 compiles and packages the source code of the software to be compiled in the compiling container to generate a software package.

[0113] The deletion module 66 is configured to delete the compilation container when it is detected that the compilation and packaging task in the compilation container is completed.

[0114] The deletion module 66 is further configured to delete the compilation virtual machine and files associated with the compilation virtual machine when it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, so as to complete the continuous integration operation.

[0115] In an optional embodiment of the present application, the script execution module 62 is specifically configured to execute a pre-stored kernel traversal script to determine kernel version information and job-specific platform information, and to determine a standard image copy file and a standard virtual machine copy file based on the kernel version information and the job-specific platform information.

[0116] In an optional embodiment of the present application, the compilation environment creation module 63 is specifically configured to create at least one compilation virtual machine based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, where each compilation virtual machine corresponds to one operating platform. The compilation virtual machine is started, and at least one compilation container is created in the compilation virtual machine based on the standard image copy file and the container template file, thereby completing the creation of at least one compilation platform.

[0117] In an optional embodiment of the present application, after the acquisition module 61 acquires the software source code to be compiled, the deletion module 66 is further configured to: upon detecting that the duration of abnormal operation information in the compilation virtual machine and / or compilation container exceeds a preset compilation timeout threshold, delete the compilation container, the compilation virtual machine, and files associated with the compilation virtual machine, and re-execute the steps of acquiring the virtual machine configuration file, the system image file, and the container template file.

[0118] refer to Figure 7 , Figure 7 Schematic diagram of the structure of the continuous integration job processing device provided in the embodiment of the present application Figure 2 .and Figure 6 The continuous integration job setup shown is different in that, Figure 7As shown, the continuous integration job processing device further includes: a test environment creation module 67 , a software package installation module 68 and a test module 69 .

[0119] Among them, the test environment creation module 67 is used to create at least one test platform based on standard virtual machine copy files, standard image copy files, virtual machine configuration files, system image files and container template files, wherein each test platform includes at least one test virtual machine, and each test virtual machine includes at least one test container.

[0120] The software package installation module 68 is used to install the kernel-mode module in the test virtual machine and install the user-mode module in the test container.

[0121] The starting module 64 is also used to start the test container.

[0122] The test module 69 is used to test the software package in the test container and generate test results.

[0123] The deletion module 66 is further configured to delete the test container when it is detected that the test task in the test container is completed.

[0124] The deletion module 66 is further configured to delete the test virtual machine and associated files when it is detected that the test tasks of all test containers in the test virtual machine are completed, so as to complete the continuous integration operation.

[0125] In an optional embodiment of the present application, the test environment creation module 67 is specifically configured to create at least one test virtual machine based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, wherein each test virtual machine corresponds to one operating platform. The test virtual machine is started, and at least one test container is created in the test virtual machine based on the standard image copy file and the container template file, thereby completing the creation of at least one test platform.

[0126] In an optional embodiment of the present application, the deletion module 66 is further specifically used to: when it is detected that the time for abnormal operation information to appear in the test virtual machine and / or the test container exceeds the preset test timeout threshold, delete the test container, the test virtual machine and the files associated with the test virtual machine, and re-execute the steps of obtaining the virtual machine configuration file, the system image file and the container template file.

[0127] In an optional embodiment of the present application, the startup module 64 is further specifically configured to: start a pre-existing container orchestration management tool to manage all compilation containers in the compilation virtual machine.

[0128] In an optional embodiment of the present application, the startup module 64 is further specifically configured to: start a pre-existing container orchestration management tool to manage all compilation containers in the compilation virtual machine.

[0129] The continuous integration job processing device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0130] Figure 8 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the system includes: at least one processor 801 and a memory 802.

[0131] The processor 801 is used to store computer-executable instructions.

[0132] The memory 802 is configured to execute computer-executable instructions stored in the memory to implement the various steps involved in the above method embodiment. For details, please refer to the relevant description in the above method embodiment.

[0133] Optionally, the memory 802 may be independent or integrated with the processor 801 .

[0134] When the memory 802 is independently provided, the device further includes a bus 803 for connecting the memory 802 and the processor 801 .

[0135] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned continuous integration job processing method is implemented.

[0136] An embodiment of the present application further provides a computer program product, including a computer program, which implements the above continuous integration job processing method when executed by a processor.

[0137] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection of some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0138] The modules described above as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these modules may be selected to implement the solution of this embodiment according to actual needs.

[0139] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each module may exist physically separately, or two or more modules may be integrated into a single unit. The units composed of the above modules may be implemented in the form of hardware or hardware plus software functional units.

[0140] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform some steps of the method of each embodiment of the present application.

[0141] It should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), or application-specific integrated circuits (ASICs). A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0142] The memory may include a high-speed RAM memory, and may also include non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk.

[0143] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0144] The storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0145] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0146] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0147] The present invention is intended to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solution from the scope of the technical solution of the embodiments of the present application.

Claims

1. A continuous integration job processing method, characterized in that: include: Obtain virtual machine configuration files, system image files, and container template files; Execute a pre-stored kernel traversal script to determine the kernel version information and job-specific platform information. The job-specific platform information can be a pre-stored code segment used to automatically specify the corresponding platform and kernel version based on the requirements of the continuous integration job; Determining a standard image copy file and a standard virtual machine copy file according to the kernel version information and the job-specified platform information; Creating at least one compilation platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container; Start the compilation container and obtain the source code of the software to be compiled; In the compilation container, compiling and packaging the software source code to be compiled to generate a software package; When it is detected that the compilation and packaging task in the compilation container is completed, deleting the compilation container; When it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, the compilation virtual machine and files associated with the compilation virtual machine are deleted to complete the continuous integration job.

2. The method according to claim 1, characterized in that The step of creating at least one compilation platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file includes: Creating at least one compiled virtual machine according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, wherein one compiled virtual machine corresponds to one operating platform; The compiling virtual machine is started, and at least one compiling container is created in the compiling virtual machine according to the standard image copy file and the container template file, so as to complete the creation of at least one compiling platform.

3. The method according to claim 1, characterized in that After obtaining the source code of the software to be compiled, the method further includes: When it is detected that the time for abnormal operation information of the compiled virtual machine and / or the compiled container exceeds a preset compilation timeout threshold, the compiled container, the compiled virtual machine and files associated with the compiled virtual machine are deleted, and the steps of obtaining the virtual machine configuration file, the system image file and the container template file are re-executed.

4. The method according to claim 1, wherein The software package includes a kernel mode module and a user mode module; Accordingly, after executing the pre-stored kernel traversal script and determining the standard image copy file and the standard virtual machine copy file, the method further includes: Creating at least one test platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each test platform includes at least one test virtual machine, and each test virtual machine includes at least one test container; Installing the kernel-mode module in the test virtual machine and installing the user-mode module in the test container; Starting the test container; In the test container, testing the software package to generate a test result; When it is detected that the test task in the test container is completed, deleting the test container; When it is detected that the test tasks of all the test containers in the test virtual machine are completed, the test virtual machine and associated files are deleted to complete the continuous integration operation.

5. The method according to claim 4, characterized in that The step of creating at least one test platform according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file includes: Creating at least one test virtual machine according to the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, and the system image file, wherein one test virtual machine corresponds to one operating platform; The test virtual machine is started, and at least one test container is created in the test virtual machine according to the standard image copy file and the container template file, so as to complete the creation of at least one test platform.

6. The method according to claim 4, characterized in that After installing the kernel state module in the test virtual machine and installing the user state module in the test container, the method further includes: When it is detected that the time for abnormal operation information to appear in the test virtual machine and / or the test container exceeds the preset test timeout threshold, the test container, the test virtual machine and the files associated with the test virtual machine are deleted, and the steps of obtaining the virtual machine configuration file, the system image file and the container template file are re-executed.

7. The method according to any one of claims 1 to 6, characterized in that After executing the pre-stored kernel traversal script to determine the standard image copy file and the standard virtual machine copy file, the method further includes: A pre-existing container orchestration management tool is started to manage all compilation containers in the compilation virtual machine.

8. A continuous integration job processing device, characterized in that: include: The acquisition module is used to obtain virtual machine configuration files, system image files, and container template files; A script execution module is configured to execute a pre-stored kernel traversal script, determine kernel version information and job-specified platform information, which may be a pre-stored code segment that automatically specifies the corresponding platform and kernel version based on the requirements of the continuous integration job; and determine a standard image copy file and a standard virtual machine copy file based on the kernel version information and the job-specified platform information. a compilation environment creation module, configured to create at least one compilation platform based on the standard virtual machine copy file, the standard image copy file, the virtual machine configuration file, the system image file, and the container template file, wherein each compilation platform includes at least one compilation virtual machine, and each compilation virtual machine includes at least one compilation container; A startup module, used to start the compilation container and obtain the source code of the software to be compiled; A compiling and packaging module compiles and packages the source code of the software to be compiled in the compilation container to generate a software package; a deletion module, configured to delete the compilation container when detecting that the compilation and packaging task in the compilation container is completed; The deletion module is further configured to delete the compilation virtual machine and files associated with the compilation virtual machine when it is detected that the compilation and packaging tasks of all compilation containers in the compilation virtual machine are completed, so as to complete the continuous integration job.

9. A computer device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the continuous integration job processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the continuous integration job processing method according to any one of claims 1 to 7 is implemented.

Images