Compilation construction method and device of client program, storage medium and electronic equipment

Abstract

The invention discloses a compiling construction method and device of a client program, a storage medium and electronic equipment, is applied to a server, and relates to the technical field of computers. Comprising the following steps: generating a dynamic link library corresponding to each kernel version according to a kernel information function of which a return value is modified into a kernel version number corresponding to each kernel version of each operating system; according to the dynamic link library and the target drive file, creating a compiling construction mirror image corresponding to each kernel version, and further constructing a compiling construction container consistent with the environment of an operating system of each kernel version in a server according to each compiling construction mirror image, and compiling and constructing the client program of each kernel version in each compiling and constructing container. Therefore, the problem that resources are occupied due to the fact that different operating systems are canned on different servers and kernel clients of corresponding versions are compiled and constructed on the operating systems of the different servers in the prior art can be solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for compiling and constructing a client program, a storage medium, and an electronic device. Background Art

[0002] To adapt to the Linux operating system (an operating system), different versions of the kernel source code need to be compiled and built on different versions of the Linux operating system to generate executable programs for use in different projects. Currently, common Linux operating systems include: Ubuntu, CentOS, Kylin, Rocky (all operating systems), etc. Each operating system has many versions. For example, CentOS, the most commonly used ones include CentOS 7.6, CentOS 7.8, and CentOS 8.4 (all different versions of the CentOS operating system). Different types and versions of Linux operating system kernel clients require different compilation and build environments.

[0003] Traditionally, different operating systems are installed on different servers, and the corresponding kernel client versions are compiled and built directly on the operating systems. However, this approach requires multiple servers if multiple versions are required, resulting in significant resource usage and poor scalability.

[0004] Therefore, the problem of resource occupation caused by canning different operating systems on different servers and then compiling and building corresponding versions of kernel clients on the operating systems of different servers in the related art has not yet been effectively solved. Summary of the Invention

[0005] The present application provides a method and device for compiling and building a client program, a storage medium, and an electronic device to at least solve the problem of resource occupation caused by canning different operating systems on different servers in the related art, and then compiling and building corresponding versions of kernel clients on the operating systems of different servers.

[0006] The present application provides a method for compiling and building a client program, which is applied to a server, comprising: generating a dynamic link library corresponding to the kernel version of each operating system according to a kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version; creating a compilation and construction image corresponding to each kernel version according to the dynamic link library and a target driver file; building a compilation and construction container corresponding to each kernel version according to each compilation and construction image, and compiling and building a client program for each kernel version in each compilation and construction container, wherein the client program for each kernel version is installed in the operating system corresponding to each kernel version.

[0007] The present application also provides a compilation and construction device for a client program, including: a generation module, used to generate a dynamic link library corresponding to the kernel version of each operating system based on a kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version; a creation module, used to create a compilation and construction image corresponding to each kernel version based on the dynamic link library and the target driver file; a construction module, used to construct a compilation and construction container corresponding to each kernel version based on each compilation and construction image, and compile and construct a client program for each kernel version in each compilation and construction container, wherein the client program for each kernel version is installed in the operating system corresponding to each kernel version.

[0008] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned client program compilation and construction methods when executing the computer program.

[0009] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned client program compilation and construction methods are implemented.

[0010] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned client program compilation and construction methods when executed by a processor.

[0011] Through this application, a dynamic link library corresponding to each kernel version is generated based on the kernel information function that modifies the return value to the kernel version number corresponding to each kernel version of each operating system; a compilation and construction image corresponding to each kernel version is created based on the dynamic link library and the target driver file; a compilation and construction container corresponding to each kernel version is constructed based on each compilation and construction image, and a client program for each kernel version is compiled and constructed in each compilation and construction container, and the client program for each kernel version can be installed on the operating system of the corresponding kernel version. In other words, this application constructs compilation and construction containers corresponding to different kernel versions on a server to create compilation and construction environments for different versions of operating systems, and then the compilation and construction container of each kernel version is the same as the compilation and construction environment of each corresponding kernel version of the operating system, without the need to can different operating systems on different servers. Therefore, the problem of resource occupation caused by canning different operating systems on different servers and then compiling and constructing corresponding versions of kernel clients on the operating systems of different servers in the related art can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a hardware structure block diagram of a computer terminal for a method for compiling and building a client program according to an embodiment of the present application;

[0014] Figure 2 This is an architectural diagram of the interaction between different components of the Ceph distributed file system in related technologies;

[0015] Figure 3 This is a flowchart of a method for compiling and building a client program according to an embodiment of the present application;

[0016] Figure 4 This is a flowchart of a compilation process solidification according to an optional embodiment of the present application;

[0017] Figure 5 It is a structural block diagram of a device for compiling and constructing a client program according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0020] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0021] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the compilation and construction method of the client program depends, the specific application environment architecture or specific hardware architecture is described here.

[0022] The method embodiments provided in the embodiments of the present application can be executed in a server corresponding to a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 This is a hardware structure diagram of a computer terminal for a method of compiling and building a client program according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. The computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0023] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the interactive state in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0024] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0025] The embodiments of the present application provide a method for compiling and building a client program. The following is an explanation of the technical terms involved in the embodiments of the present application:

[0026] A distributed file system (Ceph Distributed Object Store and File System, abbreviated as Ceph): is an object-based storage system. The so-called object-based storage means that it divides each data stream to be managed (for example, a file) into one or more object data of a fixed size (4 megabytes by default), and uses object data as atomic units (atom is the smallest unit of constituent elements) to complete data access. The underlying storage service for object data is a storage cluster composed of multiple hosts. This storage cluster is called a Reliable Automatic Distributed Object Store (RADOS) cluster. Figure 2 This is an architectural diagram of the interaction between different components of the Ceph distributed file system in related technologies, such as Figure 2 As shown: the client included in the user state may include: kernel client, user state file system client and user state user end. The client may include a command line and a graphical user interface. The user state also includes: libcephfs (Ceph file system library), MDS (metadata server), libfuse (user state file system library), and the link library related to libcephfs can be libcephfs.so (Ceph file system dynamic link library), and the link library related to libfuse can be libfuse.so (user state file system dynamic link library); the kernel state includes: vfs layer (virtual file system layer), ceph layer (distributed file system layer, including: ceph.ko (Ceph kernel module)) and fuse (user state file system module).

[0027] Librados Librado (Librados) is the application programming interface (API) for the RADOS storage cluster. It supports programming languages ​​such as C, C++, Java, Python, and PHP (all different programming languages). The RADOS Gateway (RADOSGW), Reliable Block Device (RBD), and Ceph File System (CEPHFS) are all RADOS storage service interfaces. They further abstract the RADOS storage service interface, Librados, from different perspectives and are therefore suitable for different application scenarios. Specifically:

[0028] 1. RADOSGW (for object storage) abstracts the underlying RADOS storage service into an object storage service using a Representational State Transfer (RESTful) interface. This interface is suitable for accessing object data, such as World Wide Web (WEB) services.

[0029] 2. RBD (for block storage) is a storage device that abstracts the underlying RADOS storage service into a block device. It is mainly used for virtualization, for example, providing a hard disk for a virtual machine.

[0030] 3. CEPHFS (suitable for file storage) abstracts the underlying RADOS into a file system interface for use by other hosts.

[0031] CEPHFS supports two methods of use: mounting via kernel modules and mounting via userspace FUSE. Mounting via kernel modules refers to kernel client technology. The kernel client runs on compute nodes (not storage service nodes) and can use storage cluster services just like a local file system.

[0032] Figure 3 This is a flowchart of a client program compilation and construction method according to an embodiment of the present application, which can be applied to Figure 1 In the server corresponding to the computer terminal, such as Figure 3 As shown, the process includes the following steps:

[0033] Step S302, generating a dynamic link library corresponding to the kernel version of each operating system according to the kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version;

[0034] The original return value of the kernel information function is the version number of the host machine.

[0035] Step S304: Create a compiled image corresponding to each kernel version based on the dynamic link library and the target driver file;

[0036] Step S306: Build a compile and build container corresponding to each kernel version according to each compile and build image, and compile and build a client program of each kernel version in each compile and build container, wherein the client program of each kernel version is installed in the operating system corresponding to each kernel version.

[0037] The compilation and construction container may be a Docker container.

[0038] Through the compilation and construction method of the client program of the present application, the dynamic link library corresponding to each kernel version is generated according to the kernel information function that modifies the return value to the kernel version number corresponding to each kernel version of each operating system; the compilation and construction image corresponding to each kernel version is created according to the dynamic link library and the target driver file; the compilation and construction container corresponding to each kernel version is constructed according to each compilation and construction image, and the client program of each kernel version is compiled and constructed in each compilation and construction container, and the client program of each kernel version can be installed on the operating system of the corresponding kernel version. In other words, the present application constructs compilation and construction containers corresponding to different kernel versions on a server to produce compilation and construction environments for different versions of operating systems, and then the compilation and construction container of each kernel version is the same as the compilation and construction environment of each corresponding kernel version of the operating system, without the need to can different operating systems on different servers. Therefore, it can solve the problem of resource occupation caused by canning different operating systems on different servers in the related art, and then compiling and constructing corresponding versions of kernel clients on the operating systems of different servers.

[0039] Optionally, the step S304 of creating a compile image corresponding to each kernel version according to the dynamic link library and the target driver file includes:

[0040] (1) Obtaining an image construction script template corresponding to the compiled image corresponding to each kernel version; constructing a target image construction script corresponding to each kernel version according to the dynamic link library, the target driver file, and the image construction script template; Specifically:

[0041] The target driver file is written to the first directory in the image construction script template; the first warehouse configuration file in the second directory in the image construction script template is cleared, and the second warehouse configuration file matching each kernel version is written to the second directory, wherein the second warehouse configuration file is used to indicate the software package that each kernel version depends on; the dynamic link library is written to the third directory in the image construction script template; a dynamic link preload variable is added to the image construction script template, wherein the dynamic link preload variable is used to enable the kernel information function to return the kernel version number corresponding to each kernel version; the first software package cache in the fourth directory in the image construction script template is cleared, and the second software package cache is written to the fourth directory based on the second warehouse configuration file in the second directory, wherein the second software package cache is used to install the software package that each kernel version depends on; a soft link is created in the image construction script template, and the image construction script template after the soft link is created is determined as the target image construction script, wherein the soft link is used to indicate the location of the source code in the fifth directory in the image construction script template, and the source code is used to compile and build the client program for each kernel version.

[0042] It is understandable that the steps of building the target image build script may include:

[0043] Write the target driver file to the first directory of the image build script: In the Dockerfile, use the copy command to copy the necessary driver files (for example, the Mellanox OFED driver) to a specified directory within the container (the compilation and build container, hereinafter referred to as the container). This could be a directory like / opt / or / usr / local / . This technical solution ensures the necessary drivers are present in the container environment so that the kernel compilation process can correctly identify and utilize hardware features.

[0044] Clear the repository configuration files in the second directory and add the repository configuration files that match each kernel version: Use the RUN command to first clear the default repository configuration files in the container, and then add the repository configuration files required for a specific kernel version, such as the KeyarchOS repository. This technical solution ensures that the container can access the different software packages required to compile a specific kernel version.

[0045] Write the dynamic link library to a third directory and set the preload variable: Use the copy command to copy the dynamic link library uname_wrap.so, which modifies the uname return value, to the / lib / directory within the container. Then, when the container starts, preload this library by setting the LD_PRELOAD environment variable (dynamic link preload variable). This ensures that the uname command returns the predefined kernel version number, rather than the container's default kernel version.

[0046] Clear the package cache in the fourth directory and create a second package cache based on the modified repository configuration file: Clear the YUM cache in the container (i.e., the first package cache), and then generate a new package cache (i.e., the second package cache) based on the added repository configuration file, so that you can subsequently install required software packages from a specific repository, such as kernel header files and development tools.

[0047] Create a soft link to indicate the source code location in the fifth directory: Use the ln-sf command to create a soft link to link the source code directory in the container with a specific path, such as / lib / modules and / usr / src / kernels, thereby allowing the compiler to easily find the correct source code and kernel version at build time.

[0048] The above technical solution improves the compilation and build efficiency and stability of the kernel client program. Each time a new Linux kernel version is adapted, only the configuration and repository information in the Dockerfile (i.e., the image build script template) needs to be fine-tuned to quickly generate a new compilation environment image, eliminating the need to repeatedly deploy the environment on multiple physical servers.

[0049] Among them, after adding the dynamic link preloading variable in the image construction script template, the method also includes: when the dynamic link preloading variable detects that the kernel information function is called, intercepting the original return value of the kernel information function, and determining the kernel version number corresponding to each kernel version in the dynamic link library as the return value of the kernel information function, so that the kernel information function returns the kernel version number corresponding to each kernel version.

[0050] It is understood that when the dynamic link preload variable (LD_PRELOAD) is set, the dynamic link preload variable (LD_PRELOAD) is used by the linker to load the library that is preloaded before the process starts. This allows developers to replace or enhance the behavior of library functions without modifying the application code. In the above technical solution, the uname function (i.e., the kernel information function) is selected as the target for redirection because it is the standard way to obtain system kernel version information.

[0051] The technical solutions for changing the return value of kernel information functions by dynamically linking preloaded variables include:

[0052] Writing a custom uname_wrap.so library (dynamic link library): First, you need to write a library called uname_wrap.so. This library reimplements the uname function. That is, when uname is called, it does not call the native uname function, but instead directly returns a hard-coded kernel version number, for example, "5.14.0-362.8.1.el9_3.x86_64".

[0053] Copy and set the dynamic link library in the Dockerfile: Use the copy command in the Dockerfile to copy uname_wrap.so to the standard system library directory inside the container, such as / lib. Then, use the RUN echo command to set the LD_PRELOAD environment variable in the container's startup script to point to uname_wrap.so.

[0054] Run a Docker container: When running a Docker container, since the LD_PRELOAD variable is correctly set when the container is started, any application that attempts to call the uname function will use the implementation provided by the custom library instead of the system default implementation.

[0055] The above technical solution ensures that all compilation tools and scripts in the container can access consistent kernel version information, avoiding compilation errors or warnings caused by inconsistencies between the container environment and the expected kernel version.

[0056] (2) Creating a compiled build image corresponding to each kernel version according to the target image build script.

[0057] Optionally, the step S306 of compiling and building the client program of each kernel version in each compilation and construction container includes:

[0058] (1) a first acquisition step: acquiring compilation parameters corresponding to the client program of each kernel version, wherein the compilation parameters include at least one of the following: a source code branch, a container name, and a version number, wherein the source code branch is used to indicate a set of source codes for compiling and constructing the client program of each kernel version, the container name is the name of the compilation and construction container corresponding to the client program of each kernel version, and the version number is the version number of the client version corresponding to the client program; a second acquisition step: acquiring the source code corresponding to the client program in a source code management system according to the source code branch; a copying step: copying the source code to the compilation and construction container and decompressing the source code; a compilation and construction step: compiling and constructing the decompressed source code to obtain an executable file from the decompressed source code, and packaging the executable file through a packaging command to obtain an installation package corresponding to the client program of each kernel version;

[0059] (2) constructing a programming script corresponding to the client program of each kernel version based on the first acquisition step, the second acquisition step, the copying step, and the compiling and constructing step, wherein the programming script corresponding to the client program of each kernel version is used to compile and construct the client program of each kernel version.

[0060] It is understandable that the steps of compiling and building the client program for each kernel version may include:

[0061] The first acquisition step: Get the compilation parameters. Before starting the build process, collect a set of specific parameters, mainly including: source code branch, container name and version number, etc. Among them:

[0062] Source branch: identifies a specific version or development status of the source code, for example: feature / 5.15_client_updates.

[0063] Container name: For example, ikc_build_rocky_9.3, which represents the name of the dedicated Docker container used for compilation and building.

[0064] Version number: For example, 5.14.0-362.8.1.el9_3.x86_64, specifies the target kernel version of the client program and its detailed specifications.

[0065] The second acquisition step: source code acquisition, using the acquired source code branch information to pull the source code of the client program from the source code management system (such as GitLab or GitHub).

[0066] Copying steps: Import the source code into the container, copy the source code into the corresponding Docker container, and decompress it in preparation for the build. For example, use docker cp or directly use the ADD or COPY command in the Dockerfile to copy the source code to the / source directory of the running ikc_build_rocky_9.3 container and decompress it.

[0067] Compilation and build steps: Generate an installation package, compile the decompressed source code to obtain an executable file, and then use a packaging tool (such as RPM or DEB package) to package the executable file into a client installation package. For example, run the compilation command in the container, such as using the make tool to build, and then use rpmbuild to package the generated files into an RPM installation package.

[0068] Build programming scripts: Automated compilation and construction. To further improve efficiency, the above-mentioned process of obtaining parameters, obtaining source code, copying and decompressing source code, and compiling and building is solidified into a shell script, so that each build only requires a small number of parameters to automatically complete the entire process.

[0069] The above technical solution, by encoding the compilation and build process into scripts, achieves a high degree of standardization and automation, reducing human involvement and the potential for error, and improving build efficiency and quality. Furthermore, encoding the compilation and build process into parameterized scripts allows for flexible adaptation to the compilation and build requirements of different kernel versions and operating systems.

[0070] Among them, after building the programming script corresponding to the client program of each kernel version based on the first acquisition step, the second acquisition step, the copying step and the compilation and construction step, the method also includes: when it is necessary to compile and construct the first client program of the first kernel version and the second client program of the second kernel version at the same time, creating multiple parallel compilation and construction nodes, and obtaining the first programming script corresponding to the first client program of the first kernel version and the second programming script corresponding to the second client program of the second kernel version; adding the first programming script to the first compilation and construction node among the multiple compilation and construction nodes, and adding the second programming script to the second compilation and construction node among the multiple compilation and construction nodes; compiling and constructing the first client program of the first kernel version and the second client program of the second kernel version in parallel based on the first compilation and construction node and the second compilation and construction node.

[0071] It is understandable that after constructing the programming script, the first client program of the first kernel version and the second client program of the second kernel version may be compiled and constructed simultaneously. Specifically:

[0072] Create parallel compilation and build nodes. Create multiple compilation and build nodes in continuous integration / continuous deployment tools (such as Jenkins). Each node is actually an independent build environment that can execute compilation tasks in parallel in its own context without being affected by other tasks. Suppose you need to compile the first client program of Centos 7.6 kernel version 3.10.0-957.el7 and the second client program of Centos 8.4 kernel version 4.18.0-305.3.1.el8 at the same time. Configure two build nodes in Jenkins, namely Node A and Node B. These two nodes will independently compile and build the client programs.

[0073] Obtain programming scripts. For each kernel version and corresponding client program, a dedicated programming script is required. Assign scripts to build nodes. Assign programming scripts corresponding to different kernel versions to specific compilation and build nodes.

[0074] Parallel compilation and construction: After the script is assigned, each compilation and construction node will independently execute the compilation and construction process, and finally generate installation packages of two client programs with different kernel versions at the same time.

[0075] By implementing the above-mentioned technical solution of executing multiple compilation and construction tasks in parallel, the overall construction time is significantly shortened, especially when a large number of client programs with different kernel versions need to be built, thereby improving compilation efficiency.

[0076] Among them, after building the programming script corresponding to the client program of each kernel version based on the first acquisition step, the second acquisition step, the copying step and the compilation and construction step, the method also includes: when it is necessary to compile and construct the third client program of the third kernel version on a regular basis, obtaining the third programming script corresponding to the third client program of the third kernel version; adding the third programming script to the third compilation and construction node among the multiple compilation and construction nodes; adding a timer to the target compilation project, wherein the target compilation project includes: the third compilation and construction node; setting a scheduled task for the third compilation and construction node based on the timer, wherein the scheduled task is used to indicate the target time for executing the third compilation and construction node; and compiling and constructing the third client program of the third kernel version at the target time based on the third compilation and construction node.

[0077] It is understandable that compilation and building can also be performed regularly, specifically:

[0078] Step 1: Obtain the programming script corresponding to the specific kernel version and client program. The programming script contains all the compilation and build instructions, as well as environment configuration information to ensure smooth compilation in the specific Docker container.

[0079] Step 2: Assign the script to the build node. In the continuous integration and continuous deployment tool (such as Jenkins), create a new compile and build node and add the previously prepared programming script to the node.

[0080] Step 3: Set up scheduled tasks. In your continuous integration and continuous deployment tool, add a timer to the target build project to ensure that tasks on the build node are automatically executed at the preset time. By configuring a timer, you can regularly check for source code updates and automatically trigger a build without manual intervention.

[0081] Step 4: Schedule compilation and build. According to the configuration of the scheduled task, the continuous integration and continuous deployment tool will automatically execute the compilation and build work on the third compilation and build node at the target time.

[0082] Through the above technical solution, even in an unattended environment, the latest code changes can be regularly checked and compiled to ensure the compatibility of client programs and kernel versions, as well as the synchronization of development progress and requirements.

[0083] In order to better understand the process of the compilation and construction method of the above-mentioned client program, the implementation process of the compilation and construction method of the above-mentioned client program is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of this application.

[0084] Related technologies typically install different operating systems on different servers and compile and build the corresponding kernel client versions directly on the operating systems. However, this approach requires multiple servers if multiple versions are required, resulting in significant resource usage and poor scalability. To conserve resources, virtual machines with different operating system versions are installed on a single physical machine to achieve the same effect. However, this approach consumes significant space and has limited portability. Furthermore, if too many virtual machines are installed on a single server, performance can be degraded.

[0085] Therefore, an optional embodiment of the present application proposes a method and device for batch compiling and building kernel client programs based on Docker containers, creating a set of tools based on Docker container technology, producing different versions of operating system compilation and construction environments by writing different DockerFile files, and combining Jenkins (a continuous integration server) with automated construction functions to achieve batch compilation and construction of different versions of kernel clients, solving the problems of time-consuming, resource-intensive, manpower-intensive, poor portability, and cumbersome operation under traditional methods.

[0086] Faced with a plethora of kernel versions, the traditional approach involves adding a server for each new version, installing the corresponding operating system, installing the compilation and build environment, and generating the kernel client installation package. This consumes enormous server resources, human resources, and time. Furthermore, if the kernel version corresponding to the installed operating system differs from the kernel version used in the project, the kernel version of the compilation and build server must be adjusted. This typically requires adjusting most dependencies, which can easily lead to dependency errors.

[0087] A method and apparatus for batch compiling and building a kernel client program based on a Docker container in an optional embodiment of the present application specifically includes:

[0088] 1. Preparation for compilation and construction:

[0089] (1) Prepare uname.so (kernel information dynamic link library).

[0090] Because in the docker container, when the kernel information command (unified name, abbreviated as uname command, that is, the kernel information function of this application) is executed, the kernel version number of the host machine is returned. However, when you want to compile and build different versions of kernel client programs in the docker container, you need the uname command to be able to return the kernel version number to be compiled and built, so you need to change the return result of the uname command in the docker container. Therefore, write the uname_wrap.c program, and use the LD_PRELOAD (dynamic linker preloading environment variable, that is, the dynamic link preloading variable of this application) mechanism to intercept and modify the uname system call return value. The compiler generates the uname_wrap.so dynamic link library. In the subsequent docker container startup phase, the code replaces the uname function in the standard library through the preloading of the dynamic link library, thereby customizing the returned kernel version information.

[0091] (2) Prepare the MLNX driver.

[0092] MLNX_OFED (an open Ethernet driver software collection) is a Virtual Protocol Interconnect (VPI) software stack that runs on all NVIDIA network adapter solutions, supporting InfiniBand (an infinite bandwidth network technology) and Ethernet connections. It enables users to configure, manage, and optimize network adapters and leverage their high-speed data transfer and low latency for high-performance computing and data center applications. The MLNX driver is required for compiling and building the kernel client, so you can download the corresponding MLNX driver package in advance.

[0093] (3) Create a Dockerfile (i.e., target image build script):

[0094] A Dockerfile is a text file that contains a series of commands and instructions that define how to build a Docker image (i.e., compile and build the image). All the resources and instructions required to create the compilation and build environment are defined in the Dockerfile. Specifically:

[0095] FROM rockylinux:9.3 / / Use Rocky Linux 9.3 as the base image.

[0096] COPY . / * / opt / / / Copies all files and subdirectories in the current directory to the / opt / directory of the container.

[0097] COPY . / mlnx / * / opt / mlnx / / / Copy the Mellanox driver file to the / opt / mlnx / (the first directory) directory of the container.

[0098] RUN rm-rf / etc / yum.repos.d / * / / Clear all YUM repository configuration files (i.e. the first repository configuration files) under the / etc / yum.repos.d / (i.e. the second directory) directory.

[0099] RUN cp / opt / KeyarchOS-5.8.repo / etc / yum.repos.d / / / Copy the KeyarchOS warehouse configuration file (that is, the first warehouse configuration file) to the / etc / yum.repos.d / directory.

[0100] RUN mv / opt / uname_wrap.so / lib / / / Move the uname_wrap.so dynamic link library (i.e. dynamic link library) to the / lib / (i.e. the third directory) directory.

[0101] RUN echo "export LD_PRELOAD = / lib / uname_wrap.so" >> / etc / bashrc / / Add a line to the / etc / bashrc file and set the environment variable LD_PRELOAD (dynamic link preloading variable) to point to uname_wrap.so. In this way, the uname command executed in the container will call the custom library and return the specified kernel version number.

[0102] RUN yum clean all / / Cleans the cache (i.e., the first package cache) of a repository update tool (Yellowdog Updater Modified, referred to as YUM) to ensure that the package installation starts from a clean state.

[0103] RUN yum makecache / / Generate a new YUM package cache (i.e., the second package cache) and prepare to install the new package.

[0104] RUN yum-y install kernel-headers-5.14.0-362.8.1.el9_3.x86_64;

[0105] RUN yum-y install kernel-devel-5.14.0-362.8.1.el9_3.x86_64;

[0106] RUN yum-y install kernel-modules-extra-5.14.0-362.8.1.el9_3.x86_64;

[0107] RUN yum-y install gcc gcc-c++make libtool rpm-build rpmdevtoolspciutils;

[0108] RUN yum-y install git lrzsz wget vim lsof perl elfutils-libelf-devel;

[0109] RUN yum-y install tcl tcsh tk gcc-gfortran;

[0110] RUN yum -y install python2 libnl3 ethtool openssl; / / This series of RUN commands is used to install various software packages, including:

[0111] kernel-headers: Kernel header files, used to build kernel modules and other kernel-related software.

[0112] kernel-devel: kernel development library, containing the symbols and specifications needed to compile kernel modules.

[0113] kernel-modules-extra: additional kernel modules, which may contain some special drivers or libraries.

[0114] Compilation tools include: gcc, gcc-c++, make (all are compilation tools), etc.

[0115] Other tools and libraries such as libtool, rpm-build, rpmdevtools, pciutils, etc. are necessary tools for building and packaging software.

[0116] Development-related tools such as git, wget, vim, lsof, perl, elfutils-libelf-devel, etc. are used for pulling code, text editing, debugging, etc.

[0117] More compilation tools and libraries, such as tcl, tcsh, tk, gcc-gfortran, etc., are used to support more programming languages ​​and compilation requirements.

[0118] Runtime libraries such as python2, libnl3, ethtool, openssl, etc., which may be required to run client programs.

[0119] RUN rpm-Uvh / opt / mlnx / *.rpm / / Update or upgrade the Mellanox driver to ensure that the network adaptation in the container meets the requirements of high-performance computing.

[0120] RUN cd / lib / modules / 5.14.0-362.8.1.el9_3.x86_64 / / / Switch to the module directory of a specific kernel version.

[0121] RUN ln-sf / usr / src / kernels / 5.14.0-362.8.1.el9_3.x86_64build / / Create a soft link pointing to the kernel source code directory, which helps to find the correct kernel source code when compiling the kernel module.

[0122] RUN ln-sf build source / / Create another soft link, this time linking the build directory (the fifth directory) as source (source code), which simplifies directory references in subsequent build processes.

[0123] (4) Generate a compiled build image:

[0124] After running the docker build -t ikc_build_rocky_9.3:v1. command (one command), a compilation and build image is generated. After obtaining the image, you can create a compiled and built Docker container (that is, a compilation and build container, also known as a container). In the container, you can compile and build the kernel client source code to generate the kernel client executable program.

[0125] 2. Solidification of compilation and construction process:

[0126] After creating different Docker containers according to different versions of Docker images, you can solidify the compilation process and use different shell scripts (i.e. programming scripts) to complete all compilation work. Figure 4 This is a flowchart of a compilation process solidified according to an optional embodiment of the present application, such as Figure 4 As shown, specifically:

[0127] Step S401: Compilation and construction begins, and parameters (ie, compilation parameters) are obtained.

[0128] The parameters include at least: version number, kernel version, container name, and source code branch.

[0129] Step S402: Pull the code.

[0130] Pull the latest version of source code from GIT (a source code management system) according to the name of the source code branch.

[0131] Step S403: copy the code.

[0132] After packaging the latest source code, copy it to the directory corresponding to the compiled source.

[0133] Step S404: Replace the version.

[0134] Pull the compiled SPEC template file (i.e., executable file) from the source code and replace the client version number.

[0135] Step S405: Start compiling and building a Docker container.

[0136] Step S406: Start compiling and building.

[0137] Use the spmbuild command to generate an RPM package (a software package, i.e., an installation package) according to the SPEC configuration file.

[0138] Step S407: Use the jfrog tool (a tool) to upload the rpm package to the cloud warehouse.

[0139] 3. Continuous integration adaptation:

[0140] Create a pipeline project in Jenkins and configure the code repository and branches. In the build section, add a build step and assign the pre-built shell script to it. Then, configure multiple build nodes and use the "Parallel Execution" feature to build multiple projects simultaneously. Configure scheduled builds for automated builds. This allows you to batch build multiple versions.

[0141] The optional embodiment of the present application implements batch compilation, construction, packaging, and uploading functions for kernel client programs of different versions based on Docker container technology. The originally complex and cumbersome work items with serious resource occupation and poor scalability are transformed into simple and clear general components with low resource occupation and strong scalability. For development and maintenance personnel, they only need to follow the template and make simple adjustments to complete the compilation and construction of a new adapted version of the kernel client. This greatly facilitates the continuous integration of the kernel client, shortens daily operation and maintenance time, and frees developers from the tedious compilation and construction work.

[0142] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0143] In this embodiment, a client program compilation and construction device is also provided, which is used to implement the above embodiments and preferred implementation methods. The details that have been described will not be repeated. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0144] Figure 5 This is a structural block diagram of a device for compiling and building a client program according to an embodiment of the present application. Figure 5 As shown, the device includes:

[0145] A generating module 52 is configured to generate a dynamic link library corresponding to the kernel version of each operating system according to the kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version;

[0146] A creation module 54 is used to create a compilation image corresponding to each kernel version according to the dynamic link library and the target driver file;

[0147] The construction module 56 is used to build a compilation and construction container corresponding to each kernel version according to each compilation and construction image, and compile and build the client program of each kernel version in each compilation and construction container, wherein the client program of each kernel version is installed in the operating system corresponding to each kernel version.

[0148] Through the compilation and construction device of the client program of the present application, a dynamic link library corresponding to each kernel version is generated according to the kernel information function that modifies the return value to the kernel version number corresponding to each kernel version of each operating system; a compilation and construction image corresponding to each kernel version is created according to the dynamic link library and the target driver file; a compilation and construction container corresponding to each kernel version is constructed according to each compilation and construction image, and a client program of each kernel version is compiled and constructed in each compilation and construction container, and the client program of each kernel version can be installed on the operating system of the corresponding kernel version. In other words, the present application constructs compilation and construction containers corresponding to different kernel versions on a server to produce compilation and construction environments for different versions of operating systems, and then the compilation and construction container of each kernel version is the same as the compilation and construction environment of each corresponding kernel version of the operating system, without the need to can different operating systems on different servers. Therefore, the problem of resource occupation caused by canning different operating systems on different servers and then compiling and constructing corresponding versions of kernel clients on the operating systems of different servers in the related art can be solved.

[0149] In an exemplary embodiment, the creation module 54 is also used to obtain the image construction script template corresponding to the compiled image construction corresponding to each kernel version; construct the target image construction script corresponding to each kernel version according to the dynamic link library, the target driver file and the image construction script template; and create the compiled image construction corresponding to each kernel version according to the target image construction script.

[0150] In an exemplary embodiment, the creation module 54 is further configured to write the target driver file to the first directory in the image construction script template; clear the first warehouse configuration file in the second directory in the image construction script template, and write the second warehouse configuration file matching each kernel version to the second directory, wherein the second warehouse configuration file is used to indicate the software package that each kernel version depends on; write the dynamic link library to the third directory in the image construction script template; add a dynamic link preload variable in the image construction script template, wherein the dynamic link preload variable is used to enable the kernel information function to return the kernel version number corresponding to each kernel version; clear the first software package cache in the fourth directory in the image construction script template, and write the second software package cache in the fourth directory based on the second warehouse configuration file in the second directory, wherein the second software package cache is used to install the software package that each kernel version depends on; create a soft link in the image construction script template, and determine the image construction script template after creating the soft link as the target image construction script, wherein the soft link is used to indicate the location of the source code in the fifth directory in the image construction script template, and the source code is used to compile and build the client program for each kernel version.

[0151] In an exemplary embodiment, the creation module 54 is also used to intercept the original return value of the kernel information function when the dynamic link preloading variable detects that the kernel information function is called, and determine the kernel version number corresponding to each kernel version in the dynamic link library as the return value of the kernel information function, so that the kernel information function returns the kernel version number corresponding to each kernel version.

[0152] In an exemplary embodiment, the construction module 56 is further used to perform a first acquisition step: obtaining compilation parameters corresponding to the client program of each kernel version, wherein the compilation parameters include at least one of the following: a source code branch, a container name, and a version number, the source code branch is used to indicate a set of source codes for compiling and constructing the client program of each kernel version, the container name is the name of the compilation and construction container corresponding to the client program of each kernel version, and the version number is the version number of the client version corresponding to the client program; a second acquisition step: obtaining the source code corresponding to the client program in the source code management system according to the source code branch; a copying step: copying the source code to the compilation and construction container and decompressing the source code; a compilation and construction step: compiling and constructing the decompressed source code to obtain an executable file from the decompressed source code, and packaging the executable file through a packaging command to obtain an installation package corresponding to the client program of each kernel version; and constructing a programming script corresponding to the client program of each kernel version based on the first acquisition step, the second acquisition step, the copying step, and the compilation and construction step, wherein the programming script corresponding to the client program of each kernel version is used to compile and construct the client program of each kernel version.

[0153] In an exemplary embodiment, the construction module 56 is also used to create multiple parallel compilation and construction nodes when it is necessary to compile and construct the first client program of the first kernel version and the second client program of the second kernel version at the same time, and obtain a first programming script corresponding to the first client program of the first kernel version and a second programming script corresponding to the second client program of the second kernel version; add the first programming script to the first compilation and construction node among the multiple compilation and construction nodes, and add the second programming script to the second compilation and construction node among the multiple compilation and construction nodes; compile and construct the first client program of the first kernel version and the second client program of the second kernel version in parallel based on the first compilation and construction node and the second compilation and construction node.

[0154] In an exemplary embodiment, the construction module 56 is also used to obtain a third programming script corresponding to the third client program of the third kernel version when it is necessary to compile and build the third client program of the third kernel version on a regular basis; add the third programming script to the third compilation and construction node among the multiple compilation and construction nodes; add a timer in the target compilation project, wherein the target compilation project includes: the third compilation and construction node; set a scheduled task for the third compilation and construction node based on the timer, wherein the scheduled task is used to indicate the target time for executing the third compilation and construction node; and compile and build the third client program of the third kernel version at the target time based on the third compilation and construction node.

[0155] For the description of the features in the embodiment corresponding to the device for compiling and building a client program, please refer to the relevant description of the embodiment corresponding to the method for compiling and building a client program, which will not be repeated here.

[0156] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned client program compilation and construction method embodiments.

[0157] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned client program compilation and construction method embodiments when running.

[0158] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0159] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned client program compilation and construction method embodiments are implemented.

[0160] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps in any of the above-mentioned client program compilation and construction method embodiments.

[0161] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0162] The above is a detailed introduction to the compilation and construction method of a client program provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to this application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

Claims

1. A method for compiling and building a client program, characterized in that: Applied to servers, including: Generate a dynamic link library corresponding to the kernel version of each operating system according to the kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version; Create a compile and build image corresponding to each kernel version based on the dynamic link library and target driver file; A compilation and construction container corresponding to each kernel version is constructed according to each compilation and construction image, and a client program of each kernel version is compiled and constructed in each compilation and construction container, wherein the client program of each kernel version is installed in the operating system corresponding to each kernel version.

2. The method for compiling and constructing a client program according to claim 1, wherein: Create a compilation image corresponding to each kernel version based on the dynamic link library and target driver file, including: Obtain the image building script template corresponding to the compiled image built by each kernel version; Construct a target image construction script corresponding to each kernel version according to the dynamic link library, the target driver file and the image construction script template; Create a compiled build image corresponding to each kernel version according to the target image build script.

3. The method for compiling and constructing a client program according to claim 2, wherein: Constructing a target image construction script corresponding to each kernel version according to the dynamic link library, the target driver file, and the image construction script template includes: Write the target driver file to the first directory in the image building script template; Clear the first warehouse configuration file under the second directory in the image construction script template, and write the second warehouse configuration file matching each kernel version into the second directory, wherein the second warehouse configuration file is used to indicate the software package that each kernel version depends on; Writing the dynamic link library into the third directory in the image building script template; Adding a dynamic link preloading variable in the image building script template, wherein the dynamic link preloading variable is used to enable the kernel information function to return the kernel version number corresponding to each kernel version; Clearing a first software package cache under a fourth directory in the image building script template, and writing a second software package cache under the fourth directory based on the second warehouse configuration file under the second directory, wherein the second software package cache is used to install the software packages that each kernel version depends on; A soft link is created in the image construction script template, and the image construction script template after creating the soft link is determined as the target image construction script, wherein the soft link is used to indicate the location of the source code under the fifth directory in the image construction script template, and the source code is used to compile and build the client program of each kernel version.

4. The method for compiling and constructing a client program according to claim 3, wherein: After adding the dynamic link preloading variable in the image building script template, the method further includes: When the dynamic link preloading variable detects that the kernel information function is called, the original return value of the kernel information function is intercepted, and the kernel version number corresponding to each kernel version in the dynamic link library is determined as the return value of the kernel information function, so that the kernel information function returns the kernel version number corresponding to each kernel version.

5. The method for compiling and constructing a client program according to claim 1, wherein: Compile and build the client program for each kernel version in each compilation and build container, including: The first acquisition step is to obtain compilation parameters corresponding to the client program of each kernel version, wherein the compilation parameters include at least one of the following: a source code branch, a container name, and a version number, wherein the source code branch is used to indicate a set of source codes for compiling and constructing the client program of each kernel version, the container name is the name of the compilation and construction container corresponding to the client program of each kernel version, and the version number is the version number of the client version corresponding to the client program; Second acquisition step: acquiring source code corresponding to the client program in a source code management system according to the source code branch; Copying step: copying the source code into the compilation and construction container, and decompressing the source code; Compile and build step: compile and build the decompressed source code to obtain an executable file from the decompressed source code, and package the executable file through a packaging command to obtain an installation package corresponding to the client program of each kernel version; Based on the first acquisition step, the second acquisition step, the copying step and the compilation and construction step, a programming script corresponding to the client program of each kernel version is constructed, wherein the programming script corresponding to the client program of each kernel version is used to compile and construct the client program of each kernel version.

6. The method for compiling and constructing a client program according to claim 5, wherein: After constructing the programming script corresponding to the client program of each kernel version based on the first obtaining step, the second obtaining step, the copying step, and the compiling and constructing step, the method further includes: In the case where it is necessary to compile and build a first client program of a first kernel version and a second client program of a second kernel version at the same time, creating multiple parallel compilation and construction nodes, and obtaining a first programming script corresponding to the first client program of the first kernel version and a second programming script corresponding to the second client program of the second kernel version; Adding the first programming script to a first compile construction node among a plurality of compile construction nodes, and adding the second programming script to a second compile construction node among the plurality of compile construction nodes; A first client program of the first kernel version and a second client program of the second kernel version are compiled and constructed in parallel based on the first compile and construct node and the second compile and construct node.

7. The method for compiling and constructing a client program according to claim 6, wherein: After constructing the programming script corresponding to the client program of each kernel version based on the first obtaining step, the second obtaining step, the copying step, and the compiling and constructing step, the method further includes: When the third client program of the third kernel version needs to be compiled and constructed regularly, obtaining a third programming script corresponding to the third client program of the third kernel version; adding the third programming script to a third compile build node among the plurality of compile build nodes; Adding a timer in a target compilation project, wherein the target compilation project includes: the third compilation construction node; Setting a timed task for the third compile and build node based on the timer, wherein the timed task is used to indicate a target time for executing the third compile and build node; Compile and build the third client program of the third kernel version based on the third compile and build node at the target time.

8. A client program compilation and construction device, characterized in that: include: A generation module is used to generate a dynamic link library corresponding to the kernel version of each operating system according to the kernel information function after modifying the return value, wherein the modified return value of the kernel information function is the kernel version number corresponding to each kernel version; Create a module for creating a compile and build image corresponding to each kernel version based on the dynamic link library and target driver file; A construction module is used to build a compilation and construction container corresponding to each kernel version according to each compilation and construction image, and to compile and build a client program for each kernel version in each compilation and construction container, wherein the client program for each kernel version is installed in the operating system corresponding to each kernel version.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the method for compiling and constructing the client program according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for compiling and constructing the client program according to any one of claims 1 to 7 are implemented.

Images