File compiling method, distributed compiling system and server

Abstract

The file compiling method is applied to a first server and comprises the steps that dependency package information of a target software package is obtained, and the dependency package information indicates at least one dependency package; according to the dependency package information of the target software package, first information is sent to each second server in a plurality of second servers, the first information comprises at least one dependency package name, and the first information is used for instructing the second servers to compile a dependency package according to the received dependency package name, the compiled dependent package is stored in a shared directory, and the shared directory is supported to be accessed by the plurality of second servers; and synthesizing the target software package according to the compiled dependent package stored in the shared directory. Through the distributed compiling file, the construction time of the target system can be shortened, and the construction efficiency is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a file compilation method, a distributed compilation system, and a server. Background Art

[0002] A baseboard management controller (BMC) is a small, dedicated Linux-based management system that runs independently of the server's business systems. It monitors, manages, and maintains these systems. The BMC's target product includes a complete operating system and user software.

[0003] OpenBMC is a Linux distribution for BMC that can be flexibly customized to support different versions of BMC SoCs or boards. Typically, OpenBMC can be customized through the Yocto Project. However, custom compilation of OpenBMC through the Yocto Project requires only a single compilation server. The number of concurrent compilation tasks depends on the number of CPU cores on the compilation server, and the time required to customize OpenBMC compilation is limited by the configuration of the compilation server. This can lead to excessively long compilation times. Summary of the Invention

[0004] The embodiments of the present application provide a file compilation method, a distributed compilation system, and a server. By distributing the compilation of target software package files, the construction time of the target system can be reduced and the construction efficiency can be significantly improved.

[0005] In a first aspect, an embodiment of the present application provides a file compilation method, which is characterized in that, when applied to a first server, the method includes: obtaining dependency package information of a target software package, the dependency package information indicating at least one dependency package; sending first information to each of a plurality of second servers respectively according to the dependency package information of the target software package, the first information including at least one dependency package name, the first information being used to instruct the second server to compile the dependency package according to the received dependency package name, and store the compiled dependency package in a shared directory, the shared directory supporting access by multiple second servers; synthesizing the target software package according to the compiled dependency package stored in the shared directory.

[0006] In this solution, a shared directory is configured on the first server and made accessible to multiple second servers. The multiple compiled packages required to build the target software package are then distributed to the multiple second servers for simultaneous construction, achieving distributed compilation. Building the target software package using this distributed compilation method fully utilizes server resources, reduces target software package build time, and improves build efficiency.

[0007] In a possible implementation, the dependent package information of the target software package includes at least one of: dependent package name, total number of dependent packages, compilation time of dependent packages, and an application program interface for calling the integrated scheduling system.

[0008] In this solution, the obtained dependency package information of the target software package may include information about all dependent packages required to build the target software package, such as the total number of dependent packages required to build the target software package, the name of each dependent package, the compilation time of each dependent package, etc. This is not limited in the embodiments of the present application.

[0009] In one possible implementation, before sending the first information to each of the multiple second servers based on the dependency package information of the target software package, the method includes: obtaining the number of second servers; determining the number of dependency package names carried in the first information based on the total number of dependency packages and the number of second servers, so that the number of dependency packages compiled by any second server is the same or similar.

[0010] In this solution, in order to balance the workload of each second server when compiling dependent packages, the number of dependent packages that need to be compiled by each second server can be determined based on the total number of dependent packages that need to be compiled and the number of second servers, so as to ensure that the number of dependent packages compiled by each second server is the same or similar.

[0011] In one possible implementation, before sending the first information to each of the multiple second servers based on the dependency package information of the target software package, the method includes: obtaining the number of second servers; determining the number of dependency package names carried in the first information based on the total number of dependency packages, the compilation time information of each dependency package, and the number of second servers, so that the compilation time of the dependency package compiled by any second server is the same or similar.

[0012] In this solution, the compilation time of each dependency package can vary significantly during the actual compilation process. When each server distributes the dependency packages to be built to the second server, it obtains the individual compilation time of each dependency package in advance. This, combined with the number of dependency packages, allows for a more reasonable determination of the number of dependency packages that each second server needs to compile.

[0013] In one possible implementation, the multiple second servers include the first server, and the method further includes: obtaining source code corresponding to at least one dependent package name based on the first information; compiling the source code to obtain a compiled dependent package; and storing the compiled dependent package in a mounted directory, where the mounted directory points to a shared directory.

[0014] In this solution, the first server may be one of the second servers. When compiling dependent packages of the target software package, the first server may also participate in compiling the dependent packages, so that the server can be fully utilized.

[0015] In a second aspect, an embodiment of the present application provides a distributed compilation system, comprising: multiple servers, one of the multiple servers being provided with a shared directory, the shared directory supporting access by each of the multiple servers; a first server among the multiple servers, for obtaining dependency package information of a target software package, the dependency package information indicating at least one dependency package; the first server, further for sending first information to a second server among the multiple servers, respectively, the first information including at least one dependency package name; the second server, for compiling the dependency package according to the first information, and storing the compiled dependency package in the shared directory; any one of the multiple servers, for obtaining the compiled dependency package from the shared directory to synthesize the target software package.

[0016] In one possible implementation, the first server is configured to determine the number of dependency package names carried in the first information based on the total number of dependency packages and the number of second servers, so that the number of dependency packages compiled by any second server is the same or similar.

[0017] In one possible implementation, the first server is used to determine the number of dependent package names carried in the first information based on the total number of dependent packages, the compilation time information of each dependent package, and the number of second servers, so that the compilation time of any second server compiling the dependent package is the same or similar.

[0018] In one possible implementation, the second server includes the first server, and the second server is used to: obtain source code corresponding to at least one dependent package name based on the first information; compile the source code to obtain a compiled dependent package; and store the compiled dependent package in a mounted directory, where the mounted directory points to a shared directory.

[0019] In a third aspect, an embodiment of the present application provides a server, including:

[0020] at least one memory for storing a program;

[0021] At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0022] In a fourth aspect, an embodiment of the present application provides a computing device, including:

[0023] at least one memory for storing a program;

[0024] At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0025] In a fifth aspect, an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the method described in the first aspect or any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0028] Figure 1 A schematic diagram of a process for generating a target software package;

[0029] Figure 2 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of a distributed file compilation process provided in an embodiment of the present application;

[0032] Figure 5 A flowchart of a file compilation method provided in an embodiment of the present application;

[0033] Figure 6 A flowchart of a file compilation method provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of the structure of a compilation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0036] In the description of the embodiments of this application, any embodiment or design scheme using "exemplary," "for example," or "for example" should not be understood as being more preferred or advantageous than other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0037] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly identifying the technical features being referred to. Thus, features specified as "first" or "second" may explicitly or implicitly include one or more of such features. The terms "include," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0038] Before introducing this solution, the technical terms used in this solution are first introduced.

[0039] 1. Yocto Project: An open-source collaborative project that helps developers create customized Linux systems. It provides a flexible build system (Poky) that can generate Linux distributions for a variety of hardware architectures. The Yocto Project not only supports a wide range of hardware platforms but also provides a large number of ready-made software packages and tools, making the customization process simple and efficient.

[0040] The Yocto Project build relationship consists of the following key components:

[0041] Bitbike: A task executor for executing tasks in the build process. It uses Shell and Python tasks to run tasks in parallel efficiently within complex inter-task dependency constraints. Bitbike executes tasks based on the metadata provided to the tasks. The metadata is stored in recipe (.bb) and related recipe "append" (.bbappend) files, configuration (.conf) and base include (.inc) files, and class (.bbclass) files. The metadata provides BitBake with instructions about which tasks to run and the dependencies between those tasks.

[0042] Metadata: Contains a collection of recipes, configurations, and classes that define how to build various software packages and images.

[0043] Layers: A collection of metadata used to organize and isolate recipes and configurations.

[0044] Recipes: A set of instructions that tells BitBake how to build a specific software package or application.

[0045] Configuration files: define global settings for the build process, such as machine configuration and distribution configuration.

[0046] Before building a custom Linux distribution using the Yocto Project, you need to follow these steps:

[0047] (1) Environmental preparation

[0048] You need to ensure that your development environment meets the requirements of the Yocto Project. For example, you need a computer running the Linux operating system and necessary dependent software such as Git, tar, and Python.

[0049] (2) Download the Yocto Project

[0050] Use Git to clone the Poky build system from the official Yocto Project repository.

[0051] (3) Initialize the build environment

[0052] In the Poky directory, there is a script called oe-init-build-env, which is used to initialize the build environment. This will create a directory called build and set relevant environment variables.

[0053] (4)Configuration Build

[0054] In the build / conf directory, there are two important configuration files: local.conf and bblayers.conf. You can edit these configuration files according to the target device and requirements.

[0055] local.conf: defines the specific parameters of the build, such as the target machine, build output path, etc.

[0056] blayers.conf: defines which layers will be used during the build process.

[0057] (5) Adding layers

[0058] Yocto's layer mechanism allows adding hardware support, packages, and custom features. For example, you can use the bitbake-layers tool to add or remove layers.

[0059] (6) Write the recipe

[0060] Recipe is the key element in Yocto that defines how to build a software package. A recipe contains information such as the location of the source code, build dependencies, build steps, and installation steps.

[0061] (7)Build an image

[0062] Use BitBake to build your custom Linux image.

[0063] (8) Testing and deployment

[0064] After the build is complete, you can flash the generated image to the target device for testing. Make sure all features work as expected, and then you can start deploying your custom Linux distribution.

[0065] 2. NFS: (network file system), allows different machines and different operating systems to share each other's files through the network.

[0066] Next, this solution is introduced.

[0067] The BMC hardware itself is a computer system, and its hardware resources are very limited compared to common computer systems. Therefore, OpenBMC is designed as a complete Linux distribution that can be flexibly customized to support different BMC SoCs or boards. The OpenBMC image includes a bootloader (u-boot), a Linux kernel, open source packages, and board-specific packages. Both the bootloader and the Linux kernel include various hardware drivers for the BMC SoC, including I2C drivers, USB drivers, LPC drivers, PWM drivers, and SPI drivers. Open source packages generally include common applications such as BusyBox, I2C tools, lm sensors, OpenSSH, and Python.

[0068] When customizing OpenBMC, you can usually do so through the Yocto Project, an open-source collaborative software effort that provides templates, tools, and methods for creating customized Linux systems and embedded products, regardless of the hardware architecture.

[0069] The Yocto project uses bitbake as the build system engine, and concurrently runs multiple tasks to generate the target software package. For example, Figure 1 A schematic diagram of the process of generating a target software package is shown in FIG. Figure 1As shown, in the process of generating the target software package, it is necessary to complete the following steps in sequence: parse the recipe file → obtain the source code (do_fetch) → decompress the source code (do_unpack) → patch the source code (do_patch) → configure the source code compilation option information (do_configure) → compile the source code (do_compile) → install the results (do_install) → package the results (do_package) → generate the system image (do_rootfs) → generate the image (do_image).

[0070] When using bitbake in the Yocto Project to build the target system, the number of concurrent tasks bitbake can run depends on the number of CPU cores on the build server, and the build time is limited by the build server configuration. Furthermore, in the above embodiment, when using bitbake in the Yocto Project to build the target system, compilation can only be performed on a single server, making it difficult to fully utilize the resources of build servers with various configurations.

[0071] In view of this, an embodiment of the present application provides a file compilation method, which pre-mounts multiple compilation servers to the same remote disk to achieve data sharing between the compilation servers. Then, the different dependent packages of the target software package are distributed to multiple compilation servers for compilation. During the process of the compilation server compiling the dependent packages, the directory storing the compilation results is pointed to the same remote disk. After the compilation is completed, any compilation server can obtain the compilation results of all dependent packages from the shared remote disk, and combine the obtained compilation results of all dependent packages into the target software. The multiple dependent packages of the target software package are distributed to different compilation servers for compilation, and distributed compilation is achieved to achieve the purpose of distributed compilation. By distributing the dependent packages of the target software package, server resources can be fully utilized, and the construction time of the target software package can be saved, thereby improving the construction efficiency of the target software package.

[0072] It is understood that the file compilation method provided in the embodiments of this application can be used to build the target software package of OpenBMC, and can also be used to build other open source software. In the embodiments of this application, the construction of the target software package of OpenBMC is used as an example. When building the target software package of OpenBMC, bitbake is used as the build system engine based on the Yocto Project.

[0073] For example, Figure 2 A schematic diagram of an application scenario provided by an embodiment of the present application is shown in FIG. Figure 2As shown in the figure, there are servers A, B, C, D, and E. Servers A, B, C, D, and E can share a disk directory. For example, server A can set a disk directory on server A as an NFS shared directory, and servers B, C, D, and E can mount the directory on server A.

[0074] In one case, when a target software package needs to be compiled, any server from Server A to Server E can be selected to execute the bitbake command to obtain the dependency package information of the target software package. For example, when Server B is selected, Server B can distribute the dependency package name of at least one dependency package associated with the target software package to Server A, Server B, Server C, Server D, and Server E based on the obtained dependency package information. Server A to Server E respectively obtain the source code of the corresponding dependency package based on the received dependency package name and compile the corresponding dependency package. Then, Server A to Server E stores the compiled dependency package in a designated directory in Server A to Server E, which points to a shared directory. After all dependency packages are compiled, any server from Server A to Server E can obtain the compiled dependency package from the shared directory and synthesize the target software package based on the compiled dependency package.

[0075] In another case, server A can be set up as a compilation server specifically for synthesizing the target software package. When the target software package needs to be compiled, the bitbake command can be executed on server A to obtain the dependency package information of the target software package. Then, based on the obtained dependency package information, server A distributes the dependency package name of at least one dependency package associated with the target software package to server B, server C, server D, and server E. Based on the received dependency package name, server B-server E obtains the source code corresponding to the dependency package and compiles the corresponding dependency package. Then, server B-server E stores the compiled dependency package in a designated directory on server B-server E, which points to a shared directory. After server B-server E completes compiling the received dependency package, server A can obtain the dependency package compiled by server B-server E from the shared directory and synthesize the target software package based on the compiled dependency package.

[0076] It is understandable that Figure 2 The number of servers shown in the figure is only for illustrative purposes. In actual application scenarios, more or fewer servers may be included, and the embodiments of the present invention are not limited to this.

[0077] For example, Figure 3 A schematic diagram of the structure of a server is shown. The server can be Figure 2Any server shown in . Figure 3 As shown, the server may include: a processor 310, a network interface 320, and a memory 330. The processor 310, the network interface 320, and the memory 330 may be connected via a bus or other means.

[0078] In the embodiment of the present application, the processor 310 (also known as the central processing unit (CPU)) is the computing core and control core of the server. For example, the processor 310 can execute the file compilation method by reading program instructions and data stored in the server memory (not shown).

[0079] The network interface 320 may include a standard wired interface or a wireless interface (such as Wi-Fi, a mobile communication interface, etc.), and is controlled by the processor 310 to send and receive data. For example, the server can read data from a shared directory in the NFS system through the network interface 320, or write data to a shared directory in the NFS system through the network interface 320.

[0080] The memory 330 (memory) is a memory device of the server, which is used to store programs and data. For example, when the server acts as a central NFS server in an NFS system, the memory 330 can be used to store the compiled dependency packages corresponding to the target software package.

[0081] It is understandable that the embodiments of this application Figure 3 The illustrated structure does not constitute a specific limitation on the server. In other embodiments of the present application, the server may be a cloud server. The server may include more or fewer components than shown in the figure, or may combine or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0082] For example, Figure 4 A schematic diagram of a distributed file compilation process is shown in FIG. Figure 4 The distributed file compilation process shown in the figure is based on the Yocto project and is built using bitbake as the build system engine. Figure 4 As shown, multiple servers are used as distributed compilation servers, including a target package server 10 and multiple dependent package compilation servers 11. NFS technology enables file sharing between the target package server 10 and the multiple dependent package compilation servers 11. For example, the target package server can be configured as an NFS server to provide shared storage space. The multiple dependent package compilation servers can then act as clients in the NFS system and use the storage space provided by the NFS server.

[0083] Next, based on Figure 2 The application scenarios shown and Figure 4 The distributed file compilation process shown in FIG. 1 is a detailed introduction to a file distributed compilation solution involved in an embodiment of the present application, as described below.

[0084] (1) Configuring the target software package server 10;

[0085] When configuring the target software package server 10, it is necessary to configure the target software package server 10 as an NFS server in the NFS system. The main configuration file of NFS is / etc / exports, in which parameters such as the NFS output directory (i.e., shared directory), access permissions, and hosts allowed to access can be defined. For example, each line in the exports file provides a shared directory setting, and its command format is: <output directory> [client 1 (option 1, option 2, ...)] [client 1 (option 1, option 2, ...)]; except for the output directory, which is a required parameter, all other parameters are optional. The client refers to the server in the network that can access this NFS shared directory. The client specification is very flexible and can be the IP or domain name of a single host, or a host in a subnet or domain, etc.

[0086] In one possible example, the sstate-cache directory in the target software package server 10 can be set as an NFS shared directory and shared with the network segment where other dependent package compilation servers 11 are located. Specifically, the configuration in the NFS server includes:

[0087] mkdir / sstate-cache&&chmod777 / sstate-cache / / Create a shared directory and set read and write permissions

[0088] vim / etc / exports " / sstate-cache 10.32.xx (rw, sync, no_root_squash)" / / Configure in the / etc / exports configuration file, configure the shared directory to "sstate-cache", and share the shared directory with the dependency package compilation server in the 10.32.xx network segment.

[0089] It is understandable that the sstate-cache directory is a directory in the Yocto project. It is used to cache the build status in the build structure during the customization of the system or embedded product. This directory can be placed in a public directory to improve the efficiency of each compilation.

[0090] (2) configuring the dependency package compilation server 11;

[0091] When configuring the dependency package compilation server 11, it is necessary to configure the dependency package compilation server 11 as a client server in the NFS system. That is, the dependency package compilation server 11 is mounted in the sstate-cache directory of the target software package server 10. Specifically, the configuration in the dependency package compilation server 11 includes:

[0092] mkdir / sstate-cache&&chmod777 / sstate-cache / / Create a state-cache directory to store files and set read and write permissions.

[0093] vim / etc / fstab "120.32.xx: / sstate-cache / sstate-cachenfsdefaults,netdev0 0" / / Set the dependency package compilation server to automatically mount the specified output directory on the NFS server when it is started.

[0094] mount 120.32.xx (target package server IP): / sstate-cache (output directory in the target package server) / sstate-cache (local directory) / / Mount the files in the NFS server (in the target package server) locally.

[0095] (3) Build a Jenkins scheduling system to run node services on the target package server 10 and the dependent package compilation server 11;

[0096] After configuring the target package server 10 and the dependent package compilation server 11, you need to set up a corresponding scheduling system to schedule the target package server 10 and the dependent package compilation server 11. For example, you can build a Jenkins system as a scheduling system. Jenkins uses a Master / Slave mechanism. Master / Slave is equivalent to the concepts of Server and Agent. The Master provides a web interface for users to manage Jobs and Slaves. Jobs can run on the Master itself or be assigned to Slaves. A Master can be associated with multiple Slaves to serve different Jobs or different configurations of the same Job.

[0097] When building a Jenkins system, you need to set up a Jenkins distributed environment, including:

[0098] Install the Java Development Kit (JDK) on the Master and Slave nodes.

[0099] Install and configure Jenkins on the Master node;

[0100] Add Slave node configuration on the Master node and generate the Master-Slave communication file Slaagent;

[0101] Run SlaveAgent on the Slave node and communicate with the Master node through SlaveAgent;

[0102] Manage Jenkins projects on the Master node, specify Slave scheduling strategies, and implement task allocation and result collection sources for Slave nodes.

[0103] In this embodiment of the present application, the target package server 10 and the dependent package compiling server 11 can be configured as Jenkins slave nodes and mounted into the Jenkins system. That is, the target package server 10 and the dependent package compiling server 11 serve as node servers in the Jenkins system. Jenkins does not need to be installed or configured on either the target package server 10 or the dependent package compiling server 11. Only the SlaveAgent needs to be run to communicate with the Master node through the SlaveAgent.

[0104] (4) The target software package server 10 determines the dependent package information of the target software package;

[0105] In an embodiment of the present application, the target software package can be an image file of a customized system, such as an OpenBMC image file. The target software package server 10 generates a dependency package information document for the target software package based on the name of the target software package. Specifically, the bitbake command "bitbake -g [target software package name]" can be executed on the target software package server 10 to generate a dependency package information document "pn-buildlist" for the target software package.

[0106] In a possible example, the obtained dependency package information document "pn-buildlist" includes: the name of the dependency package, the total number of dependency packages, the estimated compilation time of each dependency package, and the API interface for calling the Jenkins system.

[0107] (5) The target software package server 10 sends the name of the dependent package of the target software package to the dependent package compilation server 11;

[0108] After obtaining the dependent package information document, the target software package server 10 can determine the number of dependent packages that each dependent package compiling server 11 can compile based on the information carried in the dependent package information document and the number of dependent package compiling servers 11 .

[0109] In one possible example, the target software package server 10 may also determine the number of dependent packages to distribute to the dependent package compiling servers 11 based on the total number of dependent packages and the number of dependent package compiling servers 11. Each dependent package compiling server 11 compiles the same or a similar number of dependent packages. For example, the number of dependent packages of the target software package is M, and the number of dependent package compiling servers 11 is N. Then, the number of dependent packages distributed by the target software package server 10 to each dependent package compiling server 11 is S = M / N.

[0110] In one possible example, the target software package server 10 may further determine the number of dependent packages to be distributed to the dependent package compiling servers 11 based on the total number of dependent packages, the compilation time of each dependent package, and the number of dependent package compiling servers 11. The compilation time of each dependent package compiling server 11 is the same or similar.

[0111] (6) The dependency package compilation server 11 downloads the source code according to the received dependency package name and compiles the dependency package;

[0112] After receiving the dependency package name sent by the target software package server 10 , the dependency package compilation server 11 can download the source code corresponding to the dependency package from the code repository corresponding to OpenBMC according to the received dependency package name.

[0113] After obtaining the source code corresponding to the dependency package, the dependency package compilation server 11 can simultaneously compile the S dependency packages sent by the target software package server 10 by executing "bitbake dependency package 1 dependency package 2...dependency package S", and store the compiled dependency packages in the sstate-cache directory.

[0114] (7) The dependency package compilation server 11 stores the compiled dependencies in a shared directory on the target software server;

[0115] Before compiling the dependent package through the dependent package compilation server 11, it is also necessary to configure the corresponding environment variables on the dependent package compilation server 11. For example, the build / [machine name] directory can be generated by executing the ".setup[machine name]" command. Among them, the build / [machine name] directory can be used to set related environment variables. Specifically, by modifying the build / [machine name] / conf / local.conf file, adding the configuration "SSTATE_DIR=" / sstate-cache"", the sstate-cache directory of the bitbake cache result in the compilation server can be pointed to the NFS shared directory.

[0116] After completing the dependency package compilation, the dependency package compilation server 11 stores the compiled dependency package in the sstate-cache directory on the dependency package compilation server 11. Since the dependency compilation server acts as a client server in the NFS system, it automatically mounts the shared directory sstate-cache on the target software package server 10 to the compilation server upon startup. Therefore, when the compilation server stores the compiled dependency package in the sstate-cache directory, it actually stores it in the sstate-cache directory on the target software package server 10.

[0117] (8) The target software package server 10 synthesizes the target software package based on the compiled dependent packages stored in the shared directory;

[0118] On the target package server 10, you need to point the sstate-cache directory on the target package to the NFS share. Specifically, you can modify the build / [machine name] / conf / local.conf file on the target package server 10 and add the configuration "SSTATE_DIR=" / sstate-cache"" to point the sstate-cache directory to the NFS share. This means setting the shared directory as the directory for Bitbake cache results.

[0119] After all dependent compiled packages are compiled, the target software package server 10 can execute the "bitbake target software package name" command to combine all compiled dependent packages that have been cached in the / sstate-cache directory into the target software package.

[0120] Next, based on the distributed file compilation process described above, a file compilation method provided by an embodiment of the present application is introduced. For example, Figure 5The flowchart of a file compilation method provided by an embodiment of the present application is shown. The method can be executed by a first server, which is configured with a shared directory. The first server can be Figure 2 Server A shown in . See Figure 5 The method includes: steps 501 to 503.

[0121] Step 501: Obtain dependent package information of a target software package, wherein the dependent package information indicates at least one dependent package.

[0122] In an embodiment of the present application, the first server may be a target software package server, and the Yocto project needs to be configured on the first server. For example, the Yocto project is downloaded through the target software package server, and the build environment is initialized. For example, a build directory can be created and relevant environment configurations can be set through the "oe-init-build-env" script. It is also necessary to configure the target software server as an NFS server in the NFS system, wherein the main configuration file of NFS is / etc / exports, in which parameters such as the NFS output directory (i.e., shared directory), access permissions, and hosts allowed to access can be defined.

[0123] In one possible example, the sstate-cache directory in the target software package server can be set as an NFS shared directory and shared with the second server, where the second server can be the dependent package compilation server in the above embodiment.

[0124] The first server can obtain the target package's dependency information from the Yocto project's recipe file based on the target package's name and generate a dependency information document for the target package. For example, the bitbake command "bitbake -g [target package name]" can be executed on the first server to generate a dependency information document "pn-buildlist" for the target package.

[0125] In a possible example, the obtained dependency package information document "pn-buildlist" includes: the name of the dependency package, the total number of dependency packages, the estimated compilation time of each dependency package, and the API interface for calling the Jenkins system.

[0126] Step 502: Based on the dependency package information of the target software package, first information is sent to each of the multiple second servers respectively. The first information includes at least one dependency package name. The first information is used to instruct the second server to compile the dependency package according to the received dependency package name and store the compiled dependency package in a shared directory. The shared directory supports access by multiple second servers.

[0127] In this embodiment of the present application, the second server first needs to configure the Yocto Project on the second server. For example, the Yocto Project can be downloaded from the second server and the build environment can be initialized. For example, the "oe-init-build-env" script can be used to create a build directory and set the relevant environment configuration.

[0128] Then, you can configure the second server as a client server in the NFS system. Specifically, you need to create an sstate-cache directory on the second server to cache the files generated by the second server during the compilation of dependent packages, and mount the shared directory "sstate-cache" on the first server to "sstate-cache" on the second server. This allows the files cached by the second server during the compilation of dependent packages to be directly stored in the shared directory on the first server.

[0129] After the first server obtains the dependency information of the target software package, it can send first information to multiple second servers based on the obtained dependency package information, wherein the first information includes at least one dependency package name.

[0130] In one possible example, the first server determines the number of dependency packages that each second server needs to compile based on the number of dependency packages of the target software package and the number of second servers. The first server then allocates the corresponding number of dependency packages to the second servers for compilation. For example, if the total number of dependency packages is M and the number of second servers is N, then the number of dependency packages that each second server needs to compile is S = M / N.

[0131] In one possible example, the first server may determine the number of dependent packages that the second server needs to compile based on the number of dependent packages of the target software package, the compilation time of each dependent package, and the number of second servers, wherein the compilation time of each second server for the dependent packages is the same or similar.

[0132] After receiving the first message from the first server, the second server can download the source code corresponding to the dependency package from the OpenBMC code repository based on the dependency package name carried in the first message. After obtaining the source code corresponding to the dependency package, the second server can simultaneously compile the S dependency packages sent by the first server by executing the command "bitbake dependency package 1 dependency package 2 ... dependency package S" and store the compiled dependency packages in the sstate-cache directory.

[0133] Step 503: synthesize the target software package according to the compiled dependent packages stored in the shared directory.

[0134] In an embodiment of the present application, after the second server compiles the dependent packages, the first server can execute the "bitbake target package name" command to combine all compiled dependent packages cached in the / sstate-cache directory into a target package. In one possible example, the first server can be any second server among multiple second servers. When the first server distributes the dependent packages to the second server, it also distributes the dependent packages to the first server itself.

[0135] In a possible example, the first server may synthesize the compiled dependent packages into the target software package, or other servers among the plurality of second servers may synthesize the compiled dependent packages into the target software package.

[0136] In this embodiment, Yocto Project-based distributed file compilation is implemented based on the Yocto Project's sstate-cache mechanism, Bitbake's build functionality, and NFS technology. Multiple dependent packages of a target software package are distributed to different compilation servers, enabling distributed compilation and fully utilizing server resources. This can save target software package build time and improve build efficiency.

[0137] The present application embodiment also provides a file compilation method, which is another expression of the file compilation method provided in the above embodiment. For example, Figure 6 FIG. 1 shows a flow chart of a file compilation method provided by an embodiment of the present application. Figure 6 As shown, the method includes: steps 601 to 606.

[0138] Step 601: Configure the first server and set a shared directory in the first server.

[0139] In an embodiment of the present application, the first server may be the target software package server in the above embodiment. When configuring the first server, it is first necessary to configure the Yocto Project on the first server. For example, the Yocto Project can be downloaded through the first server and the build environment can be initialized. Specifically, the "oe-init-build-env" script can be used to create a build directory and set the relevant environment configuration.

[0140] Next, you need to configure the first server as an NFS server in the NFS system. The main NFS configuration file is / etc / exports, which defines parameters such as the NFS export directory (i.e., shared directory), access permissions, and hosts allowed to access. For example, you can set the sstate-cache directory on the first server as an NFS shared directory and share it with the network segment where the second server resides. The specific configuration process for the first server can be referenced to the configuration process for the target software package server 10 in the above embodiment and will not be repeated here.

[0141] Step 602: Configure the second server and mount the second server to the shared directory of the first server.

[0142] In an embodiment of the present application, the second server may be the dependency package compilation server in the above embodiment. When configuring the second server, it is first necessary to configure the Yocto Project in the second server. For example, the Yocto Project is downloaded through the second server and the build environment is initialized. Specifically, a build directory can be created and the relevant environment configuration can be set using the "oe-init-build-env" script.

[0143] Next, you need to configure the second server as a client server in the NFS system. Specifically, mount the second server in the sstate-cache directory of the target package server. The specific configuration process for the second server can be found in the configuration process for the dependent package compilation server in the above embodiment, and will not be repeated here.

[0144] It is understood that in the embodiment of the present application, one server can be selected from multiple second servers as the first server, that is, the NFS server in the NFS system. The first server and the multiple second servers have the same status in the process of synthesizing the target software package.

[0145] After configuring the first and second servers, you need to set up a corresponding scheduling system to schedule them. For example, you can mount the first and second servers as Jenkins slave nodes in the Jenkins system. The Jenkins system will be responsible for scheduling the first and second servers during the compilation of the first target software package.

[0146] Step 603: The first server generates dependent package information of the target software package.

[0147] In an embodiment of the present application, the target software package may be an image file of a customized system. The first server generates a dependency package information document of the target software package based on the name of the target software package. Specifically, the bitbake command "bitbake -g [target software package name]" may be executed on the first server to generate a dependency package information document "pn-buildlist" of the target software package.

[0148] In a possible example, the obtained dependency package information document "pn-buildlist" includes: the name of the dependency package, the total number of dependency packages, the estimated compilation time of each dependency package, and the API interface for calling the Jenkins system.

[0149] Step 604: The first server sends the name of at least one dependent package to multiple second servers based on the acquired dependent package information of the target software package.

[0150] In an embodiment of the present application, after the first server obtains the dependency information of the target software package, it can distribute the dependency packages of the target software to multiple second servers for compilation based on the obtained dependency package information.

[0151] In a possible example, the first server may determine the number of dependent packages that each second server needs to compile based on the number of dependent packages of the target software package and the number of second servers, and then allocate the corresponding number of dependent packages to the second servers.

[0152] In another possible example, the first server may determine the number of dependent packages that the second server needs to compile based on the number of dependent packages of the target software package, the compilation time of each dependent package, and the number of second servers, wherein the compilation time of each second server for the dependent packages is the same or similar.

[0153] In step 605, the second server obtains the source code corresponding to the received dependency package name, compiles the dependency package, and stores the compiled dependency package in a shared directory.

[0154] In an embodiment of the present application, after receiving the dependency package sent by the first server, the second server can download the source code corresponding to the dependency package from the network based on the information contained in the dependency package. After obtaining the source code corresponding to the dependency package name, the second server can simultaneously compile the S dependency packages sent by the first server by executing "bitbake dependency package 1 dependency package 2 ... dependency package S" and store the compiled dependency packages in the sstate-cache directory.

[0155] In a possible example, the second server may include the first server, that is, the first server also participates in the compilation of the dependent package.

[0156] Step 606: The first server synthesizes the target software package according to the compiled dependency packages stored in the shared directory.

[0157] In an embodiment of the present application, after all dependent compiled packages are compiled, the first server can combine all compiled dependent packages that have been cached in the / sstate-cache directory into a target software package by executing the "bitbake target software package name" command.

[0158] It is understood that in the embodiments of the present application, the first server can be any one of the multiple second servers. After all dependent packages are compiled, any one of the multiple second servers can retrieve the compiled dependent packages from the shared directory and synthesize the target software package. For example, scheduling can be performed through the Jenkins system to determine the server used to synthesize the target software package from the multiple second servers.

[0159] It is understandable that the size of the sequence number of each step in the above embodiment does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In addition, in some possible implementations, the steps in the above embodiment can be selectively executed according to actual conditions, and can be partially executed or fully executed, which is not limited here. In addition, all or part of any features in the above embodiment can be freely and arbitrarily combined without contradiction. The combined technical solution is also within the scope of this application.

[0160] Next, we will use a specific example to introduce a file compilation method provided by an embodiment of this application. The distributed compilation system includes: 9 servers, a master server in the Jenkins system, and an openbmc code repository. In this embodiment of the application, the target software package to be built is the obmc-phosphor-image software package.

[0161] First, you need to configure the build environment for building the target software package. For example, you can select server 1 among the 9 servers as the target software package server. The remaining 8 servers are used as dependency package compilation servers. Then, configure the target software package server and the dependency package compilation server separately. Specifically, you can create an sstate-cache directory in the target software package server and configure the sstate-cache directory as an NFS shared directory. Then, create an sstate-cache directory in the dependency package compilation server as a cache directory during the dependency package compilation process, and mount the shared directory (sstate-cache directory) in the target software server to the sstate-cache directory locally on the dependency package compilation server.

[0162] After configuring the target software package's build environment, you need to build the target software package's build system. For example, you can use Jenkins as the target software package's build system. Specifically, the Jenkins system mounts the nine servers used to build the target software package as slave nodes. Jenkins then creates a task pipeline to build the target software package.

[0163] The master server in the Jenkins system triggers the task pipeline for building the target software package and distributes the tasks to each server that serves as a slave node.

[0164] For example, the master server in the Jenkins system can trigger a target package server to download code from the corresponding Git repository to the target package server and trigger the processor on the target package server to execute ".setup ebv-ast2600." This command sets the Yocto project environment variables on the target software server and creates the build / ebv-ast2600 directory. The processor on the target package server then switches to the build / ebv-ast2600 directory and executes the "bitbake -g obmc-phosphor-image" command. This command generates a pn-buildlist file in the build / ebv-ast2600 directory, storing the names of all dependent packages associated with the target package.

[0165] The processor in the target package server parses the pn-buildlist file, reads the 400 dependency package names, and determines that there are currently eight available dependency package build servers. Therefore, the target package server can distribute 50 dependency packages to each dependency package build server. The Jenkins master server obtains the dependency package names that each dependency package build server needs to compile and triggers eight subtask pipelines using each of the 50 dependency package names as parameters (triggering the eight dependency package build servers to compile the dependency packages).

[0166] After receiving the 50 dependent package names from the target package server, each of the eight dependency package compilation servers downloads the corresponding source code from the OpenBMC repository based on the received dependency package names. The dependency package compilation server then executes the ".setup ebv-ast2600" command and adds the configuration "SSTATE_DIR=" / sstate-cache"" to the build / ebv-ast2600 / conf / local.conf file, pointing the sstate-cache directory on the dependency package compilation server to the shared directory on the target package server. The dependency package compilation server then executes the "bitbake dependency-package1 dependency-package2...dependency-package50" command to simultaneously build the 50 dependent packages distributed to the server. After compiling the 50 dependent packages, the dependency package compilation server stores them in the sstate-cache directory on the dependency package compilation server.

[0167] After the eight dependency package compilation servers have completed building the received dependency packages, the Jenkins master server should trigger the target package server to execute the ".setup ebv-ast2600" command. This configuration file, "SSTATE_DIR=" / sstate-cache"", should be added to the build / ebv-ast2600 / conf / local.conf file. This allows the target package server to read the files in the sstate-cache directory. The target package server then completes the build by executing the "bitbakeobmc-phosphor-image" command.

[0168] After the target software package server generates the target software package, it uploads the target software package to the specified location to complete the construction of the target software package.

[0169] For example, the present application embodiment also provides a compilation device. Figure 7 As shown, the compilation device 700 includes: an acquisition module 710, a processing module 720, a storage module 730, and a sending module 740. The acquisition module 710 is used to obtain first information, which includes at least one dependent package name. In one possible example, the acquisition module 710 can be used to obtain dependency information of the target software package, where the dependency package information indicates at least one dependent package.

[0170] When the acquisition module 710 acquires the dependency information of the target software package, the sending module 740 is configured to send first information to other compilation servers according to the acquired dependency information of the target software package, where the first information includes at least one dependent package name.

[0171] In one possible example, before sending module 740 sends the first information to other servers, processing module 720 may determine the number of dependency package names to send to each compilation server based on the total number of dependency packages, the compilation time information of each dependency package, and the number of compilation servers, so that the compilation time of the dependency packages compiled by each compilation server is the same or similar. Alternatively, processing module 720 may determine the number of dependency package names to send to each compilation server based on the total number of dependency packages and the number of second servers, so that the number of dependency packages compiled by each compilation server is the same or similar.

[0172] The processing module 720 is further configured to compile a dependency package based on the acquired first information. Specifically, the acquisition module 710 may acquire source code corresponding to at least one dependency package name based on the dependency package name carried in the first information. The processing module 720 compiles the acquired source code to obtain a compiled dependency package, and stores the compiled dependency package in a mount directory of the file compilation device 700. The mount directory in the file compilation device 700 points to a shared directory in the distributed compilation system to which the file compilation device belongs.

[0173] In a possible example, the processing module 720 may also be configured to obtain compiled dependent packages from a shared directory and synthesize the target software package.

[0174] In a possible example, when a shared directory is configured on the compilation device 700 , the storage module 730 may be used to store compiled dependent packages.

[0175] By way of example, an embodiment of the present application further provides a distributed compilation system. The distributed compilation system includes multiple servers. One of the multiple servers is configured as an NFS server in an NFS system, and the other servers in the multiple servers, except for the server configured as the NFS server, are configured as client servers in the NFS system. A shared directory is configured in the NFS server, and the shared directory can be accessed by each of the multiple servers.

[0176] During the process of building a target software package, a first server among the multiple servers is used to obtain dependent package information of the target software package, wherein the dependent package information indicates at least one dependent package.

[0177] The first server is further configured to send first information to a second server among the plurality of servers respectively, wherein the first information includes at least one dependent package name.

[0178] The second server is used to compile the dependency package according to the received first information, and store the compiled dependency package in a shared directory.

[0179] Any one of the multiple servers is used to obtain the compiled dependency packages from the shared directory and synthesize the target software package.

[0180] In a possible example, the first server may allocate the at least one dependent package to the second server based on the total number of dependent packages and the number of the second servers, so that the number of dependent packages compiled by any second server is the same or similar.

[0181] In a possible example, the first server may allocate the at least one dependent package to the second server based on the total number of dependent packages and the compilation time information of each dependent package, so that the compilation time of any second server compiling the dependent package is the same or similar.

[0182] In one possible example, the second server includes the first server. After receiving the dependency package sent by the first server, the second server retrieves the source code corresponding to the dependency package from the Openbmc code repository based on the dependency package name. The second server then compiles the retrieved source code to obtain a compiled dependency package and stores the compiled dependency package in a shared directory.

[0183] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0184] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product, characterized in that when the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0185] Based on the methods in the above embodiments, embodiments of the present application provide a computing device comprising a motherboard and a chip. The chip is integrated on the motherboard and includes at least one memory for storing programs and at least one processor for executing the programs stored in the memory. When the programs stored in the memory are executed, the processor is configured to execute the methods in the above embodiments. In embodiments of the present application, the computing device may be a server, a network device, or other similar device.

[0186] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0188] It will be understood that the various numerical numbers involved in the embodiments of the present application are merely distinctions for the convenience of description and are not intended to limit the scope of the embodiments of the present application.

Claims

1. A file compilation method, characterized in that: Applied to the first server, the method includes: Obtaining dependent package information of a target software package, wherein the dependent package information indicates at least one dependent package; sending first information to each of the plurality of second servers based on the dependency package information of the target software package, wherein the first information includes at least one dependency package name, and the first information is used to instruct the second server to compile the dependency package based on the received dependency package name and store the compiled dependency package in a shared directory, wherein the shared directory is accessible to the plurality of second servers; The target software package is synthesized according to the compiled dependent packages stored in the shared directory.

2. The method according to claim 1, characterized in that The dependent package information of the target software package includes at least one of: dependent package name, total number of dependent packages, compilation time of dependent packages, and application program interface for calling an integrated scheduling system.

3. The method according to claim 1 or 2, characterized in that Before sending the first information to each of the plurality of second servers according to the dependent package information of the target software package, the method includes: Obtain the number of the second server; The number of dependency package names carried in the first information is determined according to the total number of the dependency packages and the number of the second servers, so that the number of dependency packages compiled by any second server is the same or similar.

4. The method according to claim 1 or 2, characterized in that Before sending the first information to each of the plurality of second servers according to the dependent package information of the target software package, the method includes: Obtain the number of the second server; According to the total number of the dependent packages, the compilation time information of each dependent package, and the number of the second servers, the number of dependent package names carried in the first information is determined so that the compilation time of any second server compiling the dependent package is the same or similar.

5. The method according to claim 1, wherein The plurality of second servers include the first server, and the method further includes: Obtaining source code corresponding to the at least one dependent package name according to the first information; Compile the source code to obtain a compiled dependency package; The compiled dependency package is stored in a mount directory, and the mount directory points to the shared directory.

6. A distributed compilation system, characterized in that: include: A plurality of servers, wherein a shared directory is provided on one of the plurality of servers, and the shared directory supports access by each of the plurality of servers; A first server among the multiple servers is configured to obtain dependency package information of a target software package, wherein the dependency package information indicates at least one dependent package; The first server is further configured to send first information to each second server among the multiple servers, wherein the first information includes at least one dependent package name; The second server is configured to compile a dependency package according to the first information and store the compiled dependency package in the shared directory; Any one of the multiple servers is configured to obtain the compiled dependent packages from the shared directory and synthesize the target software package.

7. The system according to claim 6, characterized in that The first server is used for: The number of dependency package names carried in the first information is determined according to the total number of the dependency packages and the number of the second servers, so that the number of dependency packages compiled by any second server is the same or similar.

8. The system according to claim 6, wherein: The first server is used for: According to the total number of the dependent packages, the compilation time information of each dependent package, and the number of the second servers, the number of dependent package names carried in the first information is determined so that the compilation time of the dependent packages compiled by any second server is the same or similar.

9. The system according to any one of claims 6 to 8, characterized in that: The second server includes the first server, and the second server is configured to: Obtaining source code corresponding to the at least one dependent package name according to the first information; Compile the source code to obtain a compiled dependency package; The compiled dependency package is stored in a mount directory, and the mount directory points to the shared directory.

10. A server, characterized in that: include: at least one memory for storing a program; At least one processor is configured to execute a program stored in a memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 5.

Images