Distributed compilation environment cloning method, device and storage medium

By receiving cloning compiler instructions and dependency library query functions to generate dependency tables, the complexity and resource utilization of distributed compilation environments are solved, and efficient cloning and simplified deployment of compilation environments are achieved.

CN118760467BActive Publication Date: 2024-11-29KYLIN CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411222923.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-29
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

Traditional stand-alone compilation cannot meet the needs of rapid construction, and the consistency and deployment complexity of the compilation environment during distributed compilation are difficult to solve, especially the containerized solution has high requirements for network resources and storage space.

Method used

By receiving cloning compiler instructions, find the execution path of the compilation tool, use the dependency library query function to generate a dependency table, query the relative and absolute paths of the dependency library file, and clone it into a temporary folder to ensure that the temporary folder path is from the original The compilation environment is consistent and reduces storage space usage.

Benefits of technology

It realizes cloning of the compilation environment without strong technical capabilities and network resources, and only contains the dependency library files that are required by the compilation tool, which reduces storage space usage and simplifies the deployment of the compilation environment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118760467B_ABST
    Figure CN118760467B_ABST
Patent Text Reader

Abstract

The present invention discloses a distributed compilation environment cloning method, device and storage medium. The method comprises: receiving a clone compiler instruction, searching for a compilation tool execution path corresponding to the clone compiler; querying the compiler's dependent library file by using a dependent library query tool to obtain a dependent library file path query result, and writing the dependent library file path query result and the dependent library name into a key-value pair into a dependency table; cyclically using the dependent library query tool to traverse and query the dependent libraries in the dependency table until the query result is a non-ELF file, and the query result is continuously appended to the dependency table according to the key-value relationship; in the process, synchronously using an absolute path conversion function to convert the path in the query result to obtain the absolute path of each dependent library file, and adding the absolute path information to the dependency table; using the dependency table to clone the file corresponding to the key value and the absolute path corresponding to the value value into a temporary folder.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of compilation technology, and in particular to a distributed compilation environment cloning method, device and storage medium. Background Art

[0002] As software scale and complexity increase, traditional stand-alone compilation can no longer meet the needs of fast builds. Distributed compilation, as a solution, greatly improves compilation efficiency by distributing compilation tasks to multiple computing nodes for parallel execution. However, the consistency of the compilation environment and the complexity of deployment in the distributed compilation process have become the focus of research.

[0003] At present, methods based on containerization solutions such as Docker and methods that install the same operating system version on all nodes involved in compilation are common solutions. However, docker container technology generally requires that the nodes involved in computing can be connected to the Internet normally to facilitate downloading specific operating system images from the Internet, or that a self-built docker image server is required, which has high requirements for technical capabilities and network resources. And this method, like the complete copy of the operating system, requires a lot of storage space. Summary of the invention

[0004] The embodiments of the present invention provide a distributed compilation environment cloning method, device and storage medium to solve the technical problem in the prior art that distributed compilation deployment is complex and requires a large amount of resources.

[0005] In a first aspect, an embodiment of the present invention provides a distributed compilation environment cloning method, comprising:

[0006] Receiving a clone compiler instruction, and searching for a compilation tool execution path corresponding to the clone compiler instruction;

[0007] Query the dependent library file of the compiler using the dependent library query function to obtain a relative path query result of the dependent library file, and write the relative path query result of the dependent library file as the key value in the variable and the dependency relationship into the dependency relationship table;

[0008] Utilize the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and write the query result as the key value and dependency relationship in the variable into the dependency relationship table;

[0009] Query the query result using an absolute path query statement to obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table;

[0010] The dependency table is used to clone the files and key values ​​corresponding to the key values ​​into a temporary folder according to the absolute path corresponding to the key values, and the absolute path in the temporary folder is ensured to be consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0011] In a second aspect, an embodiment of the present invention further provides a distributed compilation environment cloning device, comprising:

[0012] A receiving module, used for receiving a clone compiler instruction and searching for a compilation tool execution path corresponding to the clone compiler instruction;

[0013] A writing module is used to query the dependent library files of the compiler using the dependent library query function to obtain the relative path query result of the dependent library files, and write the relative path query result of the dependent library files as the key value in the variable and the dependency relationship into the dependency relationship table;

[0014] A query module is used to query the query result of the dependent library file by using the dependent library query function until the query result is a non-binary dynamic link ELF file, and write the query result as the key value and the dependency relationship in the variable into the dependency table;

[0015] An absolute storage path query module, used to query the query result using an absolute path query statement, obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table;

[0016] The cloning module is used to use the dependency table to clone the files and key values ​​corresponding to the key values ​​into a temporary folder according to the absolute path corresponding to the key values, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0017] In a third aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the distributed compilation environment cloning method provided in the above embodiment.

[0018] The distributed compilation environment cloning method, device and storage medium provided by the embodiment of the present invention receive a clone compiler instruction, search for the compilation tool execution path corresponding to the clone compiler instruction; use the dependency library query function to query the compiler's dependency library file, obtain the dependency library file relative path query result, write the dependency library file relative path query result as the key value in the variable and the dependency relationship into the dependency table; use the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and write the query result as the key value in the variable and the dependency relationship into the dependency table; use the absolute path query statement to query the query result, obtain the absolute storage path of each dependency library file, and add the absolute storage path information to the dependency table; use the dependency table to clone the file corresponding to the key value, the key value and the absolute path corresponding to the key value into a temporary folder, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool. Analyze the dependency chain of the compilation command input by the user, then scan and analyze the dependency chain of the compilation command, generate a dependency list, and then clone a set of identical compilation environments according to the dependency list. The compilation environment can be cloned without strong technical capabilities and network resources, and only the dependent library files required by the compilation tool are included, reducing the storage space occupied. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0020] Figure 1 It is a schematic diagram of the process of the distributed compilation environment cloning method provided in the first embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a distributed compilation environment cloning method provided in Embodiment 2 of the present invention;

[0022] Figure 3 It is a schematic diagram of the structure of a distributed compilation environment cloning device provided in Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0023] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0024] Embodiment 1

[0025] Figure 1: is a flow chart of a distributed compilation environment cloning method provided in Embodiment 1 of the present invention. This embodiment is applicable to cloning a compilation environment and to distributed compilation. The method can be executed by a distributed compilation environment cloning device, and specifically includes the following steps:

[0026] Step 110: Receive a clone compiler instruction and search for a compilation tool execution path corresponding to the clone compiler instruction.

[0027] Distributed compilation is usually done by a master server and multiple compile servers that serve as working nodes. After receiving a compilation request for a target project, the master server determines the compilation commands and the file lists corresponding to the compilation commands, and then sends the file lists and compilation commands to different compile servers to compile the files corresponding to the target project.

[0028] In this embodiment, taking the gcc compiler as an example, an operating system that has been installed and can run normally can be pre-selected as the main server for distributed compilation. In the main server, the user can enter a clone compiler command, and the clone compilation command includes an instruction to start compilation. The instruction can correspond to the compiler entry. The corresponding function is used to find the execution path of the compilation tool.

[0029] Compared with the traditional use of tracing software, such as the ptrace mechanism, there is no need to run the compiler. Due to the ptrace mechanism, it is necessary to query the corresponding process and then determine the relevant files, especially the dependent library files. However, this method will inevitably introduce other parameter files, and thus fail to achieve the purpose of streamlining the compilation environment files.

[0030] Step 120, use the dependency library query function to query the dependency library file of the compiler, obtain the dependency library file path query result, and write the dependency library file path query result as the key value in the variable and the dependency relationship into the dependency relationship table.

[0031] In this embodiment, a dependency list target_files may be pre-established, and all dependency relationships may be added and recorded into variables of the dependency relationship table.

[0032] A dependency library is a file containing program code and data, which can be called by other programs or components. During code compilation, the support of the dependency library file is required to implement code compilation. In this embodiment, the main compiler includes a complete dependency library. When cloning, it is necessary to find the dependency files required for compilation.

[0033] Exemplarily, the ldd function can be used as a dependency library query function, and the compiler entry is used as a parameter of the ldd function to find the relative path query result of the corresponding dependency library file. And the found result is written into the dependency table. Exemplarily, the relative path query result of the dependency library file can be written into the dependency table as the key value and dependency in the variable. When using ldd to query the dependency library of the ELF file, the returned path is a link file, and the link file is a file that executes a relative path, such as .. / lib / xxx.so or xxx.so.2.

[0034] Step 130, use the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and write the query result as the key value in the variable and the dependency relationship into the dependency relationship table.

[0035] Exemplarily, the dependent library files obtained in the previous steps can also be queried through the ldd function, and the dependent files of the dependent library files are queried, and the dependent files are written into the dependency table. In addition, when the query result is a non-binary dynamic link ELF file, it can be determined as a non-dependent library file. Ldd can only display the dependency of the executable module. The principle is to set a series of environment variables. If a non-ELF format file is queried, it can be considered that it is not a dependent library file, but some other variables. Then terminate the query. Optionally, the dependent library files recorded in the dependency table can be used to traverse the recorded dependent library files, and continue to query through the ldd function, and write the query results into the dependency table. And traverse again. Until all the dependent library files corresponding to the key values ​​in the dependency table are traversed. Using the above traversal method can not only prevent omissions, but also avoid the query result being the same dependent library file path, further achieving the purpose of streamlining the compilation environment file.

[0036] Step 140, query the query result using an absolute path query statement to obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table.

[0037] When using ldd to query the dependent library of the ELF file, a part of the returned path is a link file, which is a file that executes a relative path, such as / lib / xxx.so or xxx.so.2, etc. The actual corresponding file may be / usr / lib / xxx.so.2.0.1. Therefore, in this embodiment, it is also necessary to find the corresponding actual path according to the relative path.

[0038] Optionally, you can use the function abs_path(file) to pass the relative path of the file as a parameter to the function and return the absolute path of the file. Add the dependent library file and the absolute path obtained by abs_path(file) to the dependency table target_files.

[0039] When cloning a compilation environment, if the cloned compilation environment only includes the relative path of the dependent file, an error will occur during the compilation process. Therefore, in this embodiment, it is necessary to further search for the absolute path of the dependent file and add its information to the dependency table target_files.

[0040] Step 150, using the dependency table to clone the file corresponding to the key value, the key value and the absolute path corresponding to the key value into a temporary folder, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0041] In this embodiment, the dependency table is used to store the dependent library files required for compilation according to the storage absolute path of the main server of distributed compilation, and the files in the temporary folder are stored accordingly, so that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0042] This embodiment receives a clone compiler instruction, searches for the compiler tool execution path corresponding to the clone compiler instruction; uses a dependency library query function to query the compiler's dependency library file, obtains a dependency library file relative path query result, and writes the dependency library file relative path query result as a key value in a variable and a dependency relationship into a dependency table; uses the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and writes the query result as a key value in a variable and a dependency relationship into a dependency table; uses an absolute path query statement to query the query result, obtains the absolute storage path of each dependency library file, and adds the absolute storage path information to the dependency table; uses the dependency table to clone the file corresponding to the key value, the key value, and the absolute path corresponding to the key value into a temporary folder, and ensures that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compiler tool. Analyze the dependency chain of the compilation command input by the user, then scan and analyze the dependency chain of the compilation command, generate a dependency list, and then clone a set of identical compilation environments according to the dependency list. The compilation environment can be cloned without strong technical capabilities and network resources, and only the dependent library files required by the compilation tool are included, reducing the storage space occupied.

[0043] In a preferred implementation of the present embodiment, the method further includes: obtaining distributed compilation statements, searching and traversing the relative storage path, absolute storage path and file of the dependent library file of the distributed compilation file; updating the dependency table based on the relative storage path, absolute storage path and file; cloning the relative storage path, absolute storage path and file of the dependency table into the temporary folder. Accordingly, the above traversal method can be used to search and traverse the relative storage path, absolute storage path and file of the dependent library file of the distributed compilation statements, such as dtstcc and distccd statements, and write them into the dependency table. While avoiding omissions, the same dependent library files can also be removed and written into the temporary folder together according to the storage absolute path of the main server of the distributed compilation.

[0044] In another preferred implementation of this embodiment, the method may further add the following steps: obtaining the compilation supporting files, searching and traversing the relative storage paths, absolute storage paths and files of the dependent library files of the compilation supporting files; updating the dependency table based on the relative storage paths, absolute storage paths and files; cloning the relative storage paths, absolute storage paths and files of the dependency table into the temporary folder. The compiler also includes supporting required files such as as, so the aforementioned method may also be used for traversal to obtain the dependent library files of the supporting files and store them in the temporary folder.

[0045] In another preferred implementation of this embodiment, the method may further add the following steps: setting permissions and user parameters for the compression and packaging tool, and using the compression and packaging tool with set permissions and user parameters to compress and package the temporary folder to form a packaged cloned distributed compilation environment. Exemplarily, the tar command may be used for compression and packaging, and the tar command will use the --same-owner and --same-permissions options to keep the original owner and permissions of the file. This allows the cloned distributed compilation environment to have the same permissions and users, so that the distributed compilation nodes have the same permissions and users, and can smoothly perform distributed compilation.

[0046] Embodiment 2

[0047] Figure 2It is a flow chart of the distributed compilation environment cloning method provided in the second embodiment of the present invention. This embodiment is optimized based on the above embodiment, and the method can also add the following steps: using remote transmission commands to send the compressed and packaged temporary folder to the editing node, and the editing node uses its own compression and packaging tool to decompress the compressed and packaged temporary folder; transmitting the distributed compilation command carrying the compilation processing file parameters to the editing node, so that the editing node compiles the compilation processing file and returns the compilation processing result.

[0048] See also Figure 2 , the distributed compilation environment cloning method comprises:

[0049] Step 210: Receive a clone compiler instruction and search for a compilation tool execution path corresponding to the clone compiler instruction.

[0050] Step 220, use the dependency library query function to query the dependency library file of the compiler, obtain the dependency library file path query result, and write the dependency library file path query result as the key value in the variable and the dependency relationship into the dependency relationship table.

[0051] Step 230, use the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and write the query result as the key value and dependency relationship in the variable into the dependency table.

[0052] Step 240, use the absolute path query statement to query the query result, obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table.

[0053] Step 250: Use the dependency table to clone the file corresponding to the key value, the key value, and the absolute path corresponding to the key value into a temporary folder, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0054] Step 260: Send the compressed and packaged temporary folder to the editing node using a remote transmission command.

[0055] Exemplarily, by using the SSH remote transmission command, the compressed and packaged temporary folder can be sent to the editing node, and the number of the editing nodes can be multiple.

[0056] Step 270: the editing node uses its own compression and packaging tool to decompress the compressed and packaged temporary folder.

[0057] After receiving the compressed and packaged temporary folder, the editing node can also use the tar command to decompress the compressed and packaged temporary folder, so as to form a cloned distributed compilation environment, which is convenient for setting the same users and permissions.

[0058] Step 280: transmit the distributed compile command carrying the compile processing file parameters to the editing node, so that the editing node compiles the compile processing file in the chroot environment and returns the compile processing result.

[0059] The compilation master server can transmit the compilation command with the file parameters to be edited to the editing node through the remote transmission tool. For example, when distcc gcc -o hello hello.c is executed on the compilation master server, distcc will send the command hello.c and gcc to the compilation node designated to run distccd. After distccd is completed, the result will be returned to the compilation master server of distcc, and the distributed compilation will be finally completed. The above method can be used to realize that each compilation node uses the cloned distributed compilation environment to compile the corresponding files respectively, and return the compilation results to the compilation master server to realize distributed compilation.

[0060] In addition, other methods all need to rely on the libc6 of the compilation node, and a system can only have a default libc6. libc6 is the abbreviation of the C standard library, which is one of the commonly used libraries in C language program development. Libc6 provides a series of functions and constants for processing common programming tasks such as strings, memory operations, input and output, mathematical operations, date and time. And the installation location of libc6 is also fixed and cannot be replaced or overwritten at will. Therefore, in this embodiment, the compilation environment is set in the chroot environment to avoid the problem of conflict between the cloned distributed compilation environment and the libc6 of the compilation node system.

[0061] This embodiment adds the following steps: using a remote transmission command, the compressed and packaged temporary folder is sent to the editing node, and the editing node uses its own compression and packaging tool to decompress the compressed and packaged temporary folder; transmitting a distributed compilation command carrying compilation file parameters to the editing node, so that the editing node compiles the compilation file and returns the compilation result. The cloned distributed compilation environment can receive different distributed compilation commands and return the compilation result to the compilation main server to achieve distributed compilation.

[0062] Embodiment 3

[0063] Figure 3is a schematic diagram of the structure of the distributed compilation environment cloning device provided in the third embodiment of the present invention, see Figure 3 , the distributed compilation environment cloning device comprises:

[0064] A receiving module 310 is used to receive a clone compiler instruction and search for a compilation tool execution path corresponding to the clone compiler instruction;

[0065] The writing module 320 is used to query the dependent library file of the compiler using the dependent library query function to obtain the relative path query result of the dependent library file, and write the relative path query result of the dependent library file as the key value in the variable and the dependency relationship into the dependency relationship table;

[0066] A query module 330 is used to query the query result of the dependent library file by using the dependent library query function until the query result is a non-binary dynamic link ELF file, and write the query result as the key value and the dependency relationship in the variable into the dependency relationship table;

[0067] An absolute storage path query module 340 is used to query the query result using an absolute path query statement to obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table;

[0068] The cloning module 350 is used to use the dependency table to clone the files and key values ​​corresponding to the key values ​​into a temporary folder according to the absolute path corresponding to the key values, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool.

[0069] The distributed compilation environment cloning device provided in this embodiment receives a clone compiler instruction and searches for the compilation tool execution path corresponding to the clone compiler instruction; uses a dependency library query function to query the compiler's dependency library file to obtain a dependency library file relative path query result, and writes the dependency library file relative path query result as a key value in a variable and a dependency relationship into a dependency table; uses the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, and writes the query result as a key value in a variable and a dependency relationship into a dependency table; uses an absolute path query statement to query the query result to obtain the absolute storage path of each dependency library file, and adds the absolute storage path information to the dependency table; uses the dependency table to clone the file corresponding to the key value, the key value, and the absolute path corresponding to the key value into a temporary folder, and ensures that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool. Analyze the dependency chain of the compilation command input by the user, then scan and analyze the dependency chain of the compilation command, generate a dependency list, and then clone a set of identical compilation environments according to the dependency list. The compilation environment can be cloned without strong technical capabilities and network resources, and only the dependent library files required by the compilation tool are included, reducing the storage space occupied.

[0070] Based on the above embodiments, the device further includes:

[0071] A compilation supporting file traversal module is used to obtain the compilation supporting file, find and traverse the relative storage path, absolute storage path and file of the dependent library file of the compilation supporting file;

[0072] A first updating module, configured to update the dependency table based on the relative storage path, the absolute storage path and the file;

[0073] Compile a matching file cloning module for cloning files into the temporary folder using the relative storage path, absolute storage path and files in the dependency table.

[0074] Based on the above embodiments, the acquisition module includes:

[0075] A distributed compilation statement traversal module is used to obtain distributed compilation statements, find and traverse the relative storage path, absolute storage path and file of the dependent library file of the distributed compilation file;

[0076] A second updating module, configured to update the dependency table based on the relative storage path, the absolute storage path and the file;

[0077] The distributed compilation statement cloning module is used to clone the relative storage path, absolute storage path and file of the dependency table into the temporary folder.

[0078] Based on the above embodiments, the device further includes:

[0079] The compression and packaging processing module is used to set permissions and user parameters for the compression and packaging tool, and use the compression and packaging tool with set permissions and user parameters to compress and package the temporary folder to form a packaging clone distributed compilation environment.

[0080] Based on the above embodiments, the device further includes:

[0081] The sending module is used to send the compressed and packaged temporary folder to the editing node using the remote transmission command.

[0082] Based on the above embodiments, the device further includes:

[0083] A decompression processing module, used to use the editing node to use its own compression and packaging tool to decompress the temporary folder after compression and packaging;

[0084] The return module is used to transmit the distributed compilation command carrying the compilation processing file parameters to the editing node, so that the editing node compiles the compilation processing file in the chroot environment and returns the compilation processing result.

[0085] The distributed compilation environment cloning device provided in the embodiment of the present invention can execute the distributed compilation environment cloning method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0086] Embodiment 4

[0087] Embodiment 4 of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute any of the distributed compilation environment cloning methods provided in the above embodiments.

[0088] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0089] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0090] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0091] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A distributed compilation environment cloning method, characterized in that: include: Receiving a clone compiler instruction, and searching for a compilation tool execution path corresponding to the clone compiler instruction; Query the compiler's dependent library files using the dependent library query function to obtain a dependent library file path query result, and write the dependent library file path query result as the key value in the variable and the dependency relationship into the dependency relationship table; Utilize the dependency library query function to query the dependency library file query result until the query result is a non-binary dynamic link ELF file, query the dependency library file through the ldd function, query the dependency file of the dependency library file, and write the dependency file into the dependency table, when the query result is a non-binary dynamic link ELF file, it is determined to be a non-dependency library file, ldd can only display the dependency of the executable module, and it sets a series of environment variables, if the query is a non-ELF format file, it is considered that it is not a dependency library file, but other variables, then terminate the query, utilize the dependency library file recorded in the dependency table, traverse the recorded dependency library file, continue to query through the ldd function, write the query result into the dependency table, and traverse again until all the dependency library files corresponding to the key values ​​in the dependency table are traversed, and write the query result into the dependency table as the key value and dependency in the variable; Query the query result using an absolute path query statement to obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table; Using the dependency table, the file corresponding to the key value, the key value and the absolute path corresponding to the key value are cloned into a temporary folder, and the absolute path in the temporary folder is guaranteed to be consistent with the absolute path in the original compilation environment folder of the compilation tool; Setting permissions and user parameters for the compression and packaging tool, and using the compression and packaging tool with the set permissions and user parameters to compress and package the temporary folder to form a packaged clone distributed compilation environment; Use the remote transmission command to send the compressed and packaged temporary folder to the compilation node; The compiling node uses its own compression and packaging tool to decompress the temporary folder after compression and packaging; The distributed compilation command carrying the compilation processing file parameters is transmitted to the compilation node, so that the compilation node compiles the compilation processing file in the chroot environment and returns the compilation processing result.

2. The method according to claim 1, characterized in that The method further comprises: Obtain a distributed compilation statement, and search and traverse the relative storage path, absolute storage path, and file of the dependent library file of the distributed compilation file; Update the dependency table based on the relative storage path, the absolute storage path and the file; The relative storage path, the absolute storage path and the file are cloned into the temporary folder using the dependency table.

3. The method according to claim 1, characterized in that The method further comprises: Obtain the compilation supporting file, and search and traverse the relative storage path, absolute storage path and file of the dependent library file of the compilation supporting file; Update the dependency table based on the relative storage path, the absolute storage path and the file; The dependency table relative storage path, absolute storage path and files are cloned into the temporary folder.

4. A distributed compilation environment cloning device, characterized in that: include: A receiving module, used for receiving a clone compiler instruction and searching for a compilation tool execution path corresponding to the clone compiler instruction; A writing module is used to query the dependent library files of the compiler using the dependent library query function to obtain the relative path query result of the dependent library files, and write the relative path query result of the dependent library files as the key value in the variable and the dependency relationship into the dependency relationship table; A query module is used to query the query result of the dependent library file by using the dependent library query function until the query result is a non-binary dynamic link ELF file, query the dependent library file by using the ldd function, query the dependent file of the dependent library file, and write the dependent file into the dependency table. When the query result is a non-binary dynamic link ELF file, it is determined to be a non-dependent library file. LDD can only display the dependency of the executable module. It sets a series of environment variables. If a non-ELF format file is queried, it is considered that it is not a dependent library file but other variables, then the query is terminated, and the dependent library files recorded in the dependency table are traversed by using the dependent library files recorded, and the query is continued by using the ldd function, and the query result is written into the dependency table, and it is traversed again until all the dependent library files corresponding to the key values ​​in the dependency table are traversed, and the query result is written into the dependency table as the key value and dependency in the variable; An absolute storage path query module, used to query the query result using an absolute path query statement, obtain the absolute storage path of each dependent library file, and add the absolute storage path information to the dependency table; A cloning module, used to clone the file corresponding to the key value and the key value into a temporary folder according to the absolute path corresponding to the key value by using the dependency table, and ensure that the absolute path in the temporary folder is consistent with the absolute path in the original compilation environment folder of the compilation tool; A compression and packaging processing module is used to set permissions and user parameters for the compression and packaging tool, and use the compression and packaging tool with set permissions and user parameters to compress and package the temporary folder to form a packaged clone distributed compilation environment; The sending module is used to send the compressed and packaged temporary folder to the editing node using the remote transmission command; A decompression processing module, used to use the editing node to use its own compression and packaging tool to decompress the temporary folder after compression and packaging; The return module is used to transmit the distributed compilation command carrying the compilation processing file parameters to the editing node, so that the editing node compiles the compilation processing file in the chroot environment and returns the compilation processing result.

5. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the distributed compilation environment cloning method as described in any one of claims 1 to 3 when executed by a computer processor.

Citation Information

Patent Citations

  • Method and apparatus for efficiently implementing a fast-copyable database

    CN113377836A

  • High-performance data cloning copy processing method and system

    CN114020764A