Distributed Compilation Method, Device, Electronic Device and Storage Medium

Abstract

The present invention provides a distributed compilation method, apparatus, electronic device and storage medium, belonging to the technical field of data processing. The method includes: sending a source code file to a server for compilation and assembly processing to obtain a compilation result returned by the server, where the compilation result includes information of the compiled target file; creating a corresponding pseudo-file based on the compilation result, and the pseudo-file includes a method for reconstructing the compilation result file; after obtaining the pseudo-files corresponding to all compilation results, sending a linking command to the server, and the linking command instructs the server to perform a file linking operation on all target files; receiving the executable file returned by the server. The present invention avoids the multiple copying of intermediate files between the client and the server, thereby reducing the network load required for file transmission and improving the distributed compilation efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a distributed compilation method, device, electronic device, and storage medium. Background Art

[0002] Compilation is a process of using a compiler to translate a source program written in a source language into a binary language that can be recognized by a computer. The compilation process is mainly divided into four major steps: preprocessing, compilation, assembly, and linking. Preprocessing is to process the preprocessing instructions in the source code, such as expanding header files, replacing macro definitions, deleting comments, etc., to obtain a preprocessed source code file in the.i format. Compilation is to perform a series of operations such as lexical analysis, syntax analysis, and semantic analysis on the preprocessed source code, generate intermediate code and then convert it into assembly code to obtain an assembly file in the.s format. Assembly is to translate the assembly code into machine code to obtain an object file in the.o format. Linking is to merge multiple object files and possibly used library files together to generate a final executable file. When compiling large-scale software, due to insufficient single-machine performance, multiple machines often need to be called in parallel. In the current distributed compilation system, certain steps can be distributed to remote machines for running, and the compilation results are sent back to the local for further processing.

[0003] Figure 1 is a flow schematic diagram of a distributed compilation method in the prior art. As Figure 1 shown, the client is a machine device responsible for coordinating work and putting forward task requirements; the server is a machine device responsible for performing actual compilation processing. The client sends the source code file to be compiled to the server. After the server completes compilation and assembly, it returns the intermediate result (object file) to the client to help the client coordinate work. When linking files, the client then sends these object files to the server to complete the file linking operation. This method causes the intermediate files to be copied multiple times between the client and the server, increasing the network load and reducing the distributed compilation efficiency.

[0004] Therefore, how to reduce the network load in the distributed compilation process and improve the distributed compilation efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0005] The present invention provides a distributed compilation method, device, electronic device, and storage medium to solve the defects of high network load and low efficiency in the distributed compilation process in the prior art.

[0006] The present invention provides a distributed compilation method applied to a client, including the following steps:

[0007] Send the source code file to the server for compilation and assembly processing to obtain the compilation result returned by the server, where the compilation result includes information about the target file obtained by compilation;

[0008] Based on the compilation result, create a pseudo-file corresponding to the compilation result, where the pseudo-file includes a method for reconstructing the target file obtained from the compilation result;

[0009] After obtaining all the pseudo-files corresponding to the compilation results, send a linking command to the server, where the linking command is used to instruct the server to perform a file linking operation on the target files corresponding to the compilation results;

[0010] Receive the executable file returned by the server.

[0011] According to a distributed compilation method provided by the present invention, after creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes:

[0012] After receiving a first call instruction for the pseudo-file, perform the following steps, where the first call instruction is an instruction for instructing to open a specified file in a specified manner:

[0013] If the first call instruction instructs to open the pseudo-file in a first manner, then after restoring the real target file based on the reconstruction method in the pseudo-file, open the real target file in the first manner, where the first manner means that when performing a write operation after opening the specified file, write according to the requirements of the write operation;

[0014] If the first call instruction instructs to open the pseudo-file in a second manner, then after restoring the real target file based on the reconstruction method in the pseudo-file, open the real target file in the second manner, where the second manner means that when opening the specified file, if the specified file does not exist, create a new file;

[0015] If the first call instruction instructs to open the pseudo-file in a third manner, then return an error message, where the third manner means that if the specified file is a directory file, call the directory file, otherwise return an error message;

[0016] If the first call instruction instructs to open the pseudo-file in a fourth manner, then delete the pseudo-file, create a new real target file, and then open the real target file, where the fourth manner means that when opening the specified file, truncate the length of the specified file to zero bytes.

[0017] According to a distributed compilation method provided by the present invention, after creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes:

[0018] After receiving a second call instruction for the pseudo-file, return the status information of the pseudo-file, where the status information includes file type, creation time, access time, modification time, file size, and user information;

[0019] Among them, the second call instruction is an instruction for obtaining the status information of a specified file, the file type is a normal file, the file size is the file size of the target file corresponding to the pseudo-file, and the user information is the user information of the target file corresponding to the pseudo-file.

[0020] According to a distributed compilation method provided by the present invention, after creating a pseudo-file corresponding to the compilation result based on the compilation result, it further includes:

[0021] After receiving a third call instruction for the pseudo-file, execute the following steps, where the third call instruction is an instruction for instructing to set the size of a specified file to a specified value:

[0022] If the specified value is zero, delete the pseudo-file and then create an empty real target file;

[0023] If the specified value is not zero, after restoring the real target file based on the reconstruction method in the pseudo-file, truncate or expand the real target file.

[0024] According to a distributed compilation method provided by the present invention, the sending the link command to the server includes:

[0025] If there is no stored real target file corresponding to the compilation result, send the link command to the server;

[0026] If there is a stored real target file corresponding to any one of all the compilation results, send the real target file and the link command to the server for the server to perform a file linking operation based on the real target file.

[0027] The present invention also provides a distributed compilation method applied to a server, including the following steps:

[0028] After receiving the source code file sent by the client, compile and assemble the source code file to obtain a target file obtained by compiling and assembling the source code file and store it;

[0029] After receiving the link command sent by the client, perform a file linking operation on the corresponding target file to obtain an executable file;

[0030] Return the executable file to the client.

[0031] The present invention also provides a distributed compilation device, including the following modules:

[0032] A file sending module, configured to: send a source code file to a server for compilation and assembly processing, obtain a compilation result returned by the server, where the compilation result includes information of the compiled target file; a pseudo-file creation module, configured to: create a pseudo-file corresponding to the compilation result based on the compilation result, where the pseudo-file includes a reconstruction method of the compiled target file; a command sending module, configured to: after obtaining the pseudo-files corresponding to all compilation results, send a linking command to the server, where the linking command is used to instruct the server to perform a file linking operation on the target files corresponding to all compilation results; a receiving module, configured to: receive the executable file returned by the server; or, including the following modules:

[0033] A compilation module, configured to: after receiving a source code file sent by a client, perform compilation and assembly processing on the source code file, obtain a target file compiled and assembled based on the source code file, and store it; a linking module, configured to: after receiving the linking command sent by the client, perform a file linking operation on the corresponding target file to obtain an executable file; a sending module, configured to: return the executable file to the client.

[0034] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where when the processor executes the computer program, the distributed compilation method described in any one of the above is implemented.

[0035] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the distributed compilation method described in any one of the above is implemented.

[0036] The present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the distributed compilation method described in any one of the above is implemented.

[0037] The distributed compilation method, device, electronic device and storage medium provided by the present invention send source code files to a server for compilation and assembly processing, and obtain a compilation result returned by the server, where the compilation result includes information of an object file compiled and assembled from the source code file; based on the compilation result, a pseudo-file corresponding to the object file of the compilation result is created, and the pseudo-file includes a reconstruction method of the object file corresponding to the compilation result; after obtaining the pseudo-files of all the object files corresponding to the compilation results, a linking command is sent to the server, and the linking command is used to instruct the server to perform a file linking operation on the object files corresponding to all the compilation results; an executable file returned by the server is received. Generally, the client only needs to know whether the intermediate files exist, as well as information such as the time and size of these intermediate files, to complete the coordination work, and the client does not need to actually read the file content. Therefore, the present invention creates a file type of pseudo-file. During the distributed compilation process, the client receives the information of the object file sent by the server and generates a pseudo-file, so that scheduling and coordination can be carried out without transmitting the entire real object file, avoiding multiple copies of the intermediate files between the client and the server, effectively reducing the network load required for file transmission, and improving the distributed compilation efficiency. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1 is a schematic flowchart of a distributed compilation method in the prior art;

[0040] Figure 2 is one of the schematic flowcharts of the distributed compilation method provided by the present invention;

[0041] Figure 3 is another schematic flowchart of the distributed compilation method provided by the present invention;

[0042] Figure 4 is a schematic diagram of the calling process of the pseudo-file by the operating system provided by the present invention;

[0043] Figure 5 is a third schematic flowchart of the distributed compilation method provided by the present invention;

[0044] Figure 6 is one of the schematic structural diagrams of the distributed compilation device provided by the present invention;

[0045] Figure 7 It is the second structural schematic diagram of the distributed compilation device provided by the present invention;

[0046] Figure 8 It is the structural schematic diagram of the electronic device provided by the present invention. Specific embodiments

[0047] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0048] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including one..." does not exclude the presence of additional identical elements in the process, method, article or device including the element. The orientation or positional relationship indicated by terms such as "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of the present invention. Unless otherwise clearly specified and limited, the terms "mounted", "connected" and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0049] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same category, and do not limit the number of objects. For example, the first object may be one or more. In addition, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates an "or" relationship between the associated objects before and after.

[0050] The following will describe Figures 2 - 8 the distributed compilation method, device, electronic device, and storage medium provided by the embodiments of the present invention.

[0051] Figure 2 is one of the schematic flowcharts of the distributed compilation method provided by the present invention. As Figure 2 shown, this distributed compilation method is applied to a client, and the method may include the following steps:

[0052] S210. Send the source code file to the server for compilation and assembly processing to obtain the compilation result returned by the server, where the compilation result includes information about the target file obtained by compilation;

[0053] S220. Based on the compilation result, create a pseudo-file corresponding to the compilation result, where the pseudo-file includes a method for reconstructing the target file obtained by compilation;

[0054] S230. After obtaining the pseudo-files corresponding to all compilation results, send a link command to the server, where the link command is used to instruct the server to perform a file linking operation on the target files corresponding to all compilation results;

[0055] S240. Receive the executable file returned by the server.

[0056] It should be noted that the execution subject of the distributed compilation method provided by the embodiments of the present invention may be a server or a computer device, such as a tablet computer, a notebook computer, a handheld computer, an ultra-mobile personal computer (UMPC), a netbook, etc.

[0057] In S210, the information of the target file may include, but is not limited to, the hash value, file time, file size, etc. of the target file; among them, the hash value of the file is a fixed-length string generated by processing the file content through a specific algorithm (such as a hash function), and this string can be regarded as the "fingerprint" of the file, used to verify the integrity and uniqueness of the file; the file time may include, but is not limited to, the creation time of the file.

[0058] In some embodiments, the client directly sends the source code file to the server, and the server performs preprocessing, compilation, and assembly on the source code file.

[0059] In other embodiments, the source code file sent by the client to the server is a preprocessed source code file, and the server only needs to perform compilation and assembly on the received file.

[0060] In S210, the client sends the source code file to the server. After the server compiles and assembles the source code file to obtain an object file with the.o suffix, it returns the information of the object file, that is, the compilation result, to the client.

[0061] It can be understood that a source code file may be compiled multiple times using various combinations of different compilers and compilation options. Here, the compilation result is the information of the object file obtained by compiling the specified source code file using the specified compiler and compilation options.

[0062] In S220, after the client obtains the compilation result corresponding to the source code file, it creates a corresponding pseudo-file according to the compilation result. The pseudo-file stores information such as the hash value, file time, and file size of the real object file, and also stores the method for reconstructing the real object file to ensure that when the client needs to use the object file, it can recover and obtain the real object file.

[0063] In the embodiments of the present invention, the pseudo-file exists on the disk and corresponds to another real file (object file), and the corresponding file can be reconstructed in a specified manner. The pseudo-file is a new file type, so it needs to be specially processed at the operating system level: when attempting to read the file content, the operating system is responsible for restoring the pseudo-file to a real file through one or more operations, such as downloading, while retaining the file time information, etc.; when restoring the file, the stored file hash value is used as a reference requirement or it is required that the hash value of the restored result file is consistent with the stored hash value; the deletion operation of the pseudo-file is the same as that of an ordinary file, that is, when deleting the file, the pseudo-file is directly deleted; if another file is used to overwrite the pseudo-file, the overwrite is directly performed.

[0064] In the embodiments of the present invention, the method for reconstructing the pseudo-file may include, but is not limited to, downloading through the network or generating through a local specific method. When the method for reconstructing the pseudo-file adopts the network download method, the network address for downloading the real object file is stored in the pseudo-file; when the method for reconstructing the pseudo-file adopts the local specific method for generation, the commands and parameters required for generating the real object file are stored in the pseudo-file.

[0065] In S230, after the client determines that all pseudo-files already exist, it sends a link command to the server, instructing the server to perform file linking on the corresponding object file.

[0066] It can be understood that the link command contains information such as the hash value and file name of the file to be linked.

[0067] In S240, the server uses the stored object file according to the link command informed by the client, links to generate an executable file, and then returns it to the client.

[0068] Optionally, after the client obtains the executable file, it deletes the corresponding dummy file to release the cache resources.

[0069] Figure 3 It is the second flowchart of the distributed compilation method provided by the present invention. As Figure 3 shown, taking the need to compile two source code files 1.c and 2.c as an example, the process of the distributed compilation method provided by the present invention will be described below.

[0070] As Figure 3 shown, taking the C language as an example, the client sends two source code files 1.c and 2.c to the server. The server performs compilation and assembly processing to obtain two object files 1.o and 2.o, and saves these two object files in the local cache;

[0071] The server notifies the client through the network that the compilation is completed. The client obtains the information of the two object files, including file size, file hash value, file time, etc. This information is usually no more than 1000 bytes (1KB), while the actual object files are usually several million bytes (several MB) to hundreds of millions of bytes (hundreds of MB);

[0072] The client creates dummy files 1.o and 2.o according to the compilation results received from the server.

[0073] The client discovers that both 1.o and 2.o are ready, determines that both 1.o and 2.o are local dummy files, and does not need to send the object files to the server. It only needs to send a link command to the server;

[0074] After receiving the link command, the server retrieves the two object files 1.o and 2.o from the local cache, and links the two object files 1.o and 2.o into an executable program a.exe according to the link command, and then returns a.exe to the client;

[0075] After receiving the executable file a.exe returned by the server, the client can delete the dummy files 1.o and 2.o.

[0076] It should be noted that although Figure 3The process shown in the figure is that the client first sends 1.c to the server, and then sends 2.c to the server after obtaining 1.o returned by the server. However, in the actual implementation process, there is no strict timing limit for the sending of source code files and the receiving of compilation results. For example, the client can send multiple source code files to the server at one time, and then obtain multiple corresponding compilation results returned by the server; the client can also first send some source code files to the server, and then send another part of the source code files to the server after obtaining the compilation results returned by the server.

[0077] The distributed compilation method provided by the embodiments of the present invention sends source code files to the server for compilation and assembly processing to obtain the compilation results returned by the server, and the compilation results include information about the compiled target files; based on the compilation results, a pseudo-file corresponding to the compilation results is created, and the pseudo-file includes the compilation results and the reconstruction method of the target files; after obtaining the pseudo-files corresponding to all compilation results, a link command is sent to the server, and the link command is used to instruct the server to perform a file linking operation on the target files corresponding to all compilation results; the executable file returned by the server is received. Generally, the client only needs to know whether the intermediate files exist, as well as information such as the time and size of these intermediate files, to complete the coordination work, and the client does not need to actually read the file content. Therefore, the present invention creates a file type of pseudo-file. During the distributed compilation process, the client receives the information of the target files sent by the server and generates pseudo-files, so that scheduling and coordination can be carried out without transmitting the entire real target file, avoiding multiple copies of the intermediate files between the client and the server, effectively reducing the network load required for file transmission, and improving the efficiency of distributed compilation.

[0078] In an alternative embodiment, after creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes:

[0079] After receiving the first call instruction for the pseudo-file, the following steps are executed, where the first call instruction is an instruction for instructing to open a specified file in a specified manner:

[0080] If the first call instruction instructs to open the pseudo-file in a first manner, then after restoring the real target file based on the reconstruction method in the pseudo-file, the real target file is opened in the first manner, and the first manner means that when writing is performed after opening the specified file, it is written according to the requirements of the writing operation;

[0081] If the first call instruction indicates to open the pseudo-file in the second way, after restoring the real target file based on the reconstruction method in the pseudo-file, open the real target file in the second way. The second way means that when opening the specified file, if the specified file does not exist, create a new file;

[0082] If the first call instruction indicates to open the pseudo-file in the third way, return an error message. The third way means that if the specified file is a directory file, call the directory file; otherwise, return an error message.

[0083] If the first call instruction indicates to open the pseudo-file in the fourth way, delete the pseudo-file, create a new real target file, and then open the real target file. The fourth way means that when opening the specified file, truncate the length of the specified file to zero bytes.

[0084] In the embodiments of the present invention, the ways required for the write operation may include, but are not limited to, appending content at the end of the specified file, inserting content into the specified file, replacing the content in the specified file, etc.

[0085] In the embodiments of the present invention, the first call instruction is the open system call, which is used to open or create a file. When performing an open system call on the pseudo-file, different modifications are made for different modes in the open system call:

[0086] 1) The first way is the O_APPEND mode. When the O_APPEND flag is set, before each write operation, the kernel will automatically move the file pointer to the end of the file to ensure that all newly written data is appended after the existing content.

[0087] In the embodiments of the present invention, the O_APPEND mode is processed according to reading, that is, after restoring the real file, open it according to the requirements of open.

[0088] 2) The second way is the O_CREAT mode. The O_CREAT flag indicates that if the file does not exist in the specified path, try to create the file.

[0089] In the embodiments of the present invention, the O_CREAT mode is processed according to reading, that is, after restoring the real file, open it according to the requirements of open.

[0090] 3) The third way is the O_DIRECTORY mode. The O_DIRECTORY flag is used to ensure that the provided path points to a directory rather than an ordinary file. If the path is not a directory, the open() call will fail and return the corresponding error code.

[0091] In the embodiments of the present invention, for the O_DIRECTORY mode, an error is reported according to the ordinary file type, that is, it is not allowed that the real file is a directory (folder).

[0092] 4) The fourth mode is the O_TRUNC mode. The function of the O_TRUNC flag is to truncate the length of an existing file to zero bytes when opening the file. That is to say, once this flag is set and the file is successfully opened, the original content in the file will be cleared.

[0093] In the embodiments of the present invention, for the O_TRUNC mode, the pseudo-file is directly deleted, and a new ordinary file is created and opened as required.

[0094] In an optional embodiment, if the hash value of the restored file does not match the hash value stored in the pseudo-file when restoring the file, corresponding measures can be taken according to the configuration information. For example, the following measures can be taken according to the configuration: directly report an error, or retry the restoration / creation, or accept different file contents, or take other relevant measures.

[0095] The distributed compilation method provided by the embodiments of the present invention modifies the opening method of the pseudo-file to ensure the normal reading of the content of the pseudo-file.

[0096] In an optional embodiment, after creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes:

[0097] After receiving the second call instruction for the pseudo-file, return the status information of the pseudo-file. The status information includes file type, creation time, access time, modification time, file size, and user information;

[0098] Wherein, the second call instruction is an instruction for obtaining the status information of a specified file. The file type is an ordinary file, the file size is the file size of the target file corresponding to the pseudo-file, and the user information is the user information of the target file corresponding to the pseudo-file.

[0099] In the embodiments of the present invention, the second call instruction is the stat system call, which is used to obtain the status information of a file or a file system object. When performing the stat system call on the pseudo-file, the corresponding fields return the following information respectively:

[0100] 1) The file type is returned as an ordinary file;

[0101] 2) The creation time, access time, and modification time are respectively returned as the recorded creation time, access time, and modification time;

[0102] 3) The file size is returned as the file size of the recorded real target file;

[0103] 4) User information, returning the user information of the real target file.

[0104] The distributed compilation method provided by the embodiments of the present invention modifies the method for obtaining the status of the pseudo file to ensure normal acquisition of the status information of the pseudo file.

[0105] In an alternative embodiment, after creating the pseudo file corresponding to the compilation result based on the compilation result, the method further includes:

[0106] After receiving a third call instruction for the pseudo file, the following steps are executed, where the third call instruction is an instruction for instructing to set the size of a specified file to a specified value:

[0107] If the specified value is zero, after deleting the pseudo file, create an empty real target file;

[0108] If the specified value is not zero, after restoring the real target file based on the reconstruction method in the pseudo file, truncate or extend the real target file.

[0109] In the embodiments of the present invention, the third call instruction is the truncate series of system call instructions. The truncate series of system calls is a set of tools for adjusting the size of a file, which allows a program to truncate a file to a specified length. When making a truncate system call to a pseudo file, different methods are adopted based on the truncation length:

[0110] 1) When the specified value is zero, that is, the truncation length is 0, directly delete the existing pseudo file and create a new empty file; or convert the pseudo file to an ordinary file inside the operating system and then clear its content and set the length to zero;

[0111] 2) When the specified value is not zero, that is, the truncation length is other lengths, first restore the file content, and then truncate or extend the restored file according to the specified value.

[0112] Figure 4 is a schematic diagram of the process of the operating system calling a pseudo file provided by the present invention. As Figure 4 shown, after opening a file, first determine whether the file is a pseudo file. If it is not a pseudo file, directly open the file in the ordinary file mode and truncate it;

[0113] If it is a pseudo file, determine whether it is truncated to 0. If it is truncated to 0, directly delete the pseudo file and create a new empty file;

[0114] If the truncation is not 0, read information such as the hash value, file time, and reconstruction method in the pseudo-file. Based on this information, reconstruct the real target file, delete the pseudo-file, and then open and truncate the real target file in the manner of an ordinary file.

[0115] The distributed compilation method provided by the embodiment of the present invention modifies the truncation method of the pseudo-file to ensure that the pseudo-file can be truncated normally.

[0116] It should be noted that other system calls related to files are processed in the manner of ordinary files.

[0117] In an optional embodiment, the sending the link command to the server includes:

[0118] If the real target file corresponding to the compilation result is not stored, send the link command to the server;

[0119] If the real target file corresponding to any one of all the compilation results is stored, send the real target file and the link command to the server for the server to perform a file linking operation based on the real target file.

[0120] Specifically, when the client needs to read or modify the content of the target file corresponding to the pseudo-file, restore the pseudo-file to obtain the real target file, and then use the real target file to overwrite the pseudo-file before reading or modifying. When the compilation system on the client determines that all pseudo-files / target files are ready, it starts to execute the linking operation. The linker informs the server of the required commands, file names, and corresponding hash values. If the linker finds that the file to be linked is not a pseudo-file but a real target file, it sends the real target file to the server together, so that the server uses the real target file received from the client for linking instead of using the target file cached before, thus ensuring that after the client modifies the target file, the modified file is used to complete the linking operation.

[0121] As Figure 3 shown, still taking the example of the client sending two source code files, 1.c and 2.c, to the server. After the client receives the compilation results returned by the server and generates pseudo-files 1.o and 2.o, if it is necessary to read or modify the content of 1.o or 2.o on the client, download it to the local area to overwrite the pseudo-file and then read or modify it;

[0122] When the software compilation system on the client sees that both 1.o and 2.o already exist, it starts to execute the linking operation: the linker informs the server of the required commands, file names, and corresponding hash values. If the linker finds that 1.o or 2.o is not a pseudo-file, it sends it to the server together;

[0123] The server retrieves the relevant target files 1.o and 2.o from the cache according to the file name and hash value informed by the client; if the server receives the file 1.o or 2.o sent by the client, it uses the received file for linking instead of the file in the cache.

[0124] After the server uses 1.o and 2.o to generate the executable program a.exe through linking according to the command line informed by the client, it sends the executable program to the client.

[0125] The distributed compilation method provided by the embodiments of the present invention, on the basis of avoiding multiple copies of intermediate files between the client and the server, when other programs modify the intermediate files, the modified files can still be sent to the server to ensure that the server uses the modified target files to generate correct executable files.

[0126] The distributed compilation method of the server provided by the embodiments of the present invention will be described below.

[0127] Figure 5 It is the third flow diagram of the distributed compilation method provided by the present invention, as Figure 5 shown, the distributed compilation method may include:

[0128] S510, after receiving the source code file sent by the client, compile and assemble the source code file to obtain the target file obtained by compiling and assembling the source code file and store it;

[0129] S520, after receiving the link command sent by the client, perform a file linking operation on the corresponding target file to obtain an executable file;

[0130] S530, return the executable file to the client.

[0131] The distributed compilation method provided by the embodiments of the present invention, after the server compiles and assembles the source code file to obtain the target file, stores the target file locally. After receiving the link command sent by the client, it directly uses the local target file for linking without obtaining the target file from the client again, avoiding multiple copies of intermediate files between the client and the server, reducing the network load, and improving the file compilation efficiency.

[0132] It should be noted that the distributed compilation method of the server provided by the embodiments of the present invention can be correspondingly referred to the distributed compilation method of the client described above, and the specific implementation manners will not be elaborated here.

[0133] The distributed compilation device provided by the embodiments of the present invention will be described below. The distributed compilation device described below can be correspondingly referred to the distributed compilation method described above.

[0134] Figure 6 is one of the schematic structural diagrams of the distributed compilation device provided by the present invention. As Figure 6 shown, the distributed compilation device may include but is not limited to:

[0135] A file sending module 610, configured to: send a source code file to a server for compilation and assembly processing, and obtain a compilation result returned by the server, where the compilation result includes information about the target file obtained by compilation;

[0136] A pseudo-file creating module 620, configured to: create a pseudo-file corresponding to the compilation result based on the compilation result, where the pseudo-file includes a reconstruction method of the target file obtained by compilation;

[0137] A command sending module 630, configured to: after obtaining the pseudo-files corresponding to all compilation results, send a linking command to the server, where the linking command is used to instruct the server to perform a file linking operation on the target files corresponding to all compilation results;

[0138] A receiving module 640, configured to: receive the executable file returned by the server.

[0139] It should be noted that the distributed compilation device provided by the embodiments of the present invention can execute the distributed compilation method of the client described in any of the above embodiments during specific operation, and details are not described in this embodiment.

[0140] Figure 7 is the second schematic structural diagram of the distributed compilation device provided by the present invention. As Figure 7 shown, the distributed compilation device may include but is not limited to:

[0141] A compilation module 710, configured to: after receiving a source code file sent by a client, perform compilation and assembly processing on the source code file, and store the target file obtained by compiling and assembling the source code file;

[0142] A linking module 720, configured to: after receiving the linking command sent by the client, perform a file linking operation on the corresponding target file to obtain an executable file;

[0143] A sending module 730, configured to: return the executable file to the client.

[0144] It should be noted that the distributed compilation device provided in the embodiments of the present invention can execute the distributed compilation method of the server described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0145] Figure 8 An example of the physical structure diagram of an electronic device is shown in Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the distributed compilation method, which includes: sending the source code file to the server for compilation and assembly processing to obtain the compilation result returned by the server, where the compilation result includes information of the compiled target file; based on the compilation result, creating a pseudo file corresponding to the compilation result, where the pseudo file includes the reconstruction method of the compiled target file; after obtaining all the pseudo files corresponding to the compilation results, sending a link command to the server, where the link command is used to instruct the server to perform a file linking operation on the target files corresponding to all the compilation results; receiving the executable file returned by the server; or, including:

[0146] After receiving the source code file sent by the client, performing compilation and assembly processing on the source code file to obtain the target file compiled and assembled based on the source code file and storing it; after receiving the link command sent by the client, performing a file linking operation on the corresponding target file to obtain an executable file; returning the executable file to the client.

[0147] In addition, when the logical instructions in the above-mentioned memory 830 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0148] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the distributed compilation method provided by each of the above methods. The method includes: sending a source code file to a server for compilation and assembly processing to obtain a compilation result returned by the server, where the compilation result contains information about the compiled target file; based on the compilation result, creating a pseudo-file corresponding to the compilation result, where the pseudo-file contains a method for reconstructing the compiled target file; after obtaining all the pseudo-files corresponding to the compilation results, sending a link command to the server, where the link command is used to instruct the server to perform a file linking operation on the target files corresponding to all the compilation results; receiving the executable file returned by the server; or, it includes:

[0149] After receiving the source code file sent by the client, performing compilation and assembly processing on the source code file to obtain a target file compiled and assembled based on the source code file and storing it; after receiving the link command sent by the client, performing a file linking operation on the corresponding target file to obtain an executable file; and returning the executable file to the client.

[0150] In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it realizes the execution of the distributed compilation method provided by each of the above methods. The method includes: sending a source code file to a server for compilation and assembly processing to obtain a compilation result returned by the server, where the compilation result contains information about the compiled target file; based on the compilation result, creating a pseudo-file corresponding to the compilation result, where the pseudo-file contains a method for reconstructing the compiled target file; after obtaining all the pseudo-files corresponding to the compilation results, sending a link command to the server, where the link command is used to instruct the server to perform a file linking operation on the target files corresponding to all the compilation results; receiving the executable file returned by the server; or, it includes:

[0151] After receiving the source code file sent by the client, performing compilation and assembly processing on the source code file to obtain a target file compiled and assembled based on the source code file and storing it; after receiving the link command sent by the client, performing a file linking operation on the corresponding target file to obtain an executable file; and returning the executable file to the client.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0153] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed compilation method, characterized in that, Including: Sending the source code file to the server for compilation and assembly processing. After receiving the source code file sent by the client, the server performs compilation and assembly processing on the source code file, obtains the target file compiled and assembled from the source code file, and stores it. After the client obtains the compilation result returned by the server, based on the compilation result, it creates a pseudo-file corresponding to the compilation result. The compilation result includes information of the compiled target file. The information of the compiled target file includes the hash value, file time, and file size of the target file. The pseudo-file includes the information of the compiled target file and the reconstruction method of the compiled target file. The reconstruction method includes downloading through the network or generating through a local preset method. When the reconstruction method uses the network download method, the network address for downloading the real target file is stored in the pseudo-file. When the reconstruction method uses the local preset method for generation, the commands and parameters required for generating the real target file are stored in the pseudo-file. When reading the content of the real target file corresponding to the pseudo-file, the real target file is restored using the reconstruction method, and the hash value of the restored real target file matches the hash value in the pseudo-file. After obtaining the pseudo-files corresponding to all compilation results, send a link command to the server. The link command is used to instruct the server to perform a file linking operation on the target files corresponding to all compilation results. After receiving the link command sent by the client, the server performs a file linking operation on the corresponding target files to obtain an executable file. The client receives the executable file returned by the server.

2. The distributed compilation method according to claim 1, wherein After creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes: After receiving the first call instruction for the pseudo-file, perform the following steps. The first call instruction is an instruction used to indicate opening a specified file in a specified manner. If the first call instruction indicates opening the pseudo-file in a first manner, then after restoring the real target file based on the reconstruction method in the pseudo-file, open the real target file in the first manner. The first manner means that when writing after opening the specified file, write according to the requirements of the write operation. If the first call instruction indicates opening the pseudo-file in a second manner, then after restoring the real target file based on the reconstruction method in the pseudo-file, open the real target file in the second manner. The second manner means that when opening the specified file, if the specified file does not exist, create a new file. If the first call instruction indicates opening the pseudo-file in a third manner, return an error message. The third manner means that if the specified file is a directory file, call the directory file, otherwise return an error message. If the first call instruction indicates to open the pseudo-file in the fourth manner, the pseudo-file is deleted, a new real target file is created, and then the real target file is opened. The fourth manner means that when opening a specified file, the length of the specified file is truncated to zero bytes.

3. The distributed compilation method according to claim 1, wherein After creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes: After receiving the second call instruction for the pseudo-file, the status information of the pseudo-file is returned. The status information includes file type, creation time, access time, modification time, file size, and user information. Among them, the second call instruction is an instruction for obtaining the status information of a specified file. The file type is a regular file, the file size is the file size of the target file corresponding to the pseudo-file, and the user information is the user information of the target file corresponding to the pseudo-file.

4. The distributed compilation method according to claim 1, wherein After creating the pseudo-file corresponding to the compilation result based on the compilation result, it further includes: After receiving the third call instruction for the pseudo-file, the following steps are executed. The third call instruction is an instruction for indicating to set the size of a specified file to a specified value: If the specified value is zero, the pseudo-file is deleted and then an empty real target file is created. If the specified value is not zero, after restoring the real target file based on the reconstruction method in the pseudo-file, the real target file is truncated or extended.

5. The distributed compilation method according to any one of claims 2-4, characterized in that The sending the link command to the server includes: If the real target file corresponding to the compilation result is not stored, the link command is sent to the server. If the real target file corresponding to any one of all the compilation results is stored, the real target file and the link command are sent to the server for the server to perform a file linking operation based on the real target file.

Images