Distributed compilation cache method and system

Abstract

The application discloses a distributed compiling cache method, which is applied to a distributed compiling cache system and comprises the following steps: the distributed compiling cache system sends compiling information through a terminal and obtains a compiling intermediate file by commanding and hijacking the compiling information through the terminal; the distributed compiling cache system receives the compiling intermediate file through a compiling client; the distributed compiling cache system calculates a hash value of the compiling intermediate file through the compiling client and sends the hash value to a compiling cache service module through the compiling client; and the distributed compiling cache system determines a distributed compiling cache strategy according to the hash value through the compiling cache service module. The distributed compiling cache method can expand parallel compiling capacity, accelerate the compiling speed and reduce disk resources caused by multi-site caching.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a distributed compilation caching method and system. Background Art

[0002] Currently, compiling source files is a time-consuming process, especially with poor C++ compiler optimization and when users import a large number of unnecessary header files. Compiling a single source file can take up to five minutes or even longer. During development, a single code modification can take up to half an hour to compile, severely impacting development efficiency. Repeated compilation also wastes machine resources. Summary of the Invention

[0003] The embodiments of the present application provide a distributed compilation caching method and system, which expands parallel compilation capabilities, accelerates compilation speed, and reduces disk resources caused by multi-location caching through the distributed compilation caching method.

[0004] In a first aspect, embodiments of the present application provide a distributed compile cache method and system, which are applied to a distributed compile cache system. The distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. The method includes: the distributed compile cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file; the distributed compile cache system receives the compilation intermediate file through the compile client, and the compilation intermediate file is sent by the terminal to the compile client; the distributed compile cache system calculates a hash value of the compilation intermediate file through the compile client, and sends the hash value to the compile cache service module through the compile client; and the distributed compile cache system determines a distributed compile cache policy based on the hash value through the compile cache service module.

[0005] In a second aspect, an embodiment of the present application provides a distributed compile cache system, characterized in that the distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server, wherein:

[0006] The terminal is used to send compilation information and to perform command hijacking on the compilation information to obtain a compilation intermediate file; the compilation client is used to receive the compilation intermediate file, and the compilation intermediate file is sent by the terminal to the compilation client; the compilation client is further used to calculate a hash value of the compilation intermediate file and send the hash value to the compilation cache service module; the compilation cache service module is used to determine a distributed compilation cache strategy based on the hash value.

[0007] It can be seen that, in the embodiment of the application, first, the distributed compilation cache system sends compilation information through the terminal, and obtains a compilation intermediate file through command hijacking of the compilation information by the terminal, second, the distributed compilation cache system receives the compilation intermediate file through the compilation client, the compilation intermediate file is sent to the compilation client by the terminal, then, the distributed compilation cache system calculates a hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client, finally, the distributed compilation cache system determines a distributed compilation cache strategy according to the hash value through the compilation cache service module. It can be seen that, through the distributed compilation cache method, the compilation time of a single terminal can be accelerated to the time of extracting the compilation intermediate file from the cache on the basis of distributed compilation, that is, the compilation speed is accelerated through parallel compilation, and when the cache is compiled, the source file is compiled once and can be globally shared, thereby releasing more compilation resources, and further reducing the waste of disk resources caused by multiple caches through unified cache. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 is a framework schematic diagram of a distributed compilation cache system provided by an embodiment of the present application;

[0010] Figure 2A is a flow schematic diagram of a distributed compilation cache method provided by an embodiment of the present application;

[0011] Figure 2B is a timing schematic diagram of a distributed compilation cache method provided by an embodiment of the present application;

[0012] Figure 2C is a timing schematic diagram of another distributed compilation cache method provided by an embodiment of the present application;

[0013] Figure 3 is a flow schematic diagram of another distributed compilation cache method provided by an embodiment of the present application;

[0014] Figure 4 is a flow schematic diagram of another distributed compilation cache method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0016] The following are detailed descriptions of each.

[0017] The terms "first," "second," "third," and "fourth," etc., in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, rather than to describe a specific order. In addition, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0018] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0019] Below, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0020] The following is a detailed introduction to the embodiments of the present application.

[0021] like Figure 1 As shown, Figure 1 This is a schematic diagram of a distributed compilation cache system framework provided by an embodiment of the present application. The distributed compilation cache system includes a compilation cache service module, a compilation client, and a terminal. The compilation cache service module includes a scheduling server and a file system of the scheduling server, wherein:

[0022] The terminal is configured to send the compilation information and to command hijack the compilation information to obtain a compilation intermediate file; the compilation client is configured to receive the compilation intermediate file, which is sent by the terminal to the compilation client; the compilation client is further configured to calculate a hash value of the compilation intermediate file and send the hash value to the compilation cache service module; and the compilation cache service module is configured to determine a distributed compilation cache strategy according to the hash value.

[0023] It can be seen that, in the embodiment of the application, first, the distributed compilation cache system sends the compilation information through the terminal, and command hijacks the compilation information through the terminal to obtain a compilation intermediate file, second, the distributed compilation cache system receives the compilation intermediate file through the compilation client, which is sent by the terminal to the compilation client, then, the distributed compilation cache system calculates a hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client, and finally, the distributed compilation cache system determines a distributed compilation cache strategy according to the hash value through the compilation cache service module. It can be seen that, through the distributed compilation cache method, the compilation time of a single terminal can be accelerated to the time of extracting the compilation intermediate file from the cache on the basis of distributed compilation, that is, the compilation speed is accelerated through parallel compilation, and in the compilation cache, the source file is compiled once and can be globally shared, thereby releasing more compilation resources, and further reducing the disk resource waste caused by multiple caches through unified cache.

[0024] In one possible example, in terms of determining a distributed compilation cache strategy by the compilation cache service module according to the hash value, the compilation cache service module is configured to determine whether the hash value is designated for cache; if yes, the distributed compilation cache system obtains a first compilation intermediate file corresponding to the hash value from the file system of the scheduling server through the compilation client; and the file system of the scheduling server is configured to send the first compilation intermediate file to the terminal.

[0025] In one possible example, in terms of the distributed compilation cache system determining the distributed compilation cache policy based on the hash value through the compilation cache service module, the compilation cache service module is used to determine whether the hash value is destined for caching; if not, the distributed compilation cache system executes the compilation command through the compilation client to generate a second compilation intermediate file; the compilation client is used to send the second compilation intermediate file to the file system of the scheduling server; the file system of the scheduling server is used to receive the second compilation intermediate file and update a preset database on the file system of the scheduling server; the file system of the scheduling server is further used to send the second compilation intermediate file to the terminal.

[0026] In one possible example, in terms of the distributed compilation cache system determining whether the hash value is destined for cache through the compilation cache service module, the compilation client is used to send the hash value to the file system of the scheduling server; the file system of the scheduling server is used to query a preset database to determine whether there is a target hash value consistent with the hash value in the preset database; if so, it is determined to be destined for cache; or, if not, it is determined to be not destined for cache.

[0027] In one possible example, before the distributed compilation cache system receives the compilation intermediate file through the compilation client, the terminal is used to communicate with the scheduling server to determine whether the scheduling server supports parallel compilation; if so, at least one compilation client is captured in the scheduling server.

[0028] In a possible example, in terms of capturing at least one compiling client in the scheduling server, the scheduling server is used to query multiple compiling clients; and is also used to screen out at least one compiling client that can support parallel compilation as a target compiling client.

[0029] In a possible example, in the distributed compilation cache system calculating the hash value of the compilation intermediate file through the compilation client, the compilation client is configured to calculate the hash value through the compilation environment information and the source file in the compilation intermediate file.

[0030] In a possible example, in the aspect of performing command hijacking on the compilation information through the terminal, the terminal is used for soft connection or execution command overlay to perform command hijacking.

[0031] In a possible example, the cache server module communicates with the compiling client; the cache server module communicates with the terminal; and the compiling client module communicates with the terminal.

[0032] In a possible example, before the distributed compilation cache system sends compilation information through the terminal and hijacks the compilation information through the terminal to obtain the compilation intermediate file, the terminal is used to obtain the compilation information through preloading.

[0033] The above distributed compilation cache system consists of three main logical components: compilation cache service module, compilation client and terminal.

[0034] The compilation cache service module mainly stores global compilation intermediate files and manages all available compilation client resources.

[0035] The compilation client mainly completes the actual compilation command work, as well as pulling and updating the compilation cache.

[0036] The terminal, also known as the (user client), is mainly used to actually initiate the compilation command end, and at the same time hijack the compilation command to communicate with the compilation service and the compilation machine.

[0037] Terminals and compilation clients can include a variety of desktops and laptops with wireless communication capabilities, as well as subnotebooks and tablets and ultrabooks.

[0038] See also Figure 2A , Figure 2A This is a flow chart of a distributed compile cache method provided by an embodiment of the present application, which is applied to a distributed compile cache system. The distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. As shown in the figure, the distributed compile cache method includes:

[0039] S201: The distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file.

[0040] Optionally, the performing command hijacking on the compilation information through the terminal includes: the distributed compilation cache system performing command hijacking through a soft link of the terminal or executing a command overlay.

[0041] The command hijacking can be performed through gcc or clang.

[0042] Among them, in one implementation method of command hijacking the compilation information, the user can directly use the make command and directly connect to the parallel compilation capability of make. Make supports local parallel compilation. By calculating whether the source file has been modified, the compilation of the unmodified file can be skipped, thereby speeding up the overall compilation speed through parallel compilation.

[0043] Among them, the calculation of intermediate target files can be implemented through the basic logic of ccache.

[0044] Optionally, the cache server module communicates with the compiling client; the cache server module communicates with the terminal; and the compiling client module communicates with the terminal.

[0045] Optionally, before the distributed compile cache system sends the compilation information through the terminal and hijacks the compilation information through the terminal to obtain the compiled intermediate file, the method further includes: the distributed compile cache system obtains the compilation information through preloading through the terminal.

[0046] The preloaded code is a compiled code preset by the terminal itself.

[0047] It can be seen that in this example, by hijacking the command of the compilation information, the required files are further extracted, avoiding the possibility of subsequent actual compilation errors.

[0048] S202: The distributed compilation cache system receives the compilation intermediate file through the compilation client. The compilation intermediate file is sent from the terminal to the compilation client.

[0049] In one possible example, before the distributed compile cache system receives the compiled intermediate file through the compile client, the method includes: the distributed compile cache system communicates with the scheduling server through the terminal to determine whether the scheduling server supports parallel compilation; if so, fetching at least one compile client from the scheduling server.

[0050] Optionally, capturing at least one compiling client from the scheduling server includes: the distributed compiling cache system querying multiple compiling clients through the scheduling server; and the distributed compiling cache system screening out at least one compiling client that can support parallel compilation as a target compiling client through the scheduling server.

[0051] It can be seen that in this example, at least one compilation client receives compilation commands and pushes or pulls compilation intermediate files to the cache service request, thereby expanding the parallel compilation capability. Further, through parallel compilation, users can enjoy a compilation environment that exceeds local resources, thereby speeding up the compilation speed.

[0052] S203: The distributed compilation cache system calculates a hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client.

[0053] Optionally, the distributed compilation cache system calculates the hash value of the compilation intermediate file through the compilation client, including: the distributed compilation cache system calculates the hash value through the compilation client using the compilation environment information and the source file in the compilation intermediate file.

[0054] The compilation environment information may include different information such as environment variables and header files, which is not limited here.

[0055] The source file is set by the developer.

[0056] S204: The distributed compile cache system determines a distributed compile cache strategy according to the hash value through the compile cache service module.

[0057] In one possible example, the distributed compile cache system determines a distributed compile cache policy based on the hash value through the compile cache service module, including: the distributed compile cache system determines, through the compile cache service module, whether the hash value is destined for caching; if so, the distributed compile cache system obtains, through the compile client, a first compile intermediate file corresponding to the hash value from the file system of the scheduling server; and the distributed compile cache system sends the first compile intermediate file to the terminal through the file system of the scheduling server.

[0058] Among them, such as Figure 2B As shown, Figure 2B A timing diagram of a distributed compilation cache method.

[0059] In a possible example, the distributed compile cache system determines, by the compile cache service module, a distributed compile cache strategy according to the hash value, including:

[0060] The distributed compile cache system determines, through the compile cache service module, whether the hash value is destined for caching. If not, the distributed compile cache system executes the compile command through the compile client to generate a second compile intermediate file. The distributed compile cache system sends the second compile intermediate file to the file system of the scheduling server through the compile client. The distributed compile cache system receives the second compile intermediate file through the file system of the scheduling server and updates a preset database on the file system of the scheduling server. The distributed compile cache system sends the second compile intermediate file to the terminal through the file system of the scheduling server.

[0061] Among them, such as Figure 2C As shown, Figure 2C A timing diagram of another distributed compilation cache method.

[0062] It can be seen that in this example, all compilation intermediate files are stored uniformly through the compilation cache service module. Once the source file is compiled, any user can share the intermediate file, further releasing more compilation resources and reducing the waste of disk resources caused by multiple caches.

[0063] Optionally, the distributed compile cache system determines whether the hash value is destined for caching through the compile cache service module, including: the distributed compile cache system sends the hash value to the file system of the scheduling server through the compile client; the distributed compile cache system queries a preset database through the file system of the scheduling server to determine whether there is a target hash value consistent with the hash value in the preset database; if so, determining that it is destined for caching; or, if not, determining that it is not destined for caching.

[0064] The preset database may be provided by the developer through data or obtained from big data, which is not the only limitation here.

[0065] It can be seen that in the embodiment of the present application, first, the distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file. Secondly, the distributed compilation cache system receives the compilation intermediate file through the compilation client, and the compilation intermediate file is sent by the terminal to the compilation client. Then, the distributed compilation cache system calculates the hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client. Finally, the distributed compilation cache system determines the distributed compilation cache strategy according to the hash value through the compilation cache service module. It can be seen that the distributed compilation cache method can accelerate the compilation time of a single terminal to the time of extracting the compilation intermediate file from the cache on the basis of distributed compilation, that is, speed up the compilation speed through parallel compilation, and when compiling and caching, the source file can be compiled once and then shared globally, thereby releasing more compilation resources, and further reducing the waste of disk resources caused by caching in multiple locations through unified caching.

[0066] With the above Figure 2A For details on the embodiment shown, please refer to Figure 3 , Figure 3 This is a flow chart of a distributed compile cache method provided by an embodiment of the present application, which is applied to a distributed compile cache system. The distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. As shown in the figure, the distributed compile cache method includes:

[0067] S301, the distributed compiling cache system sends compiling information through the terminal, and gets a compiling intermediate file by commanding hijacking the compiling information through the terminal.

[0068] S302, the distributed compiling cache system communicates with the scheduling server through the terminal to determine whether the scheduling server supports parallel compiling.

[0069] S303, if yes, at least one compiling client is grabbed in the scheduling server.

[0070] S304, the distributed compiling cache system receives the compiling intermediate file through the compiling client, which is sent to the compiling client by the terminal.

[0071] S305, the distributed compiling cache system calculates the hash value of the compiling intermediate file through the compiling client, and sends the hash value to the compiling cache service module through the compiling client.

[0072] S306, the distributed compiling cache system judges whether the hash value is defined cache through the compiling cache service module.

[0073] S307, if yes, the distributed compiling cache system gets the first compiling intermediate file corresponding to the hash value from the file system of the scheduling server through the compiling client.

[0074] S308, the distributed compiling cache system sends the first compiling intermediate file to the terminal through the file system of the scheduling server.

[0075] It can be seen that in the embodiment of the present application, first, the distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file. Secondly, the distributed compilation cache system receives the compilation intermediate file through the compilation client, and the compilation intermediate file is sent by the terminal to the compilation client. Then, the distributed compilation cache system calculates the hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client. Finally, the distributed compilation cache system determines the distributed compilation cache strategy according to the hash value through the compilation cache service module. It can be seen that the distributed compilation cache method can accelerate the compilation time of a single terminal to the time of extracting the compilation intermediate file from the cache on the basis of distributed compilation, that is, speed up the compilation speed through parallel compilation, and when compiling and caching, the source file can be compiled once and then shared globally, thereby releasing more compilation resources, and further reducing the waste of disk resources caused by caching in multiple locations through unified caching.

[0076] In addition, by receiving compilation commands through at least one compilation client and pushing or pulling compilation intermediate files to the cache service request, the parallel compilation capability is expanded. Further, through parallel compilation, users can enjoy a compilation environment that exceeds local resources and speed up the compilation speed.

[0077] With the above Figure 2A For details on the embodiment shown, please refer to Figure 4 , Figure 4 This is a flow chart of a distributed compile cache method provided by an embodiment of the present application, which is applied to a distributed compile cache system. The distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. As shown in the figure, the distributed compile cache method includes:

[0078] S401: The distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file.

[0079] S402: The distributed compilation cache system communicates with the scheduling server through the terminal to determine whether the scheduling server supports parallel compilation.

[0080] S403: If supported, fetch at least one compiling client from the scheduling server.

[0081] S404: The distributed compilation cache system receives the compilation intermediate file through the compilation client. The compilation intermediate file is sent from the terminal to the compilation client.

[0082] S405: The distributed compilation cache system calculates a hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client.

[0083] S406: The distributed compile cache system determines, through the compile cache service module, whether the hash value is destined for cache.

[0084] S407: If not, the distributed compilation cache system executes the compilation command through the compilation client to generate a second compilation intermediate file.

[0085] S408: The distributed compilation cache system sends the second compilation intermediate file to the file system of the scheduling server through the compilation client.

[0086] S409: The distributed compilation cache system receives the second compilation intermediate file through the file system of the scheduling server and updates a preset database on the file system of the scheduling server.

[0087] S410: The distributed compilation cache system sends the second compilation intermediate file to the terminal through the file system of the scheduling server.

[0088] It can be seen that in the embodiment of the present application, first, the distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file. Secondly, the distributed compilation cache system receives the compilation intermediate file through the compilation client, and the compilation intermediate file is sent by the terminal to the compilation client. Then, the distributed compilation cache system calculates the hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client. Finally, the distributed compilation cache system determines the distributed compilation cache strategy according to the hash value through the compilation cache service module. It can be seen that the distributed compilation cache method can accelerate the compilation time of a single terminal to the time of extracting the compilation intermediate file from the cache on the basis of distributed compilation, that is, speed up the compilation speed through parallel compilation, and when compiling and caching, the source file can be compiled once and then shared globally, thereby releasing more compilation resources, and further reducing the waste of disk resources caused by caching in multiple locations through unified caching.

[0089] In addition, by receiving compilation commands through at least one compilation client and pushing or pulling compilation intermediate files to the cache service request, the parallel compilation capability is expanded. Further, through parallel compilation, users can enjoy a compilation environment that exceeds local resources and speed up the compilation speed.

[0090] The above mainly introduces the solution of the embodiment of the present application from the perspective of the execution process on the method side. It can be understood that in order to realize the above functions, the distributed compilation cache system includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the steps described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0091] The embodiment of the present application can divide the distributed compilation cache system into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical functional division. In actual implementation, there may be other division methods.

[0092] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0093] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0094] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.

[0095] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0096] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0097] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0098] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0099] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A distributed compilation caching method, characterized in that: Applied to a distributed compile cache system, the distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. The method includes: The distributed compilation cache system sends compilation information through the terminal, and hijacks the compilation information through the terminal to obtain a compilation intermediate file; The distributed compilation cache system receives the compilation intermediate file through the compilation client, and the compilation intermediate file is sent from the terminal to the compilation client; The distributed compilation cache system calculates a hash value of the compilation intermediate file through the compilation client, and sends the hash value to the compilation cache service module through the compilation client; The distributed compile cache system determines a distributed compile cache policy based on the hash value through the compile cache service module, wherein the distributed compile cache system determines whether the hash value is destined for caching through the compile cache service module; if so, the distributed compile cache system obtains a first compile intermediate file corresponding to the hash value from the file system of the scheduling server through the compile client; the distributed compile cache system sends the first compile intermediate file to the terminal through the file system of the scheduling server; if not, the distributed compile cache system executes a compile command through the compile client to generate a second compile intermediate file; the distributed compile cache system sends the second compile intermediate file to the file system of the scheduling server through the compile client; the distributed compile cache system receives the second compile intermediate file through the file system of the scheduling server and updates a preset database on the file system of the scheduling server; and the distributed compile cache system sends the second compile intermediate file to the terminal through the file system of the scheduling server.

2. The method according to claim 1, characterized in that The distributed compile cache system determines, through the compile cache service module, whether the hash value is destined for caching, including: The distributed compilation cache system sends the hash value to the file system of the scheduling server through the compilation client; The distributed compile cache system queries a preset database through the file system of the scheduling server to determine whether a target hash value consistent with the hash value exists in the preset database; If it exists, the cache is determined to be destined; or, If not present, it is determined that the cache is not designated.

3. The method according to claim 1, characterized in that Before the distributed compilation cache system receives the compilation intermediate file through the compilation client, the method includes: The distributed compile cache system communicates with the scheduling server through the terminal to determine whether the scheduling server supports parallel compilation; If supported, at least one compiling client is fetched from the scheduling server.

4. The method according to claim 3, characterized in that The capturing of at least one compiling client in the scheduling server includes: The distributed compilation cache system queries multiple compilation clients through the scheduling server; The distributed compile cache system selects at least one compile client that can support parallel compilation as a target compile client through the scheduling server.

5. The method according to claim 1, wherein The distributed compilation cache system calculates the hash value of the compilation intermediate file through the compilation client, including: The distributed compilation cache system calculates a hash value through the compilation client using the compilation environment information and the source file in the compilation intermediate file.

6. The method according to claim 1, characterized in that The command hijacking of the compilation information by the terminal includes: The distributed compile cache system performs command hijacking through the soft connection of the terminal or execution command overwriting.

7. The method according to claim 1, characterized in that The cache server module communicates with the compiling client; the cache server module communicates with the terminal; and the compiling client module communicates with the terminal.

8. The method according to claim 1, characterized in that Before the distributed compilation cache system sends compilation information through the terminal and hijacks the compilation information through the terminal to obtain the compilation intermediate file, the method further includes: The distributed compilation cache system obtains compilation information through preloading via the terminal.

9. A distributed compilation cache system, characterized in that: The distributed compile cache system includes a compile cache service module, a compile client, and a terminal. The compile cache service module includes a scheduling server and a file system of the scheduling server. The terminal is used to send compilation information and to perform command hijacking on the compilation information to obtain a compilation intermediate file; the compilation client is used to receive the compilation intermediate file, and the compilation intermediate file is sent by the terminal to the compilation client; the compilation client is also used to calculate a hash value of the compilation intermediate file and send the hash value to the compilation cache service module; the compilation cache service module is used to determine a distributed compilation cache strategy based on the hash value, wherein the distributed compilation cache system determines whether the hash value is destined for caching through the compilation cache service module; if so, the distributed compilation cache system obtains the hash value from the file system of the scheduling server through the compilation client. The distributed compile cache system generates a first compiled intermediate file corresponding to the compiled program; the distributed compile cache system sends the first compiled intermediate file to the terminal through the file system of the scheduling server; if not, the distributed compile cache system executes the compile command through the compile client to generate a second compiled intermediate file; the distributed compile cache system sends the second compiled intermediate file to the file system of the scheduling server through the compile client; the distributed compile cache system receives the second compiled intermediate file through the file system of the scheduling server and updates a preset database on the file system of the scheduling server; the distributed compile cache system sends the second compiled intermediate file to the terminal through the file system of the scheduling server.

10. The system according to claim 9, characterized in that In the distributed compile cache system determining whether the hash value is destined for caching through the compile cache service module, the compile client is configured to send the hash value to the file system of the scheduling server; the file system of the scheduling server is configured to query a preset database to determine whether a target hash value consistent with the hash value exists in the preset database; If it exists, the cache is determined to be destined; Alternatively, if not present, it is determined that the cache is not designated.

11. The system according to claim 9, wherein: Before the distributed compilation cache system receives the compilation intermediate file through the compilation client, the terminal is used to communicate with the scheduling server to determine whether the scheduling server supports parallel compilation; If supported, at least one compiling client is fetched from the scheduling server.

12. The system according to claim 11, wherein: In terms of capturing at least one compiling client in the scheduling server, the scheduling server is used to query multiple compiling clients; and is also used to screen out at least one compiling client that can support parallel compilation as a target compiling client.

13. The system according to claim 9, wherein: In the aspect of the distributed compilation cache system calculating the hash value of the compilation intermediate file through the compilation client, the compilation client is used to calculate the hash value through the compilation environment information and the source file in the compilation intermediate file.

14. The system according to claim 9, wherein: In the aspect of performing command hijacking on the compilation information through the terminal, the terminal is used for soft connection or execution command overlay to perform command hijacking.

15. The system according to claim 9, wherein: The cache server module communicates with the compiling client; the cache server module communicates with the terminal; and the compiling client module communicates with the terminal.

16. The system according to claim 9, wherein: Before the distributed compilation cache system sends compilation information through the terminal and hijacks the compilation information through the terminal to obtain the compilation intermediate file, the terminal is used to obtain the compilation information through preloading.

Images