Data processing method and related apparatus

Abstract

A data processing method and related apparatus are provided in embodiments of the present disclosure. The first compiler intermediate representation is incorporated into the first shared library during the building phase of the first shared library, and it can be used for execution of LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation, and then rebuilding is performed based on the optimized first compiler intermediate representation to obtain a rebuilt first shared library. By this optimization before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.

Description

DATA PROCESSING METHOD AND RELATED APPARATUSTECHNICAL FIELD|0001] The present disclosure relates to the field of software engineering and computer science technologies, and in particular, to a data processing method and related apparatus.BACKGROUND

[0002] Usage of shared libraries (particularly in a Linux-based systems) is a common way to manage an application’s dependencies. The shared libraries would be loaded by a dynamic loader at the beginning of the execution of the application, so as to ensure correct running of the application.

[0003] This background information is provided to reveal information believed by the applicant to be of possible relevance to the present disclosure. No admission is necessarily intended, nor should be construed, that any of the preceding information constitutes prior art against the present disclosure.SUMMARY

[0004] In a first aspect, a data processing method is provided by the present disclosure, the method is applied to a processor and includes:determining, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library, where the first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple shared libraries;upon determining to perform LTO for the first shared library, performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation; andobtaining a rebuilt first shared library based on the optimized first compiler intermediate representation.

[0005] The first compiler intermediate representation is incorporated into the first shared library during the ibuilding phase of the first shared library, and it can be used for execution of LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation, and then the first shared library having the optimized first compiler intermediate representation can be rebuilt to obtain a rebuilt first shared library. By this optimization before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.[0006| In a possible implementation of the first aspect, the determining, based on a dependency for a first shared library, whether to perform LTO for the first shared library includes:when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries includes a corresponding second compiler intermediate representation, determining to perform the LTO for the first shared library.

[0007] In a case that the first library includes the first compiler intermediate representation, and at least one of its child libraries (the one or more second shared libraries) includes the second compiler intermediate representation, it is determined that LTO can be performed for the first shared library. Through this specific determination rule, a shared library for which the LTO is performed can be quickly and accurately determined.

[0008] In a possible implementation of the first aspect, the performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation includes:performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0009] The LTO for the first shared library is performed on the first compiler intermediate representation, and will use data from the second compiler intermediate representation, that is, to achieve the optimization purpose, the LTO for the first shared library is performed on its compiler intermediate representation, and is also associated with the compiler intermediate representation(s) of the at least one of its child libraries, such that functions) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0010] In a possible implementation of the first aspect, the performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation includes:extracting the first compiler intermediate representation included in the first shared library, and thesecond compiler intermediate representation in each of the at least one of the one or more second shared libraries; combining the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; andperforming LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0011] The LTO for the first shared library is performed on the combined compiler intermediate representation which is a combination of the first compiler intermediate representation and the second compiler intermediate representation, such that function(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0012] In a possible implementation of the first aspect, the first shared library is built by incorporating a build flag, and the obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation includes:obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag.

[0013] The build flag is used for the generation of the rebuilt first shared library. A shared library is built by incorporating a build flag and a compiler intermediate representation. The build flag and the first compiler intermediate representation are incorporated into the first shared library during the building phase, while the first shared library having the optimized first compiler intermediate representation can be rebuilt with the build flag of the first shared library to obtain the rebuilt first shared library.

[0014] In a possible implementation of the first aspect, before the obtaining the rebuilt first shared library, the method further includes:storing the build flag in a temporary location.

[0015] The first shared library is built by incorporating a build flag and a first compiler intermediate representation, if it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation, the build flag can be stored in a temporary location, and then the optimized first compiler intermediate representation can be rebuilt with this build flag. The storage of the build flag is beneficial for the generation of the rebuilt first shared library.

[0016] In a possible implementation of the first aspect, the method further includes:generating a merged shared library with the rebuilt first shared library.

[0017] Since optimization is performed before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.

[0018] In a possible implementation of the first aspect, before generating a merged shared library with the rebuilt first shared library, the method further includes:storing the rebuilt first shared library in a temporary location.

[0019] During the merging process, the rebuilt first shared library can be invoked from the temporary location.

[0020] In a possible implementation of the first aspect, the method further includes:building a dependency graph, where the dependency graph indicates a dependency for each of the multiple shared libraries;where the method further includes:taking each of the multiple shared libraries as the first shared library",repeating the determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for the first shared library and the obtaining of the rebuilt first shared library.

[0021] The dependency for each of the multiple shared libraries can be indicated by the dependency graph. There may be multiple shared libraries in the dependency graph, and for each of the multiple shared libraries, the determination of whether to perform LTO is executed, when a shared library is determined to be subject to LTO, LTO is performed to obtain the optimized compiler intermediate representation, and then the shared library having the optimized compiler intermediate representation is rebuilt. That is, the number of the shared libraries to be rebuilt in the dependency graph is not limited, in a case that there are multiple shared libraries that need to be rebuilt, there may be multiple rebuilding processes accordingly, and thus multiple rebuilt shared libraries are obtained.

[0022] In a possible implementation of the first aspect, the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library.

[0023] The first shared library depends on the second shared libraries, that is, the second shared libraries are child libraries of the first shared library. When a function in the first shared library is executed, since this function uses function(s) in the second shared libraries, the functions) in the second shared libraries would also be executed. In this way, functions) from one shared library (the at least one of the one or more second libraries) can be inlinedinto another shared library (the first shared library).

[0024] In a possible implementation of the first aspect, the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code.

[0025] The specific implementation form of the compiler intermediate representation can be LLVM IR code or the textual code according to actual requirements.

[0026] In a possible implementation of the first aspect, the first shared library is built by a compiler and a linker in the processor, and the LTO when performed is performed by a loader in the processor.

[0027] In a second aspect, a data processing apparatus is provided by the present disclosure, and the apparatus includes:a determining module, configured to determine, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library, where the first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple shared libraries;a performing module, configured to upon determining to perform LTO for the first shared library, perform LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation; andan obtaining module, configured to obtain a rebuilt first shared library based on the optimized first compiler intermediate representation.

[0028] The first compiler intermediate representation is incorporated into the first shared library during the building phase of the first shared library, and it can be used for execution of LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation, and then the first shared library having the optimized first compiler intermediate representation is rebuilt to obtain a rebuilt first shared library. By this optimization before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.

[0029] In a possible implementation of the second aspect, the determining module is specifically configured to:when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries includes a corresponding second compilerintermediate representation, determine to perform the LTO for the first shared library.

[0030] In a case that the first library includes the first compiler intermediate representation, and at least one of its child libraries (the one or more second shared libraries) includes the second compiler intermediate representation, it is determined that LTO can be performed for the first shared library. Through this specific determination rule, a shared library for which the LTO is performed can be quickly and accurately determined.

[0031] In a possible implementation of the second aspect, the performing module is specifically configured to:perform LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0032] The LTO for the first shared library is performed on the first compiler intermediate representation, and will use data from the second compiler intermediate representation, that is, to achieve the optimization purpose, the LTO for the first shared library is performed on its compiler intermediate representation, and is also associated with the compiler intermediate representation(s) of the at least one of its child libraries, such that function(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0033] In a possible implementation of the second aspect, the performing module is specifically configured to:extract the first compiler intermediate representation included in the first shared library, and the second compiler intermediate representation in each of the at least one of the one or more second shared libraries;combine the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; andperform LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0034] The LTO for the first shared library is performed on the combined compiler intermediate representation which is a combination of the first compiler intermediate representation and the second compiler intermediate representation, such that function(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0035] In a possible implementation of the second aspect, the first shared library is built by incorporating a build flag, and the obtaining module is specifically configured to obtain the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag.

[0036] The build flag is used for the generation of the rebuilt first shared library. A shared library is built byincorporating a build flag and a compiler intermediate representation. The build flag and the first compiler intermediate representation are incorporated into the first shared library during the building phase, while the first shared library having the optimized first compiler intermediate representation can be rebuilt with the build flag of the first shared library to obtain the rebuilt first shared library.

[0037] In a possible implementation of the second aspect, the apparatus further includes a first storing module, configured to store the build flag in a temporary location.

[0038] The first shared library is built by incorporating a build flag and a first compiler intermediate representation, if it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation, the build flag can be stored in a temporary location, and then the optimized first compiler intermediate representation can be rebuilt with this build flag. The storage of the build flag is beneficial for the generation of the rebuilt first shared library.

[0039] In a possible implementation of the second aspect, the apparatus further includes a generating module, configured to generate a merged shared library with the rebuilt first shared library.

[0040] Since optimization is performed before the merging process, when the rebuilt first shared library is meiged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.

[0041] In a possible implementation of the second aspect, the apparatus further includes a second storing module, configured to store the rebuilt first shared libraty in a temporary location.

[0042] During the merging process, the rebuilt first shared libraty can be invoked from the temporary location.

[0043] In a possible implementation of the second aspect, the apparatus further includes a building module and a processing module, where the building module is configured to build a dependency graph, where the dependency graph indicates a dependency for each of the multiple shared libraries; the processing module is configured to take each of the multiple shared libraries as the first shared library, and repeat the determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for the first shared library and the obtaining of the rebuilt first shared library.

[0044] The dependency for each of the multiple shared libraries can be indicated by the dependency graph. There may be multiple shared libraries in the dependency graph, and for each of the multiple shared libraries, the determination of whether to perform LTO is executed, when a shared library is determined to be subject to LTO,LTO is performed to obtain the optimized compiler intermediate representation, and then the shared library having the optimized compiler intermediate representation is rebuilt. That is, the number of the shared library to be optimized in the dependency graph is not limited, in a case that there are multiple shared libraries that need to be optimized, there may be multiple optimization processes accordingly, and thus multiple rebuilt shared libraries are obtained.[0045| In a possible implementation of the second aspect, the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library.

[0046] The first shared library depends on the second shared libraries, that is, the second shared libraries are child libraries of the first shared library. When a function in the first shared library is executed, since this function uses functions) in the second shared libraries, the function(s) in the second shared libraries would also be executed. In this way, function(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0047] In a possible implementation of the second aspect, the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code.

[0048] The specific inpiementation form of the compiler intermediate representation can be LLVM IR code or the textual code according to actual requirements.

[0049] In a third aspect, a data processing apparatus is provided by the present disclosure, and the apparatus is configured to execute the method according to the first aspect or any possible implementation of the first aspect.

[0050] In a fourth aspect, a computing device cluster including at least one computing device is provided by the present disclosure, and the computing device includes at least one processor and a memory coupled with the processor, where the memory stores instructions which, when executed by the at least one processor, cause the computing device cluster to execute the method according to the first aspect or any possible implementation of the first aspect.

[0051] In a fifth aspect, a computer-readable medium is provided by the present disclosure, and the computer-readable medium stores computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute the method according to the first aspect or any possible implementation of the first aspect.

[0052] In a sixth aspect, a computer program product is provided by the present disclosure, and the computerprogram product includes computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute the method according to the first aspect or any possible implementation of the first aspect.

[0053] In a seventh aspect, a computer program is provided by the present disclosure, and the computer program includes computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute the method according to the first aspect or any possible implementation of the first aspect.

[0054] A data processing method and related apparatus are provided by the present disclosure. The first compiler intermediate representation is incorporated into the first shared library during the building phase of the first shared library, and it can be used for execution of LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation, and then rebuilding is performed based on the optimized first compiler intermediate representation to obtain a rebuilt first shared library. By this optimization before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings are used to provide a further understanding of the present disclosure, constitute a part of the specification, and are used to explain the present disclosure together with the following specific embodiments, but should not be construed as limiting the present disclosure.

[0056] FIG. 1 is a schematic illustration of an optimization process for a merged shared library from documents according to one or more embodiments of the present disclosure.

[0057] FIG. 2 is a flowchart of a data processing method according to one or more embodiments of the present disclosure.

[0058] FIG. 3 is a schematic illustration of a dependency graph according to one or more embodiments of the present disclosure.

[0059] FIGS. 4A-4C are schematic illustrations of a process for performing LTO according to one or more embodiments of the present disclosure.|0060] FIG. 5 is a block diagram of a data processing apparatus according to one or more embodiments of the present disclosure.

[0061] FIG. 6 is a schematic structural diagram of a computing device according to one or more embodiments of the present disclosure.

[0062] FIG. 7 is a schematic diagram of an architecture of a computing device cluster according to one or more embodiments of the present disclosure.

[0063] FIG. 8 is a schematic diagram of a connection between computing devices over a network according to one or more embodiments of the present disclosure.DESCRIPTION OF EMBODIMENTS

[0064] In the following description, reference is made to the accompanying figures, which form part of the present disclosure, and which show, by way of illustration, specific aspects of embodiments of the present disclosure or specific aspects in which embodiments of the present disclosure may be used. It is understood that embodiments of the present disclosure may be used in other aspects and include structural or logical changes not depicted in the figures. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims.

[0065] Before describing the detail contents of the present disclosure, the following terms are explained.

[0066] Object file - a binary file that contains a machine code (also can be termed as “object code”), as well as other data and metadata, generated by a compiler from a source code. Object file is suitable to be an input for a linker.

[0067] Code segment - also known as text segment, or as text, is a part of a binary file that contains executable instructions for a specific CPU (Central processing Unit). For example, the binary file can be in an executable and linkable format (ELF) in linux systems, or in a portable execute (PE) format in windows systems, which is not limited in embodiments of the present disclosure.

[0068] Data section - is a part of a binary file that contains permanent constants and variables used in a computer program.

[0069] Shared library (also can be referred to as “shared lib / shared_lib.so”, and also can be referred to as “Dynamic Shared Object, DSO”) - a binary file in a pre-required format that contains functions designed to be used by multiple applications or contains functions designed to be used by other shared libraries at runtime, which is also referred to as executable file. Throughout the text, a specific shared library may also be referred to as shared libN.so, e.g., shared libl.so, shared_lib2.so.

[0070] Linker - a program which takes one or more object files as an input and combines them into a singleexecutable file or shared library, or takes one or more shared libraries as an input and combines them into a merged shared library. The linker can also be implemented as a static linker.10071] Loader - is a part of the operational system that is responsible for loading programs and shared libraries into a memory for execution by CPU (and for unloading abovementioned programs and libraries from the memory after execution stage). The loader can also be implemented as a dynamic linker or dynamic loader.

[0072] Inter Procedural Optimization - is a mechanism for performing optimization across function boundaries.

[0073] Link Time Optimization - is a form of inter procedural optimization that is performed at the time of linking application code.

[0074] Whole Program Optimization - is the compiler optimization of a program using information about all the modules in the program.

[0075] In the related art, usage of shared libraries (particularly in a Linux-based system) is a common way to manage an application’s dependencies. The shared libraries are loaded by a dynamic loader before the application / process runs, where the application / process has dependencies on the shared libraries. This approach allows to reduce RAM (Random-Access Memory) consumption because different applications can use same (already loaded) shared libraries. But it has its own cost: all calls between shared libraries above are performed with PLT / GOT (Procedure Linkage Table / Global Offset Table) mechanism usage, which allows to call functions with unknown addresses (at linkage stage), but leads to performance degradation at runtime - it leads to additional overhead (more instructions need to be executed, more jumps, and increased possibility of CPU cache invalidation).

[0076] The PLT / GOT mechanism has been used for many decades already and shows its applicability in complex systems. But systems became more and more complicated, and it leads to more and more dependencies on shared libraries. For example, for an open source operational system, there is a process which is responsible for starting other processes, and this process depends on almost 1000 shared libraries, which is a quite huge number. When all those shared libraries are loaded by a dynamic loader, they consume a lot of memory (said memory is fragmented because all shared libraries are loaded independently for each own memory region, without considering how it will be positioned relative to other shared libraries). In addition to inefficient memory consumption, there is a more impactful characteristic: performance degradation - because interaction with all of those 1000 shared libraries is performed using PLT / GOT mechanism, which uses additional code for calling function(s) in a shared library for another shared library, therefore code executes slower and there is higher rate of instruction cache misses during the execution.ii

[0077] To solve the above mentioned problem, in the related art, all highly tied (in terms of usage by predefined processes) shared libraries are merged into one merged shared library (MSL), which optimizes (eliminates) PLT / GOT calls inside the MSL, reduces memory consumption, changes the static linker and dynamic loader to handle MSL in a correct way. The proposed mechanism may require changes to the static linker and dynamic loader, but any inefficient memory consumptions and any changes in user code are excluded, so from the perspective of a user application, there won’t be any changes at all - MSL is allowed to be used for the user application even without rebuilding the user application.

[0078] But the above MSL approach still lacks optimization opportunities, e.g., functions from one shared library cannot be inlined into other shared libraries - it is too complex and error-prone to do on machine code level. That is, the MSL approach can be improved, the static linker can not only merge shared libraries together into one MSL, but also perform additional cross-DSO optimizations and it will allow to improve performance further compared to the default MSL.

[0079] As mentioned before, in order to solve the problem with missed optimization in MSL mechanism, it is possible to rebuild nested shared libraries (NSL) (shared libraries that are to be merged into the MSL) with link time optimization (LTO) if NSL contains necessary information for it (e.g., low level virtual machine (LLVM) intermediate representation (IR) code and build flags). Embodiments of the present disclosure propose the optimization solution for the MSL mechanism. During the shared library building stage (or referred to as the .so file building stage), a corresponding LLVM IR code (or referred to as LLVM IR) and a corresponding build flag (or referred to as build option) are included into the shared library (e.g., as a compressed non-loadable section). During merging different shared libraries into one MSL, it is needed to build a dependency graph between the shared libraries. Each shared library in the dependency graph is analyzed one-by-one starting with the leaves libraries. If a shared library has LLVM IR code and at least one of its dependency libraries has such code, LTO would be performed for this shared library and this shared library would be rebuilt Then, newly generated shared libraries are merged into the MSL. Throughout the text, a shared library having a dependency library means that execution of one or more functions in the shared library calls execution of one or more functions in the dependency library.

[0080] FIG. 1 is a schematic illustration of an optimization process for a merged shared library according to one or more embodiments of the present disclosure. As shown in FIG. 1, shared libl.so and shared_lib2.so (specific examples of two shared libraries) are respectively built with an additional section, i.e., the LLVM IR and build options. In this example, sharedjibl .so is a dependency shared library of shared_lib2.so. It should be noted that, theadditional section can be stripped after the merging process. As mentioned before, a shared library may include functions designed to be used by multiple applications or other shared libraries, as an example, sharedjibl. so includes a function Fun aQ, while shared_lib2.so a function Fun_b(), Fun b() uses / calls Fun_a(), that is, Fun_a() would also be invoked in a case that Fun_b() is invoked. In other words, shared_lib2.so depends on sharedjibl .so. When the static linker combines shared_libl.so and shared_lib2.so into a merged shared library (merged.so illustrated in FIG. 1), shared_lib2.so will be rebuilt with LTO since it depends on sharedjibl. so and both of these shared libraries have LLVM IR, while sharedjibl. so will not be rebuilt since it does not depend on other shared libraries. Then, the merged-so is generated based on sharedjibl. so and the optimized version of sharedjib2.so with the inlined function from shared Jibl.so, that is, the merged.so is generated based on shared libl.so and the rebuilt shared Jib2. so. It should be noted that, shared libl.so and shared lib2.so are only illustrative but not restrictive, more shared libraries can be analyzed for the optimization process or can be included in the merged.so, which is not limited in the embodiments of the present disclosure.

[0081] The above describes the general technical concept of LTO usage for MSL, and next details about the optimization for MSL will be elaborated.

[0082] FIG. 2 is a flowchart of a data processing method according to one or more embodiments of the present disclosure. The method may be applied to a processor, and may include the following steps.

[0083] Step 202, determining, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library.

[0084] The first shared library may be a library in a dependency graph which includes multiple shared libraries and indicates the dependency relationship between the multiple shared libraries. The dependency for the first shared library may be indicated by the dependency graph, that is, all dependency shared libraries of the first shared library would be indicated by the dependency graph. LTO is an optimization technology when performed, causes some functions to be inlined, some useless coded to be removed, the program to be globally optimized, etc.

[0085] The first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple shared libraries. The dependency for the first shared library may be shown in a dependency graph which indicates a dependency for each of the multiple shared libraries. The first shared library is a shared library to be merged into an MSL, but the one or more second shared libraries are not necessarily to be merged into one or more MSLs.

[0086] In a possible implementation, the multiple shared libraries are those to be merged into one or more MSLs. There is no limitation on the number of the multiple shared libraries, and these multiple shared libraries may be frequently used / called shared libraries, the determination of frequently used shared libraries may be made based on, e.g., profile data of users. The first shared library and the one or more second shared libraries all belong to the multiple shared libraries. In another possible implementation, in addition to the shared libraries to be merged into one or more MSLs, the multiple shared libraries also include other shared libraries, they may not be subject to merging, but they may be dependency shared libraries for NSL(s) (shared libraries which are going to be merged into one or more MSLs), in this case, the one or more second shared libraries still belong to the multiple shared libraries but may sometimes contain one or more shared libraries which will not be merged to MSL.

[0087] Further, when performing merging, it is possible to generate one MSL with all the NSLs, it is also possible to generate multiple MSLs, each of the multiple MSLs may be a merging of some NSLs, and when a shared library merged into MSL1 depends on a shared library merged into MSL2, the MSL would be designed to include an indication of such dependency between MSL1 and MSL2.

[0088] A compiler processes a source code to obtain an intermediate representation, thus an object file (.o file) or a binary file is obtained based on the intermediate representation. Then, a linker takes one or more object files as an input, and combines them into a single executable file (.so file). The .so file is input into a loader, and the loader loads the program and shared libraries into a memory for execution by CPU. In a possible implementation, the first shared library is built by a compiler and a linker in the processor, and the LTO when performed is performed by a loader in the processor. In the proposed solution, the intermediate representation generated by the compiler (referred to as compiler intermediate representation) will be kept till the linking stage, so the first shared library would contain its own compiler intermediate representation, this compiler intermediate representation will later be subject to LTO if LTO is determined to be performed. This compiler intermediate representation would be discarded after the rebuilding of the first shared library.

[0089] It is not required to always perform LTO, for example, if the dependency indicates that the first shared library does not depend on any second shared libraries in the multiple shared libraries, then LTO will not be performed. In a possible implementation, when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries includes a corresponding second compiler intermediate representation, it will be determined to perform the LTO for the first shared library.

[0090] Step 204, upon determining to perform LTO for the first shared library, performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation.

[0091] The first compiler intermediate representation is incorporated into the first shared library during the building stage of the first shared library, and it can be used for performing LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation. A specific rule for determining whether to perform LTO for the first shared library and the details about how to perform LTO would be described later.

[0092] Step 206, obtaining a rebuilt first shared library based on the optimized first compiler intermediate representation.

[0093] LTO is performed for the first shared library to obtain the optimized first compiler intermediate representation, then the first shared libraty having the optimized first compiler intermediate representation is rebuilt to obtain the rebuilt first shared library. The first shared library can be replaced with the rebuilt first shared library in the generation of the merged shared library, which optimizes the merging process for the MSL.

[0094] The first compiler intermediate representation is incorporated into the first shared library during the building phase of the first shared library, and it can be used for execution of LTO. If it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain the optimized first compiler intermediate representation, and then the optimized first compiler intermediate representation is used for rebuilding, so as to obtain the rebuilt first shared library. When the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved.

[0095] Regarding the specific rule for determining whether to perform LTO for the first shared library, in a possible implementation, when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries includes a corresponding second compiler intermediate representation, it is determined to perform the LTO for the first shared library. As mentioned before, the first shared library may be a shared library whose dependency is indicated in the dependency graph, and in this case, a second shared library is a shared library on which the first shared library depends, and the second shared library may also be referred to as a child library of the first shared library, the first shared library is the parent library of the second shared library. It should be noted that, the first compiler intermediate representation and the second compiler intermediate representation are used for identifying the intermediate representations ofdifferent shared libraries. If the first shared library includes the first compiler intermediate representation, and at least one of its child libraries indicated by the dependency graph includes the second compiler intermediate representation, it is determined to perform the LTO for the first shared library. Through this specific determination rule, a shared library for which the LTO is performed can be quickly and accurately determined.

[0096] It should be noted that it is not required to let all of the child libraries of the first shared library have corresponding compiler intermediate representations, one or more of the child libraries of the first shared library having corresponding compiler intermediate representations would be sufficient to trigger the LTO for the first shared library. In a case that a child library does not have corresponding compiler intermediate representations, it would not be included in the LTO process.

[0097] Regarding how to perform LTO, in a possible implementation, the performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation includes: performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation. The LTO for the first shared library is performed on the first compiler intermediate representation, and will use data from the second compiler intermediate representation(s) of at least one of its child libraries, that is, to achieve the optimization purpose, the LTO for the first shared library is performed on its compiler intermediate representation, and is also associated with the compiler intermediate representation(s) of the at least one of its child libraries, such that functions) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0098] In a possible implementation, the performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation includes: extracting the first compiler intermediate representation included in the first shared library, and the second compiler intermediate representation in each of the at least one of the one or more second shared libraries; combining the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; and performing LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation. When the at least one of the one or more second shared libraries includes multiple second shared libraries, a second compiler intermediate representation in each of the multiple second shared libraries will be extracted. The LTO for the first shared library is performed on the combined compiler intermediate representation which is a combination of the firstcompiler intermediate representation and the second compiler intermediate representation, such that ftinction(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0099] In a possible implementation, the first shared library is built by incorporating a build flag, and the obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation includes: obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag. The build flag is used for the generation of the rebuilt first shared library. A shared library is built by incorporating a build flag and a compiler intermediate representation. The build flag and the first compiler intermediate representation are incorporated into the first shared library during the building phase, while the first shared library having the optimized first compiler intermediate representation can be rebuilt with the build flag of the first shared library to obtain the rebuilt first shared library.

[0100] In a possible implementation, the first shared library is built by incorporating a build flag and a first compiler intermediate representation, if it is determined to perform LTO for the first shared library, LTO is performed on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation, the build flag can be stored in a temporary location, and then the optimized first compiler intermediate representation can be rebuilt with this build flag. The storage of the build flag is beneficial for the generation of the rebuilt first shared library.

[0101] The above describes the obtaining of the rebuilt first shared library, after that, the merged shared library can be generated with the rebuilt first shared library. Since optimization is performed before the merging process, when the rebuilt first shared library is merged into a merged shared library, the performance of the program / process / application being executed through the merged shared library would be improved. In a possible implementation, after obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation, the rebuilt first shared library is stored in a temporary location. During the merging process, the rebuilt first shared library can be invoked from the temporary location.

[0102] As mentioned before, a dependency graph may indicate the dependency relationship between the multiple shared libraries. In a possible implementation, a dependency graph is built, where the dependency graph indicates a dependency for each of the multiple shared libraries; and each of the multiple shared libraries is taken as the first shared library. The determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for thefirst shared library, and the obtaining of the rebuilt first shared library are repeated. There may be multiple shared libraries shown in the dependency graph, and for each of the multiple shared libraries, the determination of whether to perform LTO is executed, when a shared library is determined to be subject to LTO, LTO is performed to obtain the optimized compiler intermediate representation, and then the shared library having the optimized compiler intermediate representation is rebuilt. That is, the number of the shared libraries to be rebuilt in the dependency graph is not limited, in a case that there are multiple shared libraries that need to be rebuilt, there may be multiple rebuilding processes accordingly, and thus multiple rebuilt shared libraries are obtained. As described above, it is not necessary to merge all the multiple shared libraries into MSL(s), when some of the multiple shared libraries are not going to be used for merging, they could still be regarded as the first shared library, so that they could be subject to LTO if necessary.

[0103] In a possible implementation, the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library, or, the dependency for the first shared library is represented by execution of one or more functions in the first shared library calling execution of one or more functions in each of the one or more second shared libraries. The first shared library depends on the second shared libraries, that is, the second shared libraries are child libraries of the first shared library. When a function in the first shared library is executed, since this function uses functions) in the second shared libraries, the function(s) in the second shared libraries would also be executed. In this way, function(s) from one shared library (the at least one of the one or more second libraries) can be inlined into another shared library (the first shared library).

[0104] In a possible implementation, the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code. The specific implementation form of the compiler intermediate representation can be LLVM IR code or the textual code according to actual requirements.

[0105] In order to better explain the optimization for MSL, a specific example will be given in the following. It should be noted that, the shared libraries to be merged and the shared libraries for which LTO is performed in the example are only illustrative but not restrictive.]0106] FIG. 3 is a schematic illustration of a dependency graph according to one or more embodiments of the present disclosure. As shown in FIG. 3, shared libraries presented in the dependency graph include shared libl.so, shared_lib2.so, shared_lib3.so, shared_lib4.so and shared_lib5.so; where shared libl.so depends on shared_lib2.soand shared_lib3.so; shared_lib2. so depends on shared_lib4.so andshared_lib5.so; shared_lib3.so, shared_lib4.so and shared_lib5.so (may also be referred to as leaves libraries) do not depend on other shared libraries.

[0107] Based on the dependency graph in FIG. 3, the analysis of whether to perform LTO for the shared libraries in the dependency graph is from bottom to top. First, the leaves libraries (i.e., the libraries which are in leaves of the tree, in the example as shown in FIG. 3, they are: shared_lib3.so, shared_lib4.so, shared_lib5.so) cannot be optimized with LTO because they do not have any children (dependencies).

[0108] Then, the analysis proceeds with parents of the leaves libraries. In the example as shown in FIG. 3, the parent library includes shared_lib2.so (the parent library of shared_lib4.so and shared_lib5.so) and shared libl.so (the parent library of shared_lib2.so, shared_lib3.so). Based on the bottom-to-top principle, the analysis for shared_lib2.so comes before that of shared libl.so. If the analyzed library has a section with a compressed LLVM IR and at least one of its child libraries also has this section, then LTO may be performed for the analyzed library.

[0109] Suppose that shared_lib2.so, shared_lib4.so and shared_lib5.so in FIG. 3 all have LLVM IRs, based on the aforementioned determination of whether to perform LTO for the analyzed library, it is determined to perform LTO for shared_lib2.so (a specific example of the aforementioned first shared library). FIGS. 4A-4C is a schematic illustration of a process for performing LTO according to one or more embodiments of the present disclosure. In this example, the parent library is shared_lib2.so, and its child libraries are shared_lib4.so and shared_lib5.so. As shown in FIG. 4 first, the analyzed library (shared_lib2.so) and all its dependencies (sharedjib4.so and shared_lib5.so which would be a specific example of the aforementioned at least one of the one or more second shared libraries) are loaded, where shared_lib2.so is built with LLVM IR (a specific example of the aforementioned first compiler intermediate representation) and build options (or referred to as a build flag), shared_lib4.so and shared_lib5.so are also built with LLVM IR (a specific example of the aforementioned second compiler intermediate representation) and build options (or referred to as a build flag). As shown in FIG. 4B, LLVM IR (codes) of these three libraries are extracted and unpacked. Then, the LLVM IR codes of these three libraries are combined into one LLVM IR file (a specific example of the aforementioned combined compiler intermediate representation), and the build options for the analyzed library are unpacked and stored in a temporary location. Next, as shown in FIG. 4C, LTO optimization is performed on the combined LLVM IR file, where the optimization may include, e.g., inlining, constant propagation, other suitable optimizations, etc. In this process, data in LLVM IR (codes) of shared_lib4.so and shared_lib5.so will be used for carrying out the LTO optimization, which would make the LTO optimization much faster.

[0110] After the optimization, by using the build options stored in the temporary location the optimized LLVMIR file (a specific example of the aforementioned optimized first compiler intermediate representation) can be rebuilt to obtain a rebuilt library, and the rebuilt library (a specific example of the aforementioned rebuilt first shared library) can be stored in the temporary location. The LTO process for other libraries proceeds, for example, the LTO process for shared libl .so may proceed. After all shared libraries are analyzed, where some of them will be rebuilt with LTO, the merging process can be executed with the newly built shared libraries (e.g., from the temporary location). And through the merging process, MSL(s) is generated. The merging process for MSL can be referred to the related art, which is not limited in the embodiments of the present disclosure.

[0111] In general, the optimization pipeline may be as follows. Firstly, create a combined LLVM IR file, e.g., a combined LLVM file including LLVM IR (codes) of shared_lib2.so, shared_lib4.so and shared_lib5.so as shown in FIG. 4B. Then, optimize the necessary part of the combined LLVM IR file with LTO, e.g, as shown in FIG. 4C. Next, build the analyzed library with the necessary part of the optimized combined LLVM IR file (e.g, shared_lib2.so as shown in FIG. 4C) and build flags (e.g., the build flag of shared_lib2.so).

[0112] The solution proposed by the embodiments of the present disclosure can be applicable to various operational systems, it should be noted that, the Linux-based (or Linux-resembled) operational system is only illustrative but not restrictive. That is, a similar solution can be developed for other operational systems, and do it as fast and efficient as possible, so the application of cross-DSO optimizations for MSL approach to other operational systems (Windows, Mac) can also be done.

[0113] The solution is designed to optimize applications / processes with high number of dependencies on shared libraries, e.g., approximately more than 10-20. The performance benefit depends on features of such shared libraries, e.g., how big they are in terms of instructions, how tied those libraries. The bigger number of interaction calls between different nested DSOs, the bigger can be possible profit from the cross-DSO optimizations of the present disclosure.

[0114] To implement the proposed solution, the static linker should be changed accordingly. Other parts of compiler tool chain or operational system do not require any changes, if the merging process were not considered. And the performance benefit from cross-DSO optimizations in MSL will be bigger for the application / process which more often uses functions from MSL on its hot paths.

[0115] The solution proposed by the embodiments of the present disclosure is implemented as a part of optimization work for operational systems. It is associated with the following characteristics of the nested shared libraries. The shared libraries are built or generated with a non-standard section including the bytecode (e.g., LLVMIR, binary or textual code) and the build flag. This section may be compressed, encrypted, obfuscated, etc. The static linker (or other tools) uses LLVM IR sections from different shared libraries to create a new shared library (an optimized shared library), which is a version of the original shared library but optimized with LTO. Some of symbols which are needed for the original shared library become non-needed for the optimized shared library, since they are inlined. The newly generated shared library may be used as a nested shared library to create a merged shared library (merged.so), that is, the rebuilt shared library based on the optimization is included in the merged, so.

[0116] The solution in the present disclosure relates to a method for performing cross-DSO optimizations between different shared libraries. As mentioned before, the method relates to how the shared library suitable for the cross-DSO optimization is chosen, how the combined bytecode (e.g., LLVM IR) file is generated and optimized, and how the newly generated (and optimized) nested shared library replaces the original nested shared library in MSL.

[0117] The important and frequently used shared libraries are merged into one or more MSLs, where LTO optimizations are performed for MSL(s), and the optimizations include inlining, constant propagations, other suitable optimizations, etc. Thus, the performance of the program / process / application using the merged shared library would be improved, and the code size would also be reduced. The MSL solution with cross-DSO optimizations is provided without any requirements for changing the user code, this allows to apply the MSL solution to the existing process / application without rebuilding it and get benefits without any additional workload related to the user side.

[0118] There are a lot of optimizations, which can be more profitable with the usage of the cross-DSO approach in MSL, for example, functions specializations for usage in different shared libraries, or a combination of the profile guided optimization (PGO) and Cross-DSO optimizations. Also, the section with bytecode will allow to perform optimization on the user device exactly for user needs, MSLs can be re-optimized to be more suitable for the user most frequent tasks, such as browsing, games, development, etc.

[0119] Nexf embodiments of products related to the data processing method will be described.

[0120] FIG. 5 shows a schematic structural diagram of a data processing apparatus according to one or more embodiments of the present disclosure. As shown in FIG. 5, the apparatus includes a determining module 502, a performing module 504 and an obtaining module 506. The determining module 502 is configured to determine, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library, where the first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple sharedlibraries. The performing module 504 is configured to upon determining to perform LTO for the first shared library, perform LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation. The obtaining module 506 is configured to obtain a rebuilt first shared library based on the optimized first compiler intermediate representation.

[0121] In a possible implementation, the determining module is specifically configured to: when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries includes a corresponding second compiler intermediate representation, determine to perform the LTO for the first shared library.

[0122] In a possible implementation, the performing module is specifically configured to: perform LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0123] In a possible implementation, the performing module is specifically configured to: extract the first compiler intermediate representation included in the first shared library, and the second compiler intermediate representation in each of the at least one of the one or more second shared libraries; combine the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; and perform LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation.

[0124] In a possible implementation, the first shared library is built by incorporating a build flag, and the obtaining module is specifically configured to obtain the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag.

[0125] In a possible implementation, the apparatus further includes a first storing module, configured to store the build flag in a temporary location.

[0126] In a possible implementation, the apparatus further includes a generating module, configured to generate a merged shared library with the rebuilt first shared library.

[0127] In a possible implementation, the apparatus further includes a second storing module, configured to store the rebuilt first shared library in a temporary location.

[0128] In a possible implementation, the apparatus further includes a building module and a processing module. The building module is configured to build a dependency graph, where the dependency graph indicates a dependency for each of the multiple shared libraries. The processing module is configured to take each of the multiple sharedlibraries as the first shared library, and repeat the determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for the first shared library and the obtaining of the rebuilt first shared library.

[0129] In a possible implementation, the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library.

[0130] In a possible implementation, the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code.

[0131] It should be noted that, the first storing module and the second storing module are only illustrative for their functions, and in practice, the functions of the first storing module and the second storing module may be implemented by one storing module, which is not limited here. It should also be noted that, the functions of the determining module, the performing module and the obtaining module may also be implemented by a single module, and the functions of multiple modules may also be implemented by one or more modules, which is not limited in the present disclosure. The apparatus may be in a form of a computing device, an electronic device, etc.

[0132] The data processing apparatus may be the implemented as described in the above method embodiments. It should be understood by a person skilled in the art that, the relevant description of the above modules in the embodiments of the present disclosure may be understood with reference to the relevant description of the data processing method in the embodiments of the present disclosure.

[0133] FIG. 6 is a schematic structural diagram of a computing device according to one or more embodiments of the present disclosure. As shown in FIG. 6, the computing device 600 includes a processor 602, an interface 604 for communicating with other devices, a memory 606, and a bus 608. The processor 602, the interface 604, and the memory 606 are in communication with each other via the bus 608. The memory 606 may be stored with computer execution instructions, and the processor 602 executes computer execution instructions stored in the memory 606 to enable the computing device to execute any of the above methods. The computing device 600 can be a server or a terminal device. The number of processors and memories in the computing device 600 is not limited in the embodiments of the present disclosure.

[0134] The bus 608 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one line is used in FIG. 6, but it does not mean that there is only one bus or one type of bus. Thebus 608 may include pathways for transferring information between various components of the computing device 600 (e.g., the memory 606, the processor 602, or the interface 604).

[0135] The processor 602 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP) processor.

[0136] The memory 606 can include volatile memory, such as random-access memory (RAM). The processor 602 can also include non-volatile memory, such as read-only memory (ROM), flash memory, a mechanical hard disk (HDD), or a solid state drive (SSD).

[0137] An executable program code is stored in the memoiy 606, and the processor 602 executes the executable program code to implement the functions of the aforementioned determining module 502, the performing module 504 and the obtaining module 506, respectively, so as to implement the method described in the above embodiments. That is, the memory 606 has instructions on it for executing the method described in the above embodiments.

[0138] The structure of the computing device 600 enumerated above is only an exemplary illustration, and the present disclosure is not limited thereto. The computing device 600 of embodiments of the present application includes a variety of hardware found in computer systems in the prior art, e.g., the computing device 600 also includes memory other than the memory 606, such as, for example, a disk memory, and the like, ft should be understood by those skilled in the art that the computing device 600 can also include other devices necessary to achieve proper operation. In addition, depending on the specific needs, one of skill in the art should understand that the above-described computing device 600 may also include other hardware devices for realizing other additional functions. In addition, one of skill in the art should understand that the above-described computing device 600 may also include only the devices necessary to realize the embodiments of the present disclosure, and need not include all of the devices shown in FIG. 6.

[0139] FIG. 7 is a schematic diagram of an architecture of a computing device cluster according to one or more embodiments of the present disclosure. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.]0140] As shown in FIG. 7, the cluster of computing devices described includes at least one computing device 700. The memory 706 in one or more of the computing devices 700 in the cluster of computing devices can hold the same instructions for performing the method described in the above embodiments.

[0141] In some possible implementations, some of the instructions for performing the method described in the above embodiments can also be separately held in the memory 706 of the one or more computing devices 700 in the cluster of computing devices. In other words, a combination of the one or more computing devices 700 can jointly execute instructions for performing the method described in the above embodiments.

[0142] It is noted that the memories 706 in the different computing devices 700 in the cluster of computing devices can store different instructions for performing some of the functions of the computing devices 700, respectively. That is, the instructions stored in the memories 706 in the different computing devices 700 can realize the functions of one or more of the determining module 502, the performing module 504 and the obtaining module 506.

[0143] In some possible implementations, one or more computing devices in a cluster of computing devices can be connected via a network Among other things, the network can be a wide area network or a local area network etc. FIG. 8 illustrates one possible implementation. FIG. 8 is a schematic diagram of a connection between computing devices 800A and 800B over a network according to one or more embodiments of the present disclosure. The two computing devices 800A and 800B are connected to each other via a network Specifically, the connection to said network is made through a communication interface in each computing device. In this class of possible implementations, the memory 806A in the computing device 800A holds instructions for performing the functions of the determining module 502. At the same time, the memory 806B in the computing device 1000B holds instructions for performing the functions of the performing module 504 and the obtaining module 506.

[0144] The connection between the clusters of computing devices shown in FIG. 8 can be such that, considering that the method provided in the present disclosure requires processing a large number of data, it is contemplated that the functions realized by the determining module 502 be performed by the computing device 800B.

[0145] The functions of computing device 800A illustrated in FIG. 8 can also be accomplished by multiple computing devices. Similarly, the functions of computing device 800B can be accomplished by multiple computing devices.

[0146] In some aspects of the present disclosure, there is provided a data processing apparatus including processing circuitry for executing any of the above methods. It should be understood that the apparatus can execute the steps in the above method embodiments, which will not be repeated here.

[0147] In some aspects of the present disclosure, there is provided a computing device cluster including at least one computing device, and the computing device includes a processor and a memory coupled with the processor.The memory is storing instructions which, when executed by the processor, cause the computing device cluster to perform any of the above methods.

[0148] In some aspects of the present disclosure, there is provided a chip, including an input / output (I / O) interface and a processor, where the processor is configured to call and run computer execution instructions stored in a memory, to enable a device installing with the chip to execute any of the above methods.

[0149] In some aspects of the present disclosure, there is provided a computer-readable medium storing computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute any of the above methods.

[0150] In some aspects of the present disclosure, there is provided a computer program product including computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute any of the above methods.

[0151] In some aspects of the present disclosure, there is provided a computer program including computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute any of the above methods.

[0152] Although the present disclosure describes methods and processes with steps in a certain order, one or more steps of the methods and processes may be omitted or altered as appropriate. One or more steps may take place in an order other than that in which they are described, as appropriate.]0153] Note that the expression “at least one of A or B”, as used herein, is interchangeable with the expression “A and / or B”. It refers to a list in which you may select A or B or both A and B. Similarly, “at least one of A, B, or C”, as used herein, is interchangeable with “Aand / or B and / or C” or “A, B, and / or C”. It refers to a list in which you may select: A or B or C, or both A and B, or both A and C, or both B and C, or all of A, B and C. The same principle applies for longer lists having a same format.

[0154] Although the present disclosure is described, at least in part, in terms of methods, a person of ordinary skill in the art will understand that the present disclosure is also directed to the various components for performing at least some of the aspects and features of the described methods, be it by way of hardware components, software or any combination of the two. Accordingly, the technical solution of the present disclosure may be embodied in the form of a software product. A suitable software product may be stored in a pre-recorded storage device or other similar non-volatile or non-transitory computer readable medium, including DVDs, CD-ROMs, USB flash disk, a removable hard disk, or other storage media, for example. The software product includes instructions tangibly storedthereon that enable a processing device (e.g., a personal computer, a server, or a network device) to execute examples of the methods disclosed herein. The machine-executable instructions may be in the form of code sequences, configuration information, or other data, which, when executed, cause a machine (e.g., a processor or other processing device) to perform steps in a method according to examples of the present disclosure.

[0155] The present disclosure may be embodied in other specific forms without departing from the subject matter of the claims. The described example embodiments are to be considered in all respects as being only illustrative and not restrictive. Selected features from one or more of the above-described embodiments may be combined to create alternative embodiments not explicitly described, features suitable for such combinations being understood within the scope of this disclosure.

[0156] All values and sub-ranges within disclosed ranges are also disclosed. Also, although the systems, devices and processes disclosed and shown herein may include a specific number of elements / components, the systems, devices and assemblies could be modified to include additional or fewer of such elements / components. For example, although any of the elements / components disclosed may be referenced as being singular, the embodiments disclosed herein could be modified to include a plurality of such elements / components. The subject matter described herein intends to cover and embrace all suitable changes in technology.

[0157] Although embodiments have been described above with reference to the accompanying drawings, those of skill in the art will appreciate that variations and modifications may be made without departing from the scope thereof as defined by the appended claims.

Claims

CLAIMS1. A data processing method applied to a processor, comprising:determining, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library, wherein the first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple shared libraries;upon determining to perform LTO for the first shared library, performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation; andobtaining a rebuilt first shared library based on the optimized first compiler intermediate representation.

2. The method according to claim 1, wherein the determining, based on a dependency for a first shared library, whether to perform LTO for the first shared library comprises:when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries comprises a corresponding second compiler intermediate representation, determining to perform the LTO for the first shared library.

3. The method according to claim 2, wherein the performing LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation comprises:performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation.

4. The method according to claim 3, wherein the performing LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation comprises:extracting the first compiler intermediate representation comprised in the first shared library, and the second compiler intermediate representation in each of the at least one of the one or more second shared libraries;combining the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; andperforming LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation.

5. The method according to any one of claims 1 to 4, wherein the first shared library is built by incorporating a build flag, and the obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation comprises:obtaining the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag.

6. The method according to claim 5, before the obtaining the rebuilt first shared library, further comprising: storing the build flag in a temporary location.

7. The method according to claim 5 or 6, further comprising:generating a merged shared library with the rebuilt first shared library.

8. The method according to claim 7, before generating a merged shared library with the rebuilt first shared library, further comprising:storing the rebuilt first shared library in a temporary location.

9. The method according to any one of claims 1 to 8, further comprising:building a dependency graph, wherein the dependency graph indicates a dependency for each of the multiple shared libraries;wherein the method further comprises:taking each of the multiple shared libraries as the first shared library; andrepeating the determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for the first shared library and the obtaining of the rebuilt first shared library.

10. The method according to any one of claims 1 to 9, wherein the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library.

11. The method according to any one of claims 1 to 10, wherein the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code.

12. The method according to any one of claims 1 to 11, wherein the first shared library is built by a compiler and a linker in the processor, and the LTO when performed is performed by a loader in the processor.

13. A data processing apparatus, comprising:a determining module, configured to determine, based on a dependency for a first shared library, whether to perform link time optimization (LTO) for the first shared library, wherein the first shared library is built by incorporating a first compiler intermediate representation corresponding to the first shared library, and the dependency for the first shared library indicates whether the first shared library in multiple shared libraries depends on one or more second shared libraries in the multiple shared libraries;a performing module, configured to upon determining to perform LTO for the first shared library, perform LTO on the first compiler intermediate representation to obtain an optimized first compiler intermediate representation; andan obtaining module, configured to obtain a rebuilt first shared library based on the optimized first compiler intermediate representation.

14. The apparatus according to claim 13, wherein the determining module is specifically configured to: when the dependency indicates that the first shared library depends on the one or more second shared libraries and each of at least one of the one or more second shared libraries comprises a corresponding second compiler intermediate representation, determine to perform the LTO for the first shared library.

15. The apparatus according to claim 1 , wherein the performing module is specifically configured to: perform LTO on the first compiler intermediate representation based on the second compiler intermediate representation to obtain the optimized first compiler intermediate representation.

16. The apparatus according to claim 15, wherein the performing module is specifically configured to: extract the first compiler intermediate representation comprised in the first shared library, and the second compiler intermediate representation in each of the at least one of the one or more second shared libraries;combine the first compiler intermediate representation and the second compiler intermediate representation to obtain a combined compiler intermediate representation; andperform LTO on the combined compiler intermediate representation to obtain the optimized first compiler intermediate representation.

17. The apparatus according to any one of claims 13 to 16, wherein the first shared library is built by incorporating a build flag, and the obtaining module is specifically configured to:obtain the rebuilt first shared library based on the optimized first compiler intermediate representation and the build flag.

18. The apparatus according to claim 17, further comprising:a first storing module, configured to store the build flag in a temporary location.

19. The apparatus according to claim 18 or 17, further comprising:a generating module, configured to generate a merged shared library with the rebuilt first shared library.

20. The apparatus according to claim 19, further comprising:a second storing module, configured to store the rebuilt first shared library in a temporary location.

21. The apparatus according to any one of claims 13 to 20, further comprising:a building module, configured to build a dependency graph, wherein the dependency graph indicates a dependency for each of the multiple shared libraries;wherein the apparatus further comprises a processing module, configured to:take each of the multiple shared libraries as the first shared library; andrepeat the determination of whether to perform LTO for the first shared library, the performing of LTO to obtain the optimized first compiler intermediate representation when it is determined to perform LTO for the first shared library and the obtaining of the rebuilt first shared library.

22. The apparatus according to any one of claims 13 to 21, wherein the dependency for the first shared library is represented by execution of one or more functions in each of the one or more second shared libraries being called by execution of one or more functions in the first shared library.

23. The apparatus according to any one of claims 13 to 22, wherein the first compiler intermediate representation for the first shared library is a low level virtual machine (LLVM) intermediate representation (IR) code or a textual code.

24. A computing device cluster, comprising at least one computing device, wherein the computing device comprises at least one processor and a memory coupled with the processor, wherein the memory stores instructions which, when executed by the at least one processor, cause the computing device cluster to execute the method according to any one of claims 1 to 12.

25. A computer-readable medium storing computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute the method according to any one of claims 1 to 12.

26. A computer program product comprising computer execution instructions which, when run on a computing device cluster, cause the computing device cluster to execute the method according to any one of claims 1 to 12.

Images