Compiling method, system and electronic device implemented by build tool

Abstract

The application provides a kind of construction tool implementation's compiling method, system, electronic equipment, the method includes: obtaining the first construction file of source code compilation;Based on the first construction file, the dependency graph and dependent data of all construction targets in the first construction file are generated;Based on the dependency graph, the dependent data is output according to the dynamic library construction file format, and the second construction file for dynamic library compilation is generated;Based on the second construction file, dynamic library compilation is realized.Through source code construction file, dynamic library compilation can be realized, and the development workload is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of program compilation, and in particular to a compilation method and system implemented by a build tool and an electronic device. BACKGROUND

[0002] Large software engineering projects have always been very important, and the build includes compilation, packaging, static checking, and testing. Common build tools are maven and gradle, both of which are used for java, and the current c++ project build tools are mainly Bazel, vcpkg, and Makefile, cmakelist used before c++, which need to develop a lot of compilation rules and do not belong to the strict sense of build tools.

[0003] Using Bazel can greatly simplify the development of project compilation and deployment workload. Bazel is an open source project build tool similar to maven and gradle developed by Google. Bazel itself compiles everything based on target, and Bazel supports source code and library compilation in two compilation modes, of which library compilation is divided into dynamic library and static library. Whether it is source code or library compilation, Bazel compilation requires a BUILD file (build file), and the BUILD file needs to write a target (build target).

[0004] In some development scenarios, the continuous integration tool uses source code compilation, and the local development uses dynamic library compilation outside the repository. Dynamic library compilation is fast and does not require permission of other code repositories, and different algorithm groups may not have permission of other code repositories, so only dynamic library compilation can be used. Therefore, it is necessary to support source code compilation and dynamic library compilation at the same time, but the target of source code compilation and the target of dynamic library compilation are different, and the BUILD file written by each algorithm group is for source code compilation. If the development writes another set of BUILD files for dynamic library compilation, it will increase the workload. Therefore, a mechanism is needed to use source code compilation for all code repositories in continuous integration, and to use dynamic library compilation for other repositories in local development, and the two sets of compilation calling modes are completely the same. SUMMARY

[0005] In view of the problems in the prior art, the present application provides a compilation method and system implemented by a build tool and an electronic device.

[0006] The present application provides a compilation method implemented by a build tool, which comprises:

[0007] Obtain a first build file for source code compilation;

[0008] generating a dependency graph and dependency data of all build targets in the first build file based on the first build file;

[0009] outputting the dependency data in a dynamic library build file format based on the dependency graph, and generating a second build file for dynamic library compilation;

[0010] implementing dynamic library compilation based on the second build file.

[0011] According to the compiling method implemented by the build tool provided by the application, the dependency data comprises at least one of the following:

[0012] a header file, a source file, a dependent build target, a dependent dependent build target, a linking option, a compiling option, a build target name, and a macro definition.

[0013] According to the compiling method implemented by the build tool provided by the application, the generating of the dependency graph and the dependency data of all build targets in the first build file comprises:

[0014] topologically sorting the dependency graph, and generating the dependency data for each build target in reverse order based on the topological sorting result.

[0015] According to the compiling method implemented by the build tool provided by the application, the implementing of the dynamic library compilation based on the second build file comprises:

[0016] uploading the library file formed by the dynamic library compilation and the dependent header file to an online database, so as to be called by a user who lacks the source code permission.

[0017] According to the compiling method implemented by the build tool provided by the application, the outputting of the dependency data in a dynamic library build file format based on the dependency graph, and the generating of a second build file for dynamic library compilation comprises:

[0018] for a third build file in a package directly or indirectly dependent on the first build file and the source code, outputting the dependency data in a dynamic library build file format based on the dependency graph, and generating a second build file for dynamic library compilation.

[0019] According to the compiling method implemented by the build tool provided by the application, the implementing of the dynamic library compilation comprises:

[0020] implementing the dynamic library compilation based on the input dynamic library compilation option.

[0021] The application further provides a compiling system implemented by a build tool, and the system comprises:

[0022] An acquisition module acquires a first build file for source code compilation;

[0023] An intermediate data module generates a dependency graph and dependency data of all build targets in the first build file based on the first build file;

[0024] A generation module outputs the dependency data in a dynamic library build file format based on the dependency graph to generate a second build file for dynamic library compilation;

[0025] A compilation module realizes dynamic library compilation based on the second build file.

[0026] According to the present application, a compilation system implemented by a build tool is provided, and the system further comprises:

[0027] A source code compilation module parallel to the acquisition module is used to compile source code;

[0028] A switch module controls the system to flow into the source code compilation module and / or the dynamic library compilation module.

[0029] The present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the compilation method implemented by the build tool according to any one of the above.

[0030] The present application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the steps of the compilation method implemented by the build tool according to any one of the above.

[0031] The present application further provides a computer program product comprising a computer program, wherein the computer program is executable on a processor to implement the steps of the compilation method implemented by the build tool according to any one of the above.

[0032] The compilation method, system, and electronic device implemented by the build tool provided by the present application can realize dynamic library compilation through a source code build file, thereby reducing the development workload. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the present application or prior art, the following will briefly introduce the drawings needed in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0034] Figure 1 A flowchart of a compiling method implemented by a build tool provided by the present application is shown in the figure;

[0035] Figure 2 A dependency graph of a build file example related to a single build target provided by the present application is shown in the figure;

[0036] Figure 3 A dependency graph of a build file example related to multiple build targets provided by the present application is shown in the figure;

[0037] Figure 4 A structural diagram of a compiling system implemented by a build tool provided by the present application is shown in the figure;

[0038] Figure 5 A physical structure diagram of an electronic device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0040] The compiling method implemented by a build tool provided by the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments and application scenarios thereof.

[0041] Optionally, the embodiments of the present application are implemented under Bazel.

[0042] Bazel is a compilation tool similar to Make.

[0043] The work of Bazel includes the following three steps:

[0044] 1. Loading: Bazel traverses all subfolders under the current project directory, finds the BUILD file (build file) therein, loads external dependencies, and generates a package (a directory containing the BUILD file and all other files and subfolders under the directory) and target (build target) one by one.

[0045] 2. Analysis: According to the build targets and input and output information defined in each BUILD file, a build graph (dependency graph) in memory is established.

[0046] 3. Execution: Run each rule-defined action according to the build graph to generate the final goal.

[0047] Figure 1 A flowchart illustrating a compilation method implemented by a build tool provided by this invention is shown below. Figure 1 As shown, the present invention provides a compilation method implemented by a build tool, the method comprising:

[0048] S100: Obtain the first build file used for compiling the source code.

[0049] Optionally, the first build file is written by the developers in conjunction with the source code.

[0050] Optionally, each compilation directive in the build file is called a build target, which points to a series of source files and dependencies. A build target can also point to other build targets.

[0051] An example of a build file involving a single build target is as follows:

[0052]

[0053] S200. Based on the first build file, generate a dependency graph and dependency data for all build targets within the first build file.

[0054] Figure 2 The dependency graph provided by this invention is an example of a build file involving a single build target, such as... Figure 2 As shown, in this example, the build target hello-world instantiates Bazel's built-in rule cc_binary. This rule tells Bazel to build the executable binary from the source code hello-world.cc without any dependencies.

[0055] The example above only involves one build goal, but in practice, multiple build goals can also be involved. An example of a build file involving multiple build goals is as follows:

[0056]

[0057] Figure 3 The dependency graph provided by this invention is an example of a build file involving multiple build targets, such as... Figure 3As shown, in this example, Bazel will first build the hello-greet library, and then build the hello-world library. The `deps` property in the build target hello-world tells Bazel that the hello-greet library is needed to build the hello-world library. The build target hello-world depends on another build target hello-greet.

[0058] S300: Based on the dependency graph, output dependency data according to the dynamic library build file format, and generate a second build file for compiling the dynamic library.

[0059] Optionally, the package path where the build file is located and the name of the build target in the build file used by the dynamic library should be consistent with those in the source code.

[0060] The second build file is completely different from the first build file in the source code, except that the package path and build target name are the same.

[0061] To automatically generate a second build file, an algorithm repository needs to perform the following steps:

[0062] 1. A build rule needs to be implemented, which is used to generate the second build file.

[0063] 2. A build rule needs an aspect rule, which is used to record the dependency data required by each build target.

[0064] 3. In order to store the dependency data above, an provider object is needed in the aspect rule.

[0065] 4. Finally, in the Build rule, for each package path, output a second build file. Each second build file contains multiple build targets. The package path and the corresponding multiple build targets need to be obtained from the dependency data stored in the provider object. The provider data recorded in the aspect rule above stores the dependency data for each build target, and then the dependency data for each build target is formatted and output to the corresponding build target.

[0066] The following is an example of a build file located in the package path corresponding to the source code compilation: / / keeper / lib:

[0067]

[0068] The following is an example of a build file under the dynamically compiled package path: / / keeper / lib:

[0069]

[0070]

[0071] In this example, the second build file is completely different from the first build file in path and build target name and source code, so as to compile the dynamic library.

[0072] S400, based on the second build file, the dynamic library is compiled.

[0073] Optionally, with the second build file, and the dependent header file, and the generated dynamic library so, these files can be packaged into a compressed package and provided to other repositories.

[0074] The embodiment can realize dynamic library compilation through the source code build file, reducing the development workload.

[0075] Further, in another embodiment, the embodiment provides a compilation method implemented by a build tool, and the dependency data includes at least one of the following:

[0076] The header file, the source file, the dependent build target, the dependent dependency build target, the link option, the compilation option, the build target name, and the macro definition.

[0077] Optionally, the aspect rule records the dependency data, including: the header file, the source file (.so), the dependent build target (directly dependent), the dependent dependency build target (transitive dependency), the link option, the compilation option, the build target name, and the macro definition.

[0078] The embodiment discloses the structure of the dependency data.

[0079] Further, in another embodiment, the dependency graph and the dependency data of all build targets in the first build file are generated, including:

[0080] The dependency graph is topologically sorted, and the dependency data of each build target is generated in reverse order based on the topological sorting result.

[0081] Specifically, the first build file of the source code is analyzed to generate a dependency graph, and each node in the dependency graph corresponds to a build target.

[0082] The dependency graph is topologically sorted to obtain a topological sorting result. The dependency data is generated in reverse order from the nodes in the topological sorting result: the header file, the library file (.so), the dependent build target (directly dependent), the dependent dependency build target (transitive dependency), the link option, the compilation option, the build target name, and the macro definition.

[0083] Wherein the dependency construction target refers to the direct dependency of the node, and the transitive dependency refers to the indirect dependency of the node. For example, A depends on B, and B depends on C, B is the direct dependency of A, and C is the indirect dependency of A. The dependency data is constructed from C, and the direct dependency and the transitive dependency in the dependency data of the C node are empty. Subsequently, the dependency data is constructed for B, the direct dependency of B is C, and the transitive dependency is empty. Finally, the dependency data is constructed for A, the direct dependency of A is B, and the transitive dependency is C. By analogy.

[0084] Optionally, the transitive dependency of each node can be known based on the direct dependency of each node in the dependency relationship.

[0085] The embodiment discloses that the dependency data is generated based on the reverse sequence of the topological sorting of the dependency graph.

[0086] Further, in another embodiment, the embodiment provides a compilation method implemented by a build tool, and the dependency data is saved by a provider object of Bazel.

[0087] Optionally, the provider object provides a storage container for a simple value object.

[0088] The provider object saves the dependency data of each build target above: header file, source file (.so), dependent build target (direct dependency), dependency of dependent build target (transitive dependency), link option, compilation option, build target name, and macro definition.

[0089] The embodiment discloses a saving method of the dependency data.

[0090] Further, in another embodiment, the embodiment provides a compilation method implemented by a build tool, and the dynamic library compilation is implemented based on the second build file, and then includes:

[0091] The library file formed by the dynamic library compilation and the dependent header file are uploaded to an online database, so as to be called by a user who lacks the source code permission.

[0092] Optionally, the source code uses Gitlab, and the compiled dynamic library binary is stored in JFrog Artifactory.

[0093] Artifactory is a product of JFrog, which is used as a binary repository manager. The binary repository can uniformly host all these binaries, so that the management of the team is more efficient and simple. Gitlab is a project management software based on Git.

[0094] Gitlab is used to manage the source code, and Artifactory is used to manage the binary file.

[0095] The embodiment discloses management of binary files through an online database, and realizes sharing of dynamic library compilation files, so that a development team without source code permission can call the dynamic library compilation files.

[0096] Further, in another embodiment, the embodiment provides a compilation method implemented by a build tool, and the compilation method comprises the following steps of:

[0097] For the first build file and a third build file in a package directly or indirectly dependent on the source code, the dependency data is output in the dynamic library build file format based on the dependency graph, and a second build file for dynamic library compilation is generated.

[0098] Optionally, Bazel identifies a workspace through a workspace file, and regards each subdirectory with a build file as a package under the workspace directory.

[0099] It should be noted that, for source code compilation of a dependent package, in addition to the first build file in the directory of the source code, a third build file is also included in the package directly or indirectly dependent on the source code, and based on the dependency graph and the dependency data, a plurality of second build files need to be generated for the first build file and the third build file respectively, and the generated second build files are arranged in the same directories as the first build files or the third build files.

[0100] The embodiment discloses that the first build file to be converted and the third build file to be converted are both generated into second build files, and dynamic library compilation of the source code is realized based on the plurality of second build files.

[0101] Further, in another embodiment, the dynamic library compilation comprises the following steps of:

[0102] The dynamic library compilation is performed based on the input dynamic library compilation option.

[0103] The dynamic library compilation option and the source code compilation option are arranged in parallel, and based on external input, dynamic library compilation and / or source code compilation can be selected to be realized.

[0104] The embodiment discloses that the dynamic library compilation can be enabled or disabled, and based on external input selection, it is determined whether to realize the dynamic library compilation.

[0105] The following describes a build tool implemented compilation system provided by the embodiment, and the build tool implemented compilation system described below can be correspondingly referred to the build tool implemented compilation method described above.

[0106] Figure 4 This invention provides a schematic diagram of the structure of a compilation system implemented using a build tool, as shown below. Figure 4 As shown, a build tool-based compilation system includes:

[0107] The acquisition module acquires the first build file used for compiling the source code;

[0108] An intermediate data module generates a dependency graph and dependency data for all build targets within the first build file, based on the first build file.

[0109] A generation module, which, based on the dependency graph, outputs the dependency data according to the dynamic library build file format and generates a second build file for compiling the dynamic library;

[0110] A compilation module, which compiles dynamic libraries based on the second build file.

[0111] This embodiment can achieve dynamic library compilation by building files from source code, which reduces the amount of development work.

[0112] Furthermore, in another embodiment, this embodiment provides a compilation system implemented by a build tool, the system further comprising:

[0113] The source code compilation module, which is listed alongside the acquisition module, is used to compile source code.

[0114] The switch module controls the flow of data into the source code compilation module and / or dynamic library compilation module.

[0115] Optionally, a switch module can be used to control whether source code is pulled from GitLab or binary files are pulled from Artifatory, thereby controlling whether source code compilation or dynamic library compilation is used. If dynamic library compilation is used, the generated build file is used to construct a dependency graph, and the actions defined by each rule are executed according to the build graph to produce the final target.

[0116] This embodiment makes the source code compilation and dynamic library compilation and calling methods completely identical, so that the developers are unaware of the differences.

[0117] Figure 5 A schematic diagram of the physical structure of an electronic device provided by the present invention, such as... Figure 5As shown, the electronic device can include a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor 810, the communications interface 820, and the memory 830 complete mutual communication through the communications bus 840. The processor 810 can invoke a logical instruction in the memory 830 to execute a compilation method implemented by a build tool, the method comprising:

[0118] obtaining a first build file for source code compilation;

[0119] generating a dependency graph and dependency data of all build targets in the first build file based on the first build file;

[0120] outputting the dependency data in a dynamic library build file format based on the dependency graph, and generating a second build file for dynamic library compilation;

[0121] implementing dynamic library compilation based on the second build file.

[0122] In addition, the logical instruction in the memory 830 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0123] On the other hand, the present application also provides a computer program product, the computer program product includes a computer program stored on a non-transitory computer readable storage medium, the computer program includes program instructions, when the program instructions are executed by a computer, the computer can execute the compilation method implemented by the build tool provided by each method, the method comprising:

[0124] obtaining a first build file for source code compilation;

[0125] generating a dependency graph and dependency data of all build targets in the first build file based on the first build file;

[0126] output the dependency data in a dynamic library build file format based on the dependency graph, to generate a second build file for dynamic library compilation;

[0127] implement dynamic library compilation based on the second build file.

[0128] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned provided compilation method implemented by the build tool, and the method comprises:

[0129] obtaining a first build file for source code compilation;

[0130] generating a dependency graph and dependency data of all build targets in the first build file based on the first build file;

[0131] outputting the dependency data in a dynamic library build file format based on the dependency graph, to generate a second build file for dynamic library compilation;

[0132] implement dynamic library compilation based on the second build file.

[0133] The apparatus embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0134] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software and the necessary universal hardware platform, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in terms of contribution to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0135] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; and although the present application has been described in detail with reference to the foregoing embodiments, it should be appreciated by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A compilation method implemented using a build tool, characterized in that, The method includes: Obtain the first build file used for compiling the source code; Based on the first build file, a dependency graph and dependency data of all build targets within the first build file are generated, wherein the dependency data includes at least one of the following: header file, source file, dependent build target, dependent-dependent build target, linking options, compilation options, build target name, and macro definition; Perform a topological sort on the dependency graph, and based on the topological sort result, generate the dependency data for each of the construction targets in reverse order; Based on the dependency graph, the dependency data is output according to the dynamic library build file format to generate a second build file for compiling the dynamic library. The second build file is completely different from the first build file of the source code except for the package path and build target name. Based on the second build file, dynamic library compilation is implemented; The compiled library files and dependent header files are uploaded to an online database for users who lack access to the corresponding source code.

2. The compilation method implemented by the build tool according to claim 1, characterized in that, The process of compiling the dynamic library based on the second build file then includes: The compiled library files and dependent header files are uploaded to an online database for users who lack access to the corresponding source code.

3. The compilation method implemented by the build tool according to claim 1, characterized in that, The step of generating a second build file for compiling the dynamic library by outputting the dependency data according to the dynamic library build file format based on the dependency graph includes: For the first build file and the third build file in the package that the source code compilation directly or indirectly depends on, based on the dependency graph, the dependency data is output according to the dynamic library build file format to generate a second build file for dynamic library compilation.

4. The compilation method implemented by the build tool according to claim 1, characterized in that, The implementation of dynamic library compilation includes: Dynamic library compilation is performed based on the input dynamic library compilation options.

5. A compilation system implemented using a build tool, characterized in that, The system includes: The acquisition module acquires the first build file used for compiling the source code; An intermediate data module generates a dependency graph and dependency data for all build targets within the first build file, based on the first build file. The dependency data includes at least one of the following: header file, source file, dependent build target, dependent-dependent build target, linking options, compilation options, build target name, macro definition. The dependency graph is topologically sorted, and the dependency data is generated in reverse order for each build target based on the topological sorting result. A generation module, based on the dependency graph, outputs the dependency data according to the dynamic library build file format, and generates a second build file for compiling the dynamic library. The second build file is completely different from the first build file of the source code, except that the package path and the name of the build target are the same. The compilation module, based on the second build file, compiles a dynamic library and uploads the compiled library file and dependent header files to an online database for use by users who lack the corresponding source code permissions.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the compilation method implemented by the build tool as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the compilation method implemented by the build tool as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the compilation method implemented by the build tool as described in any one of claims 1-4.

Images