Container image building method, device and storage medium

Abstract

The embodiment of the present invention provides a container image construction method, device and storage medium, including: determining a total dependency graph corresponding to an application to reflect multiple modules on which the application depends and the dependency between the multiple modules; determining multiple dependency paths corresponding to the total dependency graph, each dependency path includes each module in the total dependency graph; determining multiple container image construction plans corresponding to the multiple dependency paths and the number of image layers respectively corresponding to the multiple container image construction plans, the container image construction plan is a multi-stage construction plan; determining a target container image construction plan from the multiple container image construction plans; generating a container image corresponding to the application according to a dockerfile generated by the target container image construction plan. This solution realizes volume compression of container images based on the total dependency graph.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a container image building method, device and storage medium. Background Art

[0002] With the popularity of container technology, container images have become the main means of application deployment. Docker is a tool that can package application code and operating environment into container images. The operating environment can include various dependencies of the application, such as dependent libraries / files. After building the container image corresponding to the application, you can create a container and run the container image in the container to deploy the application.

[0003] Generally, the process of creating a container image includes: developers write dockerfile, then use the dockerbuild instruction to build the container image, and then use the docker run instruction to run the container image. Among them, dockerfile is a text file that contains all the instructions required to build a container image, such as: the from instruction for specifying the base image, the copy instruction for copying information, the cmd instruction for specifying the default execution command for the container image, the env instruction for setting environment variables, and the add instruction for adding information. The execution of each instruction in the dockerfile will add an image layer to the container image. The more image layers a container image has, the larger the size of the container image will be, the more storage space it will occupy, and the more resources it will require at runtime.

[0004] In actual applications, the dockerfile written by developers based on their experience will inevitably contain instructions for repeatedly loading the same data due to incomplete or inaccurate understanding, which will lead to a larger container image size based on the dockerfile. For example, an application building project contains two applications, application a and application b, and both applications may have dependencies on the same function library. In the dockerfile, when loading the dependency file of application a, there may be instructions for loading the function library, and for application b, there may also be instructions for loading the function library, resulting in repeated loading of the function library, which ultimately increases the size of the container image corresponding to the project. Summary of the invention

[0005] Embodiments of the present invention provide a container image construction method, device, and storage medium to reduce the size of a container image.

[0006] In a first aspect, an embodiment of the present invention provides a method for building a container image, the method comprising:

[0007] Determine a total dependency graph corresponding to the application, where the total dependency graph is used to reflect multiple modules on which the application depends and the dependency relationships between the multiple modules;

[0008] Determine a plurality of dependency paths corresponding to the total dependency graph, wherein each dependency path includes each module in the total dependency graph;

[0009] Determine multiple container image build plans corresponding to the multiple dependency paths, and the number of image layers corresponding to the multiple container image build plans, each container image build plan being a multi-stage build plan;

[0010] Determine a target container image build plan from the multiple container image build plans;

[0011] Generate a container image corresponding to the application according to the Dockerfile generated by the target container image construction plan.

[0012] In a second aspect, an embodiment of the present invention provides a container image building device, the device comprising:

[0013] A determination module is used to determine a total dependency graph corresponding to an application, wherein the total dependency graph is used to reflect multiple modules on which the application depends and the dependency relationships between the multiple modules; determine multiple dependency paths corresponding to the total dependency graph, wherein each dependency path includes each module in the total dependency graph; determine multiple container image build plans corresponding to the multiple dependency paths, and the number of image layers corresponding to the multiple container image build plans, each container image build plan being a multi-stage build plan; determine a target container image build plan from the multiple container image build plans;

[0014] A generation module is used to generate a container image corresponding to the application according to a Dockerfile generated by the target container image building plan.

[0015] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores an executable code, and when the executable code is executed by the processor, the processor can at least implement the container image building method as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the container image building method described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present invention provides a computer program product, including: a computer program, when the computer program is executed by a processor of an electronic device, the processor can at least implement the container image building method as described in the first aspect.

[0018] In the solution provided by an embodiment of the present invention, the process of building a container image for a certain application includes: first, determining a total dependency graph corresponding to the application, the total dependency graph is used to reflect multiple modules (that is, multiple dependencies, such as: files, libraries, functions, components, etc.) on which the application depends and the dependency relationship between multiple modules; then, determining multiple dependency paths corresponding to the total dependency graph, wherein each dependency path includes each module in the total dependency graph; then, determining multiple container image build plans corresponding to the multiple dependency paths and the number of image layers corresponding to the multiple container image build plans, and determining a target container image build plan from the multiple container image build plans, wherein each container image build plan is a multi-stage build plan, and the target container image build plan is one of the multiple container image build plans; finally, generating a container image corresponding to the application according to a dockerfile generated by the target container image build plan.

[0019] In this solution, the dependency relationship between multiple modules is concisely and clearly represented through the total dependency graph corresponding to the application. Since there are no identical or redundant modules in the total dependency graph, there is no redundant build information in the multiple container image build plans corresponding to the multiple dependency paths determined based on the total dependency graph. Therefore, building a container image based on the container image build plan in this solution can effectively avoid the large size of the container image due to repeated loading of data and other reasons. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 A flowchart of a container image building method provided by an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of a total dependency graph provided by an embodiment of the present invention;

[0023] Figure 3 A schematic diagram of a container image building process provided by an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of a container image building plan provided in an embodiment of the present invention;

[0025] Figure 5 A flowchart of a method for determining a total dependency graph provided by an embodiment of the present invention;

[0026] Figure 6 A schematic diagram of a module dependency graph determination process provided by an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the structure of a container image building device provided by an embodiment of the present invention;

[0028] Figure 8 For Figure 7 A schematic diagram of the structure of an electronic device corresponding to the container image building device provided in the illustrated embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and the features in the embodiments can be combined with each other. In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0032] Container image building refers to the process of creating a container image. Container image building can be divided into two methods: single-stage build and multi-stage build. In the single-stage build method, the Dockerfile will contain all the dependency information and related processing results that the application depends on, resulting in a large size of the final generated container image. In order to reduce the size of the built container image, the multi-stage build method is currently used, that is, multiple from instructions are used in a Dockerfile, where each from instruction corresponds to a build stage. In a multi-stage build, the necessary files in the previous build stage can be selectively copied to the later build stage through the from instruction, while discarding unnecessary files, that is, some stages in the build process may only exist in the build process, and will not appear in the final container image. Since the final built container image only retains the application code and the operating environment required by the application (the operating environment contains the application's dependencies), the size of the built container image can be effectively reduced.

[0033] However, when actually performing multi-stage builds, developers usually write Dockerfiles based on experience. Inevitably, due to incomplete or inaccurate understanding, there will be instructions in the Dockerfile for repeated loading of the same data. As a result, the size of the container image built based on the Dockerfile is reduced compared to the single-stage build, but there is still room for further compression.

[0034] The container image construction method provided in the embodiment of the present invention can be executed by an electronic device, which can be a terminal device such as a PC, a laptop, a smart phone, or a server. The server can be a physical server including an independent host, or a virtual server, or a cloud server or a server cluster.

[0035] Figure 1 A flowchart of a container image building method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the following steps may be included:

[0036] 101. Determine a total dependency graph corresponding to the application, where the total dependency graph is used to reflect multiple modules that the application depends on and the dependency relationships between the multiple modules.

[0037] 102. Determine multiple dependency paths corresponding to the total dependency graph, wherein each dependency path includes each module in the total dependency graph.

[0038] 103. Determine multiple container image build plans corresponding to multiple dependency paths and the number of image layers corresponding to the multiple container image build plans, each container image build plan being a multi-stage build plan.

[0039] 104. Determine a target container image build plan from multiple container image build plans.

[0040] 105. Generate a container image corresponding to the application according to the Dockerfile generated by the target container image building plan.

[0041] In the process of building a container image, this embodiment obtains the total dependency graph corresponding to the application. The multi-stage build plan determined based on the total dependency graph can match the actual dependency structure of the application. Furthermore, when writing a Dockerfile based on multiple build stages in the multi-stage build plan, it is possible to effectively avoid writing instructions for repeatedly loading the same data in the Dockerfile, thereby greatly reducing the size of the container image built based on the Dockerfile.

[0042] The total dependency graph corresponding to the application is used to reflect the multiple modules that the application depends on and the dependency relationships between the multiple modules. The multiple modules that the application depends on can be understood as the multiple dependencies that the application depends on, including but not limited to files, libraries, functions, components, dependency packages, etc.

[0043] The total dependency graph of the application can be obtained by analyzing the dependency relationships of the modules contained in the source code file of the application. In practical applications, a corresponding analysis tool can be selected according to the language type used in the source code file, and the source code file can be analyzed by the analysis tool to obtain the total dependency graph.

[0044] For example, when the language type used in the source code in the source code file is Python, the source code file is analyzed through the Abstract Syntax Tree (AST) to obtain the AST analysis result, where the root node of the tree in the AST analysis result is the source code file, and the child nodes are multiple modules that the application depends on; based on the AST analysis result, the total dependency graph of the application is determined.

[0045] For another example, when the language type adopted by the source code in the source code file is Java language, the source code file is analyzed by Java decompilers; and the dependency graph corresponding to each module is determined based on the analysis result of the Java decompiler.

[0046] Optionally, the total dependency graph may be a directed graph, including multiple nodes and multiple directed edges, wherein the multiple nodes are used to represent multiple modules, and the nodes correspond to the modules one by one; and the multiple directed edges are used to represent the dependency relationship between the multiple nodes, that is, the dependency relationship between the multiple modules.

[0047] In practical applications, a node can be represented by the name and version number of its corresponding module as an identifier, so that two modules with the same name but different version numbers will correspond to two different nodes. For ease of description, the nodes in the total dependency graph are described by their corresponding modules.

[0048] Combination Figure 2 , an example is given to illustrate the total dependency graph, Figure 2 A schematic diagram of a total dependency graph provided by an embodiment of the present invention.

[0049] Assume that the multiple modules that the application depends on are module 1, module 2, module 3, ..., module M (M is a positive integer), where module 1 depends on module 2, module 2 depends on module 3 and module 4, module 3 depends on module 5 and module 6, module 4 depends on module 7, module 8 and module 9, ..., module 8 depends on module g, module g depends on module G, module m depends on module M (g, G and m are positive integers less than M). Based on the dependencies between modules 1 to M and between modules 1 to M that the application depends on, generate the total dependency graph corresponding to the application, as shown in Figure 2 As shown, the dependency relationship between module 1 and module M is represented by directed edges. For example, between module 1 and module 2, the directed edge from module 2 to module 1 indicates that module 1 depends on module 2.

[0050] After determining the total dependency graph of the application, based on the total dependency graph, we can have a more comprehensive and accurate understanding of the multiple modules that the application depends on and the dependencies between multiple modules, so as to perform more accurate multi-stage construction and generate the corresponding Dockerfile to further generate a smaller container image.

[0051] For ease of understanding, the following combination Figure 3 The container image building method provided in the embodiment of the present invention is exemplarily described. Figure 3 A schematic diagram of a container image building process provided by an embodiment of the present invention, such as Figure 3 As shown, after determining the total dependency graph corresponding to the application, first, multiple dependency paths corresponding to the total dependency graph are determined, such as dependency path 1, dependency path 2, ..., dependency path N (N is a positive integer). Each dependency path includes each module in the total dependency graph.

[0052] Optionally, the total dependency graph can be traversed by traversal algorithms such as pre-order traversal, in-order traversal, and post-order traversal in tree structure traversal, and the traversal path of each traversal of the total dependency graph is used as the dependency path, thereby obtaining multiple dependency paths corresponding to the dependency graph. Among them, the pre-order traversal is performed in the order of "visiting the root node, traversing the left subtree, and traversing the right subtree", the in-order traversal is performed in the order of "traversing the left subtree, visiting the root node, and traversing the right subtree", and the post-order traversal is performed in the order of "traversing the left subtree, traversing the right subtree, and visiting the root node".

[0053] Optionally, when determining multiple dependency paths corresponding to the total dependency graph through a traversal algorithm, the traversal can be performed starting from the root node of the total dependency graph, so that the multiple dependency paths obtained have the same starting point, for example: Figure 2 When traversing the illustrated total dependency graph, the traversal may be performed starting from module 1. In a specific implementation, the root node (i.e., module 1) of the total dependency graph of an application usually corresponds to the entry file of the application. The entry file of the application is the first file that the application starts to execute.

[0054] Since the order in which the modules in the total dependency graph are traversed is different in different traversal paths, the order of the modules in the multiple dependency paths corresponding to the total dependency graph is also different. For example, in two different dependency paths 1 and 2, for modules x and y that the application depends on, in dependency path 1, module x is traversed first and then module y, and in dependency path 2, module y is traversed first and then module x.

[0055] Due to the difference in the order in which multiple modules are traversed in different dependency paths, the same module in different dependency paths may be in different build stages in the container image build plan corresponding to the corresponding dependency path. In other words, the difference in the order of multiple modules in different dependency paths can make the container image build plan determined based on the dependency path different.

[0056] In the specific implementation process, Figure 3 As shown, after determining dependency path 1, dependency path 2, ..., dependency path N, further, container image build plan 1, container image build plan 2, ..., container image build plan K are determined, where K is greater than or equal to N, and each container image build plan is a multi-stage build plan.

[0057] It is understandable that when generating a container image build plan based on a dependency path, at least one container image build plan generation strategy can be used. When there is only one container image build plan generation strategy, a container image build plan can be generated for each dependency path, and K is equal to N at this time; when there are multiple container image build plan generation strategies, a container image build plan can be generated for each dependency path based on one container image build plan generation strategy, so that multiple container image build plans can be generated, and K is greater than N at this time. In different generation strategies, different rules can be set for factors such as the order of modules and the dependencies between modules.

[0058] For example, optionally, for a dependency path, the construction phase to which each module belongs can be determined in turn according to the order of the modules and the dependencies between the modules, wherein different modules with dependencies belong to different construction phases, and modules that can be executed in parallel can be in the same construction phase. For example, the order of some modules included in a dependency path is as follows: module a, module b, module c, module d, wherein module c depends on module b and module a, and module d has no dependency relationship with the other three modules. Then, assuming that module a is first determined to belong to construction phase i, since module b has no dependency relationship with module a and can be executed in parallel, it can be determined that module b also belongs to construction phase i, since module c depends on module a and module b, it can be determined that module c belongs to construction phase i+1, and since module d has no dependency relationship with the other three modules, it can be determined that module d belongs to construction phase i or construction phase i+1 or construction phase i+2. Here, the construction phase to which module d belongs can be determined according to the specific setting of the determination rule setting of the construction phase to which the "module that can be executed independently" belongs. The so-called module that can be executed independently is a module that can be executed without relying on other modules.

[0059] As an optional container image build plan generation strategy, for any dependency path among multiple dependency paths (referred to as a target dependency path), multiple build phases can be determined based on the dependencies between multiple modules in the target dependency path; wherein there is no dependency between different modules included in the same build phase, and the modules in the later build phase depend on the build products of the previous build phase; then, multiple build phases included in the container image build plan corresponding to the target dependency path are determined.

[0060] In the container image build plan, different build phases need to perform different build tasks. The result obtained after each build phase executes the build task based on the corresponding module is the build product of that build phase. For example, in the process of multi-stage container image build, it is necessary to compile the application and prepare the operating environment. Assuming that a certain build phase is used to compile the application, the compiled file compiled in that build phase is the build result of that build phase. In actual applications, the modules in the later build phase can also only depend on the modules in the previous build phase.

[0061] In the specific implementation process, according to the dependency relationship between multiple modules in the target dependency path, the modules that are not dependent on other modules can be screened out from multiple modules and placed in the first construction stage, the modules that only depend on the entry file can be screened out from multiple modules and placed in the last construction stage, and the other modules except those placed in the first construction stage and the last construction stage can be determined to belong to at least one intermediate construction stage according to the dependency relationship. Finally, it is determined that the container image construction plan corresponding to the target dependency path includes the first construction stage, at least one intermediate construction stage and the last construction stage.

[0062] For ease of understanding, for example, assume that there is a dependency path, and the multiple modules included in the dependency path are: entry file, module A, module B, module F, module C, module D, and module E. Among them, the entry file depends on module A, module A depends on modules B and module C, module C depends on module D, module D depends on module E, and module F depends on module B.

[0063] In the process of generating a container image build plan based on the container image build plan generation strategy, first, from modules A to module F, the modules that are not dependent on other modules, namely, modules E and module F, are selected and placed in the first build phase; and the module that only depends on the entry file, namely, module A, is selected from multiple modules and placed in the last build phase. Afterwards, for modules B, module C, and module D, since module B depends on module F, module D depends on module E, and there is no dependency relationship between modules B and D, modules B and module D are placed in the second build phase; since module C depends on module D and module A depends on module C, module C is placed in the third build phase. The last build phase where module A is located is the fourth build phase. Finally, the container image build plan includes the first build phase, the second build phase, the third build phase, and the fourth build phase.

[0064] Figure 4 A schematic diagram of a container image building plan provided in an embodiment of the present invention, based on Figure 4 The above container image building plan is explained as an example. Figure 4 As shown, the container image building plan includes four building stages, the first building stage includes module E and module F, the second building stage includes module B and module D, the third building stage includes module C, and the fourth building stage includes module A. Among them, module B depends on the building product of module F, module D depends on the building product of module E, module C depends on module D, and module A depends on the building product of module B and the building product of module C.

[0065] As another optional container image build plan generation strategy, for any dependency path in multiple dependency paths (referred to as the target dependency path), you can first determine multiple build phases corresponding to the modules with continuous dependencies in the order of multiple modules in the target dependency path, starting from the first module; determine the module i after the last module in the continuous dependency in the target dependency path; in the order of multiple build phases, starting from the first build phase, determine in turn whether the modules in each build phase have no dependency with module i; if the modules in the current build phase have no dependency with module i, then put module i into the current build phase; and so on, until the last module in the target dependency path is put into the corresponding build phase. The final generated container image build plan includes these multiple build phases.

[0066] For example, it is still assumed that there is a dependency path, and the multiple modules included in the dependency path are: entry file, module A, module B, module F, module C, module D and module E. Among them, the entry file depends on module A, module A depends on modules B and module C, module C depends on module D, module D depends on module E, and module F depends on module B.

[0067] Starting from the entry file, it can be determined that there is a continuous dependency relationship between module A, module B, and module F, that is, module A depends on module B, and module B depends on module F. Based on this dependency relationship, module F is placed in the first construction phase, module B is placed in the second construction phase, and module A is placed in the third construction phase. After that, the construction phase corresponding to module C is determined. Since there is no dependency relationship between module C and module F, module C can be placed in the first construction phase; then, the construction phase corresponding to module D is determined. Since module D depends on module C in the first construction phase, module D cannot be placed in the first construction phase. There is no dependency relationship between module B and module D in the second construction phase, so module D can be placed in the second construction phase; finally, the construction phase corresponding to module E is determined. Since there is no dependency relationship between module F and module C in the first construction phase and module E, module E can be placed in the first construction phase. Finally, the container image construction plan includes the first construction phase, the second construction phase, and the third construction phase.

[0068] In the above container image building plan, the building product in the previous building stage can be passed to the next building stage until all the multiple building stages in the container image building plan are executed. In the process of passing the building product between the two building stages, only the building product itself can be passed, without passing the relevant module information used to obtain the building product. For example, a certain building stage includes a module (which can be a file) storing a compilation environment, based on which the application can be compiled and a compiled file can be obtained. When passing the building product, only the compiled file can be passed to the next building stage, without passing the file storing the compilation environment to the next building stage. It can be seen that by passing the building product of each building stage, the modules that are only relied on in the intermediate process of the container image building process (for example: the module storing the compilation environment) can be eliminated, and only the final building product of each building stage is retained, thereby reducing the information required to generate the container image and reducing the size of the container image.

[0069] In addition, in the container image build plan, configuring modules that have no dependencies in the same build phase is conducive to executing corresponding build tasks for each module in the same build phase in parallel, improving the execution efficiency of the container image build plan, that is, improving the efficiency of generating container images.

[0070] After determining multiple container image build plans, such as Figure 3 As shown, a target container image building plan can be screened out from container image building plans 1 to container image building plans K to generate a dockerfile, so that a container image corresponding to the application can be generated according to the dockerfile.

[0071] It can be understood that the total dependency graph corresponding to the application can clearly and concisely represent the dependency between multiple modules. Since there are no identical or redundant modules in the total dependency graph, the multiple container image building plans corresponding to the multiple dependency paths determined based on the total dependency graph do not have redundant building information. Therefore, the container images built based on the container image building plan in this solution can effectively avoid the large container image volume caused by repeated loading of data. In other words, building a container image based on any of the container image building plans 1 to K can reduce the container image volume.

[0072] Therefore, in an optional embodiment, the target container image building plan may be any one of container image building plans 1 to container image building plans K.

[0073] In actual applications, although container image building plans 1 to container image building plans K can all achieve a reduction in the volume of container images, different container image building plans have different degrees of reduction in the volume of container images. Specifically, in different container image building plans determined based on different dependency paths, or in different container image building plans determined by different container image building plan generation strategies for the same dependency path, the number of image layers corresponding to the multiple build stages included in the container image building plan is different. Usually, the number of image layers is positively correlated with the volume of the container image, that is, the fewer the number of image layers, the smaller the volume of the built container image. Among them, the container image volume can also be understood as the amount of data contained in the container image, which can be quantified by data volume units such as MB or GB.

[0074] In order to compress the container image volume of the application as much as possible, in another optional embodiment, optionally, according to the number of image layers corresponding to each container image build plan, a container image build plan with a smaller number of image layers can be selected from container image build plan 1 to container image build plan K as the target container image build plan. In other words, among multiple container image build plans, the number of image layers corresponding to the selected target container image build plan is smaller than the number of image layers corresponding to other container image build plans.

[0075] Optionally, if there are more than two container image build plans with the same number of image layers corresponding to container image build plans 1 to container image build plan K, container images are generated respectively based on the two or more container image build plans; and based on the volume of the generated container images, the container image with a smaller volume is determined as the container image of the application.

[0076] The above only illustrates a method of selecting a target container image build plan by considering the number of image layers and the container image volume. In fact, it is not limited to this. For example, the selection can be made directly based on the container image volume, or directly based on the number of image layers.

[0077] Finally, the Dockerfile is determined according to the target container image building plan. In the specific implementation process, the code corresponding to the entry file is first loaded into the Dockerfile, and then, based on the multiple building stages in the target container image building plan, the instructions corresponding to the multiple building stages are written in the Dockerfile manually or automatically, so as to finally generate the container image corresponding to the application according to the instructions in the Dockerfile.

[0078] It should be noted that after determining the multiple construction stages in the target container image construction plan, although the dockerfile can also be written manually, unlike the developer writing the dockerfile based on experience in the related art, in this solution, when manually writing the dockerfile, the target container image construction plan determined based on the total dependency graph of the application is used. Since the target container image construction plan is obtained based on a full and accurate analysis of the dependency structure between the multiple modules that the application depends on, it is accurate and concise. Therefore, based on the multiple construction stages in the target container image construction plan, when writing the dockerfile, the instructions for repeated loading of the same data can be effectively reduced.

[0079] In addition, when writing a dockerfile, it is usually necessary to load the source code file of the application in the dockerfile, and the source code file may contain some non-essential content for building the container image due to writing reasons. In order to reduce the information redundancy in the final generated container image, optionally, the target module in the source code file of the application that does not exist in the target dependency path can be first determined, wherein the target dependency path is a dependency path in multiple dependency paths corresponding to the target container image building plan; then, the target module is deleted from the source code file to load the source code file after deleting the target module in the dockerfile. By filtering out the target modules that do not exist in the target dependency path in the source code file, the amount of information loaded into the dockerfile from the source code file is reduced.

[0080] In summary, in the process of building a container image for a certain application, this embodiment first determines the total dependency graph corresponding to the application. The total dependency graph corresponding to the application can be used to concisely and clearly represent the multiple modules on which the application depends and the dependency relationships between the multiple modules; since there are no identical or redundant modules in the total dependency graph, there is no redundant construction information in the multiple container image construction plans corresponding to the multiple dependency paths determined based on the total dependency graph. Building container images based on the multiple container image construction plans in this solution can effectively avoid the large volume of container images due to repeated loading of data and other reasons. The multiple container image construction plans in this solution are all multi-stage construction plans. In actual applications, there are differences in the number of image layers corresponding to each of them, so the degree of optimization of the container image volume is also different. The number of image layers is positively correlated with the volume of the container image. The fewer the number of image layers, the smaller the corresponding container image volume. This solution further reduces the volume of the constructed container image to a large extent by selecting the target container image construction plan with the smallest number of image layers from multiple container image construction plans, and generating the container image corresponding to the application based on the target container image construction plan. Reducing the size of container images helps reduce resource consumption during image pulling and runtime, and can improve the overall operating performance and efficiency of the application.

[0081] The above describes the process of generating a container image of an application based on the total dependency graph of the application. Next, the process of determining the total dependency graph is described in detail.

[0082] Figure 5 A flowchart of a method for determining a total dependency graph provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the following steps may be included:

[0083] 501. Obtain a directory structure of source code files of an application, where the directory structure includes multiple modules that the application depends on.

[0084] 502. Determine dependency graphs corresponding to each of the multiple modules, wherein the dependency graph corresponding to each module is used to reflect the dependency relationship between the corresponding module and other modules on which it depends.

[0085] 503. Determine a total dependency graph corresponding to the application program according to dependency graphs corresponding to the multiple modules.

[0086] The source code file of the application contains the directory structure of the source code of the application. The directory structure includes multiple modules that the application depends on. By performing dependency analysis on these modules through analysis tools such as AST and Java decompiler, the dependency relationship between multiple modules in the directory structure can be determined, thereby constructing a dependency graph corresponding to each of the multiple modules.

[0087] The following is an example directory structure:

[0088]

[0089] Among them, init__.py, conf.py, redis.py, db.py and log.py are script files in the app_sdk directory; common_view.py is a script file in the views directory, and common_service.py is a script file in the services directory. In this embodiment, the multiple modules that the application depends on include the files in the directory structure.

[0090] In an optional embodiment, the dependency graphs corresponding to the multiple modules are determined, including: determining the dependency graph corresponding to each module from the parent directory to the subdirectory in sequence according to the directory hierarchy organization method corresponding to the multiple modules in the directory structure. For example: first determine the dependency graphs corresponding to the parent directory files such as README.md, LICENSE, app.py, Dockerfile, app_sdk, views, services, etc., and then determine the dependency graphs corresponding to the subdirectory files such as init__.py and conf.py. Among them, if there is a special module in the directory hierarchy that neither depends on other modules nor is depended on by others, it is determined that the dependency graph of the special module only contains itself.

[0091] In another optional embodiment, since the directory structure of the application source code file contains a unique entry file as a specific module, the dependency graphs corresponding to the multiple modules are determined, and the method further includes: starting from the entry file, tracking the dependency graphs of each module in sequence. Specifically, the dependency graph corresponding to the entry file is first determined, wherein the dependency graph corresponding to the entry file is used to reflect the dependency relationship between the entry file and the modules on which the entry file depends; then, the dependency graphs corresponding to the modules on which the entry file depends are determined respectively; thereafter, the dependency graphs corresponding to the modules that are dependent in the newly determined dependency graph are determined in sequence, until there is no module dependent on the module in the latest determined dependency graph.

[0092] For ease of understanding, combined Figure 6 For example, Figure 6 A schematic diagram of a module dependency graph determination process provided by an embodiment of the present invention. Assuming that the source code of the application is in Python, the entry file is analyzed through AST to determine that the entry file depends on module 1 and module 2, then the dependency graph corresponding to the entry file can be determined first. Figure 6As shown, the dependency graph corresponding to the entry file includes the entry file and module 1 and module 2, wherein there is a directed edge from module 1 to the entry file between the entry file and module 1, and there is a directed edge from module 2 to the entry file between the entry file and module 2.

[0093] Next, the dependency graphs corresponding to module 1 and module 2 are determined respectively. If, based on the AST analysis results, it is determined that module 1 depends on module 3 and module 2 depends on module 4, the dependency graphs corresponding to module 1 and module 2 are determined respectively, such as Figure 6 shown.

[0094] After that, the dependency graphs corresponding to module 3 and module 4 are determined respectively. If based on the AST analysis results, it is determined that module 3 depends on module 5 and module 4 does not have any dependent modules, then for module 3, the following can be obtained: Figure 6 The dependency graph of module 3 is shown; for module 4, it is determined that the dependency graph corresponding to module 4 only includes itself, and the tracking of the dependency graph of module 4 is stopped.

[0095] Similar to the process of the dependency graph corresponding to the aforementioned entry file, module 1, module 2, module 3 and module 4, continue to track the dependency graph corresponding to module 5 until the module in the latest determined dependency graph has no dependent modules, that is, the dependency graph only contains itself.

[0096] In this solution, the dependency graph corresponding to each module is determined in sequence starting from the entry file. Since there is a certain dependency relationship between the modules in the dependency graph determined later and the modules in the dependency graph determined earlier, the dependency graph of each module is actually determined layer by layer according to the dependency relationship between the modules. In this way, the repeated generation of the dependency graph of the same module can be effectively avoided, thereby effectively improving the efficiency of determining the dependency graphs corresponding to multiple modules on which the application depends.

[0097] Optionally, in the process of determining the dependency graph corresponding to any module j among the multiple modules that the application depends on, the other modules that module j depends on and the dependency relationship between the other modules can be determined first. Then, the identification information of module j and other modules that module j depends on is determined from the dependency file corresponding to the application, and the identification information includes the module name and version number. Finally, the identification information of multiple nodes in the dependency graph corresponding to module j is determined, and the dependency graph of module j is generated based on the identification information of the multiple nodes. Among them, multiple nodes correspond to module j and other modules that module j depends on, and there are also multiple directed edges in the dependency graph corresponding to module j, corresponding to the dependency relationship between multiple nodes.

[0098] The dependency file corresponding to the application records the version numbers of the modules that the application depends on. In actual applications, the language type of the application source code is different, and the corresponding dependency files are different. For example, if the language type of the application source code is python, the dependency file is the requirements.txt file; if the language type of the application source code is Java, the dependency files are the pom.xml file and the build.gradle file.

[0099] In the process of determining the dependency graph corresponding to module j, using the module name and version number as the unique identifier of each node in the dependency graph can achieve accurate identification of modules of different versions, effectively avoid different versions of the same module being identified as the same node in the dependency graph, and improve the accuracy of the dependency graph.

[0100] After determining the dependency graphs corresponding to the multiple modules, the dependency graphs corresponding to the multiple modules are combined to obtain a total dependency graph corresponding to the application program. Combining the dependency graphs corresponding to the multiple modules is actually merging the same nodes in the dependency graphs of different modules.

[0101] Optionally, the dependency graph corresponding to each module determined according to the directory hierarchical structure organization method corresponding to multiple modules in the directory structure can be spliced ​​together to obtain a total dependency graph corresponding to the application; the dependency graphs of each module can be traced in sequence starting from the entry file and spliced ​​together to obtain a total dependency graph corresponding to the program; the dependency graph corresponding to each module determined according to the directory hierarchical structure organization method corresponding to multiple modules in the directory structure can be spliced ​​together with the dependency graph of each module traced in sequence starting from the entry file to obtain a total dependency graph corresponding to the program.

[0102] Optionally, if there are isolated nodes in the total dependency graph obtained by combining the dependency graphs corresponding to multiple modules, the isolated nodes can be removed from the total dependency graph.

[0103] In an optional embodiment, to ensure the accuracy of the total dependency graph, after determining the dependency graph corresponding to each module, the container image construction method provided by the embodiment of the present invention further includes: performing abnormal event detection on the dependency graph corresponding to the module, and if an abnormal event is detected in the dependency graph corresponding to any module, outputting corresponding prompt information so that the user can correct the total dependency graph based on the prompt information. The abnormal event includes at least one of the following: an authorization abnormal event of a node, a circular dependency event between nodes, and an operation error event of a node.

[0104] Among them, the authorization abnormality event of the node includes that the module corresponding to the node has a security vulnerability or the license of the module corresponding to the node is illegally authorized, etc. When a node authorization abnormality occurs in a node in the dependency graph corresponding to the module, the output prompt information may include relevant information of the module corresponding to the node and specific abnormal information, such as: the license of a module is suspected to be illegally authorized, etc. Based on the prompt information, a new legally authorized license can be optionally configured for the module, or the dependency graph corresponding to the module can be removed from the total dependency graph to achieve the correction of the total dependency graph.

[0105] A circular dependency event between nodes refers to the existence of two mutually dependent nodes in the dependency graph corresponding to the module, for example: module x node depends on module y node, and module y node depends on module x node. When there is a circular dependency event between nodes in the dependency graph corresponding to the module, the output prompt information may include the two mutually dependent nodes. Based on the prompt information, the circular dependency in the dependency graph corresponding to the module can be modified to a unidirectional dependency, for example: deleting the dependency relationship of "module x node depends on module y node" (i.e., deleting the directed edge from module y node to module x node), or deleting the dependency relationship of "module y node depends on module x node" (i.e., deleting the directed edge from module x node to module y node). By correcting the dependency graph of the module, the overall dependency graph can be corrected.

[0106] A node's runtime error event refers to a source code bug detected during the parsing of source code using source code analysis tools such as AST, such as syntax errors, variable name spelling errors, or function name spelling errors. When a node in the dependency graph corresponding to a module has a runtime error event, the output prompt information may include relevant information about the module corresponding to the node and specific runtime error information. For example, when a module is a function and a function name has a spelling error, the prompt information may be that the function name is spelled incorrectly. Based on the prompt information, the function name in the dependency graph corresponding to the function can be modified to achieve correction of the overall dependency graph.

[0107] In this embodiment, since the total dependency graph is obtained based on the dependency graphs of multiple modules, the total dependency graph can be corrected by performing abnormal event detection on the dependency graph corresponding to any module and correcting the dependency graph of the corresponding module based on the prompt information corresponding to the abnormal event, thereby improving the accuracy of the total dependency graph.

[0108] The above process of determining the total dependency graph based on the source code file does not include the running data generated when the application is actually running, so it is actually a static analysis. In actual applications, dynamic analysis can also be performed, that is, the total dependency graph is updated based on the running data generated when the application is actually running.

[0109] During the specific implementation process, the application can be run by first running a container image obtained by running a total dependency graph determined based on a source code file; then, the operation data record information generated during the operation of the application is obtained, wherein the operation data record information includes the calling relationship between different modules; then, according to the calling relationship between different modules, the total dependency graph is updated; finally, according to the updated total dependency graph, the container image corresponding to the application is regenerated.

[0110] Among them, updating the total dependency graph according to the calling relationships between different modules includes: modifying the dependencies in the total dependency graph that are inconsistent with the calling relationships according to the calling relationships between different modules, and / or adding the calling relationships to the total dependency graph.

[0111] For example, if the total dependency graph contains a module h node and a module g node, and the directed edge indicates that the module h node depends on the module g node, and the call relationship indicates that module g needs to call module h, that is, module g depends on module h, then the dependency relationship between the module h node and the module g node in the total dependency graph is modified to the module g node depends on the module h node. If the call relationship indicates that module g needs to call module h, that is, module g depends on module h, and the total dependency graph does not contain the dependency relationship between the module h node and the module g node, then the dependency relationship of the module g node depending on the module h node is added to the total dependency graph.

[0112] In this embodiment, since the updated total dependency graph utilizes the operation data record information generated when the application is actually running, it has higher accuracy. Therefore, the container image corresponding to the application regenerated based on the updated total dependency graph can further reduce the size of the container image.

[0113] The following will describe in detail the container image building device of one or more embodiments of the present invention. Those skilled in the art will appreciate that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0114] Figure 7 A schematic diagram of the structure of a container image building device provided by an embodiment of the present invention is shown in FIG. Figure 7 As shown, the device includes: a determination module 11 and a generation module 12.

[0115] A determination module 11 is used to determine a total dependency graph corresponding to the application, wherein the total dependency graph is used to reflect multiple modules on which the application depends and the dependencies between the multiple modules; determine multiple dependency paths corresponding to the total dependency graph, wherein each dependency path includes each module in the total dependency graph; determine multiple container image build plans corresponding to the multiple dependency paths, and the number of image layers corresponding to the multiple container image build plans, each container image build plan being a multi-stage build plan; and determine a target container image build plan from the multiple container image build plans.

[0116] The generation module 12 is used to generate a container image corresponding to the application according to the Dockerfile generated by the target container image construction plan.

[0117] Optionally, the number of image layers corresponding to the target container image building plan is less than the number of image layers corresponding to other container image building plans.

[0118] Optionally, a determination module 11 is specifically used to obtain a directory structure of the source code file of the application, wherein the directory structure includes multiple modules on which the application depends; determine a dependency graph corresponding to each of the multiple modules; wherein the dependency graph corresponding to each module is used to reflect the dependency relationship between the corresponding module and other modules on which it depends; and determine a total dependency graph corresponding to the application based on the dependency graphs corresponding to each of the multiple modules.

[0119] Optionally, the multiple modules include an entry file of the application, and the determination module 11 is also specifically used to determine a dependency graph corresponding to the entry file, and the dependency graph corresponding to the entry file is used to reflect the dependency relationship between the entry file and the modules on which the entry file depends; respectively determine the dependency graphs corresponding to the modules on which the entry file depends.

[0120] Optionally, the determination module 11 is also specifically used to determine, for any module among the multiple modules, other modules that the any module depends on and the dependency relationship between the any module and the other modules; determine the identification information of the any module and the other modules from the dependency file corresponding to the application, the identification information including the module name and version number; determine the identification information of multiple nodes in the dependency graph corresponding to the any module, the multiple nodes correspond to the any module and the other modules, and the multiple directed edges in the dependency graph corresponding to the any module correspond to the dependency relationship between the multiple nodes.

[0121] Optionally, the determination module 11 is also specifically used to output corresponding prompt information if an abnormal event is detected in the dependency graph corresponding to any of the modules, so that the user can correct the total dependency graph based on the prompt information, and the abnormal event includes at least one of the following: an authorization abnormal event of a node, a circular dependency event between nodes, and an operation error event of a node.

[0122] Optionally, the determination module 11 is also specifically used to determine multiple construction phases according to the dependency relationship between multiple modules in any dependency path; wherein there is no dependency relationship between different modules included in the same construction phase, and the modules in the latter construction phase depend on the construction products of the previous construction phase; and determine that the container image construction plan corresponding to any dependency path includes the multiple construction phases.

[0123] Optionally, the determination module 11 is also used to determine a target module in the source code file of the application that does not exist in the target dependency path, wherein the target dependency path is a dependency path among the multiple dependency paths corresponding to the target container image build plan; and delete the target module from the source code file to load the source code file after deleting the target module into the dockerfile.

[0124] Optionally, the device also includes an update module, which is used to run the application by running the container image; obtain operation data record information generated during the operation of the application, the operation data record information includes a calling relationship between different modules; according to the calling relationship, update the total dependency graph; according to the updated total dependency graph, regenerate the container image corresponding to the application.

[0125] Figure 7 The device shown can execute the steps introduced in the aforementioned embodiments. For detailed execution process and technical effects, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0126] In one possible design, the above Figure 7 The structure of the container image building device shown can be implemented as an electronic device, such as Figure 8 As shown, the electronic device may include: a memory 21, a processor 22, and a communication interface 23. The memory 21 stores executable code, and when the executable code is executed by the processor 22, the processor 22 can at least implement the container image building method provided in the above embodiment.

[0127] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the container image building method provided in the aforementioned embodiment.

[0128] An embodiment of the present invention provides a computer program product, including: a computer program, when the computer program is executed by a processor of an electronic device, the processor is caused to execute the container image building method provided in the above embodiment.

[0129] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0130] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution can essentially or in other words be embodied in the form of a computer product, and the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for building a container image, characterized in that: include: Determine a total dependency graph corresponding to the application, where the total dependency graph is used to reflect multiple modules on which the application depends and the dependency relationships between the multiple modules; Determining a plurality of dependency paths corresponding to the total dependency graph by traversing the total dependency graph, wherein each dependency path includes each module in the total dependency graph, and the plurality of modules have different traversal orders in the plurality of dependency paths; Determine multiple container image build plans corresponding to the multiple dependency paths, and the number of image layers corresponding to the multiple container image build plans, each container image build plan being a multi-stage build plan; Determine a target container image build plan from the multiple container image build plans, wherein the number of image layers corresponding to the target container image build plan is less than the number of image layers corresponding to the other container image build plans; Generate a container image corresponding to the application according to the Dockerfile generated by the target container image building plan.

2. The method according to claim 1, characterized in that The determining of the total dependency graph corresponding to the application program includes: Obtaining a directory structure of source code files of the application, wherein the directory structure includes a plurality of modules that the application depends on; Determine the dependency graphs corresponding to each of the multiple modules; wherein the dependency graph corresponding to each module is used to reflect the dependency relationship between the corresponding module and other modules it depends on; According to the dependency graphs corresponding to the multiple modules, a total dependency graph corresponding to the application is determined.

3. The method according to claim 2, characterized in that The multiple modules include an entry file of the application; The determining the dependency graphs corresponding to the multiple modules respectively includes: Determine a dependency graph corresponding to the entry file, where the dependency graph corresponding to the entry file is used to reflect the dependency relationship between the entry file and the module on which the entry file depends; Determine the dependency graphs corresponding to the modules that the entry file depends on respectively.

4. The method according to claim 2, characterized in that: The determining the dependency graphs corresponding to the multiple modules respectively includes: For any module among the multiple modules, determine other modules that the module depends on and the dependency relationship between the module and the other modules; Determine identification information of any one module and other modules from a dependency relationship file corresponding to the application, wherein the identification information includes a module name and a version number; Determine identification information of multiple nodes in a dependency graph corresponding to any one of the modules, wherein the multiple nodes correspond to the any one of the modules and the other modules, and the multiple directed edges in the dependency graph corresponding to any one of the modules correspond to dependency relationships between the multiple nodes.

5. The method according to claim 4, characterized in that The method further comprises: If an abnormal event is detected in the dependency graph corresponding to any of the modules, the corresponding prompt information is output to enable the user to correct the total dependency graph based on the prompt information. The abnormal event includes at least one of the following: an authorization abnormal event of a node, a circular dependency event between nodes, and an operation error event of a node.

6. The method according to claim 1, characterized in that The determining of multiple container image building plans corresponding to the multiple dependency paths includes: Determine multiple construction phases according to the dependency relationship between multiple modules in any dependency path; wherein different modules included in the same construction phase do not have a dependency relationship, and a module in a later construction phase depends on a construction product of a previous construction phase; Determine that a container image build plan corresponding to any dependency path includes the multiple build phases.

7. The method according to claim 1, characterized in that The method further comprises: Determine a target module in the source code file of the application that does not exist in a target dependency path, where the target dependency path is a dependency path among the multiple dependency paths that corresponds to the target container image build plan; The target module is deleted from the source code file, so that the source code file after the target module is deleted is loaded into the Dockerfile.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Running the application by running the container image; Acquire operation data record information generated during the operation of the application, wherein the operation data record information includes the calling relationship between different modules; According to the calling relationship, updating the total dependency graph; Regenerate the container image corresponding to the application according to the updated total dependency graph.

9. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the container image building method as described in any one of claims 1 to 8.

10. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the container image building method according to any one of claims 1 to 8.

11. A computer program product, characterized in that include: A computer program, when the computer program is executed by a processor of an electronic device, causes the processor to execute the container image building method according to any one of claims 1 to 8.

Images