A method and apparatus for mirror construction

Abstract

The application discloses a mirror image construction method and device, which can fully reuse the base mirror image to save the storage space of harbor warehouse and computing nodes, and can enhance the compatibility of the mirror image after solidification. The method comprises the following steps: acquiring a software resource list, wherein the software resources in the software resource list comprise first software resources related to hardware resources and second software resources related to application tasks; generating a tree topology graph based on the software resource list, wherein the tree topology graph comprises multiple levels, each level comprises at least one node, and each node corresponds to a software resource; and constructing a mirror image corresponding to the i+1th level according to the software resources corresponding to the nodes contained in the i+1th level, wherein the mirror image corresponding to the i+1th level is constructed based on the mirror images corresponding to the levels before the i+1th level in the tree topology graph.

Description

Technical Field

[0001] This invention relates to the field of computer software technology, and in particular to a method and apparatus for image construction. Background Technology

[0002] Artificial intelligence (AI) algorithm training platforms are large-scale distributed computing platforms designed for AI training tasks. The computing nodes are typically GPU-based cloud host clusters, leveraging the advantages of distributed computing to improve model training efficiency. To provide a unified AI runtime environment, Docker technology is usually used to containerize and virtualize computing resources, building images of AI algorithm training frameworks such as TensorFlow, PyTorch, and Caffe, while simultaneously installing various high-performance computing libraries to accelerate matrix calculations.

[0003] The general workflow for algorithm engineers using this platform is as follows: prepare the dataset and training script, select a specific version of the training image on the platform, specify the startup script and parameters, and submit the task. After receiving the task, the platform uses a scheduling algorithm to allocate the task to one or more computing nodes. The node's server prepares the dataset and training script according to the passed parameters, and then pulls the image from the Docker image repository and starts training.

[0004] Because the Docker training images for current AI algorithm training platforms are very large and occupy the storage space of nodes, it takes a long time for the node servers to pull images from the Docker image repository. In addition, the complex software version dependencies for development and debugging can also easily lead to poor image compatibility. Summary of the Invention

[0005] This invention provides a method and apparatus for image building, which is used to organize the entire in-use software technology stack to build multi-level images. It can fully reuse the base image to save storage space in the Harbor repository and compute nodes, and can also enhance the compatibility of the image after the version is solidified.

[0006] In a first aspect, embodiments of the present invention provide a method for image construction, the method comprising:

[0007] Obtain a list of software resources, wherein the software resources in the list include first software resources related to hardware resources and second software resources related to application tasks;

[0008] A tree topology graph is generated based on the software resource list. The tree topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0009] Based on the software resources corresponding to the nodes contained in the (i+1)th level, a mirror image corresponding to the (i+1)th level is constructed, wherein the mirror image corresponding to the (i+1)th level is constructed based on the mirror images corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0010] This embodiment constructs multi-level images by sorting out the entire software technology stack in use. Based on the dependency relationships between the images corresponding to each level, it enables the reuse of lower-level images, saving storage space in the Harbor repository and computing nodes. Since the software version matching table is selected according to the entire software technology stack in use, the compatibility of the images can be enhanced after the versions are fixed, preventing the problem of unsuitable dependency versions being automatically installed when installing software packages. This makes the entire image environment have good compatibility for the operation of training programs.

[0011] As an optional implementation, generating a tree topology graph based on the software resource list includes:

[0012] A tree topology diagram is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource.

[0013] In the tree topology graph, the longest lifespan of the software resources corresponding to the nodes in the (i+1)th level is shorter than the shortest lifespan of the software resources corresponding to the nodes in the ith level.

[0014] As an optional implementation, generating a tree topology diagram based on the dependencies between software resources in the software resource list and the lifecycle of each software resource includes:

[0015] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0016] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0017] As an optional implementation, the tree topology includes at least three levels, wherein:

[0018] The first level is determined based on the software resources related to compilation capabilities in the first software resources;

[0019] The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages.

[0020] The third level is determined based on the second software resource;

[0021] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0022] As an optional implementation, the tree topology graph also includes a bottom-level hierarchy, which is determined in the following manner:

[0023] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0024] As an optional implementation, obtaining the software resource list includes:

[0025] The first software resource is determined based on the hardware resources used by the computer cluster;

[0026] Determine the second software resource based on the software resources required for at least one application task;

[0027] A list of software resources is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

[0028] As an optional implementation, after sequentially constructing the images corresponding to each level, the method further includes:

[0029] Upload the images corresponding to each level of the build to the image repository;

[0030] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0031] As an optional implementation, the step of sequentially constructing images corresponding to each level further includes:

[0032] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0033] As an optional implementation, the mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the i-th level in the tree topology graph; wherein, the mirror corresponding to the 1st level is constructed based on a common mirror.

[0034] Secondly, an embodiment of the present invention provides an image building device, including a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and perform the following steps:

[0035] Obtain a list of software resources, wherein the software resources in the list include first software resources related to hardware resources and second software resources related to application tasks;

[0036] A tree topology graph is generated based on the software resource list. The tree topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0037] Based on the software resources corresponding to the nodes contained in the (i+1)th level, a mirror image corresponding to the (i+1)th level is constructed, wherein the mirror image corresponding to the (i+1)th level is constructed based on the mirror images corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0038] As an optional implementation, the processor is specifically configured to execute:

[0039] A tree topology diagram is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource.

[0040] In the tree topology graph, the longest lifespan of the software resources corresponding to the nodes in the (i+1)th level is shorter than the shortest lifespan of the software resources corresponding to the nodes in the ith level.

[0041] As an optional implementation, the processor is specifically configured to execute:

[0042] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0043] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0044] As an optional implementation, the tree topology includes at least three levels, and the processor is specifically configured to execute:

[0045] The first level is determined based on the software resources related to compilation capabilities in the first software resources;

[0046] The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages.

[0047] The third level is determined based on the second software resource;

[0048] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0049] As an optional implementation, the tree topology graph also includes a bottom-level hierarchy, and the processor is specifically configured to determine the bottom-level hierarchy in the following manner:

[0050] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0051] As an optional implementation, the processor is specifically configured to execute:

[0052] The first software resource is determined based on the hardware resources used by the computer cluster;

[0053] Determine the second software resource based on the software resources required for at least one application task;

[0054] A list of software resources is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

[0055] As an optional implementation, after sequentially building the images corresponding to each level, the processor is further configured to execute:

[0056] Upload the images corresponding to each level of the build to the image repository;

[0057] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0058] As an optional implementation, the processor is further configured to execute:

[0059] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0060] As an optional implementation, the mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the i-th level in the tree topology graph; wherein, the mirror corresponding to the 1st level is constructed based on a common mirror.

[0061] Thirdly, embodiments of the present invention also provide an apparatus for image construction, the apparatus comprising:

[0062] A software acquisition unit is used to acquire a software resource list, the software resource list including a first software resource related to hardware resources and a second software resource related to application tasks;

[0063] The topology graph generation unit is used to generate a tree-like topology graph based on the software resource list, wherein the tree-like topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0064] A mirror unit is constructed to construct a mirror corresponding to the (i+1)th level based on the software resources corresponding to the nodes contained in the (i+1)th level, wherein the mirror corresponding to the (i+1)th level is constructed based on the mirrors corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0065] As an optional implementation, the topology graph generation unit is specifically used for:

[0066] A tree topology diagram is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource.

[0067] In the tree topology graph, the longest lifespan of the software resources corresponding to the nodes in the (i+1)th level is shorter than the shortest lifespan of the software resources corresponding to the nodes in the ith level.

[0068] As an optional implementation, the topology graph generation unit is specifically used for:

[0069] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0070] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0071] As an optional implementation, the tree-like topology graph includes at least three levels, and the topology graph generation unit is specifically used for:

[0072] The first level is determined based on the software resources related to compilation capabilities in the first software resources;

[0073] The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages.

[0074] The third level is determined based on the second software resource;

[0075] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0076] As an optional implementation, the tree-like topology graph further includes a bottom-level hierarchy, and the topology graph generation unit is specifically used to determine the bottom-level hierarchy in the following manner:

[0077] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0078] As an optional implementation, the software acquisition unit is specifically used for:

[0079] The first software resource is determined based on the hardware resources used by the computer cluster;

[0080] Determine the second software resource based on the software resources required for at least one application task;

[0081] A list of software resources is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

[0082] As an optional implementation, after constructing the images corresponding to each level in sequence, a reusable image unit is also included, specifically for:

[0083] Upload the images corresponding to each level of the build to the image repository;

[0084] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0085] As an optional implementation, the image building unit is further used for:

[0086] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0087] As an optional implementation, the mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the i-th level in the tree topology graph; wherein, the mirror corresponding to the 1st level is constructed based on a common mirror.

[0088] Fourthly, embodiments of the present invention also provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the method described in the first aspect above.

[0089] These or other aspects of this application will become more apparent in the following description of embodiments. Attached Figure Description

[0090] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0091] Figure 1 This is a flowchart illustrating an implementation method for image construction according to an embodiment of the present invention.

[0092] Figure 2 This is a schematic diagram of the hierarchical structure of a tree topology provided in an embodiment of the present invention;

[0093] Figure 3 A flowchart illustrating a specific implementation method for image construction provided in this embodiment of the invention;

[0094] Figure 4 A mirrored hierarchical structure diagram based on a tree topology graph is provided as an embodiment of the present invention;

[0095] Figure 5 A schematic diagram of a device for image construction provided in an embodiment of the present invention;

[0096] Figure 6 This is a schematic diagram of a mirror construction device provided in an embodiment of the present invention. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0098] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0099] In this embodiment of the invention, the term "Docker" refers to an open-source virtualization technology designed to provide an automated deployment solution for applications, enabling developers to package their applications and dependencies into a portable container.

[0100] In this embodiment of the invention, the term "container" can be regarded as a lightweight virtual machine, which is obtained by instantiating a Docker image. It is a running instance created from the image. Docker uses containers to run applications. Each container is an isolated and secure platform. A container can be regarded as a lightweight Linux runtime environment.

[0101] In this embodiment of the invention, the term "image" refers to a read-only template. An image can contain a complete operating system environment, with only the applications required by the user installed. The image does not contain any dynamic data, and its content will not be changed after it is built. Images can be used to create Docker containers, and users can use existing images on their devices to install multiple identical Docker containers.

[0102] In this embodiment of the invention, the term "Docker image" is a special file system that, in addition to providing the programs, libraries, resources, configuration files required for container runtime, also contains configuration parameters prepared for runtime.

[0103] In this embodiment of the invention, the term "mirror repository" refers to a centralized location for storing image files. After creating an image, a user can upload it to a public or private image repository. When the image needs to be used on another host, it can simply be downloaded from the image repository.

[0104] In this embodiment of the invention, the term "dependency" refers to the dependency relationship that exists in software development. Each software has different functions provided and invoked, and the dependency relationship describes the functional requirements between different software.

[0105] The application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems. In the description of this invention, unless otherwise stated, "multiple" means two or more.

[0106] Example 1: Currently, Docker allows developers to package their applications and dependencies into a lightweight, portable container, which can then be deployed on any popular Linux machine, achieving virtualization. Using Docker containers, applications are platform and hardware-insensitive. If a GPU (Graphics Processing Unit) is used in the application, the Docker engine does not natively support it because an NVIDIA driver needs to be installed, and therefore GPU resources cannot be directly accessed within the container. To solve the problem of not being able to access GPU resources within the container, the NVIDIA driver is usually reinstalled within the container, and the NVIDIA GPU is passed in as a symbolic device when the container starts. However, the problem with this approach is that the NVIDIA driver versions installed on the host machine and inside the container may be inconsistent, so the Docker image cannot be shared across multiple machines, thus negating Docker's main advantage. Therefore, to maintain the portability of Docker images while facilitating the use of GPUs, NVIDIA proposed the NVIDIA Docker solution. The overall architecture of NVIDIA Docker includes the following parts:

[0107] The server is equipped with an NVIDIA GPU.

[0108] The host machine has an operating system, CUDA driver, and Docker engine installed.

[0109] A container consists of a container OS user space, a CUDA Toolkit, and user applications.

[0110] The host machine needs to have the CUDA driver installed, and the container needs to have the CUDA toolkit installed. If the container does not need to have the CUDA driver installed, NVIDIA provides official images with different versions of the CUDA toolkit pre-installed.

[0111] Artificial intelligence (AI) algorithm training platforms are large-scale distributed computing platforms for AI training tasks. The computing nodes are typically GPU-based cloud host clusters, leveraging the advantages of distributed computing to improve model training efficiency. To provide a unified AI runtime environment, Docker technology is usually used to containerize and virtualize computing resources, building images for AI algorithm training frameworks such as TensorFlow, PyTorch, and Caffe, while installing various high-performance computing libraries to accelerate matrix calculations. The general workflow for algorithm engineers using this platform is as follows: prepare the dataset and training scripts, select a specific version of the training image on the platform, specify the startup script and parameters, and submit the task. After receiving the task, the platform uses a scheduling algorithm to allocate the task to one or more computing nodes. The node's server prepares the dataset and training scripts according to the passed parameters, then pulls the image from the Docker image repository and starts training. If the preset AI algorithm training images do not meet the user's needs, the user needs to customize the training environment, either based on an existing image or by building an image from scratch, installing and configuring the necessary software, and then uploading the image to the image repository. Because the Docker training images for current AI algorithm training platforms are very large and occupy the storage space of nodes, it takes a long time for the node servers to pull images from the Docker image repository. In addition, the complex software version dependencies for development and debugging can also easily lead to poor image compatibility.

[0112] Other common existing designs are NVIDIA's NGC images. Taking the ngc-pytorch image as an example, its main drawback is that when building a new image monthly, it selects the latest CUDA toolkit version. The compiled PyTorch version is not a standard release version tag, but rather the latest commit ID under the master branch on GitHub. Other NVIDIA GPU software is also the latest version. Typically, the driver versions, communication components, and other software versions of the compute cluster nodes will not be updated for a considerable period. As a result, the latest CUDA toolkit and underlying software may not run correctly on the node machines. Furthermore, because the image is built using an unofficial PyTorch release commit ID, compatibility issues arise between different PyTorch versions when adapting the training code.

[0113] To address the aforementioned issues, this embodiment provides a method for image construction. By organizing the currently used technology stack and building images in multiple layers, multiple base images are fully reused, saving storage space in the Harbor repository and compute nodes, and reducing the time spent by compute nodes pulling new images. Furthermore, based on the identified dependencies between images, the underlying software versions are fixed, enhancing image compatibility and allowing some older, still-used code versions to run smoothly.

[0114] like Figure 1 As shown in the figure, this embodiment provides a method for image construction, and the specific implementation process of the method is as follows:

[0115] Step 100: Obtain a software resource list, which includes a first software resource related to hardware resources and a second software resource related to application tasks;

[0116] In some embodiments, this example obtains the software resource list in the following manner:

[0117] Based on the hardware resources used by the computer cluster, a first software resource is determined; based on the software resources required for at least one application task, a second software resource is determined; based on the first and second software resources, and the dependencies between the various software resources, a list of software resources is determined.

[0118] In some embodiments, the software resource list in this embodiment may also include the dependencies between the various software resources.

[0119] During implementation, the hardware resources, software resources, dependencies between software and hardware resources, dependencies between different versions of software resources, driver versions, and dependencies between driver versions and hardware resources used by the high-performance computer cluster were first identified. Secondly, the software resources, software versions, and dependencies between various types of application tasks were identified.

[0120] The hardware and software resources in this embodiment are described in detail below:

[0121] Computer clusters typically use hardware resources such as NVIDIA GPU cards, including but not limited to any one or more of the following models: Titan-V (7.0), P4 (6.1), T4 (7.5), 1080ti (6.1), 2080ti (7.5), V100 (7.0), A40 (8.6), A30 (8.0), and A100 (8.0).

[0122] The set of CUDA operator compilation and computation capabilities is determined by hardware resources. CUDA depends on hardware resources, and different hardware models support different CUDA operator compilation and computation capabilities. The CUDA operator compilation and computation capabilities include, but are not limited to, any one or more of the following compilation capabilities: 5.2, 6.0, 6.1, 7.0, 7.5, 8.0, 8.6+PTX; where PTX (parallel thread execution) represents a specific backward compatible compilation capability.

[0123] The driver version is determined by hardware resources. Since hardware is updated over time, the driver version in the computer cluster does not change frequently. In order to ensure that the previous hardware can still run, the selected driver version includes, but is not limited to, CUDA version 10.2 and 11.1+ (the Ampere architecture GPU requires a minimum CUDA version of 11.x).

[0124] Drivers and communication software are usually packaged together for communication software used in multi-machine, multi-card training. The drivers and communication software involved in this embodiment include, but are not limited to, any one or more of the following: mlnx-ofed, nccl, openmpi, openucx, hpcx, gdrcopy, nv_peer_memory.

[0125] This embodiment also relates to high-performance computing software for accelerating computation, including but not limited to any one or more of the following: cudaToolkit, cudnn, mkl, openblas, atlas, opencv.

[0126] This embodiment also involves AI framework software resources for algorithm training, model conversion and verification, including but not limited to: PyTorch for algorithm training and Caffe for model conversion and verification.

[0127] This embodiment also relates to application software, including but not limited to open-source repository code software commonly used in fields such as object detection, tracking, classification, segmentation, recognition, and NLP.

[0128] Step 101: Generate a tree topology graph based on the software resource list, wherein the tree topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0129] In implementation, this embodiment generates a tree-like topology based on the hardware resources, software resources, and application task-related software resources used in the computer cluster. The software resources are hierarchically divided, separating the first software resources related to hardware resources from the second software resources related to application services. Since the first software resources related to hardware resources are not easily changed, they are easier to reuse as a base image. Therefore, multi-level images are constructed through this hierarchical relationship, allowing images built using lower-level software resources to be reused, saving storage resources for computing nodes and image repositories.

[0130] In some embodiments, this example generates a tree topology graph based on the software resource list in the following manner:

[0131] A tree topology graph is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource. The longest lifecycle of the software resource corresponding to the node in the (i+1)th level of the tree topology graph is shorter than the shortest lifecycle of the software resource corresponding to the node in the ith level. That is, the level of the software resource with a longer lifecycle is lower than the level of the software resource with a shorter lifecycle.

[0132] In this embodiment, the lifecycle of software resources refers to the process from the creation of software until it is scrapped or stopped being used. The specific duration of a software's use is defined in the project based on actual business needs. In this embodiment, software whose versions do not need to change frequently is considered a software resource with a long lifecycle. For example, the parent image of CUDA and cuDNN included in a phased build of an image usually does not change unless there is a defect during the build process that requires the generation of a new version. In this case, the parent image is considered a software resource with a long lifecycle.

[0133] In Linux, software dependencies are structured as a topological tree. For example, A directly or indirectly depends on B, and B in turn depends on C and D, etc. Software dependencies are complex, and currently, they are generally installed automatically by package management software. This embodiment mainly considers the dependencies between critical software, such as CUDA, cuDNN, PyTorch, and Caffe. If conflicts occur between multiple versions of commonly used or critical software that they depend on, the version that is compatible with more software versions will be selected, or the code of the software in question will be manually modified to resolve the conflict.

[0134] In some embodiments, this embodiment generates a tree topology diagram based on the dependencies between software resources in the software resource list and the lifecycle of each software resource in the software resource list as follows:

[0135] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0136] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0137] In implementation, this embodiment can generate a tree-like topology based on information of interest to the user. This information refers to the information selected by the user at the system entry point, typically including hardware resources and application tasks. For example, if the user selects hardware resources, the corresponding software resource is determined to be the CUDA version. The user can also select application tasks, determining the training framework version directly related to the user's training code based on the training task, such as PyTorch 1.7, PyTorch 1.8, PyTorch 1.9, PyTorch 1.10, or PyTorch 1.11. The user is not concerned with the operating system version used (e.g., Python version) or related dependency versions. Therefore, in actual operation, the user only needs to specify the hardware resources and application tasks. This embodiment can filter the corresponding target software resources from the software list based on the user-specified hardware resources and application tasks, thereby generating a tree-like topology using these target software resources.

[0138] Step 102: Based on the software resources corresponding to the nodes contained in the (i+1)th level, construct an image corresponding to the (i+1)th level, wherein the image corresponding to the (i+1)th level is constructed based on the images corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0139] In some embodiments, the image corresponding to the (i+1)th level is constructed based on the image corresponding to the i-th level in the tree topology graph; wherein, the image corresponding to the 1st level is constructed based on a common image.

[0140] In practice, according to the hierarchical order in the tree topology diagram, and based on the software resources represented by the nodes contained in each level, mirror images corresponding to each level are constructed sequentially, wherein the mirror image corresponding to the higher level in adjacent levels depends on the mirror image corresponding to the lower level.

[0141] In some embodiments, the tree topology diagram in this embodiment includes at least three levels, which are, from low to high, the first level, the second level, and the third level, wherein:

[0142] The first level is determined based on the software resources related to compilation capabilities in the first software resources; in practice, the first level can be determined based on the CUDA and cuDNN versions.

[0143] The second level is determined based on software resources in the first software resource that are related to at least one of the following: driver, communication, accelerated computing, and programming language; in practice, the second level can be determined based on the software packages and basic Python 3 coding language required by the AI ​​development and training platform.

[0144] The third level is determined based on the second software resource; in practice, the third level can be determined based on the cudaToolkit and pytorch versions.

[0145] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0146] In some embodiments, the tree topology graph in this embodiment further includes a bottom-level hierarchy, which is determined in the following manner:

[0147] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0148] During implementation, such as Figure 2 As shown, this embodiment provides a hierarchical structure of a tree-like topology graph, comprising five levels, from lowest to highest: bottom level, first level, second level, third level, and top level; wherein:

[0149] The lowest level image is the base image, which depends on the public image. The base image contains hardware-independent runtime libraries and development compilation toolchains, and will not change for a considerable period of time after a successful build. The public image is provided by the development vendor.

[0150] The first level corresponds to the CUDA image. The CUDA image depends on the base image, and there are multiple versions depending on the CUDA and cuDNN versions.

[0151] The second level corresponds to the basic image, which depends on the CUDA image and includes the software packages and basic Python 3 coding language required for the AI ​​development and training platform.

[0152] The third level corresponds to the PyTorch image. The PyTorch image depends on the basic image. Since there are multiple official releases of PyTorch, and the highest and lowest CUDA Toolkit versions supported by each version are fixed, as are the lowest and highest CUDA Toolkit versions supported by the compute node drivers of different graphics card models, the actual number of PyTorch images built is related to the number of CUDA Toolkits and PyTorch versions used.

[0153] The top-level image is the application image, or apps image, which is the final image released to users. The apps image depends on the PyTorch image. If the apps image contains missing or faulty software, a patch image can be built based on it to apply the patches and fix the issues.

[0154] In practice, this embodiment starts with the base image and sequentially builds the CUDA image, basic image, PyTorch image, and apps image, and uploads them to an image repository such as the Harbor repository, so as to perform functional testing on the apps image to be released.

[0155] In some embodiments, after constructing the images corresponding to each level in sequence, this embodiment further includes:

[0156] Upload the images corresponding to each level of the build to the image repository;

[0157] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0158] In practice, since the first software resource is related to hardware, and hardware usually does not change for a long period of time, with a long lifespan, and the node corresponding to the first software resource is at a lower level, the image corresponding to the level of the node corresponding to the first software resource is used as the base image for reuse. This embodiment builds images in multiple stages based on the lifespan of the software resource and the dependencies between software versions, which can fully reuse the base image and save storage space for the Harbor repository and computing nodes.

[0159] In some embodiments, when constructing images corresponding to each level sequentially, this embodiment further performs the following steps:

[0160] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0161] In practice, when any computing node in the computer cluster has at least one image that has a common target image with the image currently being processed by the computing node, the computing node uses the target image to build the at least one image, thereby reducing the time it takes for the computing node to pull the target image.

[0162] In practice, after a new image is released, when a user who selects the new version image is scheduled to a compute node, if other images existing on the compute node have a common target image dependency with the currently selected new version image, the compute node can directly reuse the target image without having to pull the target image from the image repository, thus effectively reducing the time for the compute node to pull images.

[0163] This embodiment constructs images in multiple layers according to the software lifecycle and dependencies, making full use of multiple base images. This can save storage space in the Harbor repository and compute nodes, reduce the time spent by compute nodes pulling new images, and solidify the underlying software versions based on the identified software dependencies, thereby enhancing image compatibility and enabling some older versions of code in use to run well.

[0164] like Figure 3 As shown, this embodiment also provides a specific implementation method for image construction, which involves the following steps:

[0165] Step 300: Determine the first software resource based on the hardware resources used by the computer cluster, and determine the second software resource based on the software resources required by at least one application task;

[0166] Step 301: Determine the software resource list based on the first software resource, the second software resource, and the dependencies between each software resource;

[0167] Step 302: In response to the hardware resources and application tasks specified by the user, filter out the target software resources related to the specified hardware resources and application tasks from the software resource list;

[0168] Step 303: Generate a tree topology diagram based on the dependencies between the various software resources in the target software resource and the lifecycle of each software resource;

[0169] The tree topology includes multiple levels, each level including at least one node, and the node represents a software resource. The level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource. The level of a software resource with a longer lifespan is lower than the level of a software resource with a shorter lifespan.

[0170] like Figure 4 As shown, this implementation provides a mirrored hierarchical structure diagram based on a tree-like topology graph, where the mirrors from low to high are the base mirror, CUDA mirror, basic mirror, PyTorch mirror, and apps mirror, where:

[0171] The base image includes commonly used development tools, development and compilation toolchains, matrix operation libraries, decoding libraries, etc., such as Ubuntu 18.04 and Ubuntu 20.04;

[0172] CUDA images include cudaToolkit, cudann, MLNX_OFED, Gdrcopy, HPCX, NCCL, ffmpeg, etc., such as cuda10.2, cuda11.1, and cuda11.3;

[0173] The basic image includes Python 3.8, basic packages, OpenCV, Libboost, etc.

[0174] The PyTorch mirror includes tools related to PyTorch, Caffe, Torchvision, Apex, Horovod, DeepSpeed, and ONNX, such as PyTorch 1.7, PyTorch 1.8, PyTorch 1.9, PyTorch 1.10, and PyTorch 1.11.

[0175] The apps mirror includes open-source repository code dependencies for fields such as object detection, tracking, classification, segmentation, recognition, NLP, and OCR; the specific dependencies will be determined based on the actual project.

[0176] Step 304: According to the hierarchical order in the tree topology diagram, construct mirror images corresponding to each level in sequence based on the software resources represented by the nodes contained in each level;

[0177] In this context, the mirror image corresponding to a higher level in an adjacent hierarchy depends on the mirror image corresponding to a lower level.

[0178] Step 305: Upload the images corresponding to each level to the image repository;

[0179] Step 306: When there is a need to build a new image based on the first software resource, a new image is built by reusing the images corresponding to the preset levels in the image repository;

[0180] The preset level includes the level where the node corresponding to the first software resource is located.

[0181] Step 307: When at least one image exists on any computing node in the computer cluster and the image currently being processed by the computing node is built based on a common target image, the computing node uses the target image to build the at least one image.

[0182] This embodiment analyzes the entire currently used software technology stack and builds images in multiple stages according to user focus (user-specified hardware resources and application tasks), software lifecycle, and software version dependencies. This allows for full reuse of the base image, saving storage space in the Harbor repository and compute nodes. After a new image is released, users who select the new version image are scheduled to compute nodes. When other images on the compute node have dependencies on the currently selected image, the image retrieval time can be significantly reduced. Because the software version compatibility table is selected according to the entire currently used software technology stack, the image compatibility is enhanced after the version is fixed, preventing the problem of inappropriate dependency versions being automatically installed when installing software packages. This ensures that the entire image environment has good compatibility for the operation of training programs.

[0183] Example 2: Based on the same inventive concept, this embodiment of the invention also provides a device for mirror construction. Since this device is the same as the device in the method of this embodiment of the invention, and the principle of the device in solving the problem is similar to that of the method, the implementation of this device can refer to the implementation of the method, and the repeated parts will not be described again.

[0184] like Figure 5 As shown, the device includes a processor 500 and a memory 501. The memory 501 stores programs executable by the processor 500. The processor 500 reads the programs from the memory 501 and performs the following steps:

[0185] Obtain a list of software resources, wherein the software resources in the list include first software resources related to hardware resources and second software resources related to application tasks;

[0186] A tree topology graph is generated based on the software resource list. The tree topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0187] Based on the software resources corresponding to the nodes contained in the (i+1)th level, a mirror image corresponding to the (i+1)th level is constructed, wherein the mirror image corresponding to the (i+1)th level is constructed based on the mirror images corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0188] As an optional implementation, the processor 500 is specifically configured to perform:

[0189] A tree topology diagram is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource.

[0190] In the tree topology graph, the longest lifespan of the software resources corresponding to the nodes in the (i+1)th level is shorter than the shortest lifespan of the software resources corresponding to the nodes in the ith level.

[0191] As an optional implementation, the processor 500 is specifically configured to perform:

[0192] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0193] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0194] As an optional implementation, the tree topology includes at least three levels, and the processor 500 is specifically configured to execute:

[0195] The first level is determined based on the software resources related to compilation capabilities in the first software resources;

[0196] The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages.

[0197] The third level is determined based on the second software resource;

[0198] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0199] As an optional implementation, the tree topology graph also includes a bottom-level hierarchy, and the processor 500 is specifically configured to determine the bottom-level hierarchy in the following manner:

[0200] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0201] As an optional implementation, the processor 500 is specifically configured to perform:

[0202] The first software resource is determined based on the hardware resources used by the computer cluster;

[0203] Determine the second software resource based on the software resources required for at least one application task;

[0204] A list of software resources is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

[0205] As an optional implementation, after constructing the images corresponding to each level in sequence, the processor 500 is further configured to execute:

[0206] Upload the images corresponding to each level of the build to the image repository;

[0207] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0208] As an optional implementation, the processor 500 is further configured to perform:

[0209] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0210] As an optional implementation, the mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the i-th level in the tree topology graph; wherein, the mirror corresponding to the 1st level is constructed based on a common mirror.

[0211] Example 3: Based on the same inventive concept, this embodiment of the invention also provides a mirror construction device. Since this device is the same as the device in the method of this embodiment of the invention, and the principle of the device in solving the problem is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0212] like Figure 6 As shown, the device includes:

[0213] Software unit 600 is used to acquire a software resource list, the software resource list including a first software resource related to hardware resources and a second software resource related to application tasks;

[0214] The topology graph generation unit 601 is used to generate a tree-like topology graph based on the software resource list, wherein the tree-like topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0215] A mirror unit 602 is used to construct a mirror corresponding to the (i+1)th level based on the software resources corresponding to the nodes contained in the (i+1)th level, wherein the mirror corresponding to the (i+1)th level is constructed based on the mirrors corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0216] As an optional implementation, the topology graph generation unit 601 is specifically used for:

[0217] A tree topology diagram is generated based on the dependencies between the software resources in the software resource list and the lifecycle of each software resource.

[0218] In the tree topology graph, the longest lifespan of the software resources corresponding to the nodes in the (i+1)th level is shorter than the shortest lifespan of the software resources corresponding to the nodes in the ith level.

[0219] As an optional implementation, the topology graph generation unit 601 is specifically used for:

[0220] In response to the hardware resources and application tasks specified by the user, target software resources related to the specified hardware resources and application tasks are filtered from the list of software resources;

[0221] A tree-like topology diagram is generated based on the dependencies between various software resources in the target software resource and the lifecycle of each software resource.

[0222] As an optional implementation, the tree-like topology graph includes at least three levels, and the topology graph generation unit 601 is specifically used for:

[0223] The first level is determined based on the software resources related to compilation capabilities in the first software resources;

[0224] The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages.

[0225] The third level is determined based on the second software resource;

[0226] The levels, from lowest to highest, are the first level, the second level, and the third level.

[0227] As an optional implementation, the tree-like topology graph further includes a bottom-level hierarchy, and the topology graph generation unit 601 is specifically used to determine the bottom-level hierarchy in the following manner:

[0228] The lowest level of the tree topology graph is generated based on the hardware-independent runtime library and development compilation toolchain; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

[0229] As an optional implementation, the software acquisition unit 600 is specifically used for:

[0230] The first software resource is determined based on the hardware resources used by the computer cluster;

[0231] Determine the second software resource based on the software resources required for at least one application task;

[0232] A list of software resources is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

[0233] As an optional implementation, after constructing the images corresponding to each level in sequence, a reusable image unit is also included, specifically for:

[0234] Upload the images corresponding to each level of the build to the image repository;

[0235] When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

[0236] As an optional implementation, the image building unit 602 is further used for:

[0237] When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

[0238] As an optional implementation, the mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the i-th level in the tree topology graph; wherein, the mirror corresponding to the 1st level is constructed based on a common mirror.

[0239] Based on the same inventive concept, embodiments of the present invention also provide a computer storage medium storing a computer program, which, when executed by a processor, performs the following steps:

[0240] Obtain a list of software resources, wherein the software resources in the list include first software resources related to hardware resources and second software resources related to application tasks;

[0241] A tree topology graph is generated based on the software resource list. The tree topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource.

[0242] Based on the software resources corresponding to the nodes contained in the (i+1)th level, a mirror image corresponding to the (i+1)th level is constructed, wherein the mirror image corresponding to the (i+1)th level is constructed based on the mirror images corresponding to the levels preceding the (i+1)th level in the tree topology graph, and i is an integer greater than 0.

[0243] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0244] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0245] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0246] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0247] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for constructing an image, characterized in that, The method includes: Obtain a software resource list, wherein the software resources in the software resource list include first software resources related to hardware resources and second software resources related to application tasks; the software resource list includes hardware resources used by the high-performance computer cluster, software resources, dependencies between software resources and hardware resources, dependencies between different versions of software resources, driver versions, dependencies between driver versions and hardware resources; and software resources used by various types of application tasks, software versions, and dependencies between software versions. A tree-like topology graph is generated based on the software resource list, wherein the tree-like topology graph includes multiple levels, each level includes at least one node, each node corresponds to a software resource, and the level of the node corresponding to the first software resource is lower than the level of the node corresponding to the second software resource; the generation of the tree-like topology graph based on the software resource list includes: generating the tree-like topology graph according to the dependencies between the software resources in the software resource list and the lifecycle of each software resource, including: in response to the hardware resources and application tasks specified by the user, filtering out the target software resources related to the specified hardware resources and application tasks from the software resource list; generating the tree-like topology graph according to the dependencies between the software resources in the target software resources and the lifecycle of each software resource; wherein the longest lifecycle of the software resources corresponding to the nodes contained in the (i+1)th level of the tree-like topology graph is less than the shortest lifecycle of the software resources corresponding to the nodes contained in the ith level; Mirrors corresponding to each level are constructed sequentially. Based on the software resources corresponding to the nodes contained in the (i+1)th level, a mirror corresponding to the (i+1)th level is constructed. The mirror corresponding to the (i+1)th level is constructed based on the mirrors corresponding to the levels before the (i+1)th level in the tree topology graph, where i is an integer greater than 0.

2. The method according to claim 1, characterized in that, The tree topology graph includes at least 3 levels, wherein: The first level is determined based on the software resources related to compilation capabilities in the first software resources; The second level is determined based on the software resources in the first software resources that are related to at least one of the following: drivers, communication, accelerated computing, and programming languages. The third level is determined based on the second software resource; The levels, from lowest to highest, are the first level, the second level, and the third level.

3. The method according to claim 1 or 2, characterized in that, The tree topology graph also includes a bottom-level hierarchy, which is determined as follows: The lowest level of the tree topology graph is generated based on hardware-independent runtime libraries and development compilation toolchains; wherein the lowest level is lower than the level of the node corresponding to the first software resource.

4. The method according to claim 1, characterized in that, The obtained software resource list includes: The first software resource is determined based on the hardware resources used by the computer cluster; Determine the second software resource based on the software resources required for at least one application task; The software resource list is determined based on the first software resource, the second software resource, and the dependencies between the various software resources.

5. The method according to claim 1, characterized in that, After constructing the images corresponding to each level in sequence, the process also includes: Upload the images corresponding to each level of the build to the image repository; When there is a need to build a new image based on the first software resource, the new image is built by reusing the images corresponding to the preset levels in the image repository, wherein the preset levels include the level where the node corresponding to the first software resource is located.

6. The method according to claim 1, characterized in that, The process of sequentially constructing images corresponding to each level also includes: When any compute node in a computer cluster has at least one image that is built based on a common target image and the image currently being processed by the compute node, the compute node uses the target image to build the at least one image, thereby reducing the time it takes for the compute node to pull the target image.

7. The method according to claim 1, characterized in that, The mirror corresponding to the (i+1)th level is constructed based on the mirror corresponding to the ith level in the tree topology graph; wherein, the mirror corresponding to the ith level is constructed based on the common mirror.

8. A device for image construction, characterized in that, The device includes a processor and a memory for storing a program executable by the processor, and the processor for reading the program in the memory and performing the steps of the method according to any one of claims 1 to 7.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images