Model compiling auxiliary method and device, computer equipment, readable storage medium and program product
By deploying compiler containers and compilation controllers in the container orchestration platform and using multiple GPU devices for model compilation, the problem of time-consuming and insufficient resource utilization of deep learning model compilation process is solved, and efficient compilation and optimization are achieved.
Patent Information
- Application Number
- CN202411083596.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-08
AI Technical Summary
The compilation and optimization process of deep learning models takes a long time on a single node, and cannot fully utilize distributed computing resources, resulting in low compilation efficiency and inability to meet the needs of efficient compilation.
Deploy a compiler container in the container orchestration platform, use the compilation controller to pre-run model compilation tasks on multiple GPU devices, obtain the compilation results and model weights, and save them to cloud storage to form a model image, and respond to the client's task pre-order Execute instructions for precompilation and optimization model sending.
It significantly improves the compilation efficiency and resource utilization of the model, supports unified compilation and optimization of multiple hardware architectures, and meets the needs of efficient compilation.
Smart Images

Figure CN118963725A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a model compilation auxiliary method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art
[0002] With the development of computer technology, deep learning models have emerged to realize artificial intelligence tasks and applications. However, due to the increasing size and complexity of deep learning models, the compilation and optimization process of the models has become more and more complicated.
[0003] In traditional technologies, the compilation and optimization process of the model is usually performed on a single node. Common compilation tools include TVM and TensorRT. These tools can effectively improve the reasoning performance of the model on a single node. However, when faced with large-scale models and diversified hardware architectures, the compilation and optimization process is often time-consuming and cannot fully utilize distributed computing resources, resulting in low compilation efficiency and failure to meet the needs of efficient compilation. Summary of the invention
[0004] Based on this, it is necessary to provide a model compilation auxiliary method, device, computer equipment, computer readable storage medium and computer program product to address the above technical problems.
[0005] In a first aspect, the present application provides a model compilation auxiliary method, comprising:
[0006] Obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container;
[0007] A compilation controller is added to the container orchestration platform, and the target model compilation task in the task container is pre-run on various types of GPU devices by using the compilation controller to obtain the compilation results and model weights output by various types of GPU devices;
[0008] Saving the compilation result and the model weight to cloud storage to form a corresponding model image;
[0009] In response to a task pre-execution instruction sent by the client, the model image is obtained from the cloud storage, and the target model compilation task is pre-executed according to the model image to obtain a pre-compiled model;
[0010] The precompiled model is sent to the client.
[0011] In one embodiment, before obtaining the model compilation task to be executed, the method further includes:
[0012] A compiler container is deployed in the container orchestration platform, and the compiler container is installed on various types of the GPU devices; and according to a preset device management mechanism, the GPU device including the compiler container is registered in the container orchestration platform.
[0013] In one embodiment, obtaining a target model compilation task that needs to use GPU resources according to the task type of the model compilation task includes:
[0014] According to the task type of the model compilation task, determine whether the model compilation task needs to use the GPU resources to obtain a corresponding determination result; according to the determination result, identify the target model compilation task in the model compilation task that needs to use the GPU resources.
[0015] In one of the embodiments, before the target model compilation task is pre-executed according to the model image to obtain the pre-compiled model, the method further includes: identifying idle nodes in the container orchestration platform according to the node operation status in the container orchestration platform;
[0016] Pre-executing the target model compilation task according to the model image to obtain a pre-compiled model includes: deploying the model image to the idle node of the container orchestration platform, and pre-executing the target model compilation task to obtain a task pre-execution result; and obtaining the pre-compiled model according to the task pre-execution result.
[0017] In one embodiment, before sending the precompiled model to the client, the method further includes: collecting the user's identity information and access permission information, and verifying the identity information and the access permission information;
[0018] The sending the precompiled model to the client includes: when both the identity information and the access permission information are verified, sending the precompiled model to the client.
[0019] In one embodiment, the step of saving the compilation result and the model weight to cloud storage to form a corresponding model image includes:
[0020] According to the identification information of the target model compilation task, the target information entry in the cloud storage is matched; the compilation result and the model weight are entered into the target information entry in the cloud storage, and the model image is formed.
[0021] In a second aspect, the present application also provides a model compilation auxiliary device, comprising:
[0022] A resource management module is used to obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container;
[0023] A compilation control module is used to add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain the compilation results and model weights output by the various types of GPU devices;
[0024] An image generation module, used to save the compilation result and the model weight to cloud storage to form a corresponding model image;
[0025] An image management module, configured to obtain the model image from the cloud storage in response to a task pre-execution instruction sent by the client, and pre-execute the target model compilation task according to the model image to obtain a pre-compiled model;
[0026] The result sending module is used to send the precompiled model to the client.
[0027] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0028] Obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container; add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain compilation results and model weights output by various types of GPU devices; save the compilation results and the model weights into cloud storage to form a corresponding model image; in response to a task pre-execution instruction sent by a client, obtain the model image from the cloud storage, pre-execute the target model compilation task according to the model image, and obtain a pre-compiled model; and send the pre-compiled model to the client.
[0029] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0030] Obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container; add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain compilation results and model weights output by various types of GPU devices; save the compilation results and the model weights into cloud storage to form a corresponding model image; in response to a task pre-execution instruction sent by a client, obtain the model image from the cloud storage, pre-execute the target model compilation task according to the model image, and obtain a pre-compiled model; and send the pre-compiled model to the client.
[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0032] Obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container; add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain compilation results and model weights output by various types of GPU devices; save the compilation results and the model weights into cloud storage to form a corresponding model image; in response to a task pre-execution instruction sent by a client, obtain the model image from the cloud storage, pre-execute the target model compilation task according to the model image, and obtain a pre-compiled model; and send the pre-compiled model to the client.
[0033] The above-mentioned model compilation auxiliary method, device, computer equipment, computer-readable storage medium and computer program product, by adding a compilation controller in the container orchestration platform deployed with the compiler container, pre-running the target model compilation task in the task container on various types of GPU devices using the compilation controller, obtains the compilation results and model weights output by various types of GPU devices; then saves the compilation results and model weights in the cloud storage to form a corresponding model image; in response to the task pre-execution instruction sent by the client, obtains the model image from the cloud storage, pre-executes the target model compilation task according to the model image, and obtains the pre-compiled model. This solution makes full use of the resource management and scheduling capabilities of the container orchestration platform, can automatically allocate and manage and fully utilize distributed computing resources, and supports unified compilation and optimization of multiple hardware architectures, thereby significantly improving the compilation efficiency and resource utilization of the model, and effectively meeting the needs of efficient compilation. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.
[0035] FIG1 is a diagram of an application environment of a model compilation auxiliary method in one embodiment;
[0036] Figure 2 A schematic diagram of a flow chart of a model compilation auxiliary method in one embodiment;
[0037] Figure 3 A schematic diagram of a process of containerized deployment of a compiler in one embodiment;
[0038] Figure 4 A schematic diagram of a flow chart of a model compilation auxiliary method in a specific embodiment;
[0039] Figure 5 A structural block diagram of a model compilation auxiliary device in one embodiment;
[0040] FIG. 6 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0042] The model compilation auxiliary method provided in the embodiment of the present application can be applied to the application environment shown in Figure 1. Among them, the client communicates with the server through the network. The data storage system can store the data that the server needs to process. The data storage system can be integrated on the server, or it can be placed on the cloud or other network servers.
[0043] Specifically, the model compilation assistance method provided in the embodiment of the present application can be executed by a server.
[0044] Exemplarily, the server obtains a model compilation task to be executed, obtains a target model compilation task that requires GPU resources based on the task type of the model compilation task, and mounts the target model compilation task into a task container; the server adds a compilation controller to the container orchestration platform, and uses the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain the compilation results and model weights output by various types of GPU devices; the server saves the compilation results and model weights to cloud storage to form a corresponding model image; in response to a task pre-execution instruction sent by the client, the server obtains the model image from the cloud storage, pre-executes the target model compilation task based on the model image, and obtains the pre-compiled model; the server sends the pre-compiled model to the client.
[0045] In the application environment shown in FIG1 , the client may be, but is not limited to, various personal computers, laptops, smart phones, and tablet computers. The server may be implemented as an independent server or a server cluster consisting of multiple servers.
[0046] In one embodiment, Figure 2 As shown, a model compilation auxiliary method is provided, which is described by taking the method applied to the server in FIG1 as an example, and includes the following steps:
[0047] Step S201, obtain the model compilation task to be executed, obtain the target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into the task container.
[0048] Among them, the model compilation task usually refers to converting the deep learning model into an executable, optimized form for fast inference or execution on the target device.
[0049] The task type of model compilation tasks can be classified based on whether they are computationally intensive tasks. If the model compilation task requires a large amount of numerical calculations, matrix operations, and machine learning model training, it can usually benefit from the parallel computing capabilities of the GPU.
[0050] Specifically, the server obtains the model compilation task to be executed, identifies the target model compilation task that needs to use GPU resources in the model compilation task according to the task type of the model compilation task, mounts the target model compilation task and related file resources into the task container, and configures related environment variables.
[0051] Step S202, adding a compilation controller in the container orchestration platform, using the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices, and obtain the compilation results and model weights output by various types of GPU devices.
[0052] Among them, the container orchestration platform can be a Kubernetes (K8s) cluster, which is specifically an open source container orchestration engine that can be used to automate the deployment, expansion, and management of containerized applications.
[0053] A compiler controller can be a part of a control system that manages resources and tasks in a cluster to achieve a specific goal or function. In deep learning, it can refer to certain parts of a neural network model, such as the controller unit in a recurrent neural network (RNN), which manages the processing of sequence data.
[0054] Specifically, the server adds a compilation controller in the container orchestration platform, and uses the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices, and obtain the compilation results and model weights output by various types of GPU devices in advance.
[0055] Step S203, saving the compilation results and model weights to cloud storage to form a corresponding model image.
[0056] Among them, a model image can be a lightweight, executable software package that contains everything required for the software to run, including code, runtime, libraries, and configuration files.
[0057] Specifically, the server saves the compilation results and model weights to cloud storage and makes a corresponding model image.
[0058] Step S204, in response to the task pre-execution instruction sent by the client, obtain the model image from the cloud storage, pre-execute the target model compilation task according to the model image, and obtain the pre-compiled model.
[0059] Among them, cloud storage generally refers to storing data on a remote server connected via the Internet, rather than storing it on a local computer or traditional local storage device. This storage method allows users to access their stored data through the network and to access and manage this data from multiple devices.
[0060] Specifically, the server responds to the task pre-execution instruction sent by the client, obtains the model image from the cloud storage, deploys the model image to the idle node of the container orchestration platform, and pre-executes the target model compilation task to obtain the task pre-execution result; then, based on the task pre-execution result, the pre-compiled model is obtained.
[0061] Step S205: sending the precompiled model to the client.
[0062] Specifically, the server sends the precompiled model to the client, so that the client can display the precompiled model.
[0063] In the above-mentioned model compilation auxiliary method, a compilation controller is added to the container orchestration platform where the compiler container is deployed, and the compilation controller is used to pre-run the target model compilation task in the task container on various types of GPU devices to obtain the compilation results and model weights output by various types of GPU devices; the compilation results and model weights are then saved in the cloud storage to form a corresponding model image; in response to the task pre-execution instruction sent by the client, the model image is obtained from the cloud storage, and the target model compilation task is pre-executed according to the model image to obtain the pre-compiled model. This solution makes full use of the resource management and scheduling capabilities of the container orchestration platform, can automatically allocate and manage and fully utilize distributed computing resources, and supports unified compilation and optimization of multiple hardware architectures, thereby significantly improving the compilation efficiency and resource utilization of the model, and effectively meeting the needs of efficient compilation.
[0064] In one embodiment, if Figure 3 As shown, before obtaining the model compilation task to be executed, the method of the present application also includes the following steps:
[0065] Step S301: deploy a compiler container in the container orchestration platform, and install the compiler container on various types of GPU devices.
[0066] Step S302: register the GPU device including the compiler container into the container orchestration platform according to a preset device management mechanism.
[0067] Among them, the compiler container can be used to compile the trained AI model into high-performance backend code to improve the reasoning performance of the software tool. The container can be customized and optimized based on the existing AI compiler software (such as TVM, TensorRT, etc.) to support multiple hardware architectures.
[0068] The device management mechanism may be a device plugin mechanism, which can be used to manage and allocate special hardware resources in the cluster, such as GPUs, FPGAs, and network devices.
[0069] Specifically, the server deploys the compiler container in the K8s cluster and installs the compiler container on the nodes of various types of GPU devices to ensure that each node has the ability to compile AI models; then, according to the deviceplugin mechanism in the K8s cluster, the GPU device containing the compiler container is registered with the K8s cluster.
[0070] In this embodiment, through the containerized deployment of the compiler and the registration of the GPU device, each node of the GPU device has the AI model compilation capability, and the K8s cluster can identify and manage these GPU devices.
[0071] In one embodiment, in the above step S201, obtaining a target model compilation task that needs to use GPU resources according to the task type of the model compilation task specifically includes the following steps:
[0072] According to the task type of the model compilation task, whether the model compilation task needs to use GPU resources is judged to obtain a corresponding judgment result; according to the judgment result, a target model compilation task that needs to use GPU resources in the model compilation task is identified.
[0073] Specifically, the server determines the task type of the model compilation task based on whether it is a computationally intensive task, and then determines whether the model compilation task needs to use GPU resources based on the task type of the model compilation task to obtain a corresponding judgment result; then, based on the judgment result, the server identifies the target model compilation task that needs to use GPU resources in the model compilation task.
[0074] Among them, computationally intensive tasks can be large-scale data processing, scientific computing, and numerical calculations of complex models, which can usually benefit from the parallel computing capabilities of GPUs.
[0075] In this embodiment, by determining the task type of the model compilation task, the target model compilation task that needs to use GPU resources in the model compilation task is accurately identified, which is beneficial to improving the resource utilization of distributed computing resources.
[0076] In one embodiment, before pre-executing the target model compilation task according to the model image to obtain the pre-compiled model, the method of the present application further includes the following steps:
[0077] According to the running status of nodes in the container orchestration platform, identify idle nodes in the container orchestration platform;
[0078] In the above step S204, the target model compilation task is pre-executed according to the model image to obtain a pre-compiled model, which specifically includes the following steps:
[0079] The model image is deployed to an idle node of the container orchestration platform, and the target model compilation task is pre-executed to obtain the task pre-execution result; based on the task pre-execution result, the pre-compiled model is obtained.
[0080] Specifically, the server identifies idle nodes in the container orchestration platform based on the operation status of the nodes in the container orchestration platform; then deploys the model image to the idle nodes of the container orchestration platform, and pre-executes the target model compilation task to obtain the task pre-execution result; then obtains the pre-compiled model based on the task pre-execution result.
[0081] In this embodiment, when the user chooses to deploy a large model, the server will pull the model image file from the cloud storage and deploy it to the idle node scheduled by the K8s cluster; through the mirroring method, the user can quickly pull the pre-compiled and optimized model, reducing deployment time and complexity, which is conducive to improving the compilation efficiency of the model.
[0082] In one embodiment, before sending the precompiled model to the client, the method of the present application further includes the following steps:
[0083] Collect the user's identity information and access permission information, and verify the identity information and access permission information;
[0084] In the above step S205, the precompiled model is sent to the client, which specifically includes the following steps:
[0085] When both the identity information and access permission information are verified, the precompiled model is sent to the client.
[0086] The user's identity information may include the user's ID number, mobile phone number, and user's name, etc.
[0087] Specifically, the server collects the user's identity information and access permission information, and verifies the identity information and access permission information; then, when it is identified that the identity information and access permission information are both verified, the precompiled model is sent to the client.
[0088] In this embodiment, the user's identity information and access permission information are verified, and when both the identity information and the access permission information are verified, the precompiled model is sent to the client, thereby improving the security and reliability of user use.
[0089] In one embodiment, in the above step S203, the compilation result and the model weight are saved in the cloud storage to form a corresponding model image, which specifically includes the following steps:
[0090] According to the identification information of the target model compilation task, the target information entry in the cloud storage is matched; the compilation result and the model weight are entered into the target information entry in the cloud storage, and a model image is formed.
[0091] Among them, an information entry generally refers to a short information unit arranged according to certain rules in a book, document, database or encyclopedia, similar to a title or a section.
[0092] Specifically, the server matches the target information entry in the cloud storage according to the identification information of the target model compilation task; then enters the compilation result and model weight into the target information entry in the cloud storage, and forms a model image.
[0093] In this embodiment, by matching the target information entries in the cloud storage according to the identification information of the target model compilation task, and entering the compilation results and model weights into the target information entries in the cloud storage, the compilation results and model weights are entered into the target information entries in the cloud storage in a regular and orderly manner based on the identification information, which is beneficial to the efficient storage and utilization of model images.
[0094] In one embodiment, Figure 4 As shown, a model compilation auxiliary method in a specific embodiment is provided, which specifically includes the following steps:
[0095] Step S401, deploying a compiler container in a container orchestration platform, and installing the compiler container on various types of GPU devices; and registering the GPU device including the compiler container in the container orchestration platform according to a preset device management mechanism.
[0096] Step S402, obtain the model compilation task to be executed, and determine whether the model compilation task needs to use GPU resources based on the task type of the model compilation task, and obtain the corresponding judgment result; based on the judgment result, identify the target model compilation task that needs to use GPU resources in the model compilation task, and mount the target model compilation task into the task container.
[0097] Step S403, adding a compilation controller in the container orchestration platform, using the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices, and obtain the compilation results and model weights output by various types of GPU devices.
[0098] Step S404, matching the target information entry in the cloud storage according to the identification information of the target model compilation task; entering the compilation result and the model weight into the target information entry in the cloud storage, and forming a model image.
[0099] Step S405, in response to the task pre-execution instruction sent by the client, obtain the model image from the cloud storage; identify the idle nodes in the container orchestration platform according to the node operation status in the container orchestration platform; deploy the model image to the idle nodes in the container orchestration platform, and pre-execute the target model compilation task to obtain the task pre-execution result; obtain the pre-compiled model according to the task pre-execution result.
[0100] Step S406, collecting the user's identity information and access permission information, and verifying the identity information and access permission information; when both the identity information and access permission information are verified, sending the precompiled model to the client.
[0101] The above embodiment solves the problem that existing compilers are difficult to compile and optimize efficiently under various hardware types by introducing a compiler container and adding a compilation controller in the K8s cluster. Specific effects include:
[0102] 1) Improve compilation efficiency: Through distributed compilation, the compilation time of large-scale models can be significantly reduced, and the development and optimization efficiency can be improved.
[0103] 2) Improve resource utilization: Make full use of the resource management and scheduling capabilities of the K8s cluster to achieve efficient management and use of computing resources.
[0104] 3) Improve hardware compatibility: Support unified compilation and optimization of multiple hardware architectures, reducing complexity and errors in the compilation process.
[0105] 4) Simplify the deployment process: Through image management and automatic deployment of server tasks, the model deployment and usage process is simplified and the user experience is improved.
[0106] It should be noted that in practical applications, some parts of this technical solution can be replaced and adjusted according to specific needs, for example:
[0107] 1) Implementation of compiler containers: They can be customized and optimized according to different compiler software (such as TVM, TensorRT, etc.) to adapt to more hardware architectures and model types.
[0108] 2) Implementation of compilation controller: It can be dynamically adjusted according to different GPU models and user needs to improve the flexibility and efficiency of compilation and optimization.
[0109] 3) Image management method: The image generation and management method can be adjusted according to specific deployment requirements to adapt to different application scenarios.
[0110] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0111] Based on the same inventive concept, the embodiment of the present application also provides a model compilation assisting device for implementing the model compilation assisting method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the model compilation assisting device provided below can refer to the limitations of the model compilation assisting method above, and will not be repeated here.
[0112] In an exemplary embodiment, Figure 5 As shown, a model compilation auxiliary device is provided, comprising:
[0113] The resource management module 501 is used to obtain the model compilation task to be executed, obtain the target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into the task container;
[0114] The compilation control module 502 is used to add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain the compilation results and model weights output by various types of GPU devices;
[0115] The image generation module 503 is used to save the compilation results and model weights to the cloud storage to form a corresponding model image;
[0116] The image management module 504 is used to obtain the model image from the cloud storage in response to the task pre-execution instruction sent by the client, and pre-execute the target model compilation task according to the model image to obtain the pre-compiled model;
[0117] The result sending module 505 is used to send the precompiled model to the client.
[0118] In one embodiment, the model compilation auxiliary device also includes a container registration module, which is used to deploy a compiler container in a container orchestration platform and install the compiler container on various types of GPU devices; according to a preset device management mechanism, the GPU device containing the compiler container is registered in the container orchestration platform.
[0119] In one embodiment, the resource management module 501 is also used to determine whether the model compilation task needs to use GPU resources according to the task type of the model compilation task, and obtain a corresponding judgment result; based on the judgment result, identify the target model compilation task that needs to use GPU resources in the model compilation task.
[0120] In one embodiment, the model compilation auxiliary device also includes a node identification module, which is used to identify idle nodes in the container orchestration platform according to the node operation status in the container orchestration platform; the image management module 504 is also used to deploy the model image to the idle nodes of the container orchestration platform, and pre-execute the target model compilation task to obtain the task pre-execution result; according to the task pre-execution result, the pre-compiled model is obtained.
[0121] In one embodiment, the model compilation auxiliary device also includes an information verification module for collecting the user's identity information and access permission information, and verifying the identity information and access permission information; the result sending module 505 is also used to send the pre-compiled model to the client when the identity information and access permission information are verified.
[0122] In one embodiment, the image generation module 503 is also used to match the target information entry in the cloud storage according to the identification information of the target model compilation task; enter the compilation results and model weights into the target information entry in the cloud storage, and form a model image.
[0123] Each module in the above-mentioned model compilation auxiliary device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.
[0124] In an exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store compilation results and model weight data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a model compilation auxiliary method is implemented.
[0125] Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0126] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0127] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0128] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0129] It should be noted that the client information (including but not limited to client device information, client personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the client or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0130] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.
[0131] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0132] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A model compilation auxiliary method, characterized in that: The method comprises: Obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container; A compilation controller is added to the container orchestration platform, and the target model compilation task in the task container is pre-run on various types of GPU devices by using the compilation controller to obtain the compilation results and model weights output by various types of GPU devices; Saving the compilation result and the model weight to cloud storage to form a corresponding model image; In response to a task pre-execution instruction sent by the client, the model image is obtained from the cloud storage, and the target model compilation task is pre-executed according to the model image to obtain a pre-compiled model; The precompiled model is sent to the client.
2. The method according to claim 1, characterized in that Before obtaining the model compilation task to be executed, the method further includes: Deploy a compiler container in the container orchestration platform, and install the compiler container on various types of GPU devices; According to a preset device management mechanism, the GPU device including the compiler container is registered in the container orchestration platform.
3. The method according to claim 1, characterized in that The step of obtaining a target model compilation task that needs to use GPU resources according to the task type of the model compilation task includes: According to the task type of the model compilation task, determine whether the model compilation task needs to use the GPU resources, and obtain a corresponding determination result; According to the determination result, the target model compilation task that needs to use the GPU resources is identified among the model compilation tasks.
4. The method according to claim 1, characterized in that: Before pre-executing the target model compilation task according to the model image to obtain the pre-compiled model, the method further includes: Identify idle nodes in the container orchestration platform according to the node operation status in the container orchestration platform; The pre-executing the target model compilation task according to the model image to obtain a pre-compiled model includes: Deploy the model image to the idle node of the container orchestration platform, and pre-execute the target model compilation task to obtain a task pre-execution result; The precompiled model is obtained according to the task pre-execution result.
5. The method according to claim 1, characterized in that Before sending the precompiled model to the client, the method further includes: Collecting the user's identity information and access permission information, and verifying the identity information and the access permission information; The sending the precompiled model to the client includes: When both the identity information and the access permission information are verified, the precompiled model is sent to the client.
6. The method according to any one of claims 1 to 5, characterized in that: The step of saving the compilation result and the model weight to cloud storage to form a corresponding model image includes: Matching the target information entry in the cloud storage according to the identification information of the target model compilation task; The compilation result and the model weight are entered into the target information entry of the cloud storage to form the model image.
7. A model compilation auxiliary device, characterized in that: The device comprises: A resource management module is used to obtain a model compilation task to be executed, obtain a target model compilation task that needs to use GPU resources according to the task type of the model compilation task, and mount the target model compilation task into a task container; A compilation control module is used to add a compilation controller in the container orchestration platform, and use the compilation controller to pre-run the target model compilation task in the task container on various types of GPU devices to obtain the compilation results and model weights output by the various types of GPU devices; An image generation module, used to save the compilation result and the model weight to cloud storage to form a corresponding model image; An image management module, configured to obtain the model image from the cloud storage in response to a task pre-execution instruction sent by the client, and pre-execute the target model compilation task according to the model image to obtain a pre-compiled model; The result sending module is used to send the precompiled model to the client.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
Citation Information
Patent Citations
Mirror image construction method, system and device and medium
CN112363803A
Deep learning distributed compiler for cloud edge computing and construction method
CN113127203A
Continuous integration method, device and system based on container, working node and medium
CN114296871A
K8S integration and deployment method and system supporting multiple architectures and storage medium
CN114327484A
Mirror image updating method and device, electronic equipment and storage medium
CN115023693A
