Machine learning model deployment methods, devices, equipment, media, and products
By automating the deployment of multiple components of a machine learning model through file arrangement and task execution instructions, the problem of low efficiency in multi-component deployment is solved, achieving efficient, flexible model deployment and rapid response.
Patent Information
- Application Number
- CN202211185878.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-27
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-09-27
AI Technical Summary
Existing machine learning model deployment methods cannot efficiently handle the deployment requirements of multiple components, resulting in the need for in-depth development to respond to business needs and failing to achieve rapid response.
By orchestrating files to describe the multiple ordered stages of a machine learning model, task execution instructions are generated, each component is automatically deployed to the container support framework, and a proxy service is started to respond to external requests.
It enables one-stop automated deployment of multi-component machine learning models, improving deployment efficiency and flexibility, adapting to changes in model structure, and enhancing the configurability and execution efficiency of the deployment process.
Smart Images

Figure CN115599396B_ABST
Abstract
Description
Technical Field
[0001] This application relates to network technology, and more particularly to a method for deploying machine learning models and the apparatus, device, medium, and product thereof. Background Technology
[0002] As machine learning models become more widely used and complex, model deployment has evolved from deploying a single model to deploying multiple model components simultaneously. The traditional approach of deploying individual models separately and then combining them via interfaces later is no longer sufficient to meet the demands for rapid deployment.
[0003] Currently, many industry solutions are based on the Kubernetes framework for deployment (such as Kubeflow and Seldon). However, these tools focus more on model deployment itself and do not integrate well with the business logic, failing to achieve efficient deployment of machine learning models containing multiple components. This leads to a problem: when using these solutions, in-depth development based on business needs is often required, rather than rapid response to deployment requirements.
[0004] Therefore, there is a need to provide more efficient deployment methods for machine learning models that contain multiple components in order to improve the deployment effectiveness of such models. Summary of the Invention
[0005] The purpose of this application is to solve the above-mentioned problems by providing a method for deploying machine learning models and corresponding apparatus, devices, non-volatile readable storage media, and computer program products.
[0006] According to one aspect of this application, a method for deploying a machine learning model is provided, comprising the following steps:
[0007] Obtain an orchestration file for a machine learning model that includes multiple components, wherein the orchestration file includes description information of multiple tasks corresponding to multiple ordered steps in the process of deploying the machine learning model;
[0008] Based on the description information of each ordered step in the orchestration file, corresponding task execution instructions are generated in an orderly manner, and the deployment tasks of the corresponding ordered steps are executed in response to each task execution instruction, thereby completing the deployment of each component of the machine learning model in the container support framework.
[0009] The proxy service corresponding to the machine learning model is started to respond to external requests by calling the machine learning model in the container support framework and obtaining the corresponding processing results to respond to the external request.
[0010] Optionally, the description information of the arrangement file includes:
[0011] Image build description information is used to describe the program resource information required for the machine learning model to run, including code resources for generating machine learning model instances and engine resources for supporting the running of the instances;
[0012] The model publishes descriptive information, which describes the runtime resource information of the containers required by each component of the machine learning model;
[0013] Call chain description information, used to describe the dependency information of each component of the machine learning model relative to other components;
[0014] Deployment environment description information, which describes the deployment environment information of the machine learning model with unique characteristics in the container support framework.
[0015] Optionally, based on the description information of each ordered step in the orchestration file, corresponding task execution instructions are generated in an orderly manner, and the deployment tasks of the corresponding ordered steps are executed in response to each task execution instruction, thereby completing the deployment of each component of the machine learning model in the container support framework, including:
[0016] Based on the image building description information in the orchestration file, a first task execution instruction corresponding to the image building task is generated. In response to the first task execution instruction, the program resource information specified in the image building description information is called to create an image of the machine learning model.
[0017] Based on the model release description information in the orchestration file, a second task execution instruction corresponding to the model release task is generated. In response to the second task execution instruction, the corresponding running resources for each component of the machine learning model are determined according to the running resource information specified in the model release description information, and the release of each component in the image is completed.
[0018] Based on the call chain description information and deployment environment description information in the orchestration file, a third task execution instruction corresponding to the container configuration task is generated. In response to the third task execution instruction, the dependency information of each component is configured in the container support framework according to the call chain description information, and the deployment environment information of the machine learning model is configured in the container support framework according to the deployment environment description information.
[0019] Optionally, the orchestration file represents the description information of multiple tasks of the machine learning model in structured data, wherein the model release description information and call chain description information corresponding to each component are encapsulated in the same component logical unit.
[0020] Optionally, a proxy service corresponding to the machine learning model is started to respond to external requests by invoking the machine learning model in the container support framework, obtaining the corresponding processing results to respond to the external request, including:
[0021] Start the proxy service corresponding to the machine learning model, making the proxy service suitable for invoking the machine learning model in response to external requests;
[0022] The proxy service responds to an external request by invoking the machine learning model, using the source data carried in the external request as the input information of the machine learning model, and obtains the result information obtained by the machine learning model after processing the input information;
[0023] The external request is responded to with the result information.
[0024] Optionally, the step of generating corresponding task execution instructions in an orderly manner according to the description information of each ordered step in the orchestration file is executed by a preset analysis module, the step of executing the deployment task of the corresponding ordered step in response to each task execution instruction is executed by a preset execution module, and the step of deploying each component of the machine learning model in the container support framework is executed by a preset execution module. The analysis module and the execution module communicate with each other through interface calls.
[0025] According to another aspect of this application, a machine learning model deployment apparatus is provided, comprising:
[0026] The file acquisition module is configured to acquire an orchestration file of a machine learning model containing multiple components. The orchestration file includes description information of multiple tasks corresponding to multiple ordered steps in the process of deploying the machine learning model.
[0027] The deployment and execution module is configured to generate corresponding task execution instructions in an orderly manner based on the description information of each ordered step in the orchestration file, and execute the deployment tasks of the corresponding ordered steps in response to each task execution instruction, thereby completing the deployment of each component of the machine learning model in the container support framework.
[0028] The service operation module is configured to start the proxy service corresponding to the machine learning model, which is used to respond to external requests by calling the machine learning model in the container support framework and obtaining the corresponding processing results to respond to the external request.
[0029] Optionally, the description information of the arrangement file includes:
[0030] Image build description information is used to describe the program resource information required for the machine learning model to run, including code resources for generating machine learning model instances and engine resources for supporting the running of the instances;
[0031] The model publishes descriptive information, which describes the runtime resource information of the containers required by each component of the machine learning model;
[0032] Call chain description information, used to describe the dependency information of each component of the machine learning model relative to other components;
[0033] Deployment environment description information, which describes the deployment environment information of the machine learning model with unique characteristics in the container support framework.
[0034] Optionally, the deployment execution module includes:
[0035] The image building unit is configured to generate a first task execution instruction corresponding to the image building task based on the image building description information in the orchestration file, and in response to the first task execution instruction, call the program resource information specified in the image building description information to create an image of the machine learning model.
[0036] The model publishing unit is configured to generate a second task execution instruction corresponding to the model publishing task based on the model publishing description information in the orchestration file, and in response to the second task execution instruction, determine the corresponding running resources for each component of the machine learning model according to the running resource information specified in the model publishing description information and complete the publishing of each component in the image.
[0037] The container configuration unit is configured to generate a third task execution instruction corresponding to the container configuration task based on the call chain description information and deployment environment description information in the orchestration file, and in response to the third task execution instruction, configure the dependency information of each component in the container support framework based on the call chain description information, and configure the deployment environment information of the machine learning model in the container support framework based on the deployment environment description information.
[0038] Optionally, the orchestration file represents the description information of multiple tasks of the machine learning model in structured data, wherein the model release description information and call chain description information corresponding to each component are encapsulated in the same component logical unit.
[0039] Optionally, the service operation module includes:
[0040] The service startup unit is configured to start the proxy service corresponding to the machine learning model, so that the proxy service is suitable for responding to external requests and calling the machine learning model.
[0041] The model invocation unit is configured to invoke the machine learning model in response to an external request by the proxy service, using the source data carried by the external request as the input information of the machine learning model, and obtaining the result information obtained by the machine learning model after processing the input information;
[0042] The request-response unit is configured to respond to the external request with the result information.
[0043] Optionally, the deployment execution module is implemented by two processes working together. The first process is configured to generate corresponding task execution instructions in an orderly manner according to the description information of each ordered step in the orchestration file. The second process is configured to execute the deployment tasks of the corresponding ordered steps in response to each task execution instruction, thereby completing the deployment of each component of the machine learning model in the container support framework. The first process and the second process communicate with each other through interface calls.
[0044] According to another aspect of this application, a machine learning model deployment device is provided, including a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the machine learning model deployment method described in this application.
[0045] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the machine learning model deployment method in the form of computer-readable instructions, wherein the computer program, when invoked by a computer, performs the steps included in the method.
[0046] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.
[0047] Compared with existing technologies, this application has several advantages, including but not limited to:
[0048] First, this application can provide all configuration information of a multi-component machine learning model through an orchestration file. Then, based on the orchestration file, an automated deployment process is initiated to deploy each component into a container support framework and automatically start the proxy service, thereby achieving one-stop automated deployment of a multi-component machine learning model and improving model deployment efficiency.
[0049] Secondly, the orchestration file of this application includes description information of multiple tasks corresponding to multiple ordered steps in the process of deploying the machine learning model, making each ordered step in the model deployment process configurable. Therefore, it allows for effective control of each ordered step in the model deployment process by writing the orchestration file, and also allows for adjustments to the deployment process by timely adjustments to the orchestration file, thus making the model deployment process configurable and effectively adapting to the structural changes of the machine learning model.
[0050] Furthermore, in each orderly stage of the deployment process, this application decouples the parsing mechanism of the orchestration file and the execution mechanism of the task execution instructions by generating corresponding task execution instructions and responding to the task execution instructions. This makes the functions of each part relatively independent after decoupling, thereby improving the execution efficiency of each function. At the same time, this application also decouples the algorithm implementation process of the machine learning model and the deployment process through the orchestration file. This makes each process independent and efficient, thereby helping to improve the timeliness of the machine learning model going online. Attached Figure Description
[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0052] Figure 1 This is a schematic diagram of the network architecture corresponding to the exemplary operating environment of this application;
[0053] Figure 2 A flowchart illustrating one embodiment of the machine learning model deployment method of this application;
[0054] Figure 3 This is a flowchart illustrating the deployment process of the machine learning model in an embodiment of this application.
[0055] Figure 4 This is a schematic diagram illustrating the process of implementing the proxy service in the embodiments of this application;
[0056] Figure 5 A schematic diagram of the machine learning model deployment device for this application;
[0057] Figure 6 This is a schematic diagram of the structure of a machine learning model deployment device used in this application. Detailed Implementation
[0058] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.
[0059] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.
[0060] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0061] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.
[0062] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.
[0063] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.
[0064] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.
[0065] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.
[0066] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.
[0067] Please see Figure 1 , Figure 1 This describes the network architecture corresponding to an exemplary runtime environment for the machine learning model deployment method of this application. It includes a front-end server 81 for installing computer program products implemented according to the machine learning model deployment method and a container server 80 for deploying a container support framework. Both the front-end server and the container server can be a server cluster. The front-end server can open services to the terminal device 82 by starting a proxy service, receiving external requests submitted by the terminal device, calling the machine learning model running in the container server to process the external requests, obtaining corresponding results, and returning them to the corresponding terminal device.
[0068] Please see Figure 2 According to a machine learning model deployment method provided in this application, in one embodiment, the method includes the following steps:
[0069] Step S1100: Obtain an orchestration file for a machine learning model containing multiple components. The orchestration file includes description information of multiple tasks corresponding to multiple ordered steps in the process of deploying the machine learning model.
[0070] The machine learning model referred to in this application is an algorithmic model that has been pre-trained to reach a convergent state and is suitable for inference based on input information in a predetermined format to obtain corresponding result information. The machine learning model typically includes multiple functionally independent components. When each component is invoked, it completes a corresponding inference task in one stage of the machine learning model's inference process. Multiple components work together to ultimately obtain a unified result information output.
[0071] The machine learning model can be a traditional machine learning model based on mathematical modeling, or a neural network model based on deep learning. It is usually trained using training samples in a training set to enable it to acquire the ability to obtain the corresponding result information, and then deployed accordingly.
[0072] The process of deploying a machine learning model, i.e., its deployment process, is pre-planned into several ordered stages: image building, deployment execution, and environment configuration. Each ordered stage is relatively independent but interconnected, working together to complete the deployment of the various components of the machine learning model within the container support framework. Specifically:
[0073] The image building process guides the creation of an image corresponding to the machine learning model. Therefore, the orchestration file contains image building description information for this process. This image building description information describes the program resource information required by each component during the runtime of the machine learning model, including code resources (runtime package) for generating machine learning model instances and engine resources (runtime engine) for supporting the instance's execution. For the code resources, in one embodiment, they can be described by specifying the component's corresponding package name (name), the module exposed by the package (servModule), and the class exposed by the package (servClass), thus describing the code resource information required for the package's runtime. This information is parsed and loaded by the engine specified by the engine resource. To represent the engine resource, in one embodiment, the component's corresponding engine type (servEngineType) and engine dependencies (servEngineRequirements) can be specified. These descriptions configure the engines that the component's package depends on during runtime, so that the corresponding runtime engine can be packaged and imported during image building. Once an image is created based on the image build description information, such as a Docker image, the build structure information of the image can be obtained, and the network storage location where the machine learning model can be published can be known. This information can be used to improve the configuration information required for the deployment and execution process, so that each component can be published to the corresponding network storage location.
[0074] The deployment execution phase guides the specific deployment of each component of the machine learning model within the container support framework. Correspondingly, the orchestration file contains model deployment description information for this phase, describing the container runtime resources required by each component of the machine learning model. These runtime resources primarily relate to the system overhead resources occupied by the container, such as CPU and memory usage. By combining the model deployment description information with the build results of the created image, each component of the machine learning model can be deployed to a designated location. This designated location refers to the shared volume mounted by the machine learning model, i.e., the network storage location loaded by the container at runtime, used to store model or data files.
[0075] The environment configuration step is primarily used to instruct the container support framework to complete the configuration of the dependencies and environment information of each component of the machine learning model. Therefore, the orchestration file includes corresponding call chain description information and deployment environment description information. Specifically: the call chain description information describes the dependency relationship between each component of the machine learning model and other components; the deployment environment description information describes the unique deployment environment information of the machine learning model within the container support framework. After the container support framework completes the configuration of the machine learning model based on the call chain description information and environment description information, it can uniquely determine the machine learning model and subsequently invoke each component to cooperate according to the dependencies presented in the call chain description information, ensuring the effective invocation and operation of the machine learning model.
[0076] In one embodiment, for the call chain description information, the components of the machine learning model are treated as nodes. A structure is used to define the reference relationships between nodes. A tree structure is used within this structure to describe the parent-child relationships between different nodes, making the structure a recursive structure that represents the call dependencies between various components. Furthermore, parallel nodes are also allowed in the structure, as long as the type information corresponding to the triggering timing is defined in each node.
[0077] In another embodiment, the deployment environment description information can be provided to the container support framework by specifying the deployment name, deployment type, cluster name, project name, persistent volume storage path, etc., to provide the information required for configuring its implementation environment.
[0078] In one embodiment, when describing the corresponding component as a node in the structure in the orchestration file, the corresponding model release description information of the component in the deployment and execution phase can also be recorded in the node, that is, the runtime resource information required by the component, so that the description structure of the orchestration file is more concise and efficient and can improve the parsing efficiency of the orchestration file.
[0079] In one embodiment, the orchestration file of this application provides descriptive information corresponding to each ordered stage in the deployment process in the form of structured data, so as to guide the generation of task execution instructions corresponding to each ordered stage, and drive the deployment operation through task execution instructions. An example of the structured data of the orchestration file is provided below:
[0080]
[0081]
[0082]
[0083] The following explanation, based on the examples above, details the descriptive information provided by each field:
[0084] components: The component set module contained in the machine learning model, which specifies the construction information of each component of the machine learning model and consists of a list of Components.
[0085] The `graph` module specifies the call relationships between components. It consists of a `Node` and its `children` lists, which are also lists of `Nodes`, essentially forming a recursive structure. The call chain starts from the root `Node`, passes to the `Children` lists, and so on. If there are multiple parallel `Nodes` within `Children`, the `type` property in the `Node` module specifies the scheduling type, thus indicating the scheduling relationship of the parallel `Nodes`, such as calling all, random, or combined calls.
[0086] deployTarget: The deployment module provides configuration information for the actual deployment of components, including deployment name (deploymentName), deployment environment (deployType), deployment cluster name (clusterName), project name (projectName), and persistent volume storage path (pvcPath).
[0087] Therefore, based on any embodiment of this application, when the orchestration file represents the description information of multiple tasks of the machine learning model in structured data, the model publication description information and call chain description information corresponding to each component are encapsulated in the same component logical unit, namely the graph module of the above example. The call chain description information is reflected through the nesting relationship between nodes, and the model publication description information of the component is reflected through the details information defined in the Node.
[0088] Specifically, the component set module (components) specifies the component's name, type, and build in its corresponding description structure. The build includes the base image of the machine learning model (baseImage), build type (buildType), and build information (buildInfo). The build information (buildInfo) specifies the runtime package and dependencies for this deployment. The runtime package module specifies the required package name (name), the exposed modules (servModule), the exposed classes (servClass), the engine type (servEngineType), and engine dependencies (servEngineRequirements). The first three pieces of information configure the information required for the package to run, which will be parsed and loaded by the corresponding engine. The last two pieces of information configure the engines that the package depends on at runtime, which are packaged and imported by the module executing the image build during image construction.
[0089] Based on this, corresponding runtime build information can be generated according to the Component module information, triggering the corresponding task execution instruction for image building. This task execution instruction then performs the image building operation based on the runtime build information. After the image building is complete, the build result information is returned.
[0090] Specifically, the Node module contains the component name (corresponding to the name in a specific component), the component type (corresponding to the type in a specific component), runtime parameters (runParams), and models (models). The runParams parameters specify a series of parameters required by the model during runtime, such as CPU and memory. The models list contains a list of models that the Node module needs to run, specifying basic model information, including the model name, model tag, and model version.
[0091] Based on this, corresponding model publishing description information can be generated according to the models information in the Node module. Then, the corresponding task execution instruction for model publishing is triggered. The task execution instruction executes the model publishing according to the model publishing description information and returns the publishing result information.
[0092] In addition, the component scheduling relationship in the graph module can be analyzed to obtain the call chain description information represented therein. At the same time, based on the build package information and model release result information of each component, the actual deployment information can be combined, and the actual deployment operation can be executed according to the deployment information (such as deployment to the container support framework implemented by Kubernetes).
[0093] Based on the examples above, it's easy to understand that the orchestration file can be pre-written and modified, thus achieving pre-configuration, which is very convenient. The control operations of deploying a machine learning model are represented through a single orchestration file. By parsing this orchestration file to generate corresponding task execution instructions, the deployment process is automated, achieving a one-stop automated deployment operation that is highly efficient.
[0094] Step S1200: Based on the description information of each ordered link in the orchestration file, generate corresponding task execution instructions in an orderly manner, and execute the deployment tasks of the corresponding ordered links in response to each task execution instruction to complete the deployment of each component of the machine learning model in the container support framework.
[0095] In one embodiment, the deployment process can be decoupled by implementing the functions involved in the deployment process as different functional modules. These modules work together to handle the parsing of the orchestration file and the execution of corresponding operations based on the parsing results, allowing each module to focus relatively independently on its specific operation. Therefore, an analysis module and an execution module running in the memory of the front-end server can be pre-implemented. The analysis module is configured as a first process to generate task execution instructions for the corresponding ordered steps based on the description information of each ordered step in the orchestration file. The execution module is configured as a second process to execute the deployment tasks of the corresponding ordered steps in response to each task execution instruction, thereby completing the deployment of the various components of the machine learning model within the container support framework. Data communication between the analysis module and the execution module can be implemented by setting up interfaces between them, typically by the analysis module calling the data communication interface exposed by the analysis module.
[0096] During the deployment process, the deployment operations corresponding to each step are executed sequentially according to the orderly steps of the deployment process. For example, the image building step, deployment execution step, and environment configuration step in the aforementioned example can be executed sequentially. In each step, the analysis module first analyzes the description information required for the corresponding step from the orchestration file, then triggers the corresponding task execution instruction, and then the execution module responds to the task execution instruction and executes the corresponding deployment operation according to the corresponding description information. After the deployment operation is completed, the next step is processed, and so on, to ensure orderly progress and finally complete the deployment of each component of the machine learning model in the container support framework.
[0097] Step S1300: Start the proxy service corresponding to the machine learning model to respond to external requests by calling the machine learning model in the container support framework and obtaining the corresponding processing results to respond to the external request.
[0098] After the front-end server completes the deployment of the machine learning model, it can register the interface information of the machine learning model to a callable list, and then start a corresponding proxy service. This proxy service is used to implement proxy access services for the interface information pre-registered in the callable list. Thus, when the terminal device needs to call the machine learning model and submits an external request, the proxy service can receive and respond to the external request, obtain the input information encapsulated in the external request, use it as the input information of the machine learning model, call the machine learning model to obtain the corresponding processing result, and use the processing result to respond to the external request, so that the terminal device can obtain the corresponding result information.
[0099] Based on the above embodiments, it is easy to understand that this application has many advantages, including but not limited to:
[0100] First, this application can provide all configuration information of a multi-component machine learning model through an orchestration file. Then, based on the orchestration file, an automated deployment process is initiated to deploy each component into a container support framework and automatically start the proxy service, thereby achieving one-stop automated deployment of a multi-component machine learning model and improving model deployment efficiency.
[0101] Secondly, the orchestration file of this application includes description information of multiple tasks corresponding to multiple ordered steps in the process of deploying the machine learning model, making each ordered step in the model deployment process configurable. Therefore, it allows for effective control of each ordered step in the model deployment process by writing the orchestration file, and also allows for adjustments to the deployment process by timely adjustments to the orchestration file, thus making the model deployment process configurable and effectively adapting to the structural changes of the machine learning model.
[0102] Furthermore, in each orderly stage of the deployment process, this application decouples the parsing mechanism of the orchestration file and the execution mechanism of the task execution instructions by generating corresponding task execution instructions and responding to the task execution instructions. This makes the functions of each part relatively independent after decoupling, thereby improving the execution efficiency of each function. At the same time, this application also decouples the algorithm implementation process of the machine learning model and the deployment process through the orchestration file. This makes each process independent and efficient, thereby helping to improve the timeliness of the machine learning model going online.
[0103] Based on any embodiment of this application, please refer to Figure 3 Based on the description information of each ordered step in the orchestration file, corresponding task execution instructions are generated in an orderly manner. Responding to each task execution instruction, the deployment tasks of the corresponding ordered steps are executed, completing the deployment of each component of the machine learning model within the container support framework, including:
[0104] Step S1210: Based on the image building description information in the orchestration file, generate a first task execution instruction corresponding to the image building task; in response to the first task execution instruction, call the program resource information specified in the image building description information to create an image of the machine learning model.
[0105] In the first ordered step of the deployment process, namely the image building step, the analysis module first calls the orchestration file, parses and obtains the image building description information therein, and then generates its corresponding first task execution instruction.
[0106] Then, the execution module responds to the first task execution instruction. In response to the first task execution instruction, the execution module loads the corresponding code resources and engine resources based on the program resource information represented by each module and specific field specified in the image construction description information, and creates the corresponding image. The result information of the image is then obtained, which can be used for reference in subsequent ordered steps.
[0107] Step S1220: Based on the model release description information in the orchestration file, generate a second task execution instruction corresponding to the model release task; in response to the second task execution instruction, determine the corresponding running resources for each component of the machine learning model according to the running resource information specified in the model release description information and complete the release of each component in the image.
[0108] In the second ordered stage of the deployment process, namely the deployment execution stage, the analysis module first calls the orchestration file, parses and obtains the model release description information therein, and then generates its corresponding second task execution instructions.
[0109] Then, the execution module responds to the second task execution instruction. In response to the second task execution instruction, the execution module publishes the runtime resource information represented by each module and specific field specified in the model publishing description information. Based on this runtime resource information, it notifies the container support framework to allocate the corresponding runtime resources, and then allocates the corresponding components in the image to the corresponding runtime resources, thereby realizing the publishing of the image to the corresponding container for operation.
[0110] Step S1230: Based on the call chain description information and deployment environment description information in the orchestration file, generate a third task execution instruction corresponding to the container configuration task. In response to the third task execution instruction, configure the dependency information of each component in the container support framework according to the call chain description information, and configure the deployment environment information of the machine learning model in the container support framework according to the deployment environment description information.
[0111] In the third orderly stage of the deployment process, namely the environment configuration stage, the analysis module first calls the orchestration file, parses and obtains the call chain description information and deployment environment description information, and then generates the corresponding third task execution instructions.
[0112] Then, the execution module responds to the third task execution instruction. In response to the third task execution instruction, the execution module calls the relevant API interfaces exposed by the container support framework (e.g., Kubernetes) to set the call chain description information and deployment environment description information accordingly. Thus, based on the call chain description information, the dependency information of each component is configured in the container support framework, and based on the deployment environment description information, the deployment environment information of the machine learning model is configured in the container support framework. This completes the entire deployment process.
[0113] As can be seen from the above embodiments, based on the instruction response mechanism, the parsing mechanism of the orchestration file and the execution mechanism of the deployment work can be decoupled. Therefore, as long as complete descriptive information is provided in the orchestration file, the machine learning model can be deployed in an orderly manner according to the established deployment process, resulting in high execution efficiency.
[0114] Based on any embodiment of this application, please refer to Figure 4 The proxy service corresponding to the machine learning model is started to respond to external requests by invoking the machine learning model in the container support framework, obtaining the corresponding processing results to respond to the external request, including:
[0115] Step S1310: Start the proxy service corresponding to the machine learning model, so that the proxy service is suitable for responding to external requests and calling the machine learning model;
[0116] After the machine learning model is deployed on the container support framework, the front-end server starts the proxy service. The proxy service can also be started in advance by the front-end server pre-registering the interface information of the machine learning model to the list of callable interfaces accessible to the proxy service. Thus, the proxy service can act as a proxy for the pre-registered interfaces in the callable list. When an external request corresponding to a pre-registered interface is generated, the proxy service can then perform a proxy call to the machine learning model.
[0117] Step S1320: The proxy service responds to an external request by calling the machine learning model, using the source data carried by the external request as the input information of the machine learning model, and obtains the result information obtained by the machine learning model after processing the input information;
[0118] When a terminal device initiates an external request to invoke the machine learning model, the proxy service responds to the external request by parsing out the source data carried therein. The source data is determined according to the input parameters of the machine learning model and can be any type of information such as images, text, or videos. Then, the source data is used to construct the input information of the machine learning model, and the machine learning model is invoked by passing in the input information. After the machine learning model performs inference, the inference result information is returned to the proxy service.
[0119] Step S1330: Respond to the external request with the result information.
[0120] The result information is the model inference result that the external request expects to obtain. Therefore, the proxy service can respond to the external request with the result information and send the result information to the terminal device.
[0121] In particular, the terminal device in this embodiment can also be a server device deployed in the cloud. That is, the proxy service of this application can respond to any legitimate external request, whether the external request is submitted by the terminal device used by the user or by the cloud server device.
[0122] As can be understood from the above embodiments, by processing external requests to call the machine learning model through a proxy service, since the machine learning model is pre-deployed by the front-end server, the front-end server can effectively achieve organic scheduling of multiple components, thereby ensuring that the call to the machine learning model is more efficient and robust.
[0123] Please see Figure 5 According to one aspect of this application, a machine learning model deployment apparatus includes a file acquisition module 1100, a deployment execution module 1200, and a service operation module 1300. The file acquisition module 1100 is configured to acquire an orchestration file containing multiple components of a machine learning model. The orchestration file includes description information of multiple tasks corresponding to multiple ordered stages in the deployment of the machine learning model. The deployment execution module 1200 is configured to generate corresponding task execution instructions in an orderly manner based on the description information of each ordered stage in the orchestration file, and execute the deployment tasks of the corresponding ordered stages in response to each task execution instruction, thereby completing the deployment of each component of the machine learning model within a container support framework. The service operation module 1300 is configured to start a proxy service corresponding to the machine learning model, which is used to respond to external requests by invoking the machine learning model in the container support framework and obtaining corresponding processing results to respond to the external request.
[0124] Based on any embodiment of this application, the description information of the orchestration file includes: image build description information, used to describe the program resource information required for the machine learning model to run, including code resources for generating machine learning model instances and engine resources for supporting the running of the instances; model release description information, used to describe the container runtime resource information required by each component of the machine learning model; call chain description information, used to describe the dependency information of each component of the machine learning model relative to other components; and deployment environment description information, used to describe the deployment environment information of the machine learning model with unique characteristics in the container support framework.
[0125] Based on any embodiment of this application, the deployment execution module 1200 includes: an image building unit, configured to generate a first task execution instruction corresponding to an image building task according to the image building description information in the orchestration file, and in response to the first task execution instruction, call the program resource information specified in the image building description information to create an image of the machine learning model; a model publishing unit, configured to generate a second task execution instruction corresponding to a model publishing task according to the model publishing description information in the orchestration file, and in response to the second task execution instruction, determine the corresponding running resources for each component of the machine learning model according to the running resource information specified in the model publishing description information and complete the publishing of each component in the image; and a container configuration unit, configured to generate a third task execution instruction corresponding to a container configuration task according to the call chain description information and the deployment environment description information in the orchestration file, and in response to the third task execution instruction, configure the dependency information of each component in the container support framework according to the call chain description information, and configure the deployment environment information of the machine learning model in the container support framework according to the deployment environment description information.
[0126] Based on any embodiment of this application, the orchestration file represents the description information of multiple tasks of the machine learning model in structured data, wherein the model release description information and call chain description information corresponding to each component are encapsulated in the same component logical unit.
[0127] Based on any embodiment of this application, the service operation module 1300 includes: a service startup unit, configured to start a proxy service corresponding to the machine learning model, so that the proxy service is suitable for responding to external requests and calling the machine learning model; a model invocation unit, configured to have the proxy service respond to external requests and call the machine learning model, using the source data carried by the external request as the input information of the machine learning model, and obtaining the result information obtained by the machine learning model after processing the input information; and a request response unit, configured to respond to the external request with the result information.
[0128] Based on any embodiment of this application, the deployment execution module 1200 is implemented by two processes working together. The first process is configured to generate corresponding task execution instructions in an orderly manner according to the description information of each ordered link in the orchestration file. The second process is configured to execute the deployment task of the corresponding ordered link in response to each task execution instruction, and complete the deployment of each component of the machine learning model in the container support framework. The first process and the second process communicate with each other through interface calls.
[0129] Another embodiment of this application also provides a machine learning model deployment device. For example... Figure 6 The diagram shows the internal structure of a machine learning model deployment device. This device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database stores information sequences, and when executed by the processor, these computer-readable instructions enable the processor to implement a machine learning model deployment method.
[0130] The processor of the machine learning model deployment device provides computational and control capabilities to support the operation of the entire device. The memory of the device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the machine learning model deployment method of this application. The network interface of the device is used for communication with a terminal.
[0131] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the solution of this application, and does not constitute a limitation on the machine learning model deployment device on which the solution of this application is applied. The specific machine learning model deployment device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.
[0132] In this embodiment, the processor is used to execute... Figure 5 The specific functions of each module are described, and the memory stores the program code and various data required to execute the above modules or sub-modules. The network interface is used to realize data transmission between user terminals or servers. In this embodiment, the non-volatile readable storage medium stores the program code and data required to execute all modules in the machine learning model deployment device of this application, and the server can call the server's program code and data to execute the functions of all modules.
[0133] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the machine learning model deployment method of any embodiment of this application.
[0134] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the method described in any embodiment of this application.
[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).
[0136] In summary, this application optimizes the deployment of machine learning models containing multiple components. By providing descriptive information of each ordered step in the model deployment process through configurable orchestration files, it generates corresponding task execution instructions to complete one-stop automated deployment of the model. This decouples the algorithm implementation process from the model deployment process, as well as the orchestration file parsing mechanism from the task execution instruction execution mechanism. This helps to achieve efficient deployment of machine learning models and improves the implementation efficiency of related software engineering.
Claims
1. A method for deploying a machine learning model, characterized in that, include: An orchestration file for a machine learning model comprising multiple components is obtained. This orchestration file includes descriptions of multiple tasks corresponding to ordered stages in the deployment of the machine learning model. The descriptions in the orchestration file include: image build descriptions, which describe the program resources required for the machine learning model to run, including code resources for generating machine learning model instances and engine resources for supporting the instance's execution; model deployment descriptions, which describe the container runtime resources required by each component of the machine learning model; call chain descriptions, which describe the dependencies of each component of the machine learning model relative to other components; and deployment environment descriptions, which describe the unique deployment environment information of the machine learning model within the container support framework. The machine learning model comprises multiple functionally independent components. When each component is invoked, it completes a reasoning task corresponding to a stage in the machine learning model's reasoning process. The multiple components work together to ultimately obtain a unified output result. Based on the description information of each ordered step in the orchestration file, corresponding task execution instructions are generated in an orderly manner. In response to each task execution instruction, the deployment tasks of the corresponding ordered steps are executed to complete the deployment of each component of the machine learning model in the container support framework. This includes: generating a first task execution instruction corresponding to the image building task based on the image building description information in the orchestration file; in response to the first task execution instruction, calling the program resource information specified in the image building description information to create an image of the machine learning model; generating a second task execution instruction corresponding to the model publishing task based on the model publishing description information in the orchestration file; in response to the second task execution instruction, determining the corresponding running resources for each component of the machine learning model according to the running resource information specified in the model publishing description information and completing the publishing of each component in the image; generating a third task execution instruction corresponding to the container configuration task based on the call chain description information and the deployment environment description information in the orchestration file; in response to the third task execution instruction, configuring the dependency information of each component in the container support framework according to the call chain description information, and configuring the deployment environment information of the machine learning model in the container support framework according to the deployment environment description information. The proxy service corresponding to the machine learning model is started to respond to external requests by calling the machine learning model in the container support framework and obtaining the corresponding processing results to respond to the external request.
2. The machine learning model deployment method according to claim 1, characterized in that, The orchestration file uses structured data to represent the description information of multiple tasks of the machine learning model, wherein the model release description information and call chain description information corresponding to each component are encapsulated in the same component logical unit.
3. The machine learning model deployment method according to claim 1 or 2, characterized in that, The proxy service corresponding to the machine learning model is started to respond to external requests by invoking the machine learning model in the container support framework, obtaining the corresponding processing results to respond to the external request, including: Start the proxy service corresponding to the machine learning model, making the proxy service suitable for invoking the machine learning model in response to external requests; The proxy service responds to an external request by invoking the machine learning model, using the source data carried in the external request as the input information of the machine learning model, and obtains the result information obtained by the machine learning model after processing the input information; The external request is responded to with the result information.
4. The machine learning model deployment method according to claim 1 or 2, characterized in that, Based on the description information of each ordered step in the orchestration file, the step of generating corresponding task execution instructions in an orderly manner is executed by a preset analysis module. The step of deploying the corresponding ordered steps in response to each task execution instruction is executed by a preset execution module. The step of deploying each component of the machine learning model in the container support framework is executed by a preset execution module. The analysis module and the execution module communicate with each other through interface calls.
5. A machine learning model deployment device, characterized in that, include: The file acquisition module is configured to acquire an orchestration file for a machine learning model containing multiple components. The orchestration file includes descriptions of multiple tasks corresponding to multiple ordered stages in the deployment of the machine learning model. The descriptions in the orchestration file include: image build descriptions, which describe the program resources required for the machine learning model to run, including code resources for generating machine learning model instances and engine resources for supporting the instance's operation; model deployment descriptions, which describe the container runtime resources required by each component of the machine learning model; call chain descriptions, which describe the dependencies of each component of the machine learning model relative to other components; and deployment environment descriptions, which describe the unique deployment environment information of the machine learning model within the container support framework. The machine learning model includes multiple functionally independent components. When each component is invoked, it completes a reasoning task corresponding to a stage in the machine learning model's reasoning process. The multiple components work together to ultimately obtain a unified output result. The deployment execution module is configured to generate corresponding task execution instructions in an orderly manner based on the description information of each ordered step in the orchestration file, and execute the deployment tasks of the corresponding ordered steps in response to each task execution instruction to complete the deployment of each component of the machine learning model in the container support framework. This includes: generating a first task execution instruction corresponding to the image building task based on the image building description information in the orchestration file; in response to the first task execution instruction, calling the program resource information specified in the image building description information to create an image of the machine learning model; generating a second task execution instruction corresponding to the model release task based on the model release description information in the orchestration file; in response to the second task execution instruction, determining the corresponding runtime resources for each component of the machine learning model according to the runtime resource information specified in the model release description information and completing the release of each component in the image; and generating a third task execution instruction corresponding to the container configuration task based on the call chain description information and deployment environment description information in the orchestration file; in response to the third task execution instruction, configuring the dependency information of each component in the container support framework according to the call chain description information, and configuring the deployment environment information of the machine learning model in the container support framework according to the deployment environment description information. The service operation module is configured to start the proxy service corresponding to the machine learning model, which is used to respond to external requests by calling the machine learning model in the container support framework and obtaining the corresponding processing results to respond to the external request.
6. The machine learning model deployment apparatus according to claim 5, characterized in that, The orchestration file uses structured data to represent the description information of multiple tasks of the machine learning model, wherein the model release description information and call chain description information corresponding to each component are encapsulated in the same component logical unit.
7. The machine learning model deployment apparatus according to claim 5 or 6, characterized in that, The service operation module includes: The service startup unit is configured to start the proxy service corresponding to the machine learning model, so that the proxy service is suitable for responding to external requests and calling the machine learning model. The model invocation unit is configured to invoke the machine learning model in response to an external request by the proxy service, using the source data carried by the external request as the input information of the machine learning model, and obtaining the result information obtained by the machine learning model after processing the input information; The request-response unit is configured to respond to the external request with the result information.
8. A machine learning model deployment device, comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 4.
9. A non-volatile readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 4, which, when invoked by a computer, executes the steps included in the corresponding method.
10. A computer program product, characterized in that, It includes a computer program / instruction that, when invoked by a processor, performs the steps of the method described in any one of claims 1 to 4.
Citation Information
Patent Citations
Program deployment task execution method and device, computer equipment and storage medium
CN111258591A
Machine learning model automatic production line construction method and system
CN114115857A
Model deployment method and device, electronic equipment and storage medium
CN114546425A
