Model architecture design method and deployment system for multi-tenant machine learning platform

Abstract

The present invention discloses a model architecture design method and deployment system for a multi-tenant machine learning platform. The design method includes the following steps: constructing a visual model architecture design device, including a component library, a canvas, and a configuration window; generating an easily extensible model architecture description file based on the visual model architecture design device, creating a model architecture design workspace, filling in basic information of the model, or loading the model architecture design workspace; dragging components from the component library to the canvas, and configuring the layers in the configuration window, the model architecture description file adds layers to the layer set, and generates different tensors in the tensor set according to different layer configurations; using directed edges to connect the output anchor point of the starting layer to the input anchor point of the end layer, and generating and persisting the model architecture description file. The present invention provides the ability to quickly design a model architecture. Through platform task deployment, the model can be efficiently deployed to the private cloud specified by the tenant.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning platform deployment, and in particular to a model architecture design method and a deployment system for a multi-tenant machine learning platform. Background Art

[0002] With the rapid development of machine learning, especially deep learning, more and more businesses and users are looking to train machine learning models to inform business decisions. Machine learning platforms offer comprehensive capabilities, including data management, architecture design, training task management, model management, and deployment management. However, most mainstream machine learning platforms are developed internally by large enterprises and do not offer external services. Small and medium-sized enterprises cannot afford the development costs of machine learning platforms.

[0003] Thanks to the multi-tenant machine learning platforms provided by software-as-a-service providers, which are easy to expand and pay on demand, more and more small and medium-sized enterprises and users choose to rent machine learning platforms to achieve rapid model development and deployment. Unlike privately deployed machine learning platforms, multi-tenant machine learning platforms need to have universal ease of use when designing model architecture. Therefore, it is of great practical significance to study machine learning platform deployment technology to meet the service quality promised to tenants and the tenants' customized requirements for user task management when deploying training tasks, and to achieve rapid private cloud deployment when deploying models. Summary of the Invention

[0004] In order to overcome the defects and shortcomings of the existing technology, the present invention provides a model architecture design method and deployment system for a multi-tenant machine learning platform. The present invention provides the ability to quickly design the model architecture by constructing an extensible model file architecture description file; scheduling tasks through a two-level scheduling mechanism not only ensures the service quality promised by the platform to tenants, but also meets the tenants' personalized needs for user management; through platform task deployment, tenants can efficiently deploy models to the tenant-specified private cloud.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a model architecture design method for a multi-tenant machine learning platform, comprising the following steps:

[0007] Constructing a visual model architecture design device, wherein the visual model architecture design device includes a component library, a canvas, and a configuration window;

[0008] Generate an easily extensible model architecture description file based on a visual model architecture design device. The model architecture description file includes basic information of the model architecture, layer sets, and tensor sets.

[0009] Create a model architecture design workspace, fill in the basic information of the model, or load the model architecture design workspace;

[0010] Drag components from the component library to the canvas and configure the layers in the configuration window. The model architecture description file adds layers to the layer collection and generates different tensors in the tensor collection based on different layer configurations.

[0011] According to the order between layers, a directed edge is used to connect the output anchor point of the starting layer to the input anchor point of the ending layer. The model architecture description file modifies the input tensor identifier and output tensor identifier of the layer;

[0012] Generate and persist model architecture description files.

[0013] As a preferred technical solution, the component library includes multiple linear layers, convolutional layers, pooling layers, and dropout layers;

[0014] The canvas is provided with a model input layer, and the various layers of the component library are arranged and connected on the canvas;

[0015] The configuration window is used to set and adjust the parameters of the component.

[0016] As a preferred technical solution, the basic information of the model architecture includes: model ID, model name, label, author, creation time, modification time, problem type, and supported image versions;

[0017] The layer set includes: layer ID, layer input tensor identifier, layer output tensor identifier, non-input and output tensor set, layer type, and other parameter configurations of the layer;

[0018] The tensor set includes: tensor ID, tensor shape, initialization method, and tensor attributes.

[0019] The present invention also provides a training task deployment system for a multi-tenant machine learning platform, comprising: a model architecture acquisition module, an interface service module, a scheduling module, and a control module;

[0020] The architecture acquisition module of the model obtains a model architecture description file based on the above-mentioned model architecture design method for a multi-tenant machine learning platform;

[0021] The interface service module is used to receive training task requests from the client, verify the parameters of the training task, verify the architecture of the model, check the legitimacy of the training request, and forward the task to the scheduling module;

[0022] The scheduling module is used to decide the node where the training task actually runs, selects the scheduling task from the scheduling queue according to the secondary scheduling algorithm, and sends the task request to the control module;

[0023] The control module is used to respond to the scheduling results of the scheduling module, execute the operations notified by the task scheduling module, and monitor the node and task status. When there are idle resources, it notifies the task scheduling module and issues commands to the node to perform the corresponding training task preparation and training task deployment;

[0024] The training task preparation process includes: downloading the model architecture description file to the node's file system, downloading the training and test data to the node's file system;

[0025] The process of training task deployment includes: checking whether the node has a training task execution image that meets the version requirements. If not, pull it from the image repository; creating the corresponding container, mounting the model architecture description file into the container, downloading the training code image that the model depends on, starting the container and monitoring the container status, and configuring environment variables for the container according to the task request.

[0026] As a preferred technical solution, the scheduling module maintains a scheduling queue for each tenant. Each scheduling queue stores training task requests submitted by users. Each scheduling queue has a priority and has a two-level scheduling mechanism. The first-level scheduling is determined by the service quality and waiting time promised by the machine learning platform to the tenant, and the second-level scheduling is personalized by the tenant.

[0027] When the scheduling module schedules a task, the first-level scheduling selects the tenant queue. The second-level scheduling selects a training task from the tenant's scheduling queue according to the scheduling algorithm configured by the tenant. If the idle resources of CPU and memory cannot meet the requirements of the training task, rescheduling is performed until the scheduling is successful or the maximum number of retries is reached.

[0028] After finding a scheduling task that meets the requirements, the weight of the tenant queue is readjusted and the task request is sent to the control module.

[0029] As a preferred technical solution, the scheduling module calculates the priority of each tenant queue, uses the priority as a weight, and randomly selects a tenant queue for scheduling according to the weighted method. The priority is expressed as:

[0030]

[0031] Among them, len(Q t ) represents the scheduling queue Q t The length of p init(t) Indicates tenant priority, P wait(t) Indicates waiting time priority;

[0032] If the scheduling is determined to be successful, the waiting time priority corresponding to the tenant queue is set to 0, and the scheduling task is forwarded to the control module.

[0033] The present invention also provides a model deployment system for a multi-tenant machine learning platform, which is provided with the above-mentioned training task deployment system for a multi-tenant machine learning platform, and further includes: a server synchronization module, a client synchronization module, a client computing module, and a client interface module;

[0034] The server synchronization module is used to process requests from the client synchronization module, including model file synchronization requests, model basic information synchronization requests, client computing module code synchronization requests, and script synchronization requests;

[0035] The client synchronization module is used to receive and send requests to the server synchronization module, including model file synchronization requests, model basic information synchronization requests, and client computing module code synchronization requests;

[0036] The client computing module is used to read and load the model file and process the inference request from the client interface module;

[0037] The client interface module is used to process HTTP requests for online reasoning, specifically including: verifying whether the model exists, verifying whether the request parameters are legal, forwarding the reasoning request to the client computing module, and returning the processing results of the client computing module to the user.

[0038] As a preferred technical solution, the server synchronization module processes the request from the client synchronization module. When the server synchronization module receives the model file synchronization request from the client, it sends the model architecture and parameters to the client file system;

[0039] When the server-side synchronization module receives a request for synchronization of basic model information from the client, it sends the basic model information to the client;

[0040] When the server-side synchronization module receives a synchronization request from the client-side calculation module, it sends the latest calculation module code to the client, and the client updates the calculation module code;

[0041] When the server receives a script synchronization request from the client, it sends the initialization script to the client, and the client runs the initialization script.

[0042] The present invention also provides a model private cloud deployment system for a multi-tenant machine learning platform, which is provided with the above-mentioned model deployment system for the multi-tenant machine learning platform and a client database, wherein the client database is used to store basic information of the model architecture.

[0043] As a preferred technical solution, when it is determined that the user is deploying for the first time, the machine learning platform generates a command with a token for the user, which initiates a request to the server to download the initialization script;

[0044] Download the client synchronization module, client calculation module and client interface module;

[0045] Initialize the operating environment and database, including: downloading the dependencies required by the code and creating the model information data table;

[0046] Start the client program, and the client synchronization module verifies the client version once;

[0047] When it is determined that this is not the first deployment for the user and the client code already exists on the private host, a synchronization command is initiated through the client interface module. The machine learning platform generates a model file synchronization command carrying a token for the user. By calling the new model request of the client interface module, the client interface module calls the client synchronization module, and the client synchronization module establishes a connection with the server synchronization module. The server synchronization module checks the legitimacy of the model synchronization request and returns the trained model file and basic model information parameters to the client synchronization module. The client synchronization module persists the basic model information in the database and the model file in the file system.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] The present invention constructs an extensible model file architecture description file through an easy-to-use drag-and-drop construction method, providing the ability to quickly design the model architecture; scheduling tasks through a two-level scheduling mechanism not only ensures the service quality promised by the platform to tenants, but also meets the tenants' personalized needs for user management; through platform task deployment, tenants can efficiently deploy models to the tenant-specified private cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the architecture of a device for designing a visual model architecture for the present invention;

[0051] Figure 2 A schematic diagram of a process for generating an easily extensible model architecture description file for the present invention;

[0052] Figure 3 This is a schematic diagram of the architecture of a training task deployment system for a multi-tenant machine learning platform according to the present invention;

[0053] Figure 4 Schematic diagram of the process of implementing the training task deployment system for a multi-tenant machine learning platform of the present invention;

[0054] Figure 5 A schematic diagram of the process of selecting a task from a scheduling queue by the scheduling module of the present invention;

[0055] Figure 6This is a flow chart of scheduling performed by the scheduling module of the present invention based on a weighted random algorithm;

[0056] Figure 7 This is a schematic diagram of the architecture of the model deployment system of the multi-tenant machine learning platform of the present invention;

[0057] Figure 8 Schematic diagram of the process of private cloud deployment of models for the multi-tenant machine learning platform of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Example 1

[0060] This embodiment provides a model architecture design method for a multi-tenant machine learning platform, including the following steps:

[0061] S1: If Figure 1 As shown, a visual model architecture design device is constructed, and the visual model architecture design device includes a component library, a canvas, and a configuration window;

[0062] In this embodiment, the component library is a key part. It contains various layers commonly used by users, allowing users to easily build the models they need. Commonly used layers include linear layers, convolutional layers, pooling layers, dropout layers, etc. Users can select and add them to their models from the component library as needed; at the same time, the component library also supports user-defined components.

[0063] In this embodiment, the canvas is the main area for users to visually design the model architecture. There is a model input layer on the canvas by default, which provides an interactive interface that allows users to drag and drop various layers in the component library onto the canvas and arrange and connect them according to their needs. Through the canvas, users can intuitively build the model structure and view the model architecture in real time for adjustment and optimization.

[0064] In this embodiment, the configuration window is an important supplement to the canvas. It provides detailed configuration and parameter adjustment functions for model components. When the user selects a component on the canvas, the corresponding configuration window will be displayed, allowing the user to set and adjust the parameters of the component. For example, the user can specify the input dimension of a layer, the type of activation function, and other parameters related to the model structure in the configuration window.

[0065] S2: Generate an easily extensible model architecture description file based on the visual model architecture design device. The model architecture description file includes basic information of the model architecture, layer sets, and tensor sets;

[0066] like Figure 2 As shown, the specific steps of generating an easily extensible model architecture description file based on the visual model architecture design device include:

[0067] S201: Create a model architecture design workspace, fill in the basic information of the model, or load an existing model architecture design workspace;

[0068] S202: Drag components from the component library to the canvas, and configure the layers in the configuration window;

[0069] The model architecture description file adds layers to the layer set and generates different tensors in the tensor set according to the different layer configurations.

[0070] In this embodiment, when a component is dragged to the canvas, a unique id is generated and the layer information is written to the layers of the model architecture description file; depending on the layer type, different numbers of tensors are generated and added to the tensors of the model architecture description file, as well as to the related_tensor_ids of the layer; the relevant configuration of the layer is modified, and the relevant parameters are updated to the layer's layer.params. For example, when the activation function of the convolution layer is modified, the activation function in layer.params is modified; if tensors are involved, the tensor information should be modified synchronously. For example, when the convolution kernel size of the convolution layer is modified, the tensor corresponding to the convolution kernel should be modified synchronously with tensor.shape.

[0071] S203: According to the order between layers, use directed edges to connect the output anchor point of the starting layer to the input anchor point of the end layer. The number of input and output anchor points of different layers can be different. The model architecture description file modifies the input tensor identifier and output tensor identifier of the layer, generates and persists the model architecture description file.

[0072] In this embodiment, the basic information of the model architecture is information that is not related to the model architecture hierarchy, and specifically includes: model ID, model name, label, author, creation time, modification time, problem type, and supported image versions;

[0073] The layer set includes all layers. The layer set is a list that stores information about all model layers. The layers are connected through the input and output tensor identifiers. Specifically, it includes: layer ID, layer input tensor identifier, layer output tensor identifier, non-input and output tensor set, layer type, and other parameter configurations of the layer;

[0074] A tensor collection is a list that stores tensors generated during the calculation process, including tensor ID, tensor shape, initialization method, and tensor attributes.

[0075] Table 1 below shows an example of the model architecture description file meta information:

[0076] Table 1 Meta-information of model architecture description file

[0077]

[0078]

[0079] In this embodiment, the model architecture description file is used to describe the model and properties, such as the model name, the model hierarchy, and the model tensor information. A model architecture description file can be used for only one model, that is, one model corresponds to one model architecture description file; a model architecture description file can also be used for multiple models, that is, one model corresponds to multiple model architecture description files; a model architecture description file can also be regarded as a kind of "layer", and the model architecture description file can be regarded as a splicing description of various types of "layers". A type of "layer" can be used in multiple models, that is, a model architecture description file can be used for only one model or for multiple models. A model package can contain multiple sub-models, that is, the sub-model itself can also be used as an independent model; when a model contains multiple sub-models, it can be regarded as a combination of multiple independent models spliced ​​together; the sub-model can be provided in the component library or a user-designed model;

[0080] In this embodiment, the component library of the visual model architecture design device supports expansion. The model constructed by oneself can be uploaded to the component library as a component, and the shared models of users under the same tenant space can also be shared as components to the component library.

[0081] Example 2

[0082] like Figure 3 As shown, this embodiment provides a training task deployment system for a multi-tenant machine learning platform, including: a model architecture acquisition module, an interface service module, a scheduling module, and a control module;

[0083] like Figure 4 As shown, the implementation process of the training task deployment system for a multi-tenant machine learning platform in this embodiment specifically includes:

[0084] S401: The interface service module receives a request to deploy a training task T1.

[0085] S402, the interface service module performs parameter verification on T1's request;

[0086] S403: The interface service module forwards the training task request to the scheduling module;

[0087] S404: The scheduling module processes the training task request transmitted by the interface service, performs scheduling, and sends the scheduling result to the control module;

[0088] S405: The control module processes the scheduling result transmitted by the scheduling module and executes the deployment;

[0089] In this embodiment, the interface service module is used to receive training task requests from the client, verify the parameters of the training task, verify the architecture of the model, verify the legitimacy of the training request, and forward the task to the scheduling module;

[0090] In this embodiment, the parameters of the training task are verified, including: verifying idempotence, verifying whether the batch size is legal, verifying whether the number of iterations is legal, verifying whether the feature name is in the column name set of the dataset, verifying whether the label name is in the column name set of the dataset, and verifying whether the normalization method is legal;

[0091] In this embodiment, verifying the model architecture includes: reading the model architecture description file, starting from the layer with input_tensor_id of -1 in layers, finding the layer with input_tensor_id of the current value from layers according to output_tensor_id, and checking whether a directed acyclic graph (DAG) can be constructed;

[0092] Read the model architecture description file, traverse all tensors in tensors in sequence, and check whether the tensor initialization method is in the supported method set;

[0093] If the layer in the model architecture is a user-defined sub-model, recursive verification should be performed;

[0094] In this embodiment, the verification method adopted can ensure that the model can be trained normally logically.

[0095] The scheduling module is used to decide the node where the training task actually runs. It selects a task from the scheduling queue according to the secondary scheduling algorithm and notifies the control module to complete the corresponding operation.

[0096] In this embodiment, the scheduling module maintains a scheduling queue Q for each tenant. (t) , where the subscript t represents the tenant, and the scheduling queue Q (t) , with but not limited to three parameter values: tenant priority P init(t) , waiting time priority P wait(t), tenant self-configuration scheduling algorithm C (t) ;

[0097] Among them, the tenant priority P init(t) The multi-tenant machine learning platform determines the quality of service promised to the tenants, and the scheduling module obtains it by calling the interface module; the waiting time priority P wait(t) The initial value is 0 and it will be automatically incremented each time a scheduling failure occurs. The tenant-configured scheduling algorithm can include but is not limited to: first-come, first-served, shortest job first, order by the job submitter's rank, order by the urgency of the project described in the task, etc. It can also be a scheduling algorithm designed by the user.

[0098] Each scheduling queue stores training task requests submitted by users. Each scheduling queue has a priority. The priority is calculated based on the quality of service promised by the multi-tenant machine learning platform to the tenant. It has a two-level scheduling mechanism. The first-level scheduling is determined by the quality of service and waiting time promised by the machine learning platform to the tenant, and the second-level scheduling is customized by the tenant.

[0099] When the scheduling module schedules a task, it first performs a first-level scheduling to select a tenant queue, and then performs a second-level scheduling. The second-level scheduling selects a training task from the tenant's scheduling queue based on the scheduling algorithm configured by the tenant. When the idle resources of CPU and memory cannot meet the requirements of the training task, rescheduling is performed until the scheduling is successful or the maximum number of retries is reached. The two-level scheduling mechanism of the scheduling module can not only ensure the service quality promised by the machine learning platform to tenants, but also meet the tenants' personalized needs for user management.

[0100] like Figure 5 As shown, when a scheduling task is triggered, the priority of each scheduling queue is calculated, and the priority is used as a weight. A weighted random selection method is used to select a scheduling queue. The selected queue should return a tenant task according to the scheduling algorithm customized by the tenant. The scheduling module checks the resources required by the tenant. If the requirements are not met, the queue is rescheduled until the requirements are met or the maximum number of attempts is reached. If the maximum number of attempts is not successful, the current scheduling is abandoned and the next allocation is waited for. After finding a scheduling task that meets the requirements, the weight of the tenant queue is readjusted to avoid tenant starvation, and then the task request is sent to the control module.

[0101] like Figure 6 As shown, the scheduling module obtains the tenant queue Q according to the tenant identifier of task T (t) If not, call the interface service module interface to request tenant priority configuration P init(t) And tenant scheduling algorithm configuration C (t), and create a scheduling queue to check whether the cluster meets T's memory and CPU requirements. If so, it directly forwards T to the controller and sets the waiting time priority P of the tenant queue. wait(t) Set to 0; if not satisfied, add the task to the scheduling queue Q corresponding to the tenant (t) ;

[0102] When the control module detects that the training task is finished and the resources are released, the control module notifies the scheduling module to start scheduling. The scheduling module calculates the priority P of each tenant queue. (t) ,According to the following formula, the priority is used as the weight, and a tenant queue is selected for scheduling according to weighted random selection.

[0103] where len(Q t ) represents Q t Length

[0104] According to the tenant's preset scheduling algorithm configuration C, the task T to be scheduled is obtained * .

[0105] Check whether there is resource scheduling T in the cluster * If not, retry until the scheduling succeeds or the maximum number of attempts is reached.

[0106] If the scheduling is successful, the P corresponding to the tenant queue wait(t) Set to 0, and set task T * Forwarded to the control module. If the scheduling fails and reaches the maximum number of attempts, the current scheduling is abandoned and waits for the next scheduling.

[0107] In this embodiment, the tenant queue priority calculation method of the secondary scheduling algorithm can also be other algorithms that comprehensively consider tenant priority and tenant queue waiting time.

[0108] The control module is used to respond to the scheduling results of the scheduling module, execute the operations notified by the task scheduling module, and monitor the node and task status. When there are idle resources, it notifies the task scheduling module and issues commands to the node to perform the corresponding training task preparation and training task deployment;

[0109] The training task preparation process includes: downloading the model architecture description file to the node's file system, and downloading the training and test data to the node's file system;

[0110] The process of training task deployment includes: checking whether the node has a training task execution image that meets the version requirements. If not, pull it from the image repository; creating the corresponding container, mounting the model architecture description file into the container, downloading the training code image that the model depends on, starting the container and monitoring the container status, and configuring environment variables for the container according to the task request.

[0111] In this embodiment, the control module can control the nodes in the cluster through remote procedure calls. The control method is not limited to remote procedure calls, but can also be other forms of inter-node communication. The control node downloads the model architecture description file, downloads the training data and test data, and downloads the training task image. The format used by the image can be, but is not limited to, a Docker image. The image can parse the model architecture description file, convert the model architecture description file into executable training code, process the training data and test data, and persist the status and results of the model training process. The machine learning computing engine used by the image can be, but is not limited to, PyTorch and Tensorflow.

[0112] The interface service module receives the request for the training task, verifies the parameters of the training task, and forwards the task to the task scheduling module. The task scheduling module uses a two-level scheduling algorithm to forward the task to the control module for task deployment. It can fully consider the service quality commitment of the multi-tenant machine learning platform to tenants and the management needs of tenants for users.

[0113] Example 3

[0114] like Figure 7 As shown, this embodiment provides a model deployment system for a multi-tenant machine learning platform, which can quickly deploy models to a private cloud environment. The system is provided with a training task deployment system for a multi-tenant machine learning platform, and specifically also includes: a server synchronization module, a client synchronization module, a client computing module, and a client interface module;

[0115] The server-side synchronization module is used to process requests from the client-side synchronization module, including model file synchronization requests, model basic information synchronization requests, client-side calculation module code synchronization requests, and script synchronization requests;

[0116] Specifically, when the server-side synchronization module receives a model file synchronization request from the client, it sends the model's architecture and parameters to the client's file system; when the server-side synchronization module receives a model basic information synchronization request from the client, it sends the model's basic information to the client, and the client persists it in the client database; when the server-side synchronization module receives a calculation module synchronization request from the client, it sends the latest calculation module code to the client, and the client updates the calculation module code; when the server receives a script synchronization request from the client, it sends the initialization script to the client, and the client runs the initialization script to achieve automated deployment;

[0117] In this embodiment, the server-side synchronization module may also have a component for remotely operating the private cloud host.

[0118] The client synchronization module is used to receive and send requests to the server synchronization module, including model file synchronization requests, model basic information synchronization requests, and client calculation module code synchronization requests.

[0119] The client computing module is used to read and load model files and process inference requests from the client interface module.

[0120] The client interface module is used to process HTTP requests for online reasoning, specifically including: verifying whether the model exists; verifying whether the request parameters are legal; forwarding the reasoning request to the client computing module; and returning the processing results of the client computing module to the user.

[0121] Example 4

[0122] This embodiment provides a model private cloud deployment system for a multi-tenant machine learning platform, including a model deployment system for a multi-tenant machine learning platform and a client database, wherein the client database is used to store basic information of the model architecture;

[0123] like Figure 8 As shown in the figure, when the user deploys for the first time, the machine learning platform generates a command with a token for the user. The token is time-limited. The command is used to download a script file on the host and execute it. The command initiates a request to the server to download the initialization script. By obtaining the script file with the token, it can be quickly deployed to a private host. The script file includes: checking the dependency environment, such as Python, Pip, and dependency packages. If the version is incorrect or not installed, the script will automatically install the correct version; downloading the client program, including: client synchronization module, client calculation module, client interface module; initializing the operating environment and database, including downloading the dependencies required by the code, creating the model information data table, starting the client program, and the client synchronization module automatically verifies the client version. If the version is too old, it updates to the latest version and restarts the client program.

[0124] In this embodiment, when the user is not deploying for the first time and the client code already exists on the private host, the user only needs to initiate a synchronization command through the client interface module to deploy the model for inference service. The machine learning platform generates a model file synchronization command carrying a token for the user. The user logs in to the private cloud host and runs the command generated by the machine learning platform. Specifically, the user calls the client interface module to request a new model, the client interface module calls the client synchronization module, the client synchronization module establishes a connection with the server synchronization module, the server synchronization module checks the legitimacy of the model synchronization request, and returns the trained model file, basic model information and other parameters to the client synchronization module. The client synchronization module persists the basic model information in the database and persists the model file in the file system.

[0125] In this embodiment, it also includes accessing the port where the program is located and executing online reasoning steps. The user performs online reasoning including reasoning of single data, reasoning of batch data, by accessing the UI that already exists in the client program, by accessing the Restful API of the client interface module, and by using the command line.

[0126] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A model architecture design method for a multi-tenant machine learning platform, characterized in that: The steps include: Constructing a visual model architecture design device, wherein the visual model architecture design device includes a component library, a canvas, and a configuration window; The component library includes multiple linear layers, convolutional layers, pooling layers, and dropout layers; The canvas is provided with a model input layer, and the various layers of the component library are arranged and connected on the canvas; The configuration window is used to set and adjust the parameters of the component; Generate an easily extensible model architecture description file based on a visual model architecture design device. The model architecture description file includes basic information of the model architecture, layer sets, and tensor sets. The basic information of the model architecture includes: model ID, model name, tags, author, creation time, modification time, problem type, and supported image versions; The layer set includes: layer ID, layer input tensor identifier, layer output tensor identifier, non-input and output tensor set, and layer type; The tensor set includes: tensor ID, tensor shape, initialization method, and tensor attributes; The specific steps of generating an easily extensible model architecture description file based on the visual model architecture design device include: Create a model architecture design workspace, fill in the basic information of the model, or load an existing model architecture design workspace; Drag components from the component library to the canvas and configure the layers in the configuration window. The model architecture description file adds layers to the layer collection and generates different tensors in the tensor collection based on different layer configurations. According to the order between layers, a directed edge is used to connect the output anchor point of the starting layer to the input anchor point of the ending layer. The model architecture description file modifies the input tensor identifier and output tensor identifier of the layer; Generate and persist model architecture description files.

2. A training task deployment system for a multi-tenant machine learning platform, characterized in that: include: The model's architecture acquisition module, interface service module, scheduling module, and control module; The architecture acquisition module of the model obtains a model architecture description file based on the model architecture design method for a multi-tenant machine learning platform according to claim 1; The interface service module is used to receive training task requests from the client, verify the parameters of the training task, verify the architecture of the model, check the legitimacy of the training request, and forward the task to the scheduling module; The scheduling module is used to decide the node where the training task actually runs, selects the scheduling task from the scheduling queue according to the secondary scheduling algorithm, and sends the task request to the control module; The control module is used to respond to the scheduling results of the scheduling module, execute the operations notified by the task scheduling module, and monitor the node and task status. When there are idle resources, it notifies the task scheduling module and issues commands to the node to perform the corresponding training task preparation and training task deployment; The training task preparation process includes: downloading the model architecture description file to the node's file system, downloading the training and test data to the node's file system; The process of training task deployment includes: checking whether the node has a training task execution image that meets the version requirements. If not, pull it from the image repository; creating the corresponding container, mounting the model architecture description file into the container, downloading the training code image that the model depends on, starting the container and monitoring the container status, and configuring environment variables for the container according to the task request.

3. The training task deployment system for a multi-tenant machine learning platform according to claim 2, characterized in that: The scheduling module maintains a scheduling queue for each tenant. Each scheduling queue stores training task requests submitted by users. Each scheduling queue has a priority and has a two-level scheduling mechanism. The first-level scheduling is determined by the service quality and waiting time promised by the machine learning platform to the tenant, and the second-level scheduling is customized by the tenant. When the scheduling module schedules a task, the first-level scheduling selects the tenant queue. The second-level scheduling selects a training task from the tenant's scheduling queue according to the scheduling algorithm configured by the tenant. If the idle resources of CPU and memory cannot meet the requirements of the training task, rescheduling is performed until the scheduling is successful or the maximum number of retries is reached. After finding a scheduling task that meets the requirements, the weight of the tenant queue is readjusted and the task request is sent to the control module.

4. The training task deployment system for a multi-tenant machine learning platform according to claim 2, characterized in that: The scheduling module calculates the priority of each tenant queue, uses the priority as a weight, and randomly selects a tenant queue for scheduling based on the weight. The priority is expressed as: ； in, Represents a scheduling queue length, Indicates tenant priority, Indicates waiting time priority; If the scheduling is determined to be successful, the waiting time priority corresponding to the tenant queue is set to 0, and the scheduling task is forwarded to the control module.

5. A model deployment system for a multi-tenant machine learning platform, characterized in that: A training task deployment system for a multi-tenant machine learning platform according to any one of claims 2 to 4 is provided, further comprising: a server synchronization module, a client synchronization module, a client computing module, and a client interface module; The server synchronization module is used to process requests from the client synchronization module, including model file synchronization requests, model basic information synchronization requests, client computing module code synchronization requests, and script synchronization requests; The client synchronization module is used to receive and send requests to the server synchronization module, including model file synchronization requests, model basic information synchronization requests, and client computing module code synchronization requests; The client computing module is used to read and load the model file and process the inference request from the client interface module; The client interface module is used to process HTTP requests for online reasoning, specifically including: verifying whether the model exists, verifying whether the request parameters are legal, forwarding the reasoning request to the client computing module, and returning the processing results of the client computing module to the user.

6. The model deployment system for a multi-tenant machine learning platform according to claim 5, characterized in that: The server synchronization module processes the request from the client synchronization module. When the server synchronization module receives the model file synchronization request from the client, it sends the model architecture and parameters to the client file system; When the server-side synchronization module receives a request for synchronization of basic model information from the client, it sends the basic model information to the client; When the server-side synchronization module receives a synchronization request from the client-side calculation module, it sends the latest calculation module code to the client, and the client updates the calculation module code; When the server receives a script synchronization request from the client, it sends the initialization script to the client, and the client runs the initialization script.

7. A model private cloud deployment system for a multi-tenant machine learning platform, characterized in that: A model deployment system for a multi-tenant machine learning platform according to any one of claims 5-6 is provided, and a client database is also provided, wherein the client database is used to store basic information of the model architecture.

8. The model private cloud deployment system of the multi-tenant machine learning platform according to claim 7, characterized in that: When it is determined that the user is deploying for the first time, the machine learning platform generates a command with a token for the user, which initiates a request to the server to download the initialization script; Download the client synchronization module, client calculation module and client interface module; Initialize the operating environment and database, including: downloading the dependencies required by the code and creating the model information data table; Start the client program and the client synchronization module to verify the client version; When it is determined that this is not the first deployment for the user and the client code already exists on the private host, a synchronization command is initiated through the client interface module. The machine learning platform generates a model file synchronization command carrying a token for the user. By calling the new model request of the client interface module, the client interface module calls the client synchronization module, and the client synchronization module establishes a connection with the server synchronization module. The server synchronization module checks the legitimacy of the model synchronization request and returns the trained model file and basic model information parameters to the client synchronization module. The client synchronization module persists the basic model information in the database and the model file in the file system.

Images