Method, model processing method and apparatus for server deployment model

Abstract

The application relates to a method and device for deploying a model of a server, a computer device and a storage medium. The method comprises the following steps: obtaining a model deployment package corresponding to a to-be-deployed model, wherein the model deployment package comprises model configuration information corresponding to the to-be-deployed model; generating a model image corresponding to the to-be-deployed model based on the model deployment package; generating model service configuration information corresponding to the to-be-deployed model based on the model image and the model configuration information; creating a container control component and a service component corresponding to the container control component based on the model service configuration information; wherein the container control component is used for dynamically generating and controlling a model container set corresponding to the to-be-deployed model, and the container control component and the service component are used for cooperating to provide a model processing service corresponding to the to-be-deployed model, and the model processing service is used for distributing a model processing request corresponding to the to-be-deployed model into a model container in the model container set for processing. The method can improve the model management efficiency.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, model processing method, apparatus, computer device, and storage medium for deploying server models. Background Technology

[0002] With the development of computer technology, machine learning models have emerged. Machine learning models learn the inherent patterns and representational levels of sample data to predict and analyze new data, possessing powerful data processing capabilities. Therefore, machine learning models are widely used across various industries. In the application of machine learning models, model deployment is a crucial step.

[0003] In existing technologies, machine learning models are manually deployed on servers, lacking an effective model management mechanism. The operation and maintenance of machine learning models mainly rely on manual handling by operation and maintenance personnel, resulting in low model management efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, model processing method, apparatus, computer equipment, and storage medium for deploying models on a server that can improve model management efficiency in response to the above-mentioned technical problems.

[0005] A method for deploying a server model, the method comprising:

[0006] Obtain the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed;

[0007] Generate a model image corresponding to the model to be deployed based on the model deployment package;

[0008] Based on the model image and the model configuration information, generate the model service configuration information corresponding to the model to be deployed;

[0009] Based on the model service configuration information, a container control component and a corresponding service component are created.

[0010] The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component are used to cooperate to provide the model processing service corresponding to the model to be deployed. The model processing service is used to allocate the model processing request corresponding to the model to be deployed to the model container in the model container set for processing.

[0011] An apparatus for a server deployment model, the apparatus comprising:

[0012] The deployment package acquisition module is used to acquire the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed;

[0013] The image generation module is used to generate a model image corresponding to the model to be deployed based on the model deployment package;

[0014] The configuration information generation module is used to generate model service configuration information corresponding to the model to be deployed based on the model image and the model configuration information;

[0015] The component creation module is used to create a container control component and a service component corresponding to the container control component based on the model service configuration information.

[0016] The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component are used to cooperate to provide the model processing service corresponding to the model to be deployed. The model processing service is used to allocate the model processing request corresponding to the model to be deployed to the model container in the model container set for processing.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0018] Obtain the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed;

[0019] Generate a model image corresponding to the model to be deployed based on the model deployment package;

[0020] Based on the model image and the model configuration information, generate the model service configuration information corresponding to the model to be deployed;

[0021] Based on the model service configuration information, a container control component and a corresponding service component are created.

[0022] The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component are used to cooperate to provide the model processing service corresponding to the model to be deployed. The model processing service is used to allocate the model processing request corresponding to the model to be deployed to the model container in the model container set for processing.

[0023] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0024] Obtain the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed;

[0025] Generate a model image corresponding to the model to be deployed based on the model deployment package;

[0026] Based on the model image and the model configuration information, generate the model service configuration information corresponding to the model to be deployed;

[0027] Based on the model service configuration information, a container control component and a corresponding service component are created.

[0028] The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component are used to cooperate to provide the model processing service corresponding to the model to be deployed. The model processing service is used to allocate the model processing request corresponding to the model to be deployed to the model container in the model container set for processing.

[0029] A model processing method, the method comprising:

[0030] Obtain the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed;

[0031] The target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component are determined. The target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information.

[0032] Based on the target container control component, the target container is determined from the set of model containers corresponding to the target model;

[0033] The data to be processed is processed based on the target container to obtain the data processing result, and the data processing result is returned to the terminal.

[0034] A model processing apparatus, the apparatus comprising:

[0035] The request acquisition module is used to acquire model processing requests sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed;

[0036] The component determination module is used to determine the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component; the target container control component is used to dynamically generate and control the model container set corresponding to the target model; the target container control component and the target service component are used to cooperate to provide the model processing service corresponding to the target model; the target container control component and the target service component are generated based on the model service configuration information corresponding to the target model; the model service configuration information is generated based on the model image and model configuration information corresponding to the target model; the model image is generated based on the model deployment package corresponding to the target model; the model deployment package includes the model configuration information.

[0037] A container determination module is used to determine a target container from the set of model containers corresponding to the target model based on the target container control component.

[0038] The data processing module is used to process the data to be processed based on the target container to obtain the data processing result, and return the data processing result to the terminal.

[0039] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0040] Obtain the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed;

[0041] The target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component are determined. The target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information.

[0042] Based on the target container control component, the target container is determined from the set of model containers corresponding to the target model;

[0043] The data to be processed is processed based on the target container to obtain the data processing result, and the data processing result is returned to the terminal.

[0044] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0045] Obtain the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed;

[0046] The target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component are determined. The target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information.

[0047] Based on the target container control component, the target container is determined from the set of model containers corresponding to the target model;

[0048] The data to be processed is processed based on the target container to obtain the data processing result, and the data processing result is returned to the terminal.

[0049] The aforementioned server deployment model method, model processing method, apparatus, computer equipment, and storage medium, through obtaining the model deployment package corresponding to the model to be deployed (the model deployment package includes model configuration information corresponding to the model to be deployed), generating a model image corresponding to the model to be deployed based on the model deployment package, generating model service configuration information corresponding to the model to be deployed based on the model image and model configuration information, and creating a container control component and a corresponding service component based on the model service configuration information. The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component work together to provide model processing services corresponding to the model to be deployed. The model processing service is used to distribute model processing requests corresponding to the model to be deployed to model containers within the model container set for processing. In this way, the model deployment package based on the model can automatically create the corresponding container control component and service component, thereby automatically deploying the model on the server and providing model processing services to the outside world through the cooperation of the container control component and service component. Furthermore, the container control component can dynamically generate and control the model container set, realizing automatic intelligent operation and maintenance of the model, effectively improving model management efficiency. Attached Figure Description

[0050] Figure 1 This is an application environment diagram of a server deployment model method and a model processing method in one embodiment.

[0051] Figure 2 This is a flowchart illustrating a method for deploying a server model in one embodiment;

[0052] Figure 3A This is a schematic diagram of the interface for model deployment in one embodiment;

[0053] Figure 3B This is a schematic diagram of the model deployment process in another embodiment;

[0054] Figure 4 This is a schematic diagram illustrating the process of creating a container management component and a service component in one embodiment;

[0055] Figure 5 This is a flowchart illustrating a model processing method in one embodiment;

[0056] Figure 6A This is a schematic diagram illustrating the process of deploying a model on a server in one embodiment;

[0057] Figure 6B This is a schematic diagram of the architecture of the model management system in one embodiment;

[0058] Figure 7 This is a structural block diagram of an apparatus for deploying a server in one embodiment;

[0059] Figure 8 This is a structural block diagram of the model processing device in one embodiment;

[0060] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0062] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0063] Cloud computing refers to the delivery and usage model of IT infrastructure, meaning obtaining necessary resources in an on-demand and easily scalable manner through a network. In a broader sense, cloud computing also refers to the delivery and usage model of services, meaning obtaining necessary services in an on-demand and easily scalable manner through a network. These services can be IT and software related, internet-related, or other services. Cloud computing is a product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0064] With the development of the internet, real-time data streams, and the diversification of connected devices, as well as the demands for search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Unlike previous parallel distributed computing, cloud computing will revolutionize the entire internet model and enterprise management model from a conceptual standpoint.

[0065] Artificial intelligence cloud services are generally also known as AIaaS (AI as a Service). This is currently a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​frameworks and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.

[0066] The solutions provided in this application involve cloud technologies such as cloud computing and artificial intelligence cloud services, which are specifically illustrated through the following embodiments:

[0067] The server deployment model and model processing method provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices, and server 104 can be a standalone server, a server cluster consisting of multiple servers, or a cloud server.

[0068] Specifically, the server obtains the model deployment package corresponding to the model to be deployed from the terminal. The model deployment package includes the model configuration information corresponding to the model to be deployed. The server generates a model image corresponding to the model to be deployed based on the model deployment package, and generates model service configuration information corresponding to the model to be deployed based on the model image and the model configuration information. The server creates a container control component and a corresponding service component based on the model service configuration information. The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component work together to provide model processing services corresponding to the model to be deployed. The model processing services are used to distribute model processing requests corresponding to the model to be deployed to the model containers within the model container set for processing.

[0069] The server receives a model processing request sent by the terminal. This request carries the model processing service identifier corresponding to the target model and the data to be processed. The server determines the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component. Based on the target container control component, the server determines the target container from the set of model containers corresponding to the target model. Based on the target container, the server processes the data to be processed to obtain the data processing result and returns the result to the terminal. Specifically, the target container control component dynamically generates and controls the set of model containers corresponding to the target model. The target container control component and the target service component work together to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model, and the model deployment package includes the model configuration information.

[0070] The solutions provided in this application also relate to blockchain technology. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer. The model deployment package, model image, base image, and other information involved in this application can all be stored on the blockchain.

[0071] In one embodiment, such as Figure 2 As shown, a method for server deployment model is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0072] Step S202: Obtain the model deployment package corresponding to the model to be deployed. The model deployment package includes the model configuration information corresponding to the model to be deployed.

[0073] The "model to be deployed" refers to a machine learning model that is to be deployed on the server. Specifically, the machine learning model can be a model built using algorithms such as linear regression, logistic regression, and decision trees. It can be a model applied to various technical fields; for example, it can be an image recognition model or object detection model in image processing, a text classification model or text similarity calculation model in natural language processing, or a speech recognition model in audio processing. The model to be deployed can be a trained model or a model yet to be trained.

[0074] A model deployment package is a data package containing basic model data. This basic data includes model configuration information, model algorithm information, and other fundamental data used to compose the model. Model configuration information describes the model's attributes, specifically including model identifier, model type, model version information, model environment information, the model's corresponding API (Application Programming Interface), the model's corresponding port, and the model's corresponding computing resources. Model algorithm information describes the algorithm used to build the model, specifically including model structure information, model parameter information, such as the model consisting of convolutional layers, pooling layers, and fully connected layers, the model's hyperparameters, the model's loss function, and the formats of the model's input and output data. The model deployment package may further include information about the application service corresponding to the model to be deployed.

[0075] It is understandable that there can be at least one model to be deployed, and different models to be deployed can correspond to different model deployment packages.

[0076] Specifically, users can upload the model deployment package corresponding to the model to be deployed to the server via their terminals, and the server will then obtain the model deployment package. The model deployment package can be the original model data package, or it can be a compressed data package. Compression effectively reduces data transmission volume and improves data transmission efficiency. The terminal can obtain the model file and model configuration information corresponding to the model to be deployed, and package these files and information to obtain the model deployment package. After obtaining the model deployment package, the server can decompress it to obtain the model file and model configuration information, and then deploy the model based on these files and information.

[0077] Step S204: Generate the model image corresponding to the model to be deployed based on the model deployment package.

[0078] An image is a file storage format where data from one disk is copied exactly to another disk. An image file is created by formatting a specific series of files. Images can be hierarchical, consisting of layers of stacked images. A model image is the target image corresponding to the model to be deployed, used to represent the model. Model images are used to start and run model containers. One or more model containers corresponding to the model to be deployed can be started using the model image.

[0079] Specifically, the server can generate a model image corresponding to the model to be deployed based on the obtained model deployment package. The server can obtain the target base image corresponding to the model to be deployed from multiple preset base images based on the model deployment package. Using the target base image as the starting image layer and base image layer for the model to be deployed, the server further overlays image layers onto the target base image based on the model deployment package. New image layers are then overlaid on the base image layer based on the specific data of the model to be deployed in the model deployment package, thus obtaining the model image corresponding to the model to be deployed.

[0080] Step S206: Generate model service configuration information corresponding to the model to be deployed based on the model image and model configuration information.

[0081] Step S208: Create a container control component and a corresponding service component based on the model service configuration information. The container control component dynamically generates and controls the set of model containers corresponding to the model to be deployed. The container control component and the service component work together to provide model processing services for the model to be deployed. The model processing services distribute model processing requests for the model to be deployed to the model containers within the model container set for processing.

[0082] The model service configuration information refers to the configuration information corresponding to the model processing service, used to create container control components and service components that provide model processing services. The model processing service refers to the service provided by the model deployed on the server, used to process model processing requests sent by the terminal. Model processing requests can specifically be model training requests, model prediction requests, etc. The model service configuration information includes data such as version information, tag information, port information, and model container set information. Version information refers to the version number corresponding to the container control component and service component, such as V1. Tag information refers to the tags corresponding to the container control component and service component, used to bind the container control component and service component; these can be custom tags or model identifiers. Model container set information includes data such as image attribute information, computing resource information, and the number of container sets. Image attribute information refers to relevant information about the model image, such as the image name and image storage address, used to generate model containers. Port information refers to the various ports that need to be exposed, such as the ports corresponding to the model container set and the ports corresponding to the service components. Computing resource information refers to relevant information about the computing resources required by the model container set, such as the computing resource type and computing resource limits. Computing resource types can specifically include CPU (Central Processing Unit) resources, GPU (Graphics Processing Unit) resources, and CVM (Cloud Virtual Machine) resources. Computing resource limitations can specifically be usage limits, i.e., setting the scope of computing resource usage. Container set quantity information refers to the initial number of model container sets and the initial number of model containers within the model container set.

[0083] The container control component is a component that manages model container sets. It is used to dynamically generate and control model container sets, enabling operations such as scaling up, scaling down, upgrading, and rolling back. Dynamically generating model container sets refers to adding or removing model container sets, i.e., scaling up or down. The container control component can automatically monitor the working status of model container sets and dynamically adjust the number of model container sets based on the working status. If the working status is normal, the container control component can keep the number of model container sets unchanged; if the working status is busy, the container control component can add model container sets; if the working status is idle, the container control component can remove model container sets. The working status can be determined based on the container scheduling information of the model container sets, including data such as computing resource usage, request processing latency, and health status of each model container set. By dynamically generating model container sets, the container control component manages, discovers, and accesses model container sets, thus eliminating the need for application developers and operations personnel to perform complex manual configuration and processing.

[0084] A model container set is the basic unit for creation, deployment, and scheduling. A model container set represents a process running in a server cluster, encapsulating one or more model containers; that is, a model container set includes at least one model container. Within the same model container set, multiple model containers share storage, network IPs, and policy options for managing how the containers run. A container control component can create and control at least one model container set.

[0085] A container is a runtime instance of an image. A container is a process used to process data. A model container is the container that corresponds to a model.

[0086] The service component is the component that provides external access to the model container set. It defines an external access port for model processing services. Frontend applications access a cluster of model container instances through this port. External access requests are load-balanced across the various backend model containers. Frontend requests are not sent directly to the model container set but rather to the service component, which then forwards the requests to the model container set based on load balancing strategies through the container control component. The service component provides a fixed, unified external access port and load balancing capabilities for the model container set, thus solving the problem of how frontends and clients discover and access models.

[0087] Specifically, the server can generate model service configuration information corresponding to the model to be deployed based on the model image and model configuration information. The server can obtain a model service configuration template, fill in the image attribute information and model configuration information corresponding to the model image into the template, and obtain the model service configuration information. The model service configuration template can be a pre-set, standardized information configuration template, or it can generate model service configuration information corresponding to the model to be deployed based on the personalized information of the model to be deployed. For example, the server can use the model identifier corresponding to the model to be deployed as the tag information in the model service configuration information, the model port corresponding to the model to be deployed as the container port corresponding to the model container set in the model service configuration information, and the image attribute information of the model image corresponding to the model to be deployed as the image attribute information in the model service configuration information, etc. It can be understood that the model service configuration template includes default values ​​for some model service configuration information, such as container set quantity information and computing resource information. If no modification is needed, the default value is used; if a target value is configured in the model configuration information corresponding to the model to be deployed, the target value replaces the default value.

[0088] Furthermore, the server can create container control components and corresponding service components based on the model service configuration information. The server runs the model service configuration information, and based on the details within that information, it can create the container control components and their corresponding service components. It can be understood that, to improve component creation efficiency, the model service configuration information can be subdivided into container control component configuration information and service component configuration information. Container control components are created based on the container control component configuration information, and service components are created based on the service component configuration information. The container control component configuration information includes data such as version information, tag information, port information, and model container set information. The service component configuration information includes version information, tag information, and port information, but it may not include model container set information.

[0089] The container control component and service components work together to provide model processing services for the models to be deployed. It's understandable that, because the container control component can dynamically generate the set of model containers corresponding to the models to be deployed, meaning the set of model containers is dynamic and its network IP address is unstable, the server can receive requests from the terminal through a stable service component to facilitate terminal access to the model container set. This service component then forwards the requests to the currently running model container set through the container control component.

[0090] It's understandable that if there are multiple model container sets, the container control component can send requests to the target model container set for processing based on a load balancing strategy. The container control component can use routing strategies such as random, hash, and round-robin to determine the target model container set from multiple model container sets. A random routing strategy randomly selects one model container set as the target model container set from multiple model container sets. A hash routing strategy uses the hash value corresponding to the request sending terminal as the target model container set; different model container sets correspond to different hash values, which can be calculated based on the terminal identifier of the request sending terminal. A round-robin routing strategy sends requests to each model container set in turn. The container control component can also perform container performance evaluation on each model container set and select the model container set with the best performance as the target container set.

[0091] If the target container set includes multiple model containers, the container control component can further distribute requests to the target container for processing based on a load balancing strategy. The container control component can use routing strategies such as random, hash, and round-robin to determine the target container from among the multiple model containers. The container control component can also perform container performance evaluations on each model container and select the model container with the best performance as the target container. Through load balancing strategies, the management, discovery, and access to a set of application instances (model container set) can be achieved, and these details do not require complex manual configuration and processing by application developers and operations personnel.

[0092] In one embodiment, the server can call the interface of an image creation engine to generate a model image corresponding to the model to be deployed. By calling the interface of the image creation engine, the model deployment package can be automatically packaged into a portable image, which can then be deployed to computer devices. The image creation engine can be open-source or closed-source.

[0093] In one embodiment, the server can invoke the container management engine's interface to create container control components and corresponding service components. By invoking the container management engine's interface, model service configuration information can be automatically generated, and container control components and service components can be automatically generated based on this configuration information. The container management engine can be open-source or closed-source.

[0094] In one embodiment, the server can be further divided into a management server and worker servers. The management server is primarily responsible for creating container control components and service components on the worker servers. The worker servers are primarily responsible for processing model processing requests and obtaining data processing results. The management server can obtain model processing requests sent by the terminal, determine the corresponding service component based on the information carried in the model processing request, and forward the model processing request to the worker server corresponding to the service component. The worker server then uses its local container control component to distribute the model processing request to the model containers within the model container set for processing, obtaining the data processing results. The worker server can return the data processing results to the terminal. It is understood that there can be at least one worker server. Different worker servers can use different types of computing resources. The same model can be deployed on different worker servers. If there are multiple worker servers corresponding to the model to be deployed, each worker server can handle the model processing request corresponding to the model to be deployed. The management server can determine the target server from the multiple worker servers based on a load balancing strategy and send the model processing request to the target server for processing. The management server can use routing strategies such as random, hash, and round-robin to determine the target server from the multiple worker servers. The management server can also perform server performance evaluation on each worker server and select the worker server with the best performance as the target server.

[0095] It is understood that the specific processing procedure of the server for model processing requests can refer to the methods described in the various related embodiments of the subsequent model processing methods.

[0096] In the above method for deploying models on a server, the following steps are taken: First, a model deployment package corresponding to the model to be deployed is obtained. This package includes model configuration information. A model image is generated based on the deployment package. Then, model service configuration information is generated based on the model image and configuration information. Finally, a container control component and its corresponding service component are created based on the service configuration information. The container control component dynamically generates and controls the set of model containers corresponding to the model to be deployed. The container control component and service component work together to provide model processing services for the model to be deployed. These services distribute model processing requests to the model containers within the set of model containers for processing. In this way, the model deployment package can automatically create the corresponding container control component and service component, thus automatically deploying the model on the server and providing model processing services to external systems through the cooperation of the container control component and service component. Furthermore, the container control component can dynamically generate and control the set of model containers, enabling automated and intelligent model operation and maintenance, effectively improving model management efficiency.

[0097] In one embodiment, obtaining the model deployment package corresponding to the model to be deployed includes:

[0098] Obtain the model file and model configuration information corresponding to the model to be deployed; obtain the web service program created based on the web service framework, which includes a model loading function and is used to start the service interface corresponding to the model processing service; configure the model file into the model loading function, and package the configured web service program and model configuration information to obtain the model deployment package.

[0099] A model file is a file that stores the model algorithm. Model algorithms can be saved in different file formats.

[0100] A web service framework is a development framework for developing applications accessible via the web (World Wide Web). A web service program is program code developed based on this framework, used to launch the service interface corresponding to the model processing service, thus enabling access to the model processing service via the web. The service interface refers to the API corresponding to the model and is part of the link address for accessing the model processing service. Users only need a browser and can access the model processing service through the link address; no other software installation is required. For example, a user can enter the link address in their browser to access the model service page corresponding to the model processing service, upload the data to be predicted on the model service page, trigger the generation of a model prediction request, which is then sent to a server equipped with service components and container control components for data processing. The final model prediction result can also be displayed on the model service page.

[0101] The web service program can pre-load a model loading function. This function is used to load model files. The model file corresponding to the model to be deployed is configured in the model loading function, placing the model file at the location where model deployment will be triggered. Subsequently, during model deployment, the web service program can retrieve the model file based on the model loading function. In one embodiment, the model loading function can specifically configure the path address of the model file; if the model loading function is called, the model file will be automatically retrieved based on that path address.

[0102] Specifically, developers can pre-edit the model files and configuration information corresponding to the model to be deployed, and edit the web service program based on the web service framework. The web service program contains the model loading function. The model deployment package can be generated on the terminal or on the server. Taking the terminal as an example, the terminal can obtain the model files and configuration information corresponding to the model to be deployed uploaded by the user, obtain the web service program pre-loaded with the model loading function, and load the model file into the model loading function. Further, the terminal packages the configured web service program and model configuration information to obtain the model deployment package. Subsequently, the terminal uploads the model deployment package to the server, and the server can create the service components and container control components corresponding to the model to be deployed based on the model deployment package, thereby establishing a model processing service and providing model processing services to the outside world. After packaging, the model deployment package can be in zip format.

[0103] refer to Figure 3A In (a) of this example, after logging into the model deployment system, users can access the model management page. On this page, users can click the "Import Model" control and create a project to save the model's data. (See reference) Figure 3A In step (b), after the user clicks the "Import Model" control, they are taken to the import model page. On this page, the user can upload the model file to be deployed and fill in relevant information, such as model configuration information. If the user clicks the "Save" control, a model deployment request is generated. This request carries the model file and configuration information. The terminal can then package the model file and configuration information into a model deployment package and send it to the server for deployment. (See reference...) Figure 3A In step (c), if the model deployment is successful, the basic information about the model to be deployed can be displayed on the model management page. Users can click the "Publish" control to enter the model publishing page, and then publish the model to the model marketplace, making the model processing service available. (See reference) Figure 3A In step (d), users can fill in relevant descriptive information about their model on the model publishing page. When the user clicks the "Save" button, the model will be published to the model marketplace. (See reference) Figure 3A In section (e), users can search for various published models in the model marketplace and view, use, and comment on them. All actions performed on the models are recorded and displayed.

[0104] In this embodiment, by packaging the web service program with the configured model file and the model configuration information, a model deployment package can be obtained, thereby effectively reducing the amount of data transmission, improving data transmission efficiency, and improving model deployment efficiency.

[0105] In one embodiment, generating a model image corresponding to the model to be deployed based on the model deployment package includes:

[0106] Obtain the model environment version information and model environment dependency information corresponding to the model to be deployed from the model configuration information; obtain the target base image corresponding to the model environment version information; obtain the image file corresponding to the model to be deployed based on the model deployment package and the target base image; pull the model environment dependency package based on the model environment dependency information; generate the model image based on the image file and the model environment dependency package; and store the model image in the image library.

[0107] The model environment version information refers to the version information of the basic environment required by the model. It can be understood that different models are built and developed on different basic environments. The base image is the image used to build the basic environment required by the model. Different model environment versions correspond to different base images. For example, if the model algorithm is developed based on a Python environment, the Python base environment may have versions 2.7, 3.6, etc., and different versions of the base environment correspond to different base images.

[0108] Model environment dependency information refers to the information about the dependency packages required to build the virtual environment of the model. For example, if the model's training environment is based on TensorFlow, then the model environment dependency information can include information about the corresponding TensorFlow dependency packages. It can be understood that different models can correspond to the same model environment version information, but different models generally correspond to different model environment dependency information. Model environment dependency information can be specifically configured in the `requirements.txt` file. Model environment version information can be specifically configured in the `application.yaml` file. Other model configuration information can also be configured in the `application.yaml` file.

[0109] An image file is a build file used to construct a model image; it is a script consisting of a series of commands and parameters. An image repository refers to an image store that contains base images and model images.

[0110] Specifically, model images are built upon base images, generating personalized model images by continuously layering new image layers on top of the base image. After obtaining the model deployment package corresponding to the model to be deployed, the server can retrieve the model environment version information from the model configuration information, select the base image corresponding to the model environment version information from multiple candidate base images as the target base image, and retrieve the target base image from the image repository. Then, the server can obtain the image file corresponding to the model to be deployed based on the model deployment package and the target base image. To ensure the smooth operation of the model, the virtual environment required by the model also needs to be built in the model container. Therefore, the server also needs to obtain the model environment dependency information from the model configuration information, pull the model environment dependency package based on the model environment dependency information, and generate the model image corresponding to the model to be deployed based on the model environment dependency package and the image file. Subsequently, by running the model image, the server can build the virtual environment required by the model in the model container, determine the model algorithm information based on the model file, and start the service interface through the web service program. The server can store the generated model image in the image repository. Subsequently, if the model image is needed, the server can pull it from the image repository in a timely manner.

[0111] In this embodiment, an image file corresponding to the model to be deployed can be quickly generated based on the model file, model environment dependency information, and target base image, thereby enabling the rapid generation of a model image based on the image file.

[0112] In one embodiment, the service component has a corresponding external access port, and the container control component and the creation server corresponding to the service component have corresponding server addresses. The server address, external access port and service interface constitute the link address used to access the model processing service.

[0113] Specifically, creating a server refers to creating the server for the container control component and service components. The server address refers to the server's IP address. The server address, external access port, and service interface together form a complete link address used to access the model processing service. For example, the link address is http: / / 187.60.100.10:33330 / api / v1.0 / predictions, where 183.62.100.10 is the server address corresponding to the server creation, 33330 is the external access port corresponding to the service component, and api / v1.0 / predictions is the service interface corresponding to the model to be deployed, i.e., the service interface corresponding to the model processing service. Users can enter this link address in their browser to access the model service page corresponding to the model processing service of the model to be deployed. Users can upload data to be predicted on the model service page, triggering the generation of a model prediction request. The model prediction request is sent to the server with the service component and container control component for data processing, and the final model prediction result can also be displayed on the model service page.

[0114] refer to Figure 3B The terminal packages the model file and configuration information corresponding to the model to be deployed into a model deployment package. The model to be deployed is an image recognition model, specifically used to recognize digits in images. The terminal sends the model deployment package to the server. The server deploys the model based on the deployment package, creating service components and container control components corresponding to the model to be deployed. These components provide model processing services. The link address for the model processing service is the server address + external access port + service interface. Users can access the model service page corresponding to the model to be deployed through this link address. On the model service page, users can click the "Select Image" control to upload an image to be recognized, and click the "Recognize" control to trigger the generation of a model prediction request. After processing the model prediction request, the server returns the model processing result to the terminal. The terminal can then display the model processing result on the model service page.

[0115] In this embodiment, the link address composed of the server address, external access port, and service interface can quickly access the model processing service and trigger the model deployed on the server to perform data processing.

[0116] In one embodiment, model service configuration information corresponding to the model to be deployed is generated based on the model image and model configuration information, including:

[0117] Obtain the image attribute information corresponding to the model image, and obtain the model port and model identifier corresponding to the model to be deployed from the model configuration information; generate container control component configuration information based on the image attribute information, model port and model identifier; create an external access port, and generate service component configuration information based on the model port, external access port and model identifier; obtain model service configuration information based on container control component configuration information and service component configuration information.

[0118] The model service configuration information can include container control component configuration information and service component configuration information. The container control component configuration information is used to generate the container control component, and the service component configuration information is used to generate the service component. The container control component configuration information mainly includes information related to the model image, so that the container control component created based on the container control component configuration information can generate a set of model containers corresponding to the model image. Because the service component does not need to directly manage the model containers, the service component configuration information may not include information related to the model image. The service component configuration information mainly includes port information, so that the service component created based on the service component configuration information can act as a communication bridge between internal and external systems.

[0119] Image attribute information describes the basic attributes of a model image, such as its storage address in the image repository and its corresponding image name. The model port represents the internal access port of the model container. Within a server cluster, model containers can interact with each other based on the model port. However, devices outside the server cluster cannot directly access the model container through the model port and must use an external access port. The model identifier uniquely identifies the model and can be a string containing at least one character from letters, numbers, and symbols. The model identifier can specifically include information such as the model name or the name of the model processing service.

[0120] Specifically, the server can obtain the image attribute information corresponding to the model image, retrieve the model port and model identifier corresponding to the model to be deployed from the model configuration information, and generate container control component configuration information based on the image attribute information, model port, and model identifier. The server can then populate the image attribute information, model port, and model identifier into the container control component configuration template to obtain the container control component configuration information. Further, the server can create an external access port for the service component to provide external access, and generate service component configuration information based on the model port, external access port, and model identifier. The server can then populate the model port, external access port, and model identifier into the service component configuration template to obtain the service component configuration information. The service component configuration information includes the model port, which is the port exposed by the container, i.e., the internal access port. The service component configuration information also includes the external access port, which is the port exposed by the model processing service. It can be understood that both the container control component configuration information and the service component configuration information include the model identifier, thus the model identifier can serve as a tag for the container control component and the service component, binding the container control component and service component corresponding to the same model to be deployed together.

[0121] In this embodiment, the model service configuration information can be further divided into container control component configuration information and service component configuration information. The container control component configuration information and service component configuration information are generated based on the characteristics of the container control component and service component, which can improve the efficiency of creating container control components and service components based on the model service configuration information.

[0122] In one embodiment, a container control component and a corresponding service component are created based on model service configuration information, including:

[0123] Call the container management engine interface to generate a container control component and at least one model container set corresponding to the container control component based on the container control component configuration information; the model container set includes at least one model container corresponding to the model to be deployed, and the model container is generated based on the model image; call the container management engine interface to generate a service component based on the service component configuration information; the service component is used to provide the external access port corresponding to the model port; bind the container control component and the service component based on the model identifier.

[0124] The container management engine interface refers to the interface of the container management engine, used to create container control components and service components. The container management engine manages containerized applications on the server, providing a mechanism for application deployment, planning, updating, and maintenance, making the deployment of containerized applications simple and efficient.

[0125] Specifically, the container management engine provides interfaces for creating container control components and service components. The server can automatically run the container control component configuration information by calling the container management engine interface, thereby generating the container control component and its corresponding model container set. It can be understood that one container control component can correspond to at least one model container set, and a model container set can include at least one model container corresponding to a model to be deployed. Since the container control component configuration information includes the image attribute information of the model image, the model image can be obtained based on the image attribute information, and thus a model container can be generated based on the model image; at least one model image constitutes a model container set. The server can automatically run the service component configuration information by calling the container management engine interface, thereby generating service components. Since the container control component configuration information includes a model port, this model port can be set as the internal access port corresponding to the model container or model container set. Since the service component configuration information includes an external access port, this external access port can be set as the service port provided externally by the service component. The service component configuration information includes both model ports and external access ports, establishing an association between the port exposed by the model container corresponding to the model to be deployed and the port exposed by the model processing service corresponding to the model to be deployed. After generating container control components and service components, the server can bind these components based on model identifiers. Because model identifiers are used to uniquely identify models, using them as tagging information for the same group of container control components and service components ensures quick retrieval of the corresponding container control components and service components for the same model.

[0126] In one embodiment, the server can be further divided into a management server and worker servers. The management server can call the container management engine interface to create container control components and service components on the worker servers.

[0127] In this embodiment, container control components and service components can be quickly generated by calling the container management engine interface. Binding container control components and service components based on model identifiers can establish an association between container control components and service components corresponding to the same model, which facilitates mutual lookup between the two.

[0128] In one embodiment, the method further includes:

[0129] When the container control component and service component are created successfully, the model service status corresponding to the model to be deployed is determined to be running; when the container control component or service component fails to be created, the model service status corresponding to the model to be deployed is determined to be failed; a model deployment prompt message is generated based on the model service status corresponding to the model to be deployed, and the model deployment prompt message is sent to the preset terminal.

[0130] The model service status indicates whether the model processing service has been created successfully. The model deployment prompt message indicates whether the model has been deployed successfully. If the model service status is determined to be running, it means the model deployment was successful. If the model service status is determined to be failing, it means the model deployment failed. Preset terminals can be configured as needed, for example, the terminal sending the model deployment package can be used as the preset terminal, and the terminals of maintenance personnel can also be used as preset terminals.

[0131] Specifically, if the container control component and service component are created successfully, the server can determine the model service status corresponding to the model to be deployed as running. If the container control component or service component fails to be created, the server can determine the model service status corresponding to the model to be deployed as failed. Furthermore, the server can generate a model deployment prompt message based on the model service status corresponding to the model to be deployed and send the message to a preset terminal to indicate whether the model deployment was successful. In other words, if model deployment fails, the server can automatically restart the relevant model deployment operations and begin the deployment process again.

[0132] refer to Figure 4 The server retrieves the model deployment package corresponding to the model to be deployed and performs data verification on the package. Specifically, data verification checks whether the data and files in the deployment package are complete. Upon successful verification, the server uses the first interface to call the image creation engine interface to generate the model image corresponding to the model to be deployed based on the deployment package and pushes the model image to the image repository for storage. The first interface is a self-built interface call service used to call the relevant interfaces of various open-source engines. The second interface is the interface call service corresponding to the container management engine, a built-in service used to call the relevant interfaces of the container management engine. The server can use the first interface to call the deployment interface of the second interface, and then use the second interface to call the container management engine interface to create the container management component. When creating the container management component, the server can pull the model image corresponding to the model to be deployed from the image repository to generate the container management component. If the container management component is successfully created, the server further uses the second interface to call the container management engine interface to create the service component. If the service component is created successfully, the second interface call service proactively notifies the first interface call service that the model deployment is successful. The first interface call service then notifies the model deployment service to change the model service status to running. If the container management component or service component is created successfully, the second interface call service proactively notifies the first interface call service that the model deployment has failed. The first interface call service then notifies the model deployment service to change the model service status to failed. Finally, messages indicating successful or failed model deployment can be sent to a preset terminal to notify relevant personnel.

[0133] In one embodiment, there is at least one model to be deployed, and each model to be deployed has a corresponding container control component and service component.

[0134] Specifically, multiple models can be deployed on a single server. Therefore, on a single server, each model to be deployed has its corresponding service component and container control component. It can be understood that on a single server, one model corresponds to one service component and one container control component; this one-to-one management model effectively ensures the orderly management of a single model within a single server, thereby improving the effectiveness of model management. Of course, a model can also be deployed on multiple servers; therefore, in a server cluster, one model can correspond to multiple service components and multiple container control components.

[0135] In one embodiment, such as Figure 5 As shown, a model processing method is provided, which is applied to... Figure 1 Taking the server in the example, the following steps are included:

[0136] Step S502: Obtain the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed.

[0137] The model processing request is a request generated on the terminal to request the target model to process the data to be processed. The model processing service identifier refers to the service identifier of the model processing service corresponding to the target model. For example, the link address used to access the model processing service can be used as the model processing service identifier, or the model identifier corresponding to the target model can be used as the model processing service identifier. Specifically, the model processing request can be a model training request, a model prediction request, etc.

[0138] Specifically, users can access the model service page of the model processing service corresponding to the target model via a link on the terminal. Users can input the data to be processed on the model service page and trigger the terminal to generate a model processing request. The terminal can then send the model processing request to the server, carrying the model processing service identifier corresponding to the target model and the data to be processed. In this way, the server receives the model processing request sent by the terminal, and then processes the data based on the target container control component corresponding to the target model, obtaining the data processing result, and returning the data processing result to the terminal.

[0139] Step S504: Determine the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component; the target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate in providing the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information.

[0140] Among them, the target service component refers to the service component corresponding to the target model, and the target container control component refers to the container control component corresponding to the target model.

[0141] Specifically, after receiving a model processing request, the server can determine the target service component and target container control component corresponding to the target model based on the model processing service identifier carried in the request. For example, if the model processing service identifier is the link address of the model processing service corresponding to the target model, the server can determine the target service component based on the external access port in the link address, and then determine the target container control component corresponding to the target service component based on the binding information between the target service component and the target container control component.

[0142] It is understood that the process of establishing the target service component and the target container control component can refer to the methods described in the aforementioned relevant embodiments, and will not be repeated here.

[0143] Step S506: Determine the target container from the set of model containers corresponding to the target model based on the target container control component.

[0144] The target container refers to the model container that processes data for model processing requests.

[0145] Specifically, because the target container control component manages the set of model containers corresponding to the target model, the server can determine the target container from this set based on the target container control component. It can be understood that there can be at least one set of model containers corresponding to the target model, and at least one model container within each set. Taking multiple set of model containers, each containing one model container, as an example, the container control component can use routing strategies such as random, hash, or round-robin to determine the target container from multiple set of model containers. The container control component can also perform container performance evaluation on multiple set of model containers and select the model container from the set with the best performance as the target container.

[0146] Step S508: Based on the target container, perform data processing on the data to be processed to obtain the data processing result, and return the data processing result to the terminal.

[0147] Specifically, after determining the target container, the server can process the data to be processed based on the target container. This involves sending the data to the target container for processing to obtain the processing result. Finally, the server can return the processing result to the terminal.

[0148] In one embodiment, the server can be further divided into a management server and worker servers. The management server receives the model processing request sent by the terminal, determines the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component, and then determines the target server from multiple worker servers based on the target service component and the target container control component. The management server sends the model processing request to the target server, so that the target server determines the target container from the set of model containers corresponding to the target model based on the target container control component, and performs data processing on the data to be processed based on the target container to obtain the data processing result.

[0149] The aforementioned model processing method, based on the model deployment package, can automatically create corresponding container control components and service components for the model, thereby automatically deploying the model on the server and providing model processing services to external users through the cooperation of the container control components and service components. Furthermore, the container control component can dynamically generate and control model container sets, achieving automatic intelligent operation and maintenance of the model, effectively improving model management efficiency. Dynamic management based on the container control component can effectively ensure the optimal number of model container sets, thus guaranteeing rapid response to model processing requests without wasting computing resources.

[0150] In one embodiment, determining the target container from the set of model containers corresponding to the target model based on the target container control component includes:

[0151] Based on the target container control component, obtain the target container scheduling information of the initial container set corresponding to the target model; the initial container set is the initially created model container set corresponding to the target model; determine the working state of the initial container set based on the target container scheduling information; when the working state is busy, expand the initial container set based on the target container control component to obtain the first container set corresponding to the target model; when the working state is idle, shrink the initial container set based on the target container control component to obtain the second container set corresponding to the target model; when the working state is normal, use the initial container set as the third container set; determine the target container from the updated container set; the updated container set is one of the first, second, and third container sets.

[0152] Container scheduling information refers to the information generated during the use of a set of model containers, used to evaluate the working status of the set. Specifically, container scheduling information can include data such as the computational resource usage, request processing latency, and health status of the set of model containers. It can be understood that multiple set of model containers can be initially created, and the target container scheduling information can be the average of the container scheduling information corresponding to each of the individual set of model containers.

[0153] The working status of a model container set is used to determine whether the created model container set can meet the current workload. If the working status of the model container set is "busy," it indicates that the model container set is under high load and cannot respond to new requests quickly and promptly. If the working status of the model container set is "idle," it indicates that the model container set is under low load, with more computing resources idle and more model containers not in use. If the working status of the model container set is "normal," it indicates that the model container set is under moderate load.

[0154] The initial container set is the initially created set of model containers corresponding to the target model. That is, the initial container set is created based on the number of model containers configured in the container service configuration information. The first container set is obtained by expanding the initial model container set; therefore, the number of model containers corresponding to the first container set is greater than the number of model containers corresponding to the initial container set. The second container set is obtained by reducing the initial model container set; therefore, the number of model containers corresponding to the first container set is less than the number of model containers corresponding to the initial container set.

[0155] Specifically, the server can monitor the model container set corresponding to the target model through the target container control component, collecting container scheduling information for the model container set. Then, the server can determine the working status of the initial container set based on the container scheduling information. For example, the container scheduling information includes scheduling information from multiple dimensions. A weighted sum of the scheduling information from each dimension is used to obtain a container scheduling score. If the container scheduling score is in the first score interval, it indicates a busy working state; if the container scheduling score is in the second score interval, it indicates a normal working state; and if the container scheduling score is in the third score interval, it indicates an idle working state. The score in the first score interval is greater than the score in the second score interval, and the score in the second score interval is greater than the score in the third score interval. If the working status of the model container set is busy, it is expanded based on the target container control component, increasing the number of model containers in the initial container set to obtain the first container set corresponding to the target model. If the working status of the model container set is normal, the number of model containers remains unchanged, and the initial container set is used as the third container set. If the model container set is in an idle state, it is scaled down based on the target container control component, reducing the number of model containers in the initial container set to obtain the second container set corresponding to the target model. The server then determines the target container from the updated initial container set.

[0156] For example, the initial number of model container sets is three. If the model container set is in a busy state, the target container control component can dynamically increase the number of model container sets to four. If the model container set is in an idle state, the target container control component can dynamically decrease the number of model container sets to two. If the model container set is in a normal state, the target container control component can keep the number of model container sets unchanged at three.

[0157] In one embodiment, different types of models can have different initial container sets with different numbers of model container sets. For example, image processing models have lower computational complexity than text processing models, so a first number of container sets can be set for image processing models, and a second number of container sets can be set for text processing models, with the first number of container sets being greater than the second number of container sets.

[0158] In one embodiment, a work state can correspond to at least one work level, thereby enabling differentiated and precise adjustment of container sets based on the work level. Different work levels can be configured with corresponding increases or decreases in the number of container sets. For example, in a busy state, a higher work level indicates greater busyness. In an idle state, a higher work level indicates greater idleness. Therefore, a higher work level corresponds to a greater increase or decrease in the number of container sets. For instance, a first score interval can be further divided into three sub-intervals, where the score in the first sub-interval is greater than the score in the second sub-interval, and the score in the second sub-interval is greater than the score in the third sub-interval. The first sub-interval can correspond to adding three model container sets, the second sub-interval can correspond to adding two model container sets, and the third sub-interval can correspond to adding one model container set.

[0159] In this embodiment, the number of model container sets is dynamically adjusted based on the working status of the model container set, which can realize the automatic operation and maintenance of the model container set, thereby reducing the workload of operation and maintenance personnel.

[0160] In one embodiment, the update container set corresponding to the target model includes at least one intermediate container set. Determining the target container from the update container set includes:

[0161] Obtain container performance information corresponding to each intermediate container set; calculate scheduling priority parameters corresponding to each intermediate container set based on the container performance information corresponding to each intermediate container set; determine the target container set from each intermediate container set based on the scheduling priority parameters corresponding to each intermediate container set; determine the target container from the target container set.

[0162] The updated container set corresponding to the target model can include at least one intermediate container set. For example, the initial container set includes three model container sets, and the updated model container set obtained by expanding the initial container set includes four model container sets; these four model container sets constitute the intermediate container set. Global container performance information refers to the container performance information corresponding to a model container set. Container performance information can include data such as computing resource usage information, request processing latency, and request processing success rate.

[0163] Specifically, to improve the response speed of model processing requests, the optimal model container set from multiple intermediate container sets can be selected as the target container set. The server can obtain container performance information corresponding to each intermediate container set through the target container management component and calculate the scheduling priority parameters for each intermediate container set based on this information. Specifically, the server can perform a weighted summation of the performance information across various dimensions in the container performance information to obtain the scheduling priority parameters. Then, based on the scheduling priority parameters, the server selects the optimal model container set from each intermediate container set as the target container set. For example, if a higher scheduling priority parameter indicates higher container performance, the model container set with the highest scheduling priority parameter from each intermediate container set can be selected as the target container set. Furthermore, if the target container set contains only one model container, that model container is directly used as the target container. If the target container set contains multiple model containers, a model container can be randomly selected from the multiple model containers as the target container, or a model container can be selected using other methods. It can be understood that if the intermediate container set is a newly created model container set that has not yet been used, preset container performance information can be used as the container performance information for that intermediate container set.

[0164] In this embodiment, the scheduling priority parameters of the model container set can be determined based on the container performance information of the model container set. Based on the scheduling priority parameters of the model container set, the model container set with the best performance can be selected from multiple model container sets as the target container set. Thus, by processing requests through the target container set, a fast response can be achieved and data processing results can be obtained.

[0165] In one embodiment, the model processing request is a model prediction request, the data to be processed is the data to be predicted, and the data processing result is the model prediction result. Based on the target container, data processing is performed on the data to be processed to obtain the data processing result, which is then returned to the terminal, including:

[0166] The model prediction request is sent to the target container so that the target container can perform model prediction on the data to be predicted and obtain the model prediction result.

[0167] Specifically, if the target model is a trained model, the model processing service can be a model prediction service. Users can submit a model prediction request to the server via a link corresponding to the target model. After determining the target container, the server can send the model prediction request to the target container. The target container then performs model prediction on the data to be predicted carried in the model prediction request, thus obtaining the model prediction result. For example, if the target model is an image recognition model for identifying digits in an image, the user can upload the image to be recognized on the model service page and click the model prediction control to trigger the generation of a model prediction request. After receiving the model prediction request, the server can send the image to be recognized to the target container for model prediction. If the model prediction result is that the digit in the image to be recognized is 7, the model service page can ultimately display the model prediction result of 7 to the user.

[0168] In this embodiment, the model processing service can specifically be a model prediction service, which is used to input the data to be predicted into the model and output the model prediction result.

[0169] In one embodiment, the model processing request is a model training request, the data to be processed is the training data, and the data processing result is the model training result. Based on the target container, data processing is performed on the data to be processed to obtain the data processing result, which is then returned to the terminal, including:

[0170] The model training request is sent to the target container so that the target container can train the target model based on the training data to obtain the model training result; based on the model training result, the model container set corresponding to the target model is upgraded to obtain the upgraded container set corresponding to the target model.

[0171] Specifically, if the target model is a model to be trained, then the model processing service can be a model training service. Users can submit model training requests to the server via the link address corresponding to the target model. After determining the target container, the server can send the model training request to the target container. The target container then trains the target model based on the training data carried in the model training request, thereby obtaining the model training result. The model training result can be the model parameters corresponding to the trained target model. The server can upgrade the model container set corresponding to the target model based on the model training result, updating the model containers with the new model parameters, thus obtaining the upgraded container set corresponding to the target model. In essence, the trained target model can then provide model prediction services, and model prediction requests can be sent to the upgraded container set for data processing.

[0172] In this embodiment, the model processing service may specifically be a model training service, which is used to train a model based on training data.

[0173] In one specific embodiment, model deployment and model processing on the server includes the following steps:

[0174] I. Model Deployment (Reference) Figure 6A )

[0175] 1. Load and package the model files to obtain the model deployment package, which includes the model files and model configuration information.

[0176] The model file is loaded into the model loading function, which in turn loads the model loading function into a web service application developed based on the web service framework. The loaded web service application and model configuration information are then packaged to obtain the model deployment package. The web service application can be a program written in Python, for example, named `tiwisdom_moel_runner.py`. Alternatively, a file packaging program written in Python can be used for the packaging process, for example, named `tiwisdom_model_pack.py`. The model configuration information can include a `requirement.txt` file and an `application.yaml` file. The `requirement.txt` file primarily describes the model's environment dependencies.

[0177] 2. Parse the model environment version information in the model configuration information, select the target base image corresponding to the model environment version information, and pull the target base image from the image library to generate the image file.

[0178] 3. Parse the model environment dependency information in the model configuration information, pull the model environment dependency package based on the model environment dependency information, generate a model image based on the image file and the model environment dependency package, and push the model image to the image repository for storage.

[0179] 4. Obtain model image attribute information and model configuration information; generate container control component configuration information based on the model image attribute information and model configuration information; and generate service component configuration information based on the container control component configuration information.

[0180] The configuration information for the container control component and the configuration information for the service component can be stored in a YAML file.

[0181] 5. Generate container control components based on container control component configuration information.

[0182] 6. Generate service components based on service component configuration information

[0183] Deploy container control components and service components on the server

[0184] 7. Provide model processing services based on container control and service components.

[0185] 8. Update model service status

[0186] If the container control component and service component are created successfully, the model service status is updated to running. The model's calling information can also be further updated, such as storing data such as the model port information and external access port information corresponding to the model.

[0187] II. Model Processing

[0188] The container control component can automatically expand and shrink the model container set based on its working status, effectively ensuring that the number of model containers remains at an optimal level. This ensures rapid response to model processing requests without wasting computing resources, achieving efficient model management.

[0189] III. System Architecture

[0190] refer to Figure 6B The model management system comprises an infrastructure layer, a server cluster, a proxy service layer, and a platform service layer. The infrastructure layer provides computing resources. CPU, GPU, and CVM are collectively referred to as computing resources, and each represents a different type of computing resource. The three nodes (servers) in the diagram represent nodes using different resources.

[0191] A node cluster is a cluster of multiple nodes. Each node deploys a set of service components and container control components, which work together to provide model processing services. Service components and container control components are bound together by tags for routing. Service components provide external access ports to receive model processing requests. Container control components are used to dynamically generate and control at least one set of model containers.

[0192] The proxy service layer provides proxy services supported by the node cluster or container management engine, including the container management engine interface, port management, service interface management, and service monitoring. The container management engine interface is used to create service components and container control components on nodes. Port management manages the model ports corresponding to the models and the external access ports corresponding to the service components. Service interface management manages the service interfaces corresponding to the model processing services. Service monitoring monitors the traffic corresponding to the service interfaces, request processing latency, node availability, model container set availability, and computing resource usage information.

[0193] The platform service layer provides model services to the outside world. These services include model training, model deployment, model prediction, and model management. The model training service trains the model. The model deployment service deploys the model on nodes. The model prediction service uses the model to make predictions. The model management service manages model-related information, such as model images.

[0194] In this application, the model-based deployment package can automatically create container control components and service components corresponding to the model, thereby automatically deploying the model on the server and providing model processing services to external users through the cooperation of the container control components and service components. Furthermore, the container control component can dynamically generate and control model container sets, realizing automatic intelligent operation and maintenance of the model, effectively improving model management efficiency. Deploying multiple model container sets on a single node allows the model container sets to share machine resources, saving significant machine resources and reducing costs.

[0195] It should be understood that, although Figure 2 , 5 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2 , 5 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0196] In one embodiment, such as Figure 7 As shown, an apparatus for a server deployment model is provided. This apparatus can be a software module, a hardware module, or a combination of both, integrated into a computer device. Specifically, the apparatus includes: a deployment package acquisition module 702, an image generation module 704, a configuration information generation module 706, and a component creation module 708, wherein:

[0197] The deployment package acquisition module 702 is used to acquire the model deployment package corresponding to the model to be deployed. The model deployment package includes the model configuration information corresponding to the model to be deployed.

[0198] The image generation module 704 is used to generate a model image corresponding to the model to be deployed based on the model deployment package.

[0199] The configuration information generation module 706 is used to generate model service configuration information corresponding to the model to be deployed based on the model image and model configuration information.

[0200] The component creation module 708 is used to create container control components and corresponding service components based on model service configuration information.

[0201] The container control component is used to dynamically generate and control the model container set corresponding to the model to be deployed. The container control component and the service component are used to cooperate in providing the model processing service corresponding to the model to be deployed. The model processing service is used to allocate the model processing request corresponding to the model to be deployed to the model container in the model container set for processing.

[0202] In one embodiment, the deployment package acquisition module is further used to acquire the model file and model configuration information corresponding to the model to be deployed, acquire the web service program created based on the web service framework, the web service program includes a model loading function, the web service program is used to start the service interface corresponding to the model processing service, configure the model file into the model loading function, and package the configured web service program and model configuration information to obtain the model deployment package.

[0203] In one embodiment, the image generation module is further configured to obtain the model environment version information and model environment dependency information corresponding to the model to be deployed from the model configuration information, obtain the target base image corresponding to the model environment version information, obtain the image file corresponding to the model to be deployed based on the model deployment package and the target base image, pull the model environment dependency package based on the model environment dependency information, generate the model image based on the image file and the model environment dependency package, and store the model image in the image library.

[0204] In one embodiment, the service component has a corresponding external access port, and the container control component and the creation server corresponding to the service component have corresponding server addresses. The server address, external access port and service interface constitute the link address used to access the model processing service.

[0205] In one embodiment, the configuration information generation module is further configured to obtain image attribute information corresponding to the model image, obtain the model port and model identifier corresponding to the model to be deployed from the model configuration information, generate container control component configuration information based on the image attribute information, model port and model identifier, create an external access port, generate service component configuration information based on the model port, external access port and model identifier, and obtain model service configuration information based on the container control component configuration information and service component configuration information.

[0206] In one embodiment, the component creation module is further configured to call the container management engine interface to generate a container control component and at least one model container set corresponding to the container control component based on the container control component configuration information; the model container set includes at least one model container corresponding to the model to be deployed, the model container is generated based on the model image, and the container management engine interface is called to generate a service component based on the service component configuration information; the service component is configured to provide an external access port corresponding to the model port, and bind the container control component and the service component based on the model identifier.

[0207] In one embodiment, the apparatus for the server deployment model further includes:

[0208] The model service status determination module is used to determine the model service status of the model to be deployed as running when the container control component and service component are successfully created, and to determine the model service status of the model to be deployed as failed when the container control component or service component fails to be created. Based on the model service status of the model to be deployed, the module generates model deployment prompt information and sends the model deployment prompt information to the preset terminal.

[0209] In one embodiment, there is at least one model to be deployed, and each model to be deployed has a corresponding container control component and service component.

[0210] In one embodiment, such as Figure 8 As shown, a model processing device is provided. This device can be a software module, a hardware module, or a combination of both as part of a computer device. Specifically, the device includes: a request acquisition module 802, a component determination module 804, a container determination module 806, and a data processing module 808, wherein:

[0211] The request acquisition module 802 is used to acquire the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed.

[0212] The component determination module 804 is used to determine the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component. The target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information.

[0213] The container determination module 806 is used to determine the target container from the set of model containers corresponding to the target model based on the target container control component.

[0214] The data processing module 808 is used to process the data to be processed based on the target container, obtain the data processing result, and return the data processing result to the terminal.

[0215] In one embodiment, the container determination module is further configured to: obtain target container scheduling information of the initial container set corresponding to the target model based on the target container control component; the initial container set is the initially created model container set corresponding to the target model; determine the working state of the initial container set based on the target container scheduling information; when the working state is busy, expand the initial container set based on the target container control component to obtain the first container set corresponding to the target model; when the working state is idle, reduce the initial container set based on the target container control component to obtain the second container set corresponding to the target model; when the working state is normal, use the initial container set as the third container set; determine the target container from the updated container set; the updated container set is one of the first container set, the second container set, and the third container set.

[0216] In one embodiment, the updated container set corresponding to the target model includes at least one intermediate container set. The container determination module is further configured to obtain container performance information corresponding to each intermediate container set; calculate scheduling priority parameters corresponding to each intermediate container set based on the container performance information corresponding to each intermediate container set; determine the target container set from each intermediate container set based on the scheduling priority parameters corresponding to each intermediate container set; and determine the target container from the target container set.

[0217] In one embodiment, the model processing request is a model prediction request, the data to be processed is the data to be predicted, and the data processing result is the model prediction result. The data processing module is also used to send the model prediction request to the target container, so that the target container performs model prediction on the data to be predicted to obtain the model prediction result.

[0218] In one embodiment, the model processing request is a model training request, the data to be processed is training data, and the data processing result is the model training result. The data processing module is also used to send the model training request to the target container, so that the target container can perform model training on the target model based on the training data to obtain the model training result; and to upgrade the model container set corresponding to the target model based on the model training result to obtain the upgraded container set corresponding to the target model.

[0219] Specific limitations regarding the apparatus for deploying and processing server models can be found in the limitations on the methods for deploying and processing server models described above, and will not be repeated here. Each module in the aforementioned apparatus for deploying and processing server models can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0220] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores model images, container scheduling information, and container performance information. The network interface communicates with external terminals via a network connection. When executed by the processor, the computer program implements a method for deploying a server model and a model processing method.

[0221] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0222] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0223] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0224] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in the above-described method embodiments.

[0225] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0226] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0227] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for deploying a server model, characterized in that, The method includes: Obtain the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed; Generate a model image corresponding to the model to be deployed based on the model deployment package; Based on the model image and the model configuration information, generate the model service configuration information corresponding to the model to be deployed; Based on the model service configuration information, a container control component and a corresponding service component are created. The container control component dynamically generates and controls the model container set corresponding to the model to be deployed. The container control component and the service component work together to provide model processing services corresponding to the model to be deployed. The model processing services allocate model processing requests corresponding to the model to be deployed to model containers within the model container set for processing. The service component defines an external access port for the model processing service to receive requests sent by terminals. The container control component forwards the requests to the currently running model container set. If there are multiple model container sets, the container control component sends the requests to a target model container set determined from the multiple model container sets for processing based on a load balancing strategy. If the target model container set includes multiple model containers, the container control component sends the requests to a target container determined from the multiple model containers for processing based on a load balancing strategy.

2. The method according to claim 1, characterized in that, The step of obtaining the model deployment package corresponding to the model to be deployed includes: Obtain the model file and model configuration information corresponding to the model to be deployed; Obtain a web service program created based on a web service framework. The web service program includes a model loading function and is used to start the service interface corresponding to the model processing service. The model file is configured into the model loading function, and the configured web service program and the model configuration information are packaged to obtain the model deployment package.

3. The method according to claim 2, characterized in that, The step of generating the model image corresponding to the model to be deployed based on the model deployment package includes: Obtain the model environment version information and model environment dependency information corresponding to the model to be deployed from the model configuration information; Obtain the target base image corresponding to the model environment version information; Based on the model deployment package and the target base image, the image file corresponding to the model to be deployed is obtained; Based on the model environment dependency information, pull the model environment dependency package, generate the model image based on the image file and the model environment dependency package, and store the model image in the image library.

4. The method according to claim 2, characterized in that, The container control component and the service component have corresponding creation servers with corresponding server addresses. The server address, the external access port, and the service interface constitute a link address for accessing the model processing service.

5. The method according to claim 1, characterized in that, The step of generating model service configuration information corresponding to the model to be deployed based on the model image and the model configuration information includes: Obtain the image attribute information corresponding to the model image, and obtain the model port and model identifier corresponding to the model to be deployed from the model configuration information; Container control component configuration information is generated based on the image attribute information, the model port, and the model identifier; Create an external access port, and generate service component configuration information based on the model port, the external access port, and the model identifier; The model service configuration information is obtained based on the container control component configuration information and the service component configuration information.

6. The method according to claim 5, characterized in that, The creation of a container control component and a corresponding service component based on the model service configuration information includes: The container management engine interface is invoked to generate the container control component and at least one model container set corresponding to the container control component based on the container control component configuration information; the model container set includes at least one model container corresponding to the model to be deployed, and the model container is generated based on the model image. The container management engine interface is invoked to generate the service component based on the service component configuration information; the service component is used to provide the external access port corresponding to the model port. The container control component and the service component are bound based on the model identifier.

7. The method according to claim 1, characterized in that, The method further includes: When the container control component and the service component are successfully created, the model service status corresponding to the model to be deployed is determined to be running. When the container control component or the service component fails to be created, the model service status corresponding to the model to be deployed is determined to be a failure status. Based on the model service status corresponding to the model to be deployed, a model deployment prompt message is generated and sent to a preset terminal.

8. The method according to any one of claims 1 to 7, characterized in that, There is at least one model to be deployed, and each model to be deployed has a corresponding container control component and service component.

9. A model processing method, characterized in that, The method includes: Obtain the model processing request sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed; The system identifies the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component. The target container control component dynamically generates and controls the model container set corresponding to the target model. The target container control component and the target service component cooperate to provide the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model, and the model deployment package includes the model configuration information. The service component defines an external access port for the model processing service to receive requests sent by terminals. The container control component forwards the requests to the currently running model container set. If there are multiple model container sets, the container control component sends the requests to the target model container set determined from the multiple model container sets for processing based on a load balancing strategy. If the target model container set includes multiple model containers, the container control component sends the requests to the target container determined from the multiple model containers for processing based on a load balancing strategy. Based on the target container control component, the target container is determined from the set of model containers corresponding to the target model; The data to be processed is processed based on the target container to obtain the data processing result, and the data processing result is returned to the terminal.

10. The method according to claim 9, characterized in that, The step of determining the target container from the set of model containers corresponding to the target model based on the target container control component includes: Based on the target container control component, obtain the target container scheduling information of the initial container set corresponding to the target model; the initial container set is the initially created model container set corresponding to the target model. The working status of the initial container set is determined based on the target container scheduling information; When the working state is busy, the initial container set is expanded based on the target container control component to obtain the first container set corresponding to the target model. When the working state is idle, the initial container set is reduced based on the target container control component to obtain the second container set corresponding to the target model. When the working state is normal, the initial container set is used as the third container set; The target container is determined from the updated container set; the updated container set is one of the first container set, the second container set, and the third container set.

11. The method according to claim 10, characterized in that, The update container set corresponding to the target model includes at least one intermediate container set, and determining the target container from the update container set includes: Obtain container performance information for each intermediate container set; Calculate the scheduling priority parameters for each intermediate container set based on the container performance information corresponding to each intermediate container set. The target model container set is determined from each intermediate container set based on the scheduling priority parameters corresponding to each intermediate container set. The target container is determined from the set of target model containers.

12. The method according to any one of claims 9 to 11, characterized in that, The model processing request is a model prediction request, the data to be processed is the data to be predicted, and the data processing result is the model prediction result; The step of processing the data to be processed based on the target container to obtain a data processing result and returning the data processing result to the terminal includes: The model prediction request is sent to the target container so that the target container performs model prediction on the data to be predicted and obtains the model prediction result.

13. The method according to any one of claims 9 to 11, characterized in that, The model processing request is a model training request, the data to be processed is training data, and the data processing result is the model training result; The step of processing the data to be processed based on the target container to obtain a data processing result and returning the data processing result to the terminal includes: The model training request is sent to the target container so that the target container can train the target model based on the training data to obtain the model training result. Based on the model training results, the model container set corresponding to the target model is upgraded to obtain the upgraded container set corresponding to the target model.

14. An apparatus for a server deployment model, characterized in that, The device includes: The deployment package acquisition module is used to acquire the model deployment package corresponding to the model to be deployed, wherein the model deployment package includes the model configuration information corresponding to the model to be deployed; The image generation module is used to generate a model image corresponding to the model to be deployed based on the model deployment package; The configuration information generation module is used to generate model service configuration information corresponding to the model to be deployed based on the model image and the model configuration information; The component creation module is used to create a container control component and a service component corresponding to the container control component based on the model service configuration information. The container control component dynamically generates and controls the model container set corresponding to the model to be deployed. The container control component and the service component work together to provide model processing services corresponding to the model to be deployed. The model processing services allocate model processing requests corresponding to the model to be deployed to model containers within the model container set for processing. The service component defines an external access port for the model processing service to receive requests sent by terminals. The container control component forwards the requests to the currently running model container set. If there are multiple model container sets, the container control component sends the requests to a target model container set determined from the multiple model container sets for processing based on a load balancing strategy. If the target model container set includes multiple model containers, the container control component sends the requests to a target container determined from the multiple model containers for processing based on a load balancing strategy.

15. The apparatus for the server deployment model according to claim 14, characterized in that, The deployment package acquisition module is also used to acquire the model file and model configuration information corresponding to the model to be deployed; and to acquire a web service program created based on a web service framework, wherein the web service program includes a model loading function and is used to start the service interface corresponding to the model processing service. The model file is configured into the model loading function, and the configured web service program and the model configuration information are packaged to obtain the model deployment package.

16. The apparatus for a server deployment model according to claim 15, characterized in that, The image generation module is also used to obtain the model environment version information and model environment dependency information corresponding to the model to be deployed from the model configuration information; and to obtain the target base image corresponding to the model environment version information; Based on the model deployment package and the target base image, the image file corresponding to the model to be deployed is obtained; Based on the model environment dependency information, pull the model environment dependency package, generate the model image based on the image file and the model environment dependency package, and store the model image in the image library.

17. The apparatus for a server deployment model according to claim 15, characterized in that, The container control component and the service component have corresponding creation servers with corresponding server addresses. The server address, the external access port, and the service interface constitute a link address for accessing the model processing service.

18. The apparatus for a server deployment model according to claim 14, characterized in that, The configuration information generation module is also used to obtain the image attribute information corresponding to the model image, and to obtain the model port and model identifier corresponding to the model to be deployed from the model configuration information; Container control component configuration information is generated based on the image attribute information, the model port, and the model identifier; An external access port is created, and service component configuration information is generated based on the model port, the external access port, and the model identifier; the model service configuration information is obtained based on the container control component configuration information and the service component configuration information.

19. The apparatus for a server deployment model according to claim 18, characterized in that, The component creation module is also used to call the container management engine interface to generate the container control component and at least one model container set corresponding to the container control component based on the container control component configuration information; the model container set includes at least one model container corresponding to the model to be deployed, and the model container is generated based on the model image. Call the container management engine interface to generate the service component based on the service component configuration information; The service component is used to provide the external access port corresponding to the model port; The container control component and the service component are bound based on the model identifier.

20. The apparatus for a server deployment model according to claim 14, characterized in that, The device further includes a model service status determination module, which is used to determine the model service status corresponding to the model to be deployed as running when the container control component and the service component are successfully created; and to determine the model service status corresponding to the model to be deployed as failing when the container control component or the service component fails to be created. Based on the model service status corresponding to the model to be deployed, a model deployment prompt message is generated and sent to a preset terminal.

21. The apparatus for a server deployment model according to any one of claims 14 to 20, characterized in that, There is at least one model to be deployed, and each model to be deployed has a corresponding container control component and service component.

22. A model processing device, characterized in that, The device includes: The request acquisition module is used to acquire model processing requests sent by the terminal; the model processing request carries the model processing service identifier corresponding to the target model and the data to be processed; The component determination module is used to determine the target service component corresponding to the model processing service identifier and the target container control component corresponding to the target service component. The target container control component is used to dynamically generate and control the model container set corresponding to the target model. The target container control component and the target service component are used to cooperate in providing the model processing service corresponding to the target model. The target container control component and the target service component are generated based on the model service configuration information corresponding to the target model. The model service configuration information is generated based on the model image and model configuration information corresponding to the target model. The model image is generated based on the model deployment package corresponding to the target model. The model deployment package includes the model configuration information. The service component defines the external access port of the model processing service for receiving requests sent by the terminal. The container control component is used to forward the request to the currently running model container set. If there are multiple model container sets, the container control component is used to send the request to the target model container set determined from the multiple model container sets for processing based on a load balancing strategy. If the target model container set includes multiple model containers, the container control component is used to send the request to the target container determined from the multiple model containers for processing based on a load balancing strategy. A container determination module is used to determine a target container from the set of model containers corresponding to the target model based on the target container control component. The data processing module is used to process the data to be processed based on the target container to obtain the data processing result, and return the data processing result to the terminal.

23. The model processing apparatus according to claim 22, characterized in that, The container determination module is further configured to: obtain target container scheduling information of the initial container set corresponding to the target model based on the target container control component; the initial container set is the initially created model container set corresponding to the target model; determine the working state of the initial container set based on the target container scheduling information; when the working state is busy, expand the initial container set based on the target container control component to obtain the first container set corresponding to the target model; when the working state is idle, reduce the initial container set based on the target container control component to obtain the second container set corresponding to the target model; when the working state is normal, use the initial container set as the third container set; determine the target container from the updated container set; the updated container set is one of the first container set, the second container set, and the third container set.

24. The model processing apparatus according to claim 23, characterized in that, The update container set corresponding to the target model includes at least one intermediate container set. The container determination module is further configured to obtain container performance information corresponding to each intermediate container set; calculate scheduling priority parameters corresponding to each intermediate container set based on the container performance information corresponding to each intermediate container set; determine the target model container set from each intermediate container set based on the scheduling priority parameters corresponding to each intermediate container set; and determine the target container from the target model container set.

25. The model processing apparatus according to any one of claims 22 to 24, characterized in that, The model processing request is a model prediction request, the data to be processed is the data to be predicted, and the data processing result is the model prediction result; The data processing module is also used to send the model prediction request to the target container, so that the target container can perform model prediction on the data to be predicted to obtain the model prediction result.

26. The model processing apparatus according to any one of claims 22 to 24, characterized in that, The model processing request is a model training request, the data to be processed is training data, and the data processing result is the model training result; The data processing module is further configured to send the model training request to the target container, so that the target container can perform model training on the target model based on the training data to obtain the model training result; Based on the model training results, the model container set corresponding to the target model is upgraded to obtain the upgraded container set corresponding to the target model.

27. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 13.

28. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

29. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 13.

Images