Model deployment method and device, electronic equipment and storage medium

Abstract

The invention provides a model deployment method and device, electronic equipment and a storage medium, and the method comprises the steps: distributing a pod task for a target business through a container arrangement management platform, and respectively deploying corresponding container groups for a model module and a business module through the pod task according to the operation information of the model module and the business module associated with the target business, a model container based on the model mirror image and a service container based on the service mirror image are obtained, container sharing resources are distributed for the model container and the service container through a pod task, and files and data generated by the model container and the service container in the process of responding to the target service are stored through the container sharing resources. The model container and the service container respond to the target service based on the files and the data in the container shared resources.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model deployment method, device, electronic device and storage medium. Background Art

[0002] Model reasoning technology refers to the ability to predict and make decisions about unknown data by analyzing and inferring existing data in machine learning models. It involves multiple steps such as data modeling, feature extraction, parameter estimation, reasoning and prediction. In the field of machine learning, model reasoning technology has a wide range of application scenarios, such as image recognition, natural language processing, medical diagnosis, financial risk assessment, and intelligent recommendation systems.

[0003] At present, for business-related processing processes involved in the model reasoning process, such as processing processes related to the data warehouse, model warehouse, and log center, in order to achieve unified deployment and simplify the deployment process, the solution often adopted is to bundle these business processing processes with the model reasoning process as a whole container application. This can also achieve the overall performance brought by the bundled container and the security brought by the container isolation.

[0004] However, the current reasoning method couples the model operation with the business processing module, which makes the platform less inclusive of the model's dependency environment and leads to very high development or maintenance costs. Once there is a conflict between the model operation dependency and the business processing module dependency, one party needs to adjust the version, which makes maintenance difficult. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide a model deployment method, device, electronic device and storage medium, which can deploy corresponding container groups for model modules and business modules respectively according to the sidecar mode of the container orchestration management platform k8s, obtain model containers based on model images and business containers based on business images and allocate container shared resources, and the model containers and business containers respond to target businesses based on the files and data in the container shared resources. By separating model reasoning from business, independent operation of business and model is achieved, so that the platform supports the deployment of heterogeneous models. At the same time, in this way, business processing and model reasoning are decoupled, lightweight management of model images is achieved, the platform's inclusiveness of the model's dependent environment is improved, development costs are reduced, and maintenance is easier.

[0006] In a first aspect, an embodiment of the present application provides a model deployment method, the method comprising:

[0007] A pod task is assigned to the target business through the container orchestration management platform, and corresponding container groups are deployed for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on the model image and a business container based on the business image; the operation information of the model module and the business module represents the operation information bundled with the business processing process related to the business and the model reasoning process in the model reasoning process;

[0008] Allocate container shared resources to the model container and the business container through the pod task, and store files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources;

[0009] The model container and the service container respond to the target service based on the files and data in the container shared resources.

[0010] In a possible implementation, the model image is compatible with different model versions, different algorithm libraries, different programming languages, and different operating systems; the model image and the business image have independent running programming language libraries and operating system basic libraries;

[0011] Among them, the model image at least includes a model running application, a programming language standard library, an operating system basic library, and a mirror operating system; the business image at least includes a business application, a programming language standard library, an operating system basic library, and a mirror operating system.

[0012] In a possible implementation manner, the model container and the business container respond to the target business based on the files and data in the container shared resources, including:

[0013] In response to the target business, the business container determines a corresponding target reasoning model and a model container corresponding to the target reasoning model based on the target business;

[0014] The business container determines a corresponding access path based on the target business, directly obtains a model parameter file of the target inference model and target data to be predicted based on the access path, and sends the model parameter file and the target data to be predicted to a file sharing path corresponding to the container group;

[0015] The model container directly receives the model parameter file and the target data to be predicted from the file sharing path.

[0016] In a possible implementation, the business container determines a corresponding access path based on the target business, and directly obtains a model parameter file of the target reasoning model and target data to be predicted based on the access path, including:

[0017] Determine a model warehouse storing the target reasoning model based on the target business, and directly call the client of the model warehouse to download and obtain a model parameter file of the target reasoning model;

[0018] A data warehouse storing target data to be predicted is determined based on the target business, and a client or application programming interface of the data warehouse is directly called to download and obtain the target data to be predicted.

[0019] In a possible implementation, the method further includes:

[0020] In response to a change in the target business, determining a target model image that needs to be modified from all model images based on the changed target business;

[0021] The target model image is modified based on the changed target business to obtain a modified target model image.

[0022] In a possible implementation, the method further includes:

[0023] The model container infers the target data to be predicted to obtain a corresponding inference result;

[0024] The model container infers the target data to be predicted to obtain corresponding inference results, including:

[0025] Loading the model parameter file to obtain model parameters of the target reasoning model, and building a corresponding model reasoning engine based on the model parameters;

[0026] In response to the completion of the construction of the model reasoning engine, the target data to be predicted is loaded into the target reasoning model, so as to reason the target data to be predicted according to the target reasoning model to obtain a corresponding reasoning result.

[0027] In a possible implementation, the target data to be predicted includes user information of multiple users; and reasoning the target data to be predicted according to the target reasoning model to obtain corresponding reasoning results includes:

[0028] Determining an inference target for the target data to be predicted based on the target inference model; wherein the inference target represents the probability of the user being the target user;

[0029] Based on the target inference model, each user information in the target data to be predicted is inferred, and with respect to the inference target, the target probability of each user is output; wherein the target probability represents the probability of the user being the target user.

[0030] In a second aspect, an embodiment of the present application further provides a model deployment device, the device comprising:

[0031] A deployment module is used to allocate a pod task to a target business through a container orchestration management platform, and to deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on a model image and a business container based on a business image; the operation information of the model module and the business module represents the operation information bundled together with the business processing process related to the business and the model reasoning process in the model reasoning process;

[0032] an allocation module, configured to allocate container shared resources to the model container and the business container through the pod task, and store files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources;

[0033] A response module is used for the model container and the business container to respond to the target business based on the files and data in the container shared resources.

[0034] In a possible implementation, the model image is compatible with different model versions, different algorithm libraries, different programming languages, and different operating systems; the model image and the business image have independent running programming language libraries and operating system basic libraries;

[0035] Among them, the model image at least includes a model running application, a programming language standard library, an operating system basic library, and a mirror operating system; the business image at least includes a business application, a programming language standard library, an operating system basic library, and a mirror operating system.

[0036] In a possible implementation manner, the response module is specifically configured to:

[0037] In response to the target business, the business container determines a corresponding target reasoning model and a model container corresponding to the target reasoning model based on the target business;

[0038] The business container determines a corresponding access path based on the target business, directly obtains a model parameter file of the target inference model and target data to be predicted based on the access path, and sends the model parameter file and the target data to be predicted to a file sharing path corresponding to the container group;

[0039] The model container directly receives the model parameter file and the target data to be predicted from the file sharing path.

[0040] In a possible implementation manner, the response module is specifically configured to:

[0041] Determine a model warehouse storing the target reasoning model based on the target business, and directly call the client of the model warehouse to download and obtain a model parameter file of the target reasoning model;

[0042] A data warehouse storing target data to be predicted is determined based on the target business, and a client or application programming interface of the data warehouse is directly called to download and obtain the target data to be predicted.

[0043] In a possible implementation manner, the model deployment device further includes:

[0044] A determination module, configured to determine, in response to a change in the target business, a target model image that needs to be modified from among all model images based on the changed target business;

[0045] The modification module is used to modify the target model image based on the changed target business to obtain a modified target model image.

[0046] In a possible implementation manner, the model deployment device further includes:

[0047] An inference module, used for the model container to infer the target data to be predicted to obtain a corresponding inference result;

[0048] The reasoning module is specifically used for:

[0049] Loading the model parameter file to obtain model parameters of the target reasoning model, and building a corresponding model reasoning engine based on the model parameters;

[0050] In response to the completion of the construction of the model reasoning engine, the target data to be predicted is loaded into the target reasoning model, so as to reason the target data to be predicted according to the target reasoning model to obtain a corresponding reasoning result.

[0051] In a possible implementation manner, the target data to be predicted includes user information of multiple users; and the reasoning module is further specifically configured to:

[0052] Determining an inference target for the target data to be predicted based on the target inference model; wherein the inference target represents the probability of the user being the target user;

[0053] Based on the target inference model, each user information in the target data to be predicted is inferred, and with respect to the inference target, the target probability of each user is output; wherein the target probability represents the probability of the user being the target user.

[0054] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the model deployment method as described in any one of the first aspects.

[0055] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the model deployment method described in any one of the first aspects are executed.

[0056] The embodiments of the present application provide a model deployment method, device, electronic device and storage medium, which allocate a pod task to a target business through a container orchestration management platform, and deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on a model image and a business container based on a business image, allocate container shared resources to the model container and the business container through the pod task, and store the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources, and the model container and the business container respond to the target business based on the files and data in the container shared resources. In this application, according to the sidecar mode of the container orchestration management platform k8s, corresponding container groups are deployed for the model module and the business module respectively, and a model container based on the model image and a business container based on the business image are obtained and container shared resources are allocated. The model container and the business container respond to the target business based on the files and data in the container shared resources. By separating the model reasoning from the business, the independent operation of the business and the model is realized, so that the platform supports the deployment of heterogeneous models. At the same time, in this way, the business processing and model reasoning are decoupled, and the lightweight management of the model image is realized, which improves the platform's inclusiveness of the model's dependent environment, reduces the development cost, and is easier to maintain. In order to make the above-mentioned purposes, features and advantages of this application more obvious and easy to understand, the following is a preferred embodiment, and in conjunction with the attached drawings, a detailed description is given as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 is a flow chart of a model deployment method according to an embodiment of the present application;

[0059] Figure 2 This is a schematic diagram of the architecture of the model container and the business container;

[0060] Figure 3 It is a specific framework diagram of the model container and the business container;

[0061] Figure 4 is a flow chart of a model deployment method according to another embodiment of the present application;

[0062] Figure 5 It is a schematic diagram of the overall interaction process between the model container and the business container;

[0063] Figure 6 is a structural schematic diagram of a model deployment device according to an embodiment of the present application;

[0064] Figure 7 It is a structural diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0065] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0066] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0067] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0068] Considering that model reasoning technology refers to the ability to predict and make decisions on unknown data by analyzing and inferring existing data in machine learning models. It involves multiple steps such as data modeling, feature extraction, parameter estimation, reasoning and prediction. In the field of machine learning, model reasoning technology has a wide range of application scenarios, such as image recognition, natural language processing, medical diagnosis, financial risk assessment, and intelligent recommendation systems.

[0069] At present, for business-related processing processes involved in the model reasoning process, such as processing processes related to the data warehouse, model warehouse, and log center, in order to achieve unified deployment and simplify the deployment process, the solution often adopted is to bundle these business processing processes with the model reasoning process as a whole container application. This can also achieve the overall performance brought by the bundled container and the security brought by the container isolation.

[0070] However, the current reasoning method couples the model operation with the business processing module, which makes the platform less inclusive of the model's dependency environment and leads to very high development or maintenance costs. Once there is a conflict between the model operation dependency and the business processing module dependency, one party needs to adjust the version, which makes maintenance difficult.

[0071] To address this problem, the present application provides a model deployment method, device, electronic device and storage medium. According to the sidecar mode of the container orchestration management platform k8s, corresponding container groups are deployed for the model module and the business module respectively, and a model container based on the model image and a business container based on the business image are obtained and container shared resources are allocated. The model container and the business container respond to the target business based on the files and data in the container shared resources. By separating model reasoning from the business, the independent operation of the business and the model is achieved, so that the platform supports the deployment of heterogeneous models. At the same time, in this way, the business processing and model reasoning are decoupled, the lightweight management of the model image is achieved, the platform's inclusiveness of the model's dependent environment is improved, the development cost is reduced, and it is easier to maintain.

[0072] Figure 1 is a flow chart of a model deployment method according to an embodiment of the present application. Figure 1 As shown, the model deployment method of the embodiment of the present application may specifically include:

[0073] S101, assign a pod task to the target business through the container orchestration management platform, and deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, to obtain a model container based on the model image and a business container based on the business image.

[0074] S102, allocating container shared resources to the model container and the business container through the pod task, and storing the files and data generated by the model image and the business image in the process of responding to the target business through the container shared resources.

[0075] S103: The model container and the business container respond to the target business based on the files and data in the container shared resources.

[0076] The above-mentioned model deployment method deploys corresponding container groups for the model module and the business module respectively according to the sidecar mode of the container orchestration management platform k8s, obtains the model container based on the model image and the business container based on the business image and allocates the image shared resources. The model image and the business image respond to the target business based on the files and data in the image shared resources. By separating the model reasoning from the business, the independent operation of the business and the model is realized, so that the platform supports the deployment of heterogeneous models. At the same time, in this way, the business processing and model reasoning are decoupled, the lightweight management of the model image is realized, the platform's inclusiveness of the model's dependent environment is improved, the development cost is reduced, and it is easier to maintain.

[0077] The above exemplary steps of the embodiment of the present application are described below with reference to specific examples:

[0078] S101, assign a pod task to the target business through the container orchestration management platform, and deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, to obtain a model container based on the model image and a business container based on the business image.

[0079] In the embodiment of the present application, the container orchestration management platform is the Kubernetes (k8s for short) platform, the target business is the business to be processed, the pod task is the task of deploying the pod container group, the running information of the model module and the business module represents the information of the coupling of the model and the business, that is, the running information of the model module and the business module represents the running information of the business processing process related to the business and the model reasoning process bundled together in the model reasoning process, the model image is the model running image, and the business image is the business processing image. A pod task is assigned to the target business through the k8s platform, and the corresponding container groups are deployed for the model module and the business module respectively according to the running information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on the model image and a business container based on the business image. For example, Figure 2 As shown in the figure, the container group is deployed according to the pod task to obtain the model container based on the model image and the business container based on the business image. The host operating system is the real underlying operating system under the image, which will not be described in detail here.

[0080] It should be noted that the model image is compatible with different model versions, different algorithm libraries, different programming languages, and different operating systems; the model image and business image have independent running programming language libraries and operating system basic libraries. Figure 2 As shown, the model image includes at least the model running application, programming language standard library, operating system basic library, and image operating system; the business image includes at least the business application, programming language standard library, operating system basic library, and image operating system. Obviously, the model container based on the model image and the business container based on the business image are the same. In addition, different images of the same container group share storage, network, memory, and CPU resources.

[0081] It can be added here that what is obtained are multiple model containers based on model images and one business container based on business images, that is, different model images (different model containers based on model images) enable compatibility with different model versions, different algorithm libraries, different programming languages, and different operating systems.

[0082] Therefore, the business image and the model image have independent running images. The business layer maintains unified processing to complete business-related functions, while the model image is independent and can be compatible with different model versions, different programming languages, and even different operating systems. Figure 3 As shown, you can see various model versions, programming languages, etc.

[0083] It is understood by those skilled in the art that in K8s, a Pod is the smallest deployable computing unit, which contains one or more containers that share storage, network, configuration and other resources. A Pod is the basic unit for running applications in a Kubernetes cluster.

[0084] S102, allocating container shared resources to the model container and the business container through the pod task, and storing the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources.

[0085] In the embodiment of the present application, the container shared resources are storage, network, memory, and CPU resources. The pod task obtained in step S101 allocates container shared resources to the model container and business container in step S101, and stores the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources for subsequent processing. The files and data represent various files and data in the target business process, including various models and data, such as model parameter files and data to be predicted. In addition, they may also include messages and logs. For example, Figure 2 As shown, the shared resources of the containers in the pod include storage, network, memory, and central processing unit (CPU).

[0086] Therefore, in order to better adapt to the different operating environment requirements of different models, and in order to prevent the dependencies of the business module and the operating dependencies of the model reasoning from affecting each other, this application separates the business module and the model module into different containers. Through the sidecar function of k8s, the characteristics of k8s sharing storage, memory, network, CPU and other container sharing resources of different containers in the same POD are realized, so that the business module (business container) and the model module (corresponding to the model container) can have complete operating dependencies, from programming language to the operating system in the container (image), and the business module and the model module can be regarded as a whole independent task, limiting the use of resources in the entire reasoning process.

[0087] S103: The model container and the business container respond to the target business based on the files and data in the container shared resources.

[0088] In the embodiment of the present application, the model container and the business container obtained in step S101 respond to the target business based on the files and data in the container shared resources, thereby completing the reasoning.

[0089] The model deployment method provided in the embodiment of the present application allocates a pod task to the target business through a container orchestration management platform, and deploys corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on the model image and a business container based on the business image, allocates container shared resources to the model container and the business container through the pod task, and stores the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources, and the model container and the business container respond to the target business based on the files and data in the container shared resources. The model deployment method of the present application deploys corresponding container groups for the model module and the business module respectively according to the sidecar mode of the container orchestration management platform k8s, obtains a model container based on the model image and a business container based on the business image and allocates container shared resources, and the model container and the business container respond to the target business based on the files and data in the container shared resources. By separating the model reasoning from the business, the independent operation of the business and the model is realized, so that the platform supports the deployment of heterogeneous models. At the same time, in this way, the business processing and model reasoning are decoupled, the lightweight management of the model image is realized, the platform's inclusiveness of the model's dependent environment is improved, the development cost is reduced, and it is easier to maintain.

[0090] Further, such as Figure 4 As shown, step S103 in the above embodiment, “the model container and the business container respond to the target business based on the files and data in the container shared resources”, may specifically include the following steps:

[0091] S401 , in response to a target business, the business container determines a corresponding target reasoning model and a model container corresponding to the target reasoning model based on the target business.

[0092] In the embodiment of the present application, the target reasoning model is the reasoning model corresponding to the target business, and the business container determines the corresponding target reasoning model based on the target business, and the model container corresponding to the target reasoning model for subsequent processing. For example, if the target business is insurance product response prediction, the insurance product response prediction model corresponding to the target business is determined, and the model container corresponding to the insurance product response prediction model is determined in the model container in the above embodiment.

[0093] S402, the business container determines the corresponding access path based on the target business, and directly obtains the model parameter file and the target data to be predicted of the target inference model based on the access path, and sends the model parameter file and the target data to be predicted to the file sharing path corresponding to the container group.

[0094] In the embodiment of the present application, the access path is the path to access files and data, the model parameter file is the file that represents the parameters of the target reasoning model, the target data to be predicted is the data to be predicted in the target task, and the file sharing path is the file sharing path corresponding to the same container group where the business container and the model container are located. The business container determines the corresponding access path based on the target business, and directly obtains the model parameter file and the target data to be predicted of the target reasoning model based on the access path, and sends the model parameter file and the target data to be predicted to the file sharing path corresponding to the container group. For example, Figure 5 As shown, the business container obtains the inference model, that is, the model parameter file, and the inference data, that is, the target data to be predicted, and pushes the inference model and the inference data to the corresponding file sharing path of the same pod.

[0095] Optionally, based on the target business, determine the model warehouse that stores the target reasoning model, and directly call the client of the model warehouse to download the model parameter file of the target reasoning model; based on the target business, determine the data warehouse that stores the target data to be predicted, and directly call the client or application programming interface of the data warehouse to download the target data to be predicted. The model warehouse is a warehouse that stores the target reasoning model, and the data warehouse is a warehouse that stores the data to be predicted. For example, Figure 5 As shown in the figure, when the target business is started, the business container calls the client of the model warehouse. Generally, the model will store components in some S3 file objects. The corresponding client of the model warehouse is the S3 client. The business container calls the S3 client to obtain the model parameter file from the model warehouse. The data warehouse generally stores the data to be predicted in HDFS. The business container interacts through the client of the data warehouse, the HDFS client, or the corresponding application programming interface, namely the API, to obtain the target data file to be predicted.

[0096] S403: The model container directly receives the model parameter file and the target data to be predicted from the file sharing path.

[0097] In an embodiment of the present application, the model container corresponding to the target inference model directly receives the model parameter file and the target data to be predicted from the file sharing path corresponding to the container group.

[0098] Therefore, in the prior art, there is a method of publishing model reasoning and business processing as independent http services, in which the business processing module is responsible for data pulling and calling the model reasoning http in batches. However, all requests must go through the http service and the requests must go through the http process analysis, which makes the processing speed very slow. In addition, because of the transmission of an additional http service, the problem location chain becomes longer, which brings greater maintenance costs. However, this application directly obtains the model parameter file and the target data to be predicted by responding to the target business, without the process analysis and transmission of the http service, and has a faster processing speed and higher maintainability.

[0099] Further, in response to a change in the target business, a target model image that needs to be modified is determined in all model images based on the changed target business; the target model image is modified based on the changed target business to obtain a modified target model image.

[0100] It should be noted that when the target business changes, it is no longer necessary to modify the operating information of the model and business coupling as in the prior art. The present application realizes the separation of model reasoning and business interaction. It is only necessary to determine the target model image that needs to be modified among various model images, and then modify it based on the changed target business, thereby realizing lightweight management of model images.

[0101] Furthermore, the model container infers the target data to be predicted to obtain corresponding inference results.

[0102] As a possible implementation, the model parameter file is loaded to obtain the model parameters of the target reasoning model, and the corresponding model reasoning engine is constructed based on the model parameters; in response to the completion of the construction of the model reasoning engine, the target data to be predicted is loaded into the target reasoning model, so as to reason the target data to be predicted according to the target reasoning model to obtain the corresponding reasoning result. The target data to be predicted includes user information of multiple users.

[0103] Optionally, determine the inference target for the target data to be predicted based on the target inference model; perform inference on each user information in the target data to be predicted based on the target inference model, and output the target probability of each user for the inference target. The inference target represents the probability of the user being the target user; the target probability represents the probability of the user being the target user. For example, if the target inference model is an insurance product response prediction model, the corresponding inference target is determined to be the probability of the user being a "potential insurance product" customer. The target user is a "potential insurance product" customer. The probability of each user being a "potential insurance product" customer is inferred based on this inference target.

[0104] Further, it is determined whether the target inference model has been completed; if the target inference model has been completed, the business container pushes the inference result to the data warehouse. For example, after the target inference model has been completed, the inference result is returned to the file sharing path, and the business container then writes the inference result back to the corresponding data warehouse from the file sharing path.

[0105] Taking the "insurance product response prediction model" as an example, the specific process is described as a whole. Taking the data storage as a hive database as an example, the business container interacts with the hive database to download the data to be predicted, which in this case is the wide table data composed of user basic information, income information, insurance information, etc. to the local; taking the model stored in s3 object storage as an example, the business container interacts with the s3 object storage component to download the model parameter file to the local; after the model parameter file and the data to be predicted are ready, the model container starts to load the model parameters and build a model inference engine. After the engine is built, the data is loaded into the model. The model infers each user information in the data to be predicted, and outputs whether each user can be a "potential insurance product" customer, as well as the corresponding probability; after the model inference is completed, the business container pushes the result list back to the corresponding data platform for business personnel to extract, so as to assist sales personnel in improving sales success rate.

[0106] Therefore, this application uses a lightweight way to manage different model running environments. Any Python version can be used, including but not limited to Python2.7, Python3.5, Python3.6, Python3.7, Python3.9, Python3.10, etc.; any version of any algorithm framework can be used, including but not limited to scikit-learn, xgboost, lightbgm, catboost, etc., which can better adapt to business changes.

[0107] Figure 6 is a flow chart of a model deployment device according to an embodiment of the present application. Figure 6 As shown, the model deployment device 600 of the embodiment of the present application may specifically include:

[0108] The deployment module 601 is used to assign a pod task to the target business through the container orchestration management platform, and deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on the model image and a business container based on the business image; the operation information of the model module and the business module represents the operation information bundled together with the business processing process related to the business and the model reasoning process in the model reasoning process.

[0109] The allocation module 602 is used to allocate container shared resources to the model container and the business container through the pod task, and store the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources.

[0110] The response module 603 is used for the model container and the business container to respond to the target business based on the files and data in the container shared resources.

[0111] In a possible implementation, the model image is compatible with different model versions, different algorithm libraries, different programming languages, and different operating systems; the model image and the business image have independent running programming language libraries and operating system basic libraries;

[0112] Among them, the model image at least includes the model running application, the programming language standard library, the operating system basic library, and the mirror operating system; the business image at least includes the business application, the programming language standard library, the operating system basic library, and the mirror operating system.

[0113] In a possible implementation, the response module is specifically configured to:

[0114] In response to the target business, the business container determines a corresponding target reasoning model and a model container corresponding to the target reasoning model based on the target business;

[0115] The business container determines the corresponding access path based on the target business, and directly obtains the model parameter file and the target data to be predicted of the target inference model based on the access path, and sends the model parameter file and the target data to be predicted to the file sharing path corresponding to the container group;

[0116] The model container directly receives the model parameter file and the target data to be predicted from the file sharing path.

[0117] In a possible implementation, the response module is specifically configured to:

[0118] Determine the model warehouse that stores the target reasoning model based on the target business, and directly call the client of the model warehouse to download the model parameter file of the target reasoning model;

[0119] Based on the target business, a data warehouse is determined to store the target data to be predicted, and the client or application programming interface of the data warehouse is directly called to download and obtain the target data to be predicted.

[0120] In a possible implementation, the model deployment device further includes:

[0121] A determination module, configured to determine a target model image that needs to be modified from among all model images based on the changed target business in response to a change in the target business;

[0122] The modification module is used to modify the target model image based on the changed target business to obtain the modified target model image.

[0123] In a possible implementation, the model deployment device further includes:

[0124] The inference module is used by the model container to infer the target data to be predicted and obtain the corresponding inference results;

[0125] Reasoning module, specifically used for:

[0126] Load the model parameter file to obtain the model parameters of the target inference model, and build the corresponding model inference engine based on the model parameters;

[0127] In response to the completion of the construction of the model inference engine, the target data to be predicted is loaded into the target inference model, so as to infer the target data to be predicted according to the target inference model to obtain corresponding inference results.

[0128] In a possible implementation, the target data to be predicted includes user information of multiple users; and the reasoning module is further specifically used for:

[0129] Determining an inference target for the target data to be predicted based on the target inference model; wherein the inference target represents the probability that the user is the target user;

[0130] Based on the target inference model, each user information in the target data to be predicted is inferred, and the target probability of each user is output for the inference target; wherein the target probability represents the probability of the user being the target user.

[0131] The model deployment device provided in the embodiment of the present application allocates a pod task to the target business through the container orchestration management platform, and deploys corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, obtains a model container based on the model image and a business container based on the business image, allocates container shared resources for the model container and the business container through the pod task, and stores the files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources, and the model container and the business container respond to the target business based on the files and data in the container shared resources. The model deployment device of the present application deploys corresponding container groups for the model module and the business module respectively according to the sidecar mode of the container orchestration management platform k8s, obtains the model container and the business container and allocates container shared resources, and the model container and the business container respond to the target business based on the files and data in the container shared resources, and separates the model reasoning from the business, so as to realize the independent operation of the business and the model, so that the platform supports the deployment of heterogeneous models, and at the same time, decouples the business processing from the model reasoning in this way, realizes the lightweight management of the model image, improves the inclusiveness of the platform to the model's dependent environment, reduces the development cost, and is easier to maintain.

[0132] like Figure 7 As shown, an electronic device 700 provided in an embodiment of the present application includes: a processor 701, a memory 702 and a bus, wherein the memory 702 stores machine-readable instructions executable by the processor 701. When the electronic device is running, the processor 701 communicates with the memory 702 through the bus, and the processor 701 executes the machine-readable instructions to perform the steps of the above-mentioned model deployment method.

[0133] Specifically, the above-mentioned memory 702 and processor 701 can be general-purpose memory and processor, which are not specifically limited here. When the processor 701 runs the computer program stored in the memory 702, the above-mentioned model deployment method can be executed.

[0134] Corresponding to the above-mentioned model deployment method, an embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned model deployment method are executed.

[0135] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0136] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0137] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the deployment method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0139] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model deployment method, characterized in that: The method comprises: A pod task is assigned to the target business through the container orchestration management platform, and corresponding container groups are deployed for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on the model image and a business container based on the business image; the operation information of the model module and the business module represents the operation information bundled with the business processing process related to the business and the model reasoning process in the model reasoning process; Allocate container shared resources to the model container and the business container through the pod task, and store files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources; The model container and the service container respond to the target service based on the files and data in the container shared resources.

2. The method according to claim 1, characterized in that The model image can be compatible with different model versions, different algorithm libraries, different programming languages, and different operating systems; the model image and the business image have independent running programming language libraries and operating system basic libraries; Among them, the model image at least includes a model running application, a programming language standard library, an operating system basic library, and a mirror operating system; the business image at least includes a business application, a programming language standard library, an operating system basic library, and a mirror operating system.

3. The method according to claim 1, characterized in that The model container and the business container respond to the target business based on the files and data in the container shared resources, including: In response to the target business, the business container determines a corresponding target reasoning model and a model container corresponding to the target reasoning model based on the target business; The business container determines a corresponding access path based on the target business, directly obtains a model parameter file of the target inference model and target data to be predicted based on the access path, and sends the model parameter file and the target data to be predicted to a file sharing path corresponding to the container group; The model container directly receives the model parameter file and the target data to be predicted from the file sharing path.

4. The method according to claim 3, characterized in that The business container determines a corresponding access path based on the target business, and directly obtains a model parameter file of the target reasoning model and target data to be predicted based on the access path, including: Determine a model warehouse storing the target reasoning model based on the target business, and directly call the client of the model warehouse to download and obtain a model parameter file of the target reasoning model; A data warehouse storing target data to be predicted is determined based on the target business, and a client or application programming interface of the data warehouse is directly called to download and obtain the target data to be predicted.

5. The method according to claim 3, characterized in that: The method further comprises: In response to a change in the target business, determining a target model image that needs to be modified from all model images based on the changed target business; The target model image is modified based on the changed target business to obtain a modified target model image.

6. The method according to claim 3, characterized in that The method further comprises: The model container infers the target data to be predicted to obtain a corresponding inference result; The model container infers the target data to be predicted to obtain corresponding inference results, including: Loading the model parameter file to obtain model parameters of the target reasoning model, and building a corresponding model reasoning engine based on the model parameters; In response to the completion of the construction of the model reasoning engine, the target data to be predicted is loaded into the target reasoning model, so as to reason the target data to be predicted according to the target reasoning model to obtain a corresponding reasoning result.

7. The method according to claim 6, characterized in that The target data to be predicted includes user information of multiple users; and the target data to be predicted is reasoned according to the target reasoning model to obtain corresponding reasoning results, including: Determining an inference target for the target data to be predicted based on the target inference model; wherein the inference target represents the probability of the user being the target user; Based on the target inference model, each user information in the target data to be predicted is inferred, and with respect to the inference target, the target probability of each user is output; wherein the target probability represents the probability of the user being the target user.

8. A model deployment device, characterized in that: The device comprises: A deployment module is used to allocate a pod task to a target business through a container orchestration management platform, and to deploy corresponding container groups for the model module and the business module respectively according to the operation information of the model module and the business module associated with the target business through the pod task, so as to obtain a model container based on a model image and a business container based on a business image; the operation information of the model module and the business module represents the operation information bundled together with the business processing process related to the business and the model reasoning process in the model reasoning process; An allocation module, used to allocate mirror shared resources to the model container and the business container through the pod task, and store files and data generated by the model container and the business container in the process of responding to the target business through the container shared resources; A response module is used for the model container and the business container to respond to the target business based on the files and data in the container shared resources.

9. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the model deployment method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the model deployment method as described in any one of claims 1 to 7.

Images