Machine learning model deployment method, device, equipment and storage medium
Through the combination of distributed coordination middleware and Spring Boot and JPMML frameworks, the problem of service interruption during machine learning model deployment and update is solved, and online real-time distributed deployment and high availability are achieved.
Patent Information
- Application Number
- CN202411073854.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-08-07
AI Technical Summary
The existing machine learning model deployment method needs to stop the service during update or maintenance, restart the entire cluster, affecting the continuity of the service.
Through distributed coordination middleware monitoring service nodes, the model is built using Spring Boot and JPMML frameworks, and the prediction model markup language files are stored on the distributed file storage system to achieve isolated deployment of model changes and avoid restarting all models.
It realizes dynamic changes in the deployment model without stopping service, ensuring the continuity and high availability of services.
Smart Images

Figure CN118585227B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and in particular to a machine learning model deployment method, apparatus, device and storage medium. Background Art
[0002] With the rapid development of artificial intelligence technology, machine learning is increasingly being used in the field of financial risk. PMML (Predictive Model Markup Language) files, as a standardized model representation language carrier for machine learning, are currently being widely used for cross-platform machine learning model deployment. In most actual use cases of machine learning models, users can only continue with subsequent operations after the model's prediction results or effects are available in mobile applications or displayed on web pages, which makes real-time machine learning increasingly popular in use cases.
[0003] The existing machine learning model computing service is to load the PMML file into the Java program process memory and run it. When the PMML file is deployed, the file is packaged in the JAR package and loaded into the memory when the Java program is started to provide model prediction computing capabilities. However, this deployment method has the following defects: Since the model file is packaged in the JAR package, if you want to update or maintain the deployed model, you must first stop the service, then manually update the jar package of each node in the entire service cluster, and then restart the entire cluster service, which will affect the continuity of the service. Summary of the invention
[0004] The main purpose of this application is to provide a machine learning model deployment method, device, equipment and storage medium, aiming to solve the technical problem that in the traditional machine learning model deployment method, if you want to update or maintain the deployed model, you must first stop the service, then manually update the jar package of each node in the entire service cluster, and then restart the entire cluster service, thereby affecting the continuity of the service.
[0005] To achieve the above objectives, the present application proposes a machine learning model deployment method, the method comprising:
[0006] Monitor each service node through a distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system;
[0007] When a model change instruction is received, the change interface corresponding to the machine learning model on each service node is called in sequence according to the model change instruction;
[0008] Obtaining an updated prediction model markup language file from the distributed file storage system through the change interface;
[0009] According to the updated prediction model markup language file, the machine learning model is changed and deployed on each service node.
[0010] In one embodiment, the step of performing a change deployment of the machine learning model on each service node according to the updated prediction model markup language file includes:
[0011] Generate a prediction entity set according to the updated prediction model markup language file;
[0012] Obtaining a model identifier corresponding to the change interface;
[0013] The model identifier and the predicted entity set are cached and changed until each service node is traversed, thereby completing the change deployment of the machine learning model.
[0014] In one embodiment, the step of caching the model identifier and the predicted entity set until each service node is traversed and the change deployment of the machine learning model is terminated includes:
[0015] Perform cache change on each service node according to the model identifier and the predicted entity set to obtain a cache change state;
[0016] When the cache change state is in a deployment failure state, the cache change operation is stopped and a preset rollback mechanism is triggered;
[0017] The changed service node is rolled back according to the preset rollback mechanism, and after the rollback process, the change deployment of the machine learning model is ended.
[0018] In one embodiment, the model change instruction includes a model modification instruction, and the step of sequentially calling the change interface corresponding to the machine learning model on each service node according to the model change instruction includes:
[0019] When the model change instruction is a model modification instruction, according to the model modification instruction, the modification interface corresponding to the machine learning model on each service node is called in sequence;
[0020] According to the modification interface, the latest model version of the machine learning model is obtained from the Oracle database, and the step of obtaining the updated prediction model markup language file from the distributed file storage system is performed. The Oracle database stores deployment metadata of different model versions of the machine learning model.
[0021] In one embodiment, the model change instruction includes a delete model instruction, and the step of sequentially calling the change interface corresponding to the machine learning model on each service node according to the model change instruction further includes:
[0022] When the model change instruction is a model deletion instruction, according to the model deletion instruction, the deletion interface corresponding to the machine learning model on each service node is called in sequence;
[0023] Through the deletion interface, the corresponding model identifier is removed from the memory of the service node, and the deletion status is returned to end the change deployment of the machine learning model.
[0024] In one embodiment, after the step of monitoring each service node by the distributed coordination middleware, the method further includes:
[0025] Collecting the message traffic of each service node;
[0026] Perform node calculation according to the message traffic to determine the number of nodes to be increased or decreased corresponding to each service node;
[0027] Based on the increase or decrease in the number of nodes, each service node is turned on and off through the node controller.
[0028] In one embodiment, the step of opening and closing each service node through a node controller based on the increase or decrease in the number of nodes includes:
[0029] When the number of nodes increased or decreased is the number of newly added nodes, the machine learning model is started on the new service node through the node controller, and the new service node is loaded onto the load balancer and the distributed coordination middleware;
[0030] When the increase or decrease in the number of nodes is the number of nodes reduced, the service nodes for which the machine learning model has been started are stopped through the node controller, and the service nodes corresponding to the number of nodes reduced are deleted through the load balancer and the distributed coordination middleware.
[0031] In addition, to achieve the above purpose, the present application also proposes a machine learning model deployment device, the device comprising:
[0032] A node monitoring module is used to monitor each service node through a distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system;
[0033] A node calling module is used to call the change interface corresponding to the machine learning model on each service node in sequence according to the model change instruction when receiving the model change instruction;
[0034] A file acquisition module, used to acquire an updated prediction model markup language file from the distributed file storage system through the change interface;
[0035] The change deployment module is used to change the deployment of the machine learning model to each service node according to the updated prediction model markup language file.
[0036] In addition, to achieve the above-mentioned objectives, the present application also proposes a machine learning model deployment device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the machine learning model deployment method described above.
[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the machine learning model deployment method described above are implemented.
[0038] One or more technical solutions proposed in this application have at least the following technical effects: this application deploys a machine learning model on each service node, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system; each service node is monitored by a distributed coordination middleware; then, when a model change instruction is received, the change interface corresponding to the machine learning model on each service node is called in sequence according to the model change instruction; then, through the change interface, an updated prediction model markup language file is obtained from the distributed file storage system; finally, according to the updated prediction model markup language file, the machine learning model is changed and deployed to each service node. Since this application monitors each service node where the machine learning model is deployed through a distributed coordination middleware, when the machine learning model needs to be changed, the corresponding change interface can be called in sequence through the distributed coordination middleware without restarting all models, thereby achieving the isolation effect of the model change, and will not affect other machine learning models running online, thereby achieving the dynamic change of the machine learning model to the service node without stopping the service, ensuring the continuity of the service. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0041] Figure 1 A flowchart of the first embodiment of the machine learning model deployment method of the present application is provided;
[0042] Figure 2 A schematic diagram of the process of preloading a machine learning model provided in Example 1 of the present application;
[0043] Figure 3 A flowchart diagram of the second embodiment of the machine learning model deployment method of the present application;
[0044] Figure 4 A schematic diagram of online real-time distributed deployment of a machine learning model provided in Example 2 of the present application;
[0045] Figure 5 A flowchart diagram of the third embodiment of the machine learning model deployment method of this application;
[0046] Figure 6 A schematic diagram of the deployment of traffic monitoring and dynamic expansion and reduction of service nodes provided in Example 3 of the present application;
[0047] Figure 7 This is a schematic diagram of the module structure of the machine learning model deployment device according to an embodiment of the present application;
[0048] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the machine learning model deployment method in the embodiment of the present application.
[0049] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0050] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.
[0051] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.
[0052] It should be noted that the execution subject of this embodiment can be a computing service device with interface calling, file acquisition and model deployment functions, such as a personal computer, server, virtual machine, etc., or an electronic device that can realize the above functions, a machine learning model deployment device (hereinafter referred to as deployment device) that executes the machine learning model deployment method of this application, etc., and this embodiment does not limit this. The following takes the deployment device as an example to illustrate this embodiment and the following embodiments.
[0053] Based on this, the present application embodiment provides a method for deploying a machine learning model. Figure 1 , Figure 1 A flowchart of the first embodiment of the machine learning model deployment method of the present application is provided.
[0054] In this embodiment, the machine learning model deployment method includes steps S10 to S40:
[0055] Step S10: monitor each service node through the distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in the distributed file storage system.
[0056] It should be noted that the existing machine learning model deployment methods lack automated deployment processes and standardized interfaces, and the automation capabilities are insufficient, which makes the migration of models from development to production environments not smooth enough, and often requires a lot of manual intervention and adjustments, resulting in a time-consuming and error-prone deployment process.
[0057] Therefore, in order to ensure the high availability of the machine learning model and the high reliability of model deployment and operation, this embodiment uses the Spring Boot framework (an open source framework based on the Spring framework, used to simplify Spring application development) and the JPMML framework to build a prediction engine for the machine learning model. At the same time, the PMML file of the machine learning model is stored in a distributed file storage system, and multiple copies can be stored on multiple service nodes in the cluster of deployed devices to cope with hardware failures or data loss. In addition, the distributed coordination middleware is innovatively introduced, and all service nodes are registered as servers to the distributed coordination middleware for recording and maintenance. When a service node is changed, the distributed coordination middleware can automatically identify the change to avoid omissions when manually configuring the maintenance node and causing problems. In this way, the online real-time distributed deployment function of the model can be realized.
[0058] It should be noted that the JPMML framework is a framework for converting machine learning models into a deployable Predictive Model Markup Language (PMML) format. The PMML framework enables machine learning models to be more easily shared and deployed in different systems and environments, enabling cross-platform use and integration of models.
[0059] It should be noted that distributed coordination middleware is a distributed system that provides a series of coordination services for distributed applications, which can help multiple components or services in a distributed system to communicate, collaborate and synchronize effectively, so that the distributed application of the above machine learning model can run more orderly and reliably, avoiding inconsistencies or confusion.
[0060] Exemplarily, the distributed coordination middleware may include ZooKeeper distributed application coordination middleware, Etcd distributed key-value storage system, etc., which is not limited in this embodiment.
[0061] It should be noted that the machine learning model is deployed on multiple service nodes (which can be understood as hardware devices such as computers or servers, or multiple virtual service nodes running on virtual machines, which are not limited in this embodiment) in the cluster corresponding to the deployment device, so that the distributed computing and processing of the machine learning model can be realized. By deploying the model to multiple service nodes, data can be processed in parallel, which improves computing efficiency and processing speed, and also helps to improve the reliability and fault tolerance of the system. When a node fails, other nodes can continue to work.
[0062] It should be noted that the distributed file storage system is a distributed system for storing and managing large-scale data, such as HDFS (Hadoop Distributed File System) distributed file system, Ceph distributed file system, etc., which is not limited in this embodiment. The distributed file storage system has the characteristics of high fault tolerance, high reliability, high scalability, etc., and can store data in multiple nodes, ensure data security and availability through data redundancy, process very large amounts of data, and support data storage and access in large-scale cluster environments.
[0063] By introducing a distributed file storage system to store PMML files, high data reliability can be ensured and data loss caused by hardware failure can be avoided.
[0064] It should be noted that the Predictive Model Markup Language (PMML) file is a standard format file used to represent predictive analysis models (i.e. machine learning models). PMML files contain detailed descriptions of the structure, parameters, variable definitions, etc. of machine learning models, which can easily migrate and use these models between different applications or platforms, promoting the reusability and interoperability of models.
[0065] For easier understanding, refer to Figure 2 , Figure 2 This is a flow chart of the preloading process of the machine learning model provided in the first embodiment of the present application. Figure 2 As shown in the figure, the process content of the machine learning model pre-loaded as a prediction engine includes:
[0066] 1. When the machine learning model is in development, after the machine learning modeling training is completed, the Python sklearn2pmml function (a function or related tool for converting machine learning models into PMML files) can be used to output the model file (i.e., PMML model file). This file can be converted and compiled into a class object supported by Java, thereby providing model prediction and reasoning capabilities that can be called by external systems.
[0067] 2. The location of the PMML model file is then stored on the distributed file storage system HDFS to ensure high data availability. The model version is controlled by Oracle Database (a relational database management system) and stores the model deployment metadata.
[0068] 3. Then the model enters the running state. After the prediction engine of the machine learning model based on the Spring Boot framework is started, it will first access the Oracle database to load the metadata information of the model deployment, including the model ID (i.e. the identifier of the machine learning model), the version number, and the storage address of the file in HDFS. Then load the location of the model file, create a Configuration object, set the DFS (Distributed FileSystem) client of Hadoop (a distributed system infrastructure), and obtain the PMML file from HDFS.
[0069] 4. After obtaining the PMML file, use the JPMML module (a Java package for parsing and loading PMML files) to parse and compile the Evaluator object (a Java object used to calculate the model in the JPMML module), and define the corresponding model input parameter InputField and output parameter OutputField. Cache the predicted entity set of these models into the memory of the deployment device (that is, the Java instance set Map, a virtual storage object). In subsequent calls, the calculation method of the corresponding machine learning model can be quickly obtained according to the model ID.
[0070] 5. When connecting to the external interface of the front-end service, pass in the corresponding model ID and the feature data set of the model input parameters. Call the prediction interface of the prediction engine of the machine learning model to evaluate the model. The prediction engine obtains the corresponding model instance according to the model ID, executes the prediction Evaluator object, and returns the output prediction result.
[0071] In one embodiment, a machine learning model is deployed on each service node. The machine learning model is constructed based on the Spring Boot framework and the JPMML framework. The prediction model markup language file of the machine learning model is stored in a distributed file storage system. Each service node is monitored by a distributed coordination middleware. Through the above method, the online real-time distributed deployment function of the model is realized.
[0072] Step S20: When a model change instruction is received, the change interface corresponding to the machine learning model on each service node is called in sequence according to the model change instruction.
[0073] It should be noted that model change instructions are specific instructions for modifying, adjusting or updating machine learning models on each service node.
[0074] Exemplarily, the model change instructions may include instructions for adding a new model, deleting a model, modifying a model, rolling back a model, etc., and may also be other change instructions, which are not limited in this embodiment.
[0075] It should be noted that the change interface is the interface for changing the machine learning model in each service node. Based on different model change instructions, there are also different interfaces, such as insert interface (i.e., new interface, corresponding to the instruction of adding a new model), delete interface (i.e., delete interface, corresponding to the instruction of deleting a model), update interface (i.e., modification interface, corresponding to the instruction of modifying a model), rollback interface (i.e., rollback interface, corresponding to the instruction of rolling back a model), etc. This embodiment does not limit this.
[0076] In one embodiment, when the deployment device receives a model change instruction, it calls different change interfaces corresponding to the machine learning models on each service node according to different model change instructions, and performs subsequent corresponding operations. The deployment device will traverse all service nodes in turn to perform the same operation to ensure the integrity of the change. This method can effectively manage model changes and ensure the stability and accuracy of the system.
[0077] Step S30: Obtaining an updated prediction model markup language file from the distributed file storage system through the change interface.
[0078] It should be noted that the updated prediction model markup language file is the PMML file of the required changes stored in the distributed file storage system when the machine learning model on the service node is changed.
[0079] Step S40: According to the updated prediction model markup language file, the machine learning model is changed and deployed on each service node.
[0080] It should be noted that the change deployment process can be deployed differently depending on the interface being changed. For example, for a deletion interface, the corresponding model ID can be directly removed from the instance collection Map in memory, and then the deletion state is returned. For a new interface, the above model deployment process needs to be executed, that is, according to the latest PMML file, the Evaluator object, input parameter InputField, and output parameter OutputField are generated, the model ID and prediction instance are passed to the collection Map, and cached in the memory of the deployment device, and then the new state is returned.
[0081] In this embodiment, the deployment device can obtain the PMML file that needs to be changed from the distributed file storage system through the change interface, and finally perform different changes to the machine learning model deployment on each service node according to the PMML file that needs to be changed and the different change interfaces. By using Zookeeper technology to maintain changes in server nodes, each machine learning model is preloaded and cached in each server node to reduce the time consumption of online real-time calculation of the model, reduce hardware resource usage, and avoid omissions that may occur during manual configuration.
[0082] This embodiment provides a machine learning model deployment method, deploying machine learning models on each service node, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system; and each service node is monitored by a distributed coordination middleware, and the online real-time distributed deployment function of the model is realized by the above method. When the deployment device receives a model change instruction, different change interfaces corresponding to the machine learning models on each service node can be called according to different model change instructions. Through the change interface, the PMML file required to be changed is obtained from the distributed file storage system, and finally, according to the PMML file required to be changed, different changes are deployed to the machine learning model for each service node according to different change interfaces. Since this embodiment monitors each service node where the machine learning model is deployed through a distributed coordination middleware, when the machine learning model needs to be changed, the corresponding change interface can be called in sequence through the distributed coordination middleware without restarting all models, so as to achieve the isolation effect of the model change, and will not affect other machine learning models running online, thereby realizing the dynamic change of the machine learning model to the service node without stopping the service, and ensuring the continuity of the service.
[0083] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 1 and Figure 3 , Figure 2 A flowchart diagram is provided for Example 2 of the machine learning model deployment method of this application.
[0084] In this example, step S40 includes steps S41 to S43:
[0085] Step S41: Generate a prediction entity set according to the updated prediction model markup language file.
[0086] It should be noted that the prediction entity set is a set of objects or elements that the machine learning model targets when making predictions, such as Evaluator objects, input parameters InputField, and output parameters OutputField.
[0087] Step S42: Obtain the model identifier corresponding to the change interface.
[0088] It should be noted that the model ID is an identifier used to uniquely identify and distinguish different machine learning models.
[0089] Step S43: cache the model identifier and the predicted entity set until each service node is traversed, thereby completing the change deployment of the machine learning model.
[0090] In this embodiment, for ease of understanding, please refer to Figure 4 , Figure 4 Schematic diagram of online real-time distributed deployment of machine learning models provided in Example 2 of this application. When a user needs to update or deploy a machine learning model, the client service manager (such as client) can call the http interface to complete the update of a single model and deploy it to all computing service nodes. At this time, the deployed machine learning model instance corresponds to the program thread, and each model is deployed to all model computing service nodes (non-HDFS distributed nodes). Figure 4 As shown in the figure, server node 1 and server node 2 simultaneously deploy PMML models (i.e. machine learning models) 1, model 2, model 3, and model 4. Each node starts four instances (i.e. instance 1, instance 2, instance 3, and instance 4) for parallel computing. This ensures that each service node can handle all model requests. In the case of high concurrent requests, the load balancing system can also be used to distribute traffic without considering the nodes where the model is deployed, thereby significantly improving the concurrent performance and stability of the middleware.
[0091] At the same time, all service nodes are automatically registered as servers and deployed to the ZooKeeper component for recording and maintenance. When a service node is added or reduced, ZooKeeper can automatically identify the change. This method can avoid problems caused by omissions when manually configuring and maintaining nodes;
[0092] When the PMML model changes (including addition, deletion, and modification), the node addresses of all server-side service nodes can be obtained through ZooKeeper. Then the corresponding operations are performed through the change interface exposed by the model prediction engine. All service nodes are traversed in turn to perform the same operations to ensure the integrity of the changes. This method can effectively manage model changes and ensure the stability and accuracy of the system.
[0093] In another feasible implementation, the model change instruction includes a model modification instruction. Step S20 of this embodiment may include the following steps: when the model change instruction is a model modification instruction, calling the modification interface corresponding to the machine learning model on each service node in sequence according to the model modification instruction; obtaining the latest model version of the machine learning model from the Oracle database according to the modification interface, and executing the step of obtaining an updated prediction model markup language file from the distributed file storage system, wherein the Oracle database stores deployment metadata of different model versions of the machine learning model.
[0094] It should be noted that the model modification instruction is a specific instruction used to modify the machine learning model of each service node.
[0095] It should be noted that the modification interface (such as the update interface) is the interface for modifying the machine learning model in each service node.
[0096] In this implementation, when the model change instruction is a model modification instruction, the update interface is called to obtain the latest model version of the corresponding machine learning model from the Oracle database, and perform the same operations as the above-mentioned newly added model.
[0097] In another feasible implementation, the model change instruction includes a model deletion instruction. Step S40 of this embodiment may further include the following steps: when the model change instruction is a model deletion instruction, calling the deletion interface corresponding to the machine learning model on each service node in turn according to the model deletion instruction; through the deletion interface, removing the corresponding model identifier from the memory of the service node, and returning to the deletion state, thereby ending the change deployment of the machine learning model.
[0098] It should be noted that the delete model instruction is a specific instruction used to delete the machine learning model of each service node.
[0099] It should be noted that the deletion interface (such as the delete interface) is the interface for deleting the machine learning model in each service node.
[0100] In this implementation, when the model change instruction is a model deletion instruction, the delete interface may be called to remove the corresponding model ID from the instance set Map in the memory, and then return to the deletion state.
[0101] Furthermore, when the model change instruction is a model rollback instruction, the rollback interface can also be called to obtain the last changed version in the historical version of the corresponding machine learning model from the Oracle database, and then perform the same operation as the newly added model.
[0102] In a feasible implementation, step S43 of this embodiment may include the steps of: performing cache changes on each service node according to the model identifier and the predicted entity set to obtain a cache change status; when a deployment failure occurs in the cache change status, stopping the cache change operation and triggering a preset rollback mechanism; rolling back the changed service nodes according to the preset rollback mechanism, and ending the change deployment of the machine learning model after the rollback process.
[0103] It should be noted that the preset rollback mechanism is a fallback mechanism set when the deployment of the model fails.
[0104] In this implementation, various unexpected situations may occur when implementing changes through the interface, such as network anomalies, system loading anomalies, etc. Therefore, state identification and a preset rollback mechanism can be used to deal with the above possible situations:
[0105] 1. If the model deployment change is successful, a success status is returned.
[0106] 2. If the model deployment change fails, the preset rollback mechanism is triggered to roll back the service node that has been successfully changed. For example, if the model fails to be added, the above-mentioned model deletion operation is performed in the service node that has been successfully added; if the model fails to be deleted, the above-mentioned model addition operation is performed in the service node that has been successfully deleted; if the model modification fails, the above-mentioned model rollback operation is performed in the service node that has been modified.
[0107] Therefore, when updating nodes, the database can be used to cache the update status of each node. If all nodes are updated successfully, the model will be formally submitted for release for other systems to call. If a node fails to update due to network or configuration reasons, the above-mentioned preset rollback mechanism can be used to roll back the successfully updated nodes to ensure the consistency of the model status of all nodes.
[0108] When constructing a machine learning model deployment instance, this embodiment corresponds it one-to-one with the thread of the Java program, and deploys each instance to all service nodes indiscriminately. Creatively, each deployed service node is automatically registered with the zookeeper server, and the distributed application coordination capability of zookeeper is used to maintain the nodes where the machine learning model is deployed. When the machine learning model needs to be updated, the address of the deployment node is obtained from the zookeeper client, and then the corresponding machine learning model on each deployment node is polled and updated through the HTTP interface without restarting all models. This method achieves the isolation effect of model updates, does not affect other machine learning models running online, and does not interfere with real-time machine learning model requests.
[0109] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those of the first and second embodiments can be referred to the above description, and will not be described in detail later. Figure 1 and Figure 5 , Figure 5 A flowchart of the third embodiment of the machine learning model deployment method of this application is provided.
[0110] Considering that the traditional machine learning model deployment does not fully consider horizontal expansion and load balancing during design, resources cannot be effectively allocated. When the amount of data or user requests increase, the traditional model is difficult to flexibly expand and cannot meet the needs of high concurrency and large data volume. After step S10, this example also includes steps S21~S23:
[0111] Step S21: Collect the message traffic of each service node.
[0112] It should be noted that this embodiment further innovatively introduces the F5 load balancing solution to realize the online real-time distributed deployment function of the model. Among them, F5 load balancing is a hardware device specially designed to optimize network traffic distribution. When a user request reaches the F5 load balancer, the load balancer decides which backend server to distribute the traffic to based on a preset load balancing algorithm, so as to ensure that each server can make full use of its resources and avoid a server overload resulting in performance degradation or service unavailability.
[0113] For easier understanding, refer to Figure 6 , Figure 6 A schematic diagram of the deployment of traffic monitoring and dynamic expansion and reduction of service nodes provided in Example 3 of the present application. The prediction service nodes of multiple machine learning models deployed in a distributed manner provide services to external systems through the F5 load balancing solution. The traffic will be evenly distributed to each service node (such as multiple dynamic nodes such as node 1, node 2, node 3, etc.) for load, and the service nodes will be registered with zookeeper for maintenance. Because all machine learning models are deployed on each service node, there is no need to develop a scheduling system for routing and forwarding. By using F5 load balancing to evenly distribute requests to each computing service node, overload of a single node can be avoided, and the overall concurrent processing capability and stability of the system can be improved.
[0114] It should be noted that the message flow is the amount of messages (data packets) received and sent by each node providing services. Messages are the basic unit of data transmission. Service nodes will continuously exchange data, and these interactive data exist in the form of messages. Message flow reflects the amount of data communication processed by each node within a certain period of time.
[0115] Step S22: Perform node calculation according to the message flow to determine the number of nodes to be increased or decreased corresponding to each service node.
[0116] It should be noted that the increase or decrease in the number of nodes refers to the number of service nodes that need to be increased or decreased in each service node.
[0117] Specifically, traffic monitoring and dynamic increase or decrease of service nodes can be calculated based on the TPS (processing volume per second) of the traffic and the interval traffic of nearly 300 seconds to calculate the number of service nodes that need to be expanded or reduced. The specific calculation formula is as follows:
[0118] ;
[0119] in Indicates the number of nodes that need to be increased (or decreased). When the value is positive, it means that the number of nodes needs to be increased (rounded up). When the value is negative, it means that the number of nodes needs to be decreased (rounded up). Indicates the number of requests in the past 300 seconds; Indicates the deployed nodes (the number of registered nodes can be obtained from Zookeeper); Indicates the average throughput per second of each node.
[0120] Step S23: Based on the increase or decrease in the number of nodes, each service node is turned on or off by the node controller.
[0121] It should be noted that the node controller is a device or software responsible for managing and controlling multiple service nodes, and is responsible for configuring, monitoring, coordinating, and making some decisions on the nodes. The node controller ensures that the nodes run according to predetermined rules and policies.
[0122] When an activity has a flow that exceeds the threshold, the deployment of service nodes can be automatically triggered to balance the flow and avoid delays caused by backlogs; when the flow decreases, the service nodes are automatically reduced to save resource utilization.
[0123] In this implementation mode, for ease of understanding, reference may be made to Figure 6, the message traffic requested by the external system (such as the client) will be collected asynchronously through Kafka (a distributed message queue system used to handle large-scale real-time data processing and streaming data transmission), and then flow to the traffic monitoring service for statistics, and the number of service nodes that need to be increased or decreased will be determined by the TPS (transactions per second) of each node. When the traffic monitoring service detects a relatively large change in traffic and needs to increase or decrease nodes, it will notify the node controller (used to interact with zookeeper to obtain instance service nodes and open and close applications on service nodes) to obtain node addresses from zookeeper for dynamic increase or decrease. Therefore, using the dynamic deployment function of traffic monitoring and service nodes, traffic peaks and troughs can be identified, and service nodes can be automatically increased or decreased, while ensuring the throughput of the entire system and maximizing the use of resources. At the same time, when there is a change in service nodes, the increase or decrease of nodes can also be automatically identified through zookeeper, ensuring the accuracy of the number of deployed nodes, and also avoiding production accidents caused by omissions in manual maintenance of node lists.
[0124] In a feasible implementation, step S23 of this embodiment may include the following steps: when the increase or decrease in the number of nodes is the number of newly added nodes, starting the machine learning model on the new service node through the node controller, and loading the new service node onto the load balancer and the distributed coordination middleware; when the increase or decrease in the number of nodes is the number of reduced nodes, stopping the service node for which the machine learning model has been started through the node controller, and deleting the service nodes corresponding to the number of reduced nodes through the load balancer and the distributed coordination middleware.
[0125] In this implementation, when a node is added, the node controller starts the PMML model on the new service node, loads it to the F5 load balancer, and registers it with Zookeeper for maintenance. It should be noted that the same applies to the newly registered service node with Zookeeper. Figure 4 Online real-time distributed deployment of PMML models in . When the number of nodes decreases, the node controller stops the PMML model started on the service node, and the F5 load balancer and zookeeper automatically delete the corresponding node. The F5 load balancer and zookeeper can automatically identify and increase or decrease service nodes to ensure the accuracy of the number of deployed nodes.
[0126] In this embodiment, when the traffic monitoring service detects a significant change in traffic and needs to increase or decrease nodes, it will notify the node controller to obtain the node address from Zookeeper for dynamic increase or decrease. Therefore, using the F5 load balancer, traffic peaks and troughs can be identified, and service nodes can be automatically increased or decreased, while ensuring the throughput of the entire system and maximizing the use of resources. At the same time, when there is a change in the service node, the increase or decrease of the node can also be automatically identified through Zookeeper, ensuring the accuracy of the number of deployed nodes, while also avoiding production accidents caused by omissions in the manual maintenance of the node list.
[0127] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the machine learning model deployment method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.
[0128] This application also provides a machine learning model deployment device, please refer to Figure 7 , Figure 7 This is a schematic diagram of the module structure of the machine learning model deployment device according to an embodiment of the present application; the machine learning model deployment device includes:
[0129] A node monitoring module 701 is used to monitor each service node through a distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on a Spring Boot framework and a JPMML framework, and a prediction model markup language file of the machine learning model is stored in a distributed file storage system;
[0130] The node calling module 702 is used to call the change interface corresponding to the machine learning model on each service node in sequence according to the model change instruction when receiving the model change instruction;
[0131] A file acquisition module 703, used to acquire an updated prediction model markup language file from the distributed file storage system through the change interface;
[0132] The change deployment module 704 is used to change the deployment of the machine learning model to each service node according to the updated prediction model markup language file.
[0133] The machine learning model deployment device provided in the present application adopts the machine learning model deployment method in the above-mentioned embodiment, which can solve the technical problem that in the traditional machine learning model deployment method, if the deployed model is to be updated or maintained, the service must be stopped first, and then the jar package of each node in the entire service cluster must be manually updated, and then the entire cluster service must be restarted, thereby affecting the continuity of the service. Compared with the prior art, the beneficial effects of the machine learning model deployment device provided in the present application are the same as the beneficial effects of the machine learning model deployment method provided in the above-mentioned embodiment, and the other technical features in the machine learning model deployment device are the same as the features disclosed in the above-mentioned embodiment method, which will not be repeated here.
[0134] The present application provides a machine learning model deployment device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the machine learning model deployment method in the above-mentioned embodiment one.
[0135] Reference below Figure 8 , Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the machine learning model deployment method in the embodiment of the present application, which shows a schematic diagram of the structure of the machine learning model deployment device suitable for implementing the embodiment of the present application. The machine learning model deployment device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions: tablet computers), PMPs (Portable Media Players: portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The machine learning model deployment device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present application.
[0136] like Figure 8As shown, the machine learning model deployment device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the machine learning model deployment device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the machine learning model deployment device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a machine learning model deployment device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.
[0137] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.
[0138] The machine learning model deployment device provided by the present application adopts the machine learning model deployment method in the above embodiment, which can solve the technical problem that if the traditional machine learning model deployment method wants to update or maintain the deployed model, it is necessary to stop the service first, then manually update the jar package of each node in the entire service cluster, and then restart the entire cluster service, thereby affecting the continuity of the service. Compared with the prior art, the beneficial effects of the machine learning model deployment device provided by the present application are the same as the beneficial effects of the machine learning model deployment method provided by the above embodiment, and the other technical features in the machine learning model deployment device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.
[0139] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.
[0140] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0141] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the machine learning model deployment method in the above-mentioned embodiment.
[0142] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.
[0143] The above-mentioned computer-readable storage medium may be included in the machine learning model deployment device; or it may exist independently without being assembled into the machine learning model deployment device.
[0144] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the machine learning model deployment device, the machine learning model deployment device enables the following: monitor each service node through the distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the SpringBoot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in the distributed file storage system; when receiving a model change instruction, call the change interface corresponding to the machine learning model on each service node in turn according to the model change instruction; obtain an updated prediction model markup language file from the distributed file storage system through the change interface; and perform change deployment of the machine learning model for each service node according to the updated prediction model markup language file.
[0145] Computer program code for performing the operations of the present application may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0146] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0147] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.
[0148] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned machine learning model deployment method, and can solve the technical problem that in the traditional machine learning model deployment method, if the deployed model is to be updated or maintained, the service must be stopped first, and then the jar package of each node of the entire service cluster must be manually updated, and then the entire cluster service must be restarted, thereby affecting the continuity of the service. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the machine learning model deployment method provided in the above-mentioned embodiment, which will not be repeated here.
[0149] The present application also provides a computer program product, including a computer program, which implements the steps of the machine learning model deployment method as described above when executed by a processor.
[0150] The computer program product provided by this application can solve the technical problem that in the traditional machine learning model deployment method, if you want to update or maintain the deployed model, you must first stop the service, then manually update the jar package of each node in the entire service cluster, and then restart the entire cluster service, thereby affecting the continuity of the service. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the machine learning model deployment method provided in the above embodiment, which will not be repeated here.
[0151] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.
Claims
1. A machine learning model deployment method, characterized in that: The method includes: Monitor each service node through distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system, wherein multiple copies are stored on multiple service nodes in the cluster, and a distributed coordination middleware is introduced, and all service nodes are registered as service ends on the distributed coordination middleware for recording and maintenance. When a service node is changed, the distributed coordination middleware automatically identifies the change, and the distributed coordination middleware includes ZooKeeper distributed application coordination middleware or Etcd distributed key-value storage system; When a model change instruction is received, the change interface corresponding to the machine learning model on each service node is called in sequence according to the model change instruction; Obtaining an updated prediction model markup language file from the distributed file storage system through the change interface; According to the updated prediction model markup language file, change and deploy the machine learning model on each service node; Wherein, after the step of monitoring each service node through the distributed coordination middleware, the method further includes: Collecting the message traffic of each service node; Node calculation is performed based on the message traffic to determine the number of nodes to be increased or decreased corresponding to each service node. Traffic monitoring and dynamic increase or decrease of service nodes can be calculated based on the traffic processing volume per second and the interval traffic of nearly 300 seconds to calculate the number of service nodes that need to be expanded or reduced. The specific calculation formula is as follows: ; in, Indicates the number of nodes that need to be increased or decreased. When the value is positive, it means that the number of nodes needs to be increased, and when the value is negative, it means that the number of nodes needs to be decreased. Indicates the number of requests in the past 300 seconds; Indicates deployed nodes; Indicates the average throughput per second of each node; Based on the increase or decrease in the number of nodes, each service node is opened or closed by a node controller; The step of performing a change deployment of the machine learning model on each service node according to the updated prediction model markup language file includes: Generate a prediction entity set according to the updated prediction model markup language file; Obtaining a model identifier corresponding to the change interface; The model identifier and the predicted entity set are cached and changed until each service node is traversed, and the change deployment of the machine learning model is completed; The step of caching and changing the model identifier and the predicted entity set until each service node is traversed and the change deployment of the machine learning model is ended includes: Perform cache change on each service node according to the model identifier and the predicted entity set to obtain a cache change state; When the cache change state is in a deployment failure state, the cache change operation is stopped and a preset rollback mechanism is triggered; The changed service node is rolled back according to the preset rollback mechanism, and after the rollback process, the change deployment of the machine learning model is ended.
2. The method according to claim 1, characterized in that The model change instruction includes a model modification instruction, and the step of sequentially calling the change interface corresponding to the machine learning model on each service node according to the model change instruction includes: When the model change instruction is a model modification instruction, according to the model modification instruction, the modification interface corresponding to the machine learning model on each service node is called in sequence; According to the modification interface, the latest model version of the machine learning model is obtained from the Oracle database, and the step of obtaining the updated prediction model markup language file from the distributed file storage system is performed. The Oracle database stores deployment metadata of different model versions of the machine learning model.
3. The method according to claim 1, characterized in that The model change instruction includes a delete model instruction, and the step of sequentially calling the change interface corresponding to the machine learning model on each service node according to the model change instruction also includes: When the model change instruction is a model deletion instruction, according to the model deletion instruction, the deletion interface corresponding to the machine learning model on each service node is called in sequence; Through the deletion interface, the corresponding model identifier is removed from the memory of the service node, and the deletion status is returned to end the change deployment of the machine learning model.
4. The method according to claim 1, characterized in that The step of opening and closing each service node through a node controller based on the increase or decrease in the number of nodes includes: When the number of nodes increased or decreased is the number of newly added nodes, the machine learning model is started on the new service node through the node controller, and the new service node is loaded onto the load balancer and the distributed coordination middleware; When the increase or decrease in the number of nodes is the number of nodes reduced, the service nodes for which the machine learning model has been started are stopped through the node controller, and the service nodes corresponding to the number of nodes reduced are deleted through the load balancer and the distributed coordination middleware.
5. A machine learning model deployment device, characterized in that: The device applies the machine learning model deployment method according to any one of claims 1 to 4, and the device includes: A node monitoring module is used to monitor each service node through a distributed coordination middleware, each service node is deployed with a machine learning model, the machine learning model is constructed based on the Spring Boot framework and the JPMML framework, and the prediction model markup language file of the machine learning model is stored in a distributed file storage system; A node calling module is used to call the change interface corresponding to the machine learning model on each service node in sequence according to the model change instruction when receiving the model change instruction; A file acquisition module, used to acquire an updated prediction model markup language file from the distributed file storage system through the change interface; The change deployment module is used to change the deployment of the machine learning model to each service node according to the updated prediction model markup language file.
6. A machine learning model deployment device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the machine learning model deployment method as described in any one of claims 1 to 4.
7. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the machine learning model deployment method as described in any one of claims 1 to 4 are implemented.
