Method for systematically managing and publishing machine learning model

Abstract

The invention discloses a method for systematically managing and publishing a machine learning model, which comprises the following steps of: constructing a resource service system in three modes of containerized resources, physical machine / virtual machine resources and big data cluster resources, and dynamically selecting configuration according to business requirements; registering a model file developed by multiple programming languages to a unified model warehouse, automatically generating a version identifier, and binding a mapping relationship between a feature field and a data set; model deployment is verified through a multi-role collaborative approval process, one-key automatic deployment is realized based on a containerization technology, and version information is recorded to support rapid rollback; issuing an online / offline prediction service interface, collecting service data in real time, and monitoring and evaluating model performance, service traffic and decision logic through a visual tool; according to the invention, a model unified management, rapid deployment and release and visual effect evaluation management system constructed by multiple types of programming languages is supported, and models generated under different technology stacks can be conveniently and rapidly maintained through the system.

Description

Technical Field

[0001] The present invention relates to the field of learning models, and specifically to a method for systematically managing and publishing machine learning models. Background Art

[0002] Driven by the wave of digital transformation, machine learning, as a core technology in the field of artificial intelligence, is deeply integrated into various industries such as finance, medical care, e-commerce, and manufacturing, creating huge value for enterprises. From financial institutions using machine learning models for accurate risk assessment and credit approval, to medical institutions using them to achieve early diagnosis and prediction of diseases; from e-commerce platforms relying on models to provide personalized product recommendations and improve user purchase conversion rates, to manufacturing industries using models to optimize production processes and predict equipment failures, machine learning has become a key driving force for enterprises to enhance their competitiveness and achieve innovative development.

[0003] With the rapid development of technology, machine learning algorithms are constantly being innovated, and new model architectures such as Transformer and GPT are constantly emerging, greatly improving the performance and application scope of the models. At the same time, computing hardware is also continuously upgrading, gradually evolving from traditional CPU computing to high-performance computing chips such as GPU and TPU, providing strong support for large-scale data processing and complex model training. However, behind the rapid development of this technology, there are serious model management challenges.

[0004] In actual enterprise applications, model development is often a dynamic and continuous process. With the expansion of business and changes in the market environment, the performance and functional requirements of the model are constantly increasing, and the model needs to be updated and iterated frequently. However, most companies currently lack systematic planning in model management. Models developed by different teams are stored in a decentralized manner, and version management is chaotic, making it difficult to clearly trace the model's change history and performance. For example, due to the lack of unified identification and standardized management, multiple versions of models may lead to the mistaken selection of low-performance versions during deployment, affecting business operation results.

[0005] In the model deployment phase, the complex technology stack and diverse operating environment make the migration process of the model from development to production difficult. Different models have different requirements for hardware resources and software dependencies. If there is a lack of effective resource management and environment adaptation mechanisms, deployment failures or unstable operations are very likely to occur. In addition, when the business volume suddenly surges, the existing model cannot quickly respond to changes in resource demand and achieve elastic expansion, resulting in reduced service performance and worse user experience.

[0006] The continuous optimization of the model also faces difficulties. Due to the lack of an effective mechanism for collecting, analyzing, and providing feedback on the large amount of service data generated during the model operation, it is difficult to accurately identify the performance bottlenecks and improvement directions of the model. This makes the model optimization work often rely on manual experience and trial-and-error, with low efficiency and poor results, and unable to fully unleash the potential of machine learning technology. In summary, establishing a systematic and efficient machine learning model management and release system has become an urgent need for enterprises to achieve sustainable development in the digital age. Summary of the Invention

[0007] In view of the above situation, to overcome the defects of the prior art, the present invention provides a method for systematically managing and releasing machine learning models, effectively solving the problems mentioned in the above background art.

[0008] To achieve the above object, the present invention provides the following technical solutions: The present invention includes the following steps:

[0009] Construct a resource service system in three modes: containerized resources, physical machine / virtual machine resources, and big data cluster resources, and dynamically select and configure according to business requirements;

[0010] Register model files developed in multiple programming languages into a unified model repository, automatically generate version identifiers, and bind the mapping relationship between feature fields and data sets;

[0011] Verify model deployment through a multi-role collaborative approval process, achieve one-click automated deployment based on containerization technology, and record version information to support rapid rollback;

[0012] Publish online / offline prediction service interfaces, collect service data in real time, and monitor and evaluate model performance, service traffic, and decision-making logic through visualization tools;

[0013] Based on the statistical analysis results of service data, generate interpretable feature iteration suggestions through built-in recommendation algorithms to support semi-supervised model optimization.

[0014] According to the above technical solution: The method for generating the version identifier includes:

[0015] Generate a basic version number according to the model file upload timestamp;

[0016] Perform a hash calculation on the model file content, generate a unique verification code, and combine it with the basic version number to form a complete version identifier.

[0017] According to the above technical solution: The multi-role collaborative approval process includes:

[0018] The modeling engineer submits a deployment request and associates the model version, resource requirements, and feature configuration;

[0019] System administrators review the compliance of resource allocation and the matching of the deployment environment;

[0020] Business experts validate the business suitability of the model output based on predefined evaluation metrics.

[0021] According to the above technical solution: the monitoring content provided by the visualization tool at least includes:

[0022] Dynamic statistical charts of real-time service request volume, response time and error rate;

[0023] Visual display of model decision logic, including decision tree path analysis and feature weight heat map;

[0024] The periodic service effect comparison report supports the analysis of model performance fluctuation trends by time dimension.

[0025] According to the above technical solution: the containerization technology is specifically based on Docker's image packaging and Kubernetes' distributed cluster orchestration to achieve elastic expansion and contraction and load balancing of the model.

[0026] A system for implementing a method for systematically managing and publishing machine learning models, comprising:

[0027] Resource management module: used to configure containerized resource pools, physical / virtual machine resource pools, and big data cluster resource pools, and supports dynamic switching;

[0028] Model warehouse module: provides an interface for uploading multi-language model files, and performs version storage and feature field binding functions;

[0029] Deployment engine module: realizes automatic model deployment based on container orchestration tools, and integrates version rollback and abnormal alarm mechanisms;

[0030] Monitoring and evaluation module: collects service log data in real time, generates a visual analysis interface, and provides feature iteration optimization suggestions;

[0031] Approval collaboration module: supports multi-role process-based approval, records approval opinions and executes operations in conjunction with the deployment engine.

[0032] According to the above technical solution: the model warehouse module also includes:

[0033] Model file format verification unit, used to detect the integrity and compatibility of uploaded files;

[0034] Feature mapping configuration unit supports explicit definition of feature order, weights, and data source association rules.

[0035] According to the above technical solution: the monitoring and evaluation module further includes:

[0036] Anomaly detection unit, triggering service status alarms based on preset thresholds;

[0037] Feature recommendation unit, generating interpretable feature optimization suggestions through clustering algorithms and association rule mining.

[0038] Beneficial effects: 1. Efficient management of model assets: Realize versioned centralized unified management of model assets, establish a model asset warehouse, systematically record and manage each version of the model, facilitating query and traceability. At the same time, through information visualization, the model-related information is clear at a glance, making it convenient for team members to understand the model status and historical changes.

[0039] 2. Quick response to business needs: The dynamic resource configuration and one-key automated deployment functions can quickly respond to changes in business needs, promptly put the optimized model online, and improve business processing efficiency and competitiveness. For example, before an e-commerce promotion event, the resource configuration can be quickly adjusted and a more accurate recommendation model can be deployed to increase the sales conversion rate.

[0040] 3. Ensure model quality and stability: The multi-role collaborative approval process verifies model deployment from different perspectives, ensuring the business applicability, reasonable resource allocation, and deployment environment matching of the model. At the same time, the version rollback mechanism and anomaly warning mechanism ensure the stability of the model during deployment and operation, reducing the risks brought by model failures.

[0041] 4. Continuously optimize model performance: Through real-time monitoring and statistical analysis based on service data, model performance problems can be detected in a timely manner, and interpretable feature iteration suggestions can be provided to support semi-supervised model optimization, continuously improving model performance and better adapting to the needs of business development. Description of the Drawings

[0042] The drawings are used to provide further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0043] Figure 1 is the overall process block diagram of the present invention;

[0044] Figure 2 is the flow chart of the present invention;

[0045] Figure 3 is the schematic diagram of the service application structure of the present invention;

[0046] Figure 4 is the schematic diagram of the model trend effect evaluation of the present invention. Detailed Embodiments

[0047] The following combines the attached Figures 1-4A further detailed description is given of the specific embodiments of the present invention.

[0048] Embodiment 1 is given by Figures 1-4 The present invention provides a method for systematically managing and publishing machine learning models, including the following steps:

[0049] (1) Example of resource service system configuration

[0050] Suppose an enterprise has a data analysis business. In the initial stage, the data volume is small and the demand for computing resources is relatively stable. At this time, the physical machine / virtual machine resource pool can be selected through the resource management module, and an appropriate amount of virtual machine resources can be configured to provide a basic environment for model development and testing. As the business develops, the data volume gradually increases, and the real-time requirement for data processing increases. The enterprise can use the dynamic switching function of the resource management module to adjust the resource mode to the containerized resource mode. Create a containerized environment in the Kubernetes cluster, and dynamically adjust the number of containers according to the business load to achieve efficient utilization and rapid deployment of resources.

[0051] (2) Model file registration and version management process

[0052] After the modeling engineer develops the model, the model file is uploaded to the unified model repository using the upload interface provided by the model repository module. The system automatically generates a basic version number according to the timestamp of the model file upload, such as "20230101100000" (indicating that it was uploaded at 10:00:00 on January 1, 2023). At the same time, a hash calculation is performed on the content of the model file to generate a unique verification code, such as "abcdef1234567890". The two are combined to form a complete version identifier "20230101100000-abcdef1234567890". The modeling engineer also needs to explicitly define the feature order, weight, and data source association rules in the feature mapping configuration unit to complete the binding of the feature field and dataset mapping relationship.

[0053] (3) Model deployment approval and automated deployment process

[0054] After the modeling engineer completes model development and registration, a deployment request is submitted to the approval collaboration module. The request is associated with the model version, required resources (such as CPU, memory, storage, etc.), and feature configurations. After receiving the request, the system administrator reviews the compliance of resource allocation to ensure that resource usage complies with enterprise regulations and checks the compatibility between the deployment environment (such as operating system, software dependencies, etc.) and the model. Based on predefined evaluation metrics such as prediction accuracy and recall rate, business experts verify the business applicability of the model output. After approval, the deployment engine module packages the model based on the Docker image and uses Kubernetes for distributed cluster orchestration to achieve one-click automated deployment. During the deployment process, the model version information is recorded so that it can be quickly rolled back to the previous stable version in case of problems.

[0055] (IV) Service Interface Release and Monitoring and Evaluation Operations

[0056] After the model is deployed, online / offline prediction service interfaces are released for business systems to call. The monitoring and evaluation module collects service log data in real time and generates a visual analysis interface. For example, real-time service request volume, response time, and error rate are displayed through dynamic statistical charts, enabling operation and maintenance personnel and business personnel to intuitively understand the service running status. The decision-making logic of the model is visually displayed through decision tree path analysis and feature weight heat maps to help modeling engineers deeply understand the model behavior. Service effect comparison reports are periodically generated to analyze the performance fluctuation trend of the model in terms of time dimension. For example, a report is generated weekly to observe the performance changes of the model in different time periods, providing data support for model optimization.

[0057] (V) Model Optimization Implementation Cases

[0058] Based on the statistical analysis results of service data, the monitoring and evaluation module generates interpretable feature iteration suggestions through built-in recommendation algorithms (such as clustering algorithms and association rule mining). Suppose in the monitoring of an e-commerce recommendation model, it is found that the co-purchase behavior between certain product categories is frequent, but these association features are not fully considered in the current model. The feature recommendation unit analyzes the data and suggests adding these association features to the model and adjusting the corresponding feature weights. The modeling engineer optimizes the model semi-supervised according to the suggestions, retrains the model and deploys it online. After a period of operation and observation, it is found that the recommendation accuracy and user click-through rate of the model have been significantly improved, proving the effectiveness of model optimization.

[0059] Beneficial effects: Efficient management of model assets: Realize versioned centralized unified management of model assets, establish a model asset warehouse, systematically record and manage each version of the model, facilitate query and traceability. At the same time, through information visualization, the model-related information is clear at a glance, making it easy for team members to understand the model status and historical changes.

[0060] Rapidly respond to business requirements: The dynamic resource allocation and one-key automated deployment functions can quickly respond to changes in business requirements, promptly launch the optimized model, and enhance business processing efficiency and competitiveness. For example, before an e-commerce promotion event, resources can be quickly adjusted and a more accurate recommendation model can be deployed to increase the sales conversion rate.

[0061] Ensure model quality and stability: The multi-role collaborative approval process verifies model deployment from different perspectives to ensure the business applicability, reasonable resource allocation, and deployment environment matching of the model. At the same time, the version rollback mechanism and exception warning mechanism ensure the stability of the model during deployment and operation, reducing the risks brought by model failures.

[0062] Continuously optimize model performance: Through real-time monitoring and statistical analysis based on service data, model performance issues can be promptly detected, and interpretable feature iteration suggestions can be provided to support semi-supervised model optimization, continuously improving model performance to better meet the needs of business development.

[0063] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for systematically managing and rapidly publishing machine learning models, characterized in that: The following steps are involved: Build a resource service system with three modes: containerized resources, physical / virtual machine resources, and big data cluster resources, and dynamically select configurations based on business needs; Register model files developed in multiple programming languages ​​to a unified model repository, automatically generate version identifiers, and bind feature fields and data set mappings; Verify model deployment through a multi-role collaborative approval process, implement one-click automated deployment based on containerization technology, and record version information to support rapid rollback; Release online / offline prediction service interfaces, collect service data in real time, and monitor and evaluate model performance, service traffic, and decision logic through visualization tools; Based on the statistical analysis results of service data, the built-in recommendation algorithm generates interpretable feature iterative suggestions and supports semi-supervised model optimization.

2. The method according to claim 1, characterized in that The method for generating the version identifier includes: Generate a basic version number based on the model file upload timestamp; Perform hash calculation on the model file content to generate a unique verification code and combine it with the base version number to form a complete version identifier.

3. The method according to claim 2, characterized in that The multi-role collaborative approval process includes: Modeling engineers submit deployment requests and associate model versions, resource requirements, and feature configurations; System administrators review the compliance of resource allocation and the matching of the deployment environment; Business experts validate the business suitability of the model output based on predefined evaluation metrics.

4. The method according to claim 3, characterized in that The monitoring content provided by the visualization tool at least includes: Dynamic statistical charts of real-time service request volume, response time and error rate; Visual display of model decision logic, including decision tree path analysis and feature weight heat map; The periodic service effect comparison report supports the analysis of model performance fluctuation trends by time dimension.

5. The method according to claim 4, characterized in that The containerization technology is specifically based on Docker image packaging and Kubernetes distributed cluster orchestration to achieve elastic expansion and contraction and load balancing of the model.

6. A system for implementing the method of claims 1-5, characterized in that: include: Resource management module: used to configure containerized resource pools, physical / virtual machine resource pools, and big data cluster resource pools, and supports dynamic switching; Model warehouse module: provides an interface for uploading multi-language model files, and performs version storage and feature field binding functions; Deployment engine module: realizes automatic model deployment based on container orchestration tools, and integrates version rollback and abnormal alarm mechanisms; Monitoring and evaluation module: collects service log data in real time, generates a visual analysis interface, and provides feature iteration optimization suggestions; Approval collaboration module: supports multi-role process-based approval, records approval opinions and executes operations in conjunction with the deployment engine.

7. The system according to claim 6, characterized in that The model warehouse module also includes: Model file format verification unit, used to detect the integrity and compatibility of uploaded files; Feature mapping configuration unit supports explicit definition of feature order, weights, and data source association rules.

8. The system according to claim 7, characterized in that The monitoring and evaluation module further includes: Anomaly detection unit, triggering service status alarm based on preset thresholds; The feature recommendation unit generates explainable feature optimization suggestions through clustering algorithms and association rule mining.

Images