Machine learning model development and deployment method and apparatus based on mlops platform

Abstract

A machine learning model development and deployment method and apparatus based on an MLOps platform, which method and apparatus relate to the technical field of computer software. The method comprises: creating, in a user interface or an API of an MLOps platform, a target experimental task for iteratively training a machine learning model, and performing target parameter configuration on the target experimental task, so as to obtain a configured target experimental task; executing the configured target experimental task in a Kubernetes cluster on the basis of training sample data, so as to obtain a machine learning model having optimal performance, and recording model parameters generated by the machine learning model during the execution of the configured target experimental task; deploying, on the basis of the deployment and management capabilities of Kubernetes, the machine learning model having optimal performance to a production environment, and by means of the API, enabling the machine learning model having optimal performance to provide external services; and on the basis of a continuous integration / continuous deployment pipeline, automatically deploying the latest trained machine learning model in the Kubernetes cluster to the production environment, so as to replace the machine learning model having optimal performance. The development and deployment efficiency is improved.

Description

Machine learning model development and deployment method and device based on MLOps platform

[0001] The present application claims priority to the Chinese patent application No. 202411029800.6, filed on July 30, 2024, and entitled "Machine learning model development and deployment method and device based on MLOps platform", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of computer software, and particularly relates to a machine learning model development and deployment method and device based on an MLOps platform. BACKGROUND

[0003] With the rapid development of artificial intelligence (AI) and machine learning (ML) technologies, more and more enterprises and research institutions have begun to use artificial intelligence and machine learning technologies to solve practical problems. However, the development and deployment process of a machine learning project is usually very complex, involving multiple links such as data preparation, model training, evaluation, deployment and monitoring. This results in a long cycle time and low efficiency from development to deployment of the machine learning project. SUMMARY

[0004] The present disclosure provides a machine learning model development and deployment method and device based on an MLOps platform, which is mainly aimed at solving the problem of long cycle time and low efficiency from development to deployment of a machine learning project in the related art.

[0005] According to a first aspect of the present disclosure, a machine learning model development and deployment method based on an MLOps platform is provided, which comprises:

[0006] collecting original sample data from a preset data source;

[0007] preprocessing the original sample data to obtain training sample data;

[0008] creating a target experiment task for iterative training of a machine learning model based on a user interface or an API interface of an MLOps platform, and configuring target parameters of the target experiment task to obtain a configured target experiment task;

[0009] executing the configured target experiment task based on the training sample data in a Kubernetes cluster to obtain a machine learning model with optimal performance and record model parameters generated by the machine learning model during execution of the configured target experiment task;

[0010] The Kubernetes-based deployment and management capability deploys the performance-optimized machine learning model into a production environment and enables the performance-optimized machine learning model to provide external services through an API interface.

[0011] The latest trained machine learning model in the Kubernetes cluster is automatically deployed to the production environment based on a continuous integration / continuous deployment pipeline to replace the performance-optimized machine learning model.

[0012] Optionally, the target parameter configuration for the target experiment task includes:

[0013] The hyperparameters, model architecture, and dataset division parameters of the target experiment task are configured to obtain the configured target experiment task, the target parameters include the hyperparameters, dataset division parameters, and model architecture, and the training sample data is divided into datasets according to the dataset division parameters.

[0014] Optionally, the method includes:

[0015] When the configured target experiment task is executed in the Kubernetes cluster based on the training sample data, an experiment based on the target parameters and the code version of the configured target experiment task is recorded to obtain an experimental result, and the target parameters and the code version are recorded.

[0016] The hyperparameters are automatically adjusted based on an automated hyperparameter tuning tool, and an intelligent algorithm is used to automatically explore and compare the configurations of multiple generations of model parameters generated by iteratively training the machine learning model, select the optimal combination of model parameters from the configurations of the multiple generations of model parameters, and determine the performance-optimized machine learning model based on the optimal combination of model parameters.

[0017] Optionally, the recording of the target parameters and the code version includes:

[0018] Different commit versions of the experimental code corresponding to the target experiment tasks are recorded and managed through Git to ensure that the code version corresponding to each target experiment task is traceable and reproducible.

[0019] The dataset division parameters are recorded using DVC technology.

[0020] The hyperparameters, model parameters, and model architecture of each machine learning model corresponding to different target experiment tasks are recorded.

[0021] Optionally, after the Kubernetes-based deployment and management capability deploys the performance-optimized machine learning model into a production environment, the method comprises:

[0022] Real-time monitoring of the running state and performance indicators of the MLOps platform based on customized monitoring indicators and alarm rules.

[0023] Optionally, the MLOps platform can be used to design and define a workflow of a complex task, including using a workflow orchestration tool to orchestrate a workflow of automatically executed data preprocessing, model training, evaluation, and deployment.

[0024] According to a second aspect of the present disclosure, an MLOps platform-based machine learning model development and deployment apparatus is provided, comprising:

[0025] A collection unit configured to collect original sample data from a preset data source;

[0026] A processing unit configured to preprocess the original sample data to obtain training sample data;

[0027] A creating unit configured to create, based on a user interface or an API interface of an MLOps platform, a target experimental task of iterative training of a machine learning model, and configure target parameters of the target experimental task to obtain a configured target experimental task;

[0028] An executing unit configured to execute the configured target experimental task based on the training sample data in a Kubernetes cluster to obtain a performance-optimized machine learning model and record model parameters generated by the machine learning model in the process of executing the configured target experimental task;

[0029] A first deployment unit configured to deploy, based on a Kubernetes-based deployment and management capability, the performance-optimized machine learning model into a production environment, and enable the performance-optimized machine learning model to provide external services through an API interface;

[0030] A second deployment unit configured to automatically deploy, based on a continuous integration / continuous deployment pipeline, a latest machine learning model trained in the Kubernetes cluster into the production environment to replace the performance-optimized machine learning model.

[0031] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0032] at least one processor; and a memory connected to the at least one processor in communication;

[0033] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

[0034] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of the first aspect.

[0035] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the first aspect.

[0036] The method and device for developing and deploying a machine learning model based on an MLOps platform provided by the present disclosure collect original sample data from a preset data source; the original sample data is preprocessed to obtain training sample data; a target experiment task for iterative training of a machine learning model is created based on a user interface or an API interface of an MLOps platform, and target parameter configuration is performed on the target experiment task to obtain a configured target experiment task; the configured target experiment task is executed in a Kubernetes cluster based on the training sample data to obtain a machine learning model with optimal performance and record model parameters generated by the machine learning model during execution of the configured target experiment task; the machine learning model with optimal performance is deployed into a production environment based on deployment and management capabilities of Kubernetes, and the machine learning model with optimal performance provides external services through an API interface; the latest machine learning model trained in the Kubernetes cluster is automatically deployed into the production environment based on a continuous integration / continuous deployment pipeline to replace the machine learning model with optimal performance. Compared with related technologies, the machine learning model is iteratively trained by executing the configured target experiment task in the Kubernetes cluster, which realizes full-process monitoring and management of the training process, and the training resources can be scheduled and the model parameters generated during the training process can be recorded. The latest machine learning model trained is automatically deployed to replace the machine learning model with optimal performance in the production environment through the continuous integration / continuous deployment channel, thereby reducing the cycle time of a machine learning project from development to deployment and improving the efficiency of development and deployment. BRIEF DESCRIPTION OF DRAWINGS

[0037] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated herein in conjunction with the description of the application. In the drawings:

[0038] FIG. 1 is a flowchart of a method for developing and deploying a machine learning model based on an MLOps platform according to an embodiment of the present disclosure;

[0039] FIG. 2 is a structural diagram of an apparatus for developing and deploying a machine learning model based on an MLOps platform according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] The present application will be described in detail below with reference to the drawings and in conjunction with embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0041] The following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise defined, all technical terms used in the present application have the same meanings as those generally understood by those skilled in the art. The terms used in the present application are only for the purpose of describing the specific embodiments of the present application and are not intended to limit the exemplary embodiments according to the present application.

[0042] In addition, the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein.

[0043] The method and apparatus for developing and deploying a machine learning model based on an MLOps platform according to an embodiment of the present disclosure will be described below with reference to the drawings.

[0044] In order to at least solve the problem that a large number of requests for one application in a low-code platform may cause other applications to be temporarily unavailable. The present embodiment provides a method for developing and deploying a machine learning model based on an MLOps platform.

[0045] FIG. 1 is a flowchart of a method for developing and deploying a machine learning model based on an MLOps platform according to an embodiment of the present disclosure. As shown in FIG. 1, the method comprises the following steps:

[0046] Step 101, collecting original sample data from a preset data source;

[0047] As a refinement of the above-mentioned step 101, in order to construct rich original sample data, the original data is automatically collected from diversified preset data sources (such as a database, an API interface (Application Programming Interface), a file system, etc.).

[0048] Step 102, preprocessing the original sample data to obtain training sample data;

[0049] As a refinement of step 102, cleaning and preprocessing operations are performed on the original sample data, including but not limited to deduplication, missing value filling, data format conversion, etc., to ensure data quality and consistency, and lay a solid foundation for subsequent analysis.

[0050] Step 103, creating a target experiment task for iterative training of a machine learning model based on a user interface or API interface of an MLOps platform, and configuring target parameters for the target experiment task to obtain a configured target experiment task;

[0051] Step 104, executing the configured target experiment task based on the training sample data in a Kubernetes cluster to obtain a performance-optimized machine learning model and record model parameters generated by the machine learning model during execution of the configured target experiment task;

[0052] Step 105, deploying the performance-optimized machine learning model to a production environment based on Kubernetes deployment and management capabilities, and enabling the performance-optimized machine learning model to provide external services through an API interface;

[0053] Step 106, automatically deploying the latest trained machine learning model in the Kubernetes cluster to the production environment based on a continuous integration / continuous deployment pipeline to replace the performance-optimized machine learning model.

[0054] The machine learning model development and deployment method based on the MLOps platform provided by the disclosure collects original sample data from a preset data source; the original sample data is preprocessed to obtain training sample data; a target experiment task for iterative training of a machine learning model is created based on a user interface or an API interface of an MLOps platform, and target parameter configuration is performed on the target experiment task to obtain a configured target experiment task; the configured target experiment task is executed in a Kubernetes cluster based on the training sample data to obtain a machine learning model with optimal performance and record model parameters generated by the machine learning model during execution of the configured target experiment task; the machine learning model with optimal performance is deployed into a production environment based on the deployment and management capability of Kubernetes, and the machine learning model with optimal performance provides external services through an API interface; the latest machine learning model trained in the Kubernetes cluster is automatically deployed into the production environment based on a continuous integration / continuous deployment pipeline to replace the machine learning model with optimal performance. Compared with related technologies, the machine learning model is iteratively trained by executing the configured target experiment task in the Kubernetes cluster, which realizes full-process monitoring and management of the training process, and the training resources can be scheduled and the model parameters generated during the training process can be recorded. The latest machine learning model trained is automatically deployed to replace the machine learning model with optimal performance in the production environment through the continuous integration / continuous deployment channel, thereby reducing the cycle time of a machine learning project from development to deployment and improving the efficiency of development and deployment.

[0055] It should be understood that the MLOps platform involved in the disclosure combines the methodology of machine learning (ML) and DevOps (Development Operations) practices, aiming to promote the efficient, reliable and continuous deployment and operation of machine learning models. The goal of MLOps is to seamlessly transition machine learning models from a laboratory environment to a production environment and ensure automation, standardization and manageability in all stages of the entire life cycle, from data preparation, model development, training, evaluation, deployment to monitoring and maintenance. MLOps not only accelerates the time to market of AI applications, but also improves the quality, maintainability and scalability of models, while reducing risks and costs, and is a key support for modern enterprises to implement and scale AI strategies. In order to solve these complexities, the MLOps (machine learning operation) platform is designed, which aims to improve the development efficiency and management level of machine learning projects through automated and standardized processes.

[0056] As a refinement of the embodiments of the present disclosure, when performing the target parameter configuration on the target experiment task in step 102 to obtain the configured target experiment task, the following implementation manners can be used, but are not limited to, for example: configuring hyperparameters, model architecture and dataset division parameters of the target experiment task to obtain the configured target experiment task, wherein the target parameters include the hyperparameters, dataset division parameters and the model architecture, and the training sample data is divided into datasets according to the dataset division parameters.

[0057] In some embodiments, when the configured target experiment task is executed based on the training sample data in the Kubernetes cluster, an experiment result obtained based on the target parameters and a code version of the configured target experiment task is recorded, and the target parameters and the code version are recorded; and the hyperparameters are automatically adjusted based on an automated hyperparameter tuning tool, and a plurality of generations of model parameters generated by iteratively training the machine learning model are automatically explored and compared by an intelligent algorithm, and a combination of optimal model parameters is selected from the plurality of generations of model parameter configurations to determine the performance-optimal machine learning model according to the combination of optimal model parameters. That is, the configured target experiment task is deployed and run in the Kubernetes cluster environment to realize dynamic scheduling and management of resources. At the same time, key information in the experiment process, such as parameter settings, code versions, dataset details and experiment results, is recorded in real time. An automated hyperparameter tuning tool is integrated, and different parameter configurations are automatically explored and compared by an intelligent algorithm to select an optimal model parameter combination to significantly improve the performance of the model.

[0058] In some embodiments, when the target parameters and the code version are recorded, the following implementation manners can be used, but are not limited to, for example: different commit versions of the experiment code corresponding to the target experiment task are recorded and managed by Git to ensure that the code version corresponding to each target experiment task is traceable and reproducible; the dataset division parameters are recorded by DVC technology; and the hyperparameters, model parameters and model architecture of each machine learning model corresponding to different target experiment tasks are recorded. That is, a Git version control system is used to strictly manage the versions of the experiment code to ensure that the code version of each experiment is traceable and reproducible. DVC (Data Version Control) technology is used to control the version of the dataset to record each step and change of data processing, thereby enhancing the transparency and traceability of data processing. Model registration and version control functions are provided to record the training parameters, performance indicators and key features of each model version in detail, thereby facilitating subsequent analysis and comparison.

[0059] In some embodiments, after the Kubernetes-based deployment and management capability deploys the performance-optimized machine learning model into a production environment, the following implementation manners can also be adopted, but are not limited to, for example: real-time monitoring of the running state and performance indicators of the MLOps platform based on custom monitoring indicators and alarm rules. To ensure the stability and reliability of the MLOps platform. Once an abnormal situation is found, an alarm mechanism is triggered immediately and corresponding processing measures are taken.

[0060] In some embodiments, the MLOps platform can be used to design and define the workflow of complex tasks, explicitly define the dependency relationship and scheduling strategy between tasks, including: using workflow orchestration tools to orchestrate the workflow of automatically executed data preprocessing, model training, evaluation and deployment, to reduce manual intervention and improve overall efficiency.

[0061] In order to facilitate the understanding of the above-mentioned embodiments, an exemplary illustration is provided, including: collecting raw data, i.e. the original sample data, from various preset data sources (such as databases, APIs, file systems, etc.). Cleaning and preprocessing the collected original sample data, including deduplication, missing value filling, data format conversion, etc. Create a target experiment task using the platform interface or API, configure experiment parameters (such as hyperparameters, model architecture, dataset, etc.) and track and manage the experiment task. Run the configured target experiment task in the Kubernetes cluster, perform resource scheduling and management, and record parameters, code versions, datasets and results in real time. Use the automated hyperparameter tuning tool integrated by the MLOps platform to compare and select the optimal parameter configuration and improve model performance. Record and manage the committed version of the experiment code using Git, ensuring that the corresponding code version of each experiment is traceable and reproducible. Use DVC to record and manage dataset versions, ensuring transparency and traceability of data processing. The platform provides model registration and version control functions to record the training parameters and performance indicators of each model version. Use the workflow orchestration tool provided by the MLOps platform to define the dependency relationship and scheduling strategy of complex tasks, create an automated workflow for data processing, model training, evaluation and deployment. Use the CI / CD pipeline to implement automated deployment of models and code, integrating Kubernetes deployment functions. Deploy the trained model to a production environment and provide an API interface for application programs to call. Set custom monitoring indicators and alarm rules to promptly identify and handle problems and ensure stable operation of the system.

[0062] In summary, the embodiments of the present disclosure can achieve the following effects:

[0063] 1. Through the execution of the target experiment task of the configuration in the Kubernetes cluster to iteratively train the machine learning model, the whole process of the training process is monitored and managed, the resources of the training can be scheduled, and the model parameters generated in the training process are recorded. Through the continuous integration / continuous deployment channel, the latest trained machine learning model is automatically deployed to replace the performance optimal machine learning model in the production environment, thereby reducing the cycle time of the machine learning project from development to deployment and improving the efficiency of development and deployment.

[0064] 2. The present disclosure realizes the whole process automation and efficient management of the machine learning model from data collection, preprocessing, experiment management, automatic optimization, version control to workflow orchestration and CI (Continuous Integration) / CD (Continuous Deployment) deployment, significantly improves the efficiency and performance optimization capability of model development, and enhances the traceability and reproducibility of the model.

[0065] 3. The present disclosure seamlessly combines experiment management and optimization process, significantly improves experiment efficiency and model performance, and provides a one-stop experiment management solution for users. Meanwhile, a complete version control chain is provided, covering all key elements of the machine learning project, realizing the all-round version control of code, model and data, and the code, data and model version of each experiment are recorded in detail, ensuring the repeatability and traceability of the experiment results.

[0066] 4. The present disclosure also provides an open ecosystem, allowing users to extend and customize platform functions as needed. A unified API interface is provided for application calling, realizing the full automation of the deployment process and ensuring efficient and reliable model publishing and service.

[0067] Corresponding to the above-mentioned machine learning model development and deployment method based on the MLOps platform, the present disclosure also provides a machine learning model development and deployment device based on the MLOps platform. Since the system embodiment of the present disclosure corresponds to the above-mentioned method embodiment, for the details not disclosed in the system embodiment, reference can be made to the above-mentioned method embodiment, which will not be described herein again.

[0068] Fig. 2 is a structural schematic diagram of a machine learning model development and deployment device based on the MLOps platform provided by an embodiment of the present disclosure, as shown in Fig. 2, which comprises:

[0069] The acquisition unit 21 is configured to acquire original sample data from a preset data source;

[0070] The processing unit 22 is configured to preprocess the original sample data to obtain training sample data;

[0071] The creating unit 23 is configured to create a target experiment task for iterative training of a machine learning model based on a user interface or an API interface of an MLOps platform, and configure target parameters of the target experiment task to obtain a configured target experiment task.

[0072] The executing unit 24 is configured to execute the configured target experiment task based on the training sample data in a Kubernetes cluster to obtain a performance-optimized machine learning model and record model parameters generated by the machine learning model in the process of executing the configured target experiment task.

[0073] The first deploying unit 25 is configured to deploy the performance-optimized machine learning model into a production environment based on Kubernetes deployment and management capabilities, and enable the performance-optimized machine learning model to provide external services through an API interface.

[0074] The second deploying unit 26 is configured to automatically deploy the latest trained machine learning model in the Kubernetes cluster into the production environment based on a continuous integration / continuous deployment pipeline to replace the performance-optimized machine learning model.

[0075] Corresponding to the above-mentioned MLOps platform-based machine learning model development and deployment method, the present disclosure provides an MLOps platform, and the MLOps platform-based machine learning model development and deployment method is applied to the MLOps platform. The MLOps platform comprises an experiment management module, a version control module, an ecosystem integration module, and a deployment and service module.

[0076] The experiment management module is configured to realize experiment management and hyperparameter tuning functions through a unified interface and interface. Users can create and manage experiments, record experiment parameters and results, perform hyperparameter tuning, and generate visual reports through the platform interface.

[0077] The version control module is configured to integrate Git and DVC, automatically record and manage versions of codes, data, and models, and ensure repeatability and traceability of experiment results. Users can view and roll back different versions, compare model performance, and manage project development history.

[0078] The ecosystem integration module is configured to integrate with data storage systems such as S3, GCS, and HDFS to provide a unified data reading and storage interface. The system is monitored by integrating Prometheus and Grafana, and users can set custom monitoring indicators and alarm rules and realize complex task scheduling and automatic management.

[0079] The deployment and service module is used for realizing automatic deployment of models and codes through a CI / CD pipeline, integrating a deployment function of Kubernetes, and supporting online and offline inference. A user can quickly deploy a trained model to a production environment and provide an API interface for application calling.

[0080] It should be noted that the foregoing explanations of the method embodiments are also applicable to the device of the present embodiment, and the principles are the same, which are not limited in the present embodiment.

[0081] The present disclosure provides an electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the machine learning model development and deployment method based on the MLOps platform as described in the foregoing embodiments.

[0082] The present disclosure provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the machine learning model development and deployment method based on the MLOps platform as described in the foregoing embodiments.

[0083] The present disclosure provides a computer program product comprising a computer program which, when executed by a processor, implements the machine learning model development and deployment method based on the MLOps platform as described in the foregoing embodiments.

[0084] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer usable program code.

[0085] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce the functions specified in one or more flows in the flowcharts and / or one or more blocks in the block diagrams.

[0086] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0087] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart or flowsheets and / or block or blocks of the block diagrams.

[0088] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, but not to limit it. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the technical solutions of the present application can still be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement should be covered in the protection scope of the claims of the present application.

Claims

1. A method for developing and deploying a machine learning model based on an MLOps platform, characterized in that, The method comprises: collecting original sample data from a preset data source; preprocessing the original sample data to obtain training sample data; creating a target experiment task for iterative training of a machine learning model based on a user interface or an API interface of an MLOps platform, and configuring target parameters of the target experiment task to obtain a configured target experiment task; executing the configured target experiment task based on the training sample data in a Kubernetes cluster to obtain a performance-optimal machine learning model and record model parameters generated by the machine learning model during execution of the configured target experiment task; deploying the performance-optimal machine learning model to a production environment based on Kubernetes deployment and management capabilities, and enabling the performance-optimal machine learning model to provide external services through an API interface; automatically deploying the latest trained machine learning model in the Kubernetes cluster to the production environment based on a continuous integration / continuous deployment pipeline to replace the performance-optimal machine learning model.

2. The method of claim 1, wherein, The method comprises: configuring hyperparameters, model architectures, and dataset division parameters of the target experiment task to obtain the configured target experiment task, wherein the target parameters include the hyperparameters, dataset division parameters, and model architectures, and the training sample data is divided into datasets according to the dataset division parameters.

3. The method of claim 2, wherein, The method comprises: when executing the configured target experiment task based on the training sample data in the Kubernetes cluster, recording experiment results obtained based on the target parameters and a code version of the configured target experiment task, and recording the target parameters and the code version; automatically adjusting the hyperparameters based on an automated hyperparameter tuning tool, and automatically exploring and comparing configurations of multiple generations of model parameters generated by iterative training of the machine learning model through intelligent algorithms, selecting an optimal combination of model parameters from the configurations of the multiple generations of model parameters, and determining the performance-optimal machine learning model based on the optimal combination of model parameters.

4. The method of claim 3, wherein, The method comprises: recording and managing different commit versions of experiment codes corresponding to different target experiment tasks through Git to ensure that the code versions corresponding to each target experiment task are traceable and reproducible; recording the dataset division parameters using DVC technology; recording the hyperparameters, model parameters, and model architectures of each machine learning model corresponding to different target experiment tasks.

5. The method of claim 4, wherein, After deploying the performance-optimal machine learning model to the production environment based on the Kubernetes deployment and management capabilities, the method comprises: monitoring the running state and performance indicators of the MLOps platform in real time based on custom monitoring indicators and alarm rules.

6. The method of claim 5, wherein, The MLOps platform can be used to design and define a workflow of a complex task, including orchestrating a workflow of data preprocessing, model training, evaluation and deployment for automated execution by using a workflow orchestration tool. 7.A device for developing and deploying a machine learning model based on an MLOps platform, characterized in that, The method comprises the following steps: The acquisition unit is configured to acquire original sample data from a preset data source. The processing unit is configured to preprocess the original sample data to obtain training sample data. The creation unit is configured to create a target experiment task for iterative training of a machine learning model based on a user interface or an API interface of the MLOps platform, and configure target parameters of the target experiment task to obtain a configured target experiment task. The execution unit is configured to execute the configured target experiment task based on the training sample data in a Kubernetes cluster to obtain a machine learning model with optimal performance and record model parameters generated by the machine learning model during execution of the configured target experiment task. The first deployment unit is configured to deploy the machine learning model with optimal performance to a production environment based on deployment and management capabilities of Kubernetes, and enable the machine learning model with optimal performance to provide external services through an API interface. The second deployment unit is configured to automatically deploy a latest machine learning model trained in the Kubernetes cluster to the production environment based on a continuous integration / continuous deployment pipeline to replace the machine learning model with optimal performance.

8. An electronic device, comprising: The method comprises the following steps: At least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-6.

10. A computer program product, characterised in that, The computer program, when executed by a processor, implements the method of any one of claims 1-6.

Images