Mlops extension method and apparatus in multi-cloud native environment, and medium

Abstract

An MLOps extension method and apparatus in a multi-cloud native environment, and a medium, relating to the technical field of multi-cloud heterogeneity. The method comprises: receiving a user-defined infrastructure template, configuration parameters, and access credentials of different cloud platforms, and collecting cloud computing resources from a plurality of clouds; putting model code and a data processing script into a version control system, and uploading data to a data storage service; pre-allocating computing resources and storage resources, and installing and verifying all dependencies in a pre-deployment environment; creating isolated sandbox environments on the cloud platforms to run comprehensive simulation tests; packaging models and applications, and deploying same in production environments on the cloud platforms; configuring a monitoring tool to monitor the performance of the models and the applications in real time, and on the basis of monitoring data, optimizing model parameters and application configurations. The present invention can reduce incompatibility and failures in a multi-cloud environment, and improve the reliability and efficiency of model services and continuous application deployment.

Description

MLOps extension method, device and medium in multi-cloud native environment

[0001] The present application claims priority to the Chinese patent application No. 202410846840.3, filed on June 27, 2024, and entitled "MLOps extension method, device and medium in multi-cloud native environment", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to an MLOps extension method, device and medium in multi-cloud native environment, belonging to the technical field of multi-cloud heterogeneous technology. BACKGROUND

[0003] In recent years, with the rapid development of cloud computing and artificial intelligence technology, enterprises' demand for computing resources is increasing, especially in the aspects of artificial intelligence model training and service deployment. To meet this demand, major cloud service providers have launched a variety of cloud service resources, from computing, storage to network services, without exception. However, due to differences in underlying implementation principles, technology stacks and architectures among different cloud service providers, the behavior and performance of applications on different cloud platforms may not be consistent.

[0004] Under this background, how to effectively manage, deploy and maintain machine learning model services across multiple cloud platforms has become a problem to be solved. Although the traditional machine learning operation (MLOps) method combines the practice and principles of machine learning and operation, and improves the efficiency and reliability of machine learning projects through automation, collaboration and continuous improvement, it still faces serious challenges in the face of multi-cloud heterogeneous environment.

[0005] Firstly, the differences in multi-cloud computing resources make the deployment of model services and applications complex. Different cloud providers may use different technology stacks and architectures, resulting in differences in performance and performance of model services on different cloud platforms. In addition, different cloud platforms have different support for Kubernetes (K8s), and the differences in K8s version and compatibility further increase the difficulty of deploying model services in multi-cloud environment.

[0006] Secondly, the continuous deployment of model services and applications needs to solve the problems of multi-cloud computing resource differences and K8s version compatibility. This requires developers and operation personnel to deeply understand the characteristics of each cloud platform to ensure the stability and reliability of model services in multi-cloud heterogeneous environment. However, this process is often time-consuming and labor-intensive, and prone to errors. SUMMARY

[0007] The application aims to provide a MLOps extension method, device and medium in a multi-cloud native environment, enhance the model service and application deployment capability of the MLOps platform in a multi-cloud heterogeneous environment, and enhance the application availability and flexibility through a virtualization environment.

[0008] To achieve the above-mentioned purpose, the application is implemented through the following technical solutions:

[0009] Receiving user-defined infrastructure templates, configuration parameters and access credentials of different cloud platforms, collecting cloud computing resources of multiple clouds;

[0010] Putting the model code and data processing script into a version control system, and uploading the data to a data storage service;

[0011] Pre-allocating computing resources and storage resources, and installing and verifying all dependencies in the pre-deployment environment;

[0012] Creating an isolated sandbox environment on each cloud platform and running comprehensive simulation tests;

[0013] Packaging the model and application, and deploying them to the production environment of each cloud platform using the CI / CD pipeline;

[0014] Configuring monitoring tools, setting up a log collection and analysis system, and monitoring the performance of the model and application in real time, and optimizing the model parameters and application configuration according to the monitoring data.

[0015] Preferably, the version control system is Git.

[0016] Preferably, the CI / CD pipeline is configured with an automatic rollback strategy and a failure handling mechanism.

[0017] Preferably, the optimized model parameters and application configuration are iteratively deployed through the CI / CD pipeline.

[0018] Preferably, the information of each computing node in the multi-cloud computing resource environment is collected, and the collected cloud computing resources of multiple clouds are abstracted and unified through a multi-cloud adapter to establish a unified virtualized computing resource pool; the computing node information includes CPU model, kernel version, operating system version, runtime version, storage type and capacity information.

[0019] Preferably, the data storage service is an external distributed storage system, and a storage virtualization is configured to create a data storage persistent volume to provide a unified storage access interface.

[0020] Preferably, the computing resources and storage resources are dynamically pre-allocated according to the workload and demand; the simulation test includes unit test, integration test, performance test and security test.

[0021] Preferably, the cloud and local resources are defined in readable configuration files, and the files are version controlled and shared.

[0022] The application has the advantages that the application supports the deployment of artificial intelligence model services and applications in a multi-cloud heterogeneous environment, introduces pre-deployment environment configuration in the pipeline deployment stage, virtually manages the basic infrastructure of different cloud providers through a multi-cloud adapter, and performs adaptive environment adaptation through the virtual underlying cloud environment basic resources, application running environment, and cloud service protocol. The pre-deployment arrangement is introduced in the MLOps pipeline, and the pre-deployment stage allows the infrastructure pre-configuration, environment consistency check, cross-cloud communication configuration, and test environment simulation of the deployment environment to be completed before the actual model service release and application deployment. The incompatibility and failure in the multi-cloud environment are reduced, and the reliability and efficiency of the continuous deployment of model services and applications are improved. The model service and application deployment capability of the MLOps platform in the multi-cloud heterogeneous environment is enhanced, and the application availability and flexibility are enhanced through the virtualization environment. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, which together with the embodiments of the application, serve to explain the application, and do not constitute a limitation on the application.

[0024] FIG. 1 is a flowchart of the method of the application. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the application, but not all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0026] As shown in FIG. 1, a MLOps extension method in a multi-cloud native environment is implemented through the following technical solutions:

[0027] S1: receiving user-defined infrastructure templates, configuration parameters, and access credentials of different cloud platforms, and collecting cloud computing resources of multi-cloud.

[0028] S2: placing model code and data processing scripts into a version control system, and uploading data to a data storage service;

[0029] S3: pre-allocating computing resources and storage resources, and installing and verifying all dependencies in the pre-deployment environment.

[0030] S4: Create isolated sandbox environments on each cloud platform and run comprehensive simulation tests.

[0031] S5: Package the model and application and deploy them to the production environment of each cloud platform using the CI / CD pipeline.

[0032] S6: Configure monitoring tools, set up a log collection and analysis system, and monitor the performance of the model and application in real time. Optimize model parameters and application configurations based on monitoring data.

[0033] As a refinement of the above embodiment, in step S1, the information of each computing node in the multi-cloud computing resource environment is collected, and the collected cloud computing resources of the multi-cloud are abstracted and unified through a multi-cloud adapter to establish a unified virtualized computing resource pool. The computing node information includes CPU model, kernel version, operating system version, runtime version, storage type and capacity information.

[0034] As a refinement of the above embodiment, in step S2, the version control system is Git, which ensures version control and centralized management of code and data. The data storage service is an external distributed storage system, which configures storage virtualization to create data storage persistent volumes to provide a unified storage access interface.

[0035] As a refinement of the above embodiment, in step S3, computing resources and storage resources are dynamically pre-allocated according to workload and demand, and configuration management tools are used to install and verify all dependencies in the pre-deployment environment to ensure consistency of environment configuration and sufficiency of resources. Reduce incompatibility and failure in multi-cloud environment, improve the reliability of deployment.

[0036] As a refinement of the above embodiment, in step S4, the simulation test includes unit test, integration test, performance test and security test. Verify the performance of the model and application in the pre-deployment environment. Ensure the stability and performance of the model and application on different cloud platforms.

[0037] As a refinement of the above embodiment, in step S5, the CI / CD pipeline is used to deploy the model and application to the production environment of each cloud platform, realizing one-key deployment of the model and application, ensuring fast and reliable production environment online. And configure the automatic rollback strategy in the CI / CD pipeline to ensure that it can be rolled back quickly when problems are found; Set up an automatic alarm and fault handling mechanism to ensure that problems can be discovered and solved in a timely manner. Improve the speed and efficiency of fault handling to ensure the high availability of the system.

[0038] As a refinement of the above embodiment, in step S6, the optimized model and application are iteratively deployed through the CI / CD pipeline. Continuously improve the performance of the model and application to ensure that the system can continuously meet business needs.

[0039] In this embodiment, to ensure the above characteristics, infrastructure tools are introduced to safely and efficiently build, change, virtualize multi-cloud computing resource environments by defining cloud and local resources in readable configuration files and version controlling and sharing these files. Pre-deployment and testing of the deployment process before actual application deployment is allowed, which helps to reduce problems that may occur in multi-cloud heterogeneous environments and improve the success rate of model service and application continuous deployment.

[0040] It should be noted that the multi-cloud adapter is used to abstract and unify the cloud computing resources provided by different cloud vendors. The detailed information of each computing node in the API, SDK and computing resource environment provided by the target cloud vendor, including CPU model, kernel version, operating system version, runtime version, storage type and capacity, etc., is collected, including CPU model, kernel version, operating system version, runtime version, storage type and capacity, etc. The kernel version of each computing node is determined, and different kernel versions may support different functions and performance optimization. This adapter will serve as an intermediate layer, shielding the differences between different cloud platforms, standardizing the computing, storage, network and other resources of different cloud platforms, and providing a unified interface, so that users can more conveniently manage and use cross-cloud resources without worrying about the specific implementation details of each cloud platform. The specific subclasses of the adapter are implemented, and each subclass corresponds to a cloud platform, such as AWS, Azure, etc. The methods in the interface are implemented. In the subclass, the SDK or API of the cloud platform is used to interact with the specific cloud resources.

[0041] The adapter is used to obtain resource information on different cloud platforms and aggregate it into a unified data structure. According to the aggregated resource information, a unified virtualized computing resource pool is established, and the resources on different cloud platforms are abstracted as unified computing units. The cloud computing resource environment of the cloud vendor is virtually clouded using KVM virtual machines, Docker containers or other lightweight virtualization technologies, and a virtual network (VPC) is created to isolate the virtualization environment and provide secure network communication. Virtual machine instances or container instances are created and configured according to demand, and resource management and scheduling are automated using policies and rules, resources are dynamically allocated and adjusted according to workloads and demand, including allocation of CPU, memory and storage resources, selection of corresponding kernel version, operating system version, runtime version.

[0042] The embodiment of the present disclosure further provides a device for MLOps expansion in a multi-cloud native environment, comprising a processor and a memory. Optionally, the device can further comprise a communication interface and a bus. The processor, the communication interface and the memory can communicate with each other through the bus. The communication interface can be used for information transmission. The processor can invoke the logic instructions in the memory to execute the MLOps expansion method in the multi-cloud native environment of the above-mentioned embodiments.

[0043] In addition, the logic instructions in the memory described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium.

[0044] The memory, as a computer readable storage medium, can be used to store software programs, computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes the function application and data processing by running the program instructions / modules stored in the memory, that is, implements the MLOps expansion method in the multi-cloud native environment in the above-mentioned embodiments.

[0045] The memory can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the terminal device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory.

[0046] The embodiment of the present disclosure provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are set to execute the MLOps expansion method in the multi-cloud native environment.

[0047] The computer readable storage medium described above can be a transitory computer readable storage medium or a non-transitory computer readable storage medium.

[0048] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium can be a non-transitory storage medium, including a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc. various media that can store program codes, or a transitory storage medium.

[0049] It should be pointed out finally that the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for MLOps extension in a multi-cloud native environment, characterized in that, The method comprises the following steps: Receiving user-defined infrastructure templates, configuration parameters, and access credentials for different cloud platforms to collect multi-cloud computing resources; Putting model code and data processing scripts into a version control system and uploading data to a data storage service; Pre-allocating computing resources and storage resources and installing and verifying all dependencies in a pre-deployment environment; Creating isolated sandbox environments on each cloud platform and running comprehensive simulation tests; Packaging the model and application and deploying them to production environments on each cloud platform using a CI / CD pipeline; Configuring monitoring tools, setting up a log collection and analysis system, and monitoring the performance of the model and application in real time to optimize model parameters and application configurations based on monitoring data.

2. The MLOps extension method in a multi-cloud native environment according to claim 1, characterized in that, The version control system is Git.

3. The MLOps extension method in a multi-cloud native environment according to claim 1, characterized in that, The CI / CD pipeline includes an automated rollback strategy and a failure handling mechanism.

4. The MLOps extension method in a multi-cloud native environment according to claim 3, characterized in that, Iteratively deploying optimized model parameters and application configurations through the CI / CD pipeline.

5. The MLOps extension method in the multi-cloud native environment according to claim 1, characterized in that, Collecting information about each computing node in a multi-cloud computing resource environment, abstracting and unifying the collected multi-cloud computing resources through a multi-cloud adapter, and establishing a unified virtualized computing resource pool; the computing node information includes CPU model, kernel version, operating system version, runtime version, storage type, and capacity information.

6. The MLOps extension method in a multi-cloud native environment according to claim 1, characterized in that, The data storage service is an external distributed storage system that configures storage virtualization to create data storage persistent volumes to provide a unified storage access interface.

7. The MLOps extension method in a multi-cloud native environment according to claim 1, characterized in that, Dynamically pre-allocate computing resources and storage resources based on workload and demand; the simulation test includes unit testing, integration testing, performance testing, and security testing.

8. The MLOps extension method in a multi-cloud native environment according to claim 1, characterized in that, Define cloud and local resources in readable configuration files and version control and share these files.

9. An MLOps extension device in a multi-cloud native environment, comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to execute the MLOps extension method in the multi-cloud native environment as claimed in any one of claims 1-8 when running the program instructions.

10. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which is executed by a processor to implement the method as claimed in any one of claims 1-8.

Images