Method and device for storing, transmitting and loading cloud native machine learning model file

Abstract

The invention relates to a cloud native machine learning model file storage, transmission and loading method and device, and belongs to the technical field of model file storage, transmission and loading, and the method comprises the steps: making a model weight mirror image and a model application mirror image; pushing the model weight mirror image and the model application mirror image to a central Harbor mirror image warehouse; the central Harbor mirror image warehouse distributes the model weight mirror images and the model application mirror images to Harbor mirror image warehouses across the country; deploying a model service, and specifying a model weight mirror image required by a model application program; downloading a model weight mirror image and a model application mirror image; mounting the model weight mirror image and the model application mirror image to a specified directory of a cloud host; constructing a virtual file system on the basis of taking a specified directory as a data source; and mounting the virtual file system on the model application program, loading the model weight file and the environment and configuration required by the operation of the model application, and starting the model application service. The operation and maintenance difficulty is reduced, and the loading speed is increased.

Description

Technical Field

[0001] The present invention belongs to the technical field of storage, transmission and loading of model files, and in particular to a method and device for storing, transmitting and loading cloud native machine learning model files. Background Art

[0002] With the rise and development of large models, the weight files of large models are getting larger and larger; at the same time, the deployment scope of large models is getting wider and wider; currently, the weight files of large models are mainly stored in object storage based on the S3 communication protocol, and the weight files are downloaded from the object storage when the model is loaded. This method has the following problems: the object storage system does not have a complete version management capability; although object storage supports version management of a single file, in the model deployment scenario, a model uses multiple files, and the version should be based on the model, and different versions of the model can reuse files with the same content without having to store two copies; but these requirements are not available in the object storage itself. Private object storage has an independent set of authorization and authentication processes. Model application deployment requires the addition of object storage authentication keys, which is troublesome to maintain and cumbersome to configure. The technical standards for cross-computer room synchronization of object storage systems are different, and automatic scheduling requires DNS capabilities. The overall architecture is too heavy and the operation and maintenance difficulty is high. In the scenario of multiple copies of AI models deployed, even if other machines in the same computer room have downloaded data, if this machine does not have data, it will still download from the public network, resulting in a high center load.

[0003] In the cloud-native scenario, the model application is packaged into an image. There are currently three solutions for the weight file that needs to be loaded during runtime: 1) Download it locally before starting. If it is scheduled to other machines, it needs to be downloaded again. 2) When the model application container starts, a cloud disk is mounted. Before the application starts, check whether the weight file exists in the cloud disk. If it does not exist, download it from the object storage. If it exists, load the file directly. The advantage of this method is that the weight file does not need to be downloaded again after the container is scheduled to other machines. The disadvantage is that the speed of loading files from the cloud disk to the memory is slower than that of loading files from the local disk to the memory. 3) Package the model weight file and the application into an image. The advantage is that it is easy to deploy and does not need to rely on object storage to save weight files. The weight file can be loaded directly when the application starts. The disadvantage is that the image storage occupies a large area, downloading the image is slow, and due to the layered storage characteristics of the image, updating the weight file or application is not flexible enough, which easily leads to duplicate storage of some files. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the invention is to provide a method and device for storing, transmitting and loading cloud-native machine learning model files, which not only solves the problem of multi-file and multi-version management, but also reduces the difficulty of operation and maintenance and improves the loading speed.

[0005] The first aspect of the present invention provides a method for storing, transmitting and loading a cloud native machine learning model file, including:

[0006] Package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image;

[0007] Push model weight images and model application images to the central Harbor image repository;

[0008] The central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country;

[0009] Deploy the model service, specifying the model weight image required by the model application;

[0010] Alluxio Worker calls Alluxio UFS Plugin, which calls the container runtime interface to download the model weight image and model application image.

[0011] Alluxio UFS Plugin mounts the model weight image and model application image to the specified directory of the cloud host;

[0012] Alluxio Runtime builds a virtual file system based on the specified directory as the data source;

[0013] The model application mounts the virtual file system, loads the model weight file and the environment and configuration required for the model application to run, and starts the model application service.

[0014] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission and loading method, the model weight file is packaged into a model weight image, and the environment and configuration required for the model application to run are packaged into a model application image, including:

[0015] Package the model weight file into a model weight image according to the container image specification, and package the environment and configuration required for the model application to run into a model application image according to the container image specification;

[0016] Among them, each model file is saved as a layer of the image.

[0017] Furthermore, in the above-mentioned method for storing, transmitting and loading cloud-native machine learning model files, the central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country, including:

[0018] The central Harbor image warehouse distributes the model weight image and the model application image to Harbor image warehouses across the country based on the distribution technology of the Harbor image warehouse and the distribution strategy.

[0019] Furthermore, in the above-mentioned method for storing, transmitting and loading cloud-native machine learning model files, deploying a model service includes:

[0020] After receiving the user's operation to deploy the model service on the k8s in the computer room, the model service is deployed on the k8s in the computer room.

[0021] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, before the AlluxioUFS Plugin calls the container runtime interface to download the model weight image and the model application image, it also includes:

[0022] The cloud host's container runtime configures mirror and configures the addresses of Harbor image repositories across the country in order of distance.

[0023] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, AlluxioUFS Plugin calls the container runtime interface to download the model weight image and model application image, including:

[0024] The container runtime downloads the model weight image and model application image from the nearest Harbor image repository.

[0025] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, the AlluxioUFS Plugin mounts the model weight image and the model application image to the specified directory of the cloud host, including:

[0026] The Alluxio UFS Plugin uses overlayfs technology to mount the model weight image and model application image to the specified directory of the cloud host.

[0027] The second aspect of the present invention further proposes a storage, transmission and loading device for a cloud-native machine learning model file, including:

[0028] Packaging module: used to package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image;

[0029] Push module: used to push model weight images and model application images to the central Harbor image repository;

[0030] Distribution module: used by the central Harbor image warehouse to distribute model weight images and model application images to Harbor image warehouses across the country;

[0031] Deployment module: used to deploy model services and specify the model weight image required by the model application;

[0032] Calling module: used by Alluxio Worker to call Alluxio UFS Plugin. Alluxio UFS Plugin calls the container runtime interface to download the model weight image and model application image.

[0033] Mounting module: used by Alluxio UFS Plugin to mount the model weight image and model application image to the specified directory of the cloud host;

[0034] Build module: used by Alluxio Runtime to build a virtual file system based on the specified directory as the data source;

[0035] Loading module: used for model application to mount virtual file system, load model weight file and the environment and configuration required for model application to run, and start model application service.

[0036] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory;

[0037] The processor calls the program or instructions stored in the memory to execute any of the above methods for storing, transmitting, and loading cloud-native machine learning model files.

[0038] In a fourth aspect of the present invention, a computer-readable storage medium is further proposed, which stores a program or instruction, and the program or instruction enables a computer to execute any of the above methods for storing, transmitting and loading cloud-native machine learning model files.

[0039] The beneficial effects of the present invention are as follows: the present invention packages the model weight file into a model weight image, and packages the environment and configuration required for the model application to run into a model application image; pushes the model weight image and the model application image to the central Harbor image warehouse; the central Harbor image warehouse distributes the model weight image and the model application image to Harbor image warehouses across the country; deploys the model service, specifies the model weight image required by the model application; Alluxio Worker calls Alluxio UFS Plugin, and Alluxio UFS Plugin calls the interface of the container runtime to download the model weight image and the model application image; Alluxio UFS Plugin mounts the model weight image and the model application image to the specified directory of the cloud host; Alluxio Runtime builds a virtual file system based on the specified directory as the data source; the model application mounts the virtual file system, loads the model weight file and the environment and configuration required for the model application to run, and starts the model application service. The present invention not only solves the problem of multi-file and multi-version management, but also reduces the difficulty of operation and maintenance and improves the loading speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings are only used to illustrate specific embodiments and are not considered to limit the present invention. In the entire drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0041] Figure 1 A diagram of a method for storing, transmitting, and loading a cloud-native machine learning model file provided in an embodiment of the present invention;

[0042] Figure 2 A diagram of a storage, transmission and loading device for a cloud-native machine learning model file provided in an embodiment of the present invention;

[0043] Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present invention.

[0045] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0046] In the description of the present invention, the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0047] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0048] The present invention proposes a method, device, electronic device and storage medium for storing, transmitting and loading cloud-native machine learning model files, which not only solves the problem of multi-file and multi-version management, but also reduces the difficulty of operation and maintenance and improves the loading speed.

[0049] Before introducing the embodiments of the present invention, professional terms involved in the present invention are first introduced.

[0050] Model weights: the weight file of the artificial neural network.

[0051] Large model (LLM): a language model with a large number of parameters.

[0052] Kubernetes: A container orchestration management system.

[0053] Alluxio: A cache-based high-speed read and write file system.

[0054] Container image: stores all the files required for the container to run.

[0055] Image layer: Container images are stored in layers. Each layer of files is called a layer, and the sha256 summary is used to distinguish the contents of the layer files.

[0056] Overlayfs: A joint mount file system provided by the Linux kernel. The joint mount technology can mount multiple file systems into one directory.

[0057] Object Storage Service (OSS): A cloud service infrastructure that is typically used to store unstructured data.

[0058] S3: A communication protocol for object storage systems. It is the most widely used object storage communication protocol.

[0059] Container runtime: A tool for managing containers and container images at the bottom of k8s. It supports mirror image capabilities and can download image repositories from the mirror repository list by selecting the first repository that contains the target image.

[0060] Method Embodiment

[0061] Figure 1 A diagram of a method for storing, transmitting, and loading cloud-native machine learning model files provided in an embodiment of the present invention.

[0062] In the first aspect of the present invention, a method for storing, transmitting and loading cloud native machine learning model files is proposed, combined with Figure 1 , including eight steps from S1 to S8:

[0063] S1: Package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image.

[0064] Specifically, in an embodiment of the present invention, the model weight file is packaged into a model weight image according to the container image specification, and the environment and configuration required for the model application to run are packaged into a model application image according to the container image specification, and each model file is saved as a layer of the image.

[0065] S2: Push the model weight image and model application image to the central Harbor image repository.

[0066] Specifically, in the embodiment of the present invention, the model weight image and the model application image are pushed to the central Harbor image warehouse for storage.

[0067] S3: The central Harbor image repository distributes the model weight image and model application image to Harbor image repositories across the country.

[0068] Specifically, in an embodiment of the present invention, after the central Harbor image warehouse stores the model weight image and the model application image, the weight image and the model application image are distributed to Harbor image warehouses across the country. The Harbor image warehouses across the country are distributed in computer rooms across the country.

[0069] S4: Deploy the model service and specify the model weight image required by the model application.

[0070] Specifically, in an embodiment of the present invention, an operation of deploying a model service on the k8s in the computer room is received, and the model service is deployed on the k8s in the computer room. An operation of specifying a model weight image required by the model application on the k8s in the computer room is received, and the model weight image required by the model application is specified.

[0071] S5: Alluxio Worker calls Alluxio UFS Plugin, which calls the container runtime interface to download the model weight image and model application image.

[0072] Specifically, in an embodiment of the present invention, the cloud host's container runtime configures a mirror, configures the addresses of Harbor image repositories across the country in order of distance, and uses the container runtime mirror function to download the model weight image and model application image from the nearest Harbor image repository.

[0073] S6: Alluxio UFS Plugin mounts the model weight image and model application image to the specified directory of the cloud host.

[0074] Specifically, in an embodiment of the present invention, after downloading the model weight image and the model application image, Alluxio UFSPlugin uses overlayfs technology to mount the downloaded model weight image and the model application image to the specified directory of the cloud host. When Alluxio UFS Plugin is connected to the container runtime, Alluxio is used as a cloud native cache layer to provide model file cache for model applications, and automatically synchronize files from other machines in the computer room that have downloaded model files without accessing the public network again, thereby improving loading speed and reducing central bandwidth pressure and load.

[0075] S7: Alluxio Runtime builds a virtual file system based on the specified directory as the data source.

[0076] Specifically, in an embodiment of the present invention, after mounting the downloaded model weight image and model application image to a specified directory of the cloud host, Alluxio Runtime builds a virtual file system based on the specified directory as a data source.

[0077] S8: The model application mounts the virtual file system, loads the model weight file and the environment and configuration required for the model application to run, and starts the model application service.

[0078] It should be understood that the present invention uses the same image repository authentication mechanism as the application image, and no additional authentication configuration is required.

[0079] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission and loading method, the model weight file is packaged into a model weight image, and the environment and configuration required for the model application to run are packaged into a model application image, including:

[0080] Package the model weight file into a model weight image according to the container image specification, and package the environment and configuration required for the model application to run into a model application image according to the container image specification;

[0081] Among them, each model file is saved as a layer of the image.

[0082] Specifically, in the embodiment of the present invention, each model file is saved as a layer of the image, so that multiple versions of the same file can reuse the same layer, and the model weight file version corresponds to the image version one by one, solving the problem of multi-file and multi-version management.

[0083] Furthermore, in the above-mentioned method for storing, transmitting and loading cloud-native machine learning model files, the central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country, including:

[0084] The central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country based on the distribution technology of the Harbor image warehouse and the distribution strategy.

[0085] Specifically, in the embodiment of the present invention, the central Harbor image warehouse distributes the model weight files to the Harbor image warehouses in computer rooms across the country based on the mature distribution technology of the central Harbor image warehouse, avoiding the heavy and complex deployment architecture of object storage and reducing the difficulty of operation and maintenance.

[0086] Furthermore, in the above-mentioned method for storing, transmitting and loading cloud-native machine learning model files, deploying a model service includes:

[0087] After receiving the user's operation to deploy the model service on the k8s in the computer room, the model service is deployed on the k8s in the computer room.

[0088] Specifically, in the embodiment of the present invention, the computer room may be any computer room across the country, such as a computer room in Hebei Province or a computer room in Tianjin City.

[0089] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, before the AlluxioUFS Plugin calls the container runtime interface to download the model weight image and the model application image, it also includes:

[0090] The cloud host's container runtime configures mirror and configures the addresses of Harbor image repositories across the country in order of distance.

[0091] Specifically, in the embodiment of the present invention, configuring the addresses of mirror and Harbor image repositories across the country lays the foundation for the subsequent downloading of model weight images and model application images nearby.

[0092] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, AlluxioUFS Plugin calls the container runtime interface to download the model weight image and model application image, including:

[0093] The container runtime downloads the model weight image and model application image from the nearest Harbor image repository.

[0094] Specifically, in the embodiment of the present invention, the mirror function of the container runtime is used to automatically download the model weight file image nearby without the need for external DNS capabilities.

[0095] Furthermore, in the above-mentioned cloud-native machine learning model file storage, transmission, and loading method, the AlluxioUFS Plugin mounts the model weight image and the model application image to the specified directory of the cloud host, including:

[0096] Alluxio UFS Plugin uses overlayfs technology to mount the model weight image and model application image to the specified directory of the cloud host.

[0097] Specifically, in the embodiment of the present invention, the overlayfs technology is a joint mount file system provided by the Linux kernel. The joint mount technology can realize mounting multiple file systems into one directory.

[0098] Device Embodiment

[0099] Figure 2 A diagram of a storage, transmission, and loading device for a cloud-native machine learning model file provided in an embodiment of the present invention.

[0100] In a second aspect of the present invention, a storage, transmission and loading device for a cloud native machine learning model file is also proposed. Figure 2 ,include:

[0101] Packaging module 21: used to package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image.

[0102] Specifically, in an embodiment of the present invention, the packaging module 21 packages the model weight file into a model weight image according to the container image specification, and packages the environment and configuration required for the model application to run into a model application image according to the container image specification, and each model file is saved as a layer of the image.

[0103] Push module 22: used to push model weight images and model application images to the central Harbor image repository.

[0104] Specifically, in the embodiment of the present invention, the push module 22 pushes the model weight image and the model application image to the central Harbor image warehouse for storage.

[0105] Distribution module 23: The central Harbor image repository distributes the model weight image and model application image to Harbor image repositories across the country.

[0106] Specifically, in an embodiment of the present invention, after the central Harbor image warehouse stores the model weight image and the model application image, the distribution module 23 distributes the weight image and the model application image to Harbor image warehouses across the country, and the Harbor image warehouses across the country are distributed in computer rooms across the country.

[0107] Deployment module 24: used to deploy the model service and specify the model weight image required by the model application.

[0108] Specifically, in an embodiment of the present invention, an operation of deploying a model service on the k8s in the computer room is received from the user, and the deployment module 24 deploys the model service on the k8s in the computer room. An operation of specifying a model weight image required by the model application on the k8s in the computer room is received from the user, and the model weight image required by the model application is specified.

[0109] Calling module 25: Alluxio Worker calls Alluxio UFS Plugin. Alluxio UFSPlugin calls the container runtime interface to download the model weight image and model application image.

[0110] Specifically, in the embodiment of the present invention, the container runtime of the cloud host configures the mirror, and configures the addresses of the Harbor image repositories across the country in order of distance. The calling module 25 is used for Alluxio Worker to call Alluxio UFSPlugin, and Alluxio UFS Plugin calls the interface of the container runtime to use the mirror function of the container runtime to download the model weight image and model application image from the nearest Harbor image repository.

[0111] Mount module 26: Used by Alluxio UFS Plugin to mount the model weight image and model application image to the specified directory of the cloud host.

[0112] Specifically, in an embodiment of the present invention, after downloading the model weight image and the model application image, the mounting module 26 of Alluxio UFSPlugin uses the overlayfs technology to mount the downloaded model weight image and the model application image to the specified directory of the cloud host. When Alluxio UFS Plugin is docked with the container runtime, Alluxio is used as a cloud native cache layer to provide model file cache for the model application, and automatically synchronize files from other machines in the computer room that have downloaded the model files, without having to access the public network again, thereby improving the loading speed and reducing the central bandwidth pressure and load.

[0113] Build module 27: Used by Alluxio Runtime to build a virtual file system based on the specified directory as the data source.

[0114] Specifically, in an embodiment of the present invention, after mounting the downloaded model weight image and model application image to the specified directory of the cloud host, the construction module 27 of Alluxio Runtime builds a virtual file system based on the specified directory as a data source.

[0115] Loading module 28: used for the model application to mount the virtual file system, load the model weight file and the environment and configuration required for the model application to run, and start the model application service.

[0116] According to a third aspect of the present invention, an electronic device is provided, comprising: a processor and a memory;

[0117] The processor calls the program or instructions stored in the memory to execute any of the above methods for storing, transmitting, and loading cloud-native machine learning model files.

[0118] In a fourth aspect of the present invention, a computer-readable storage medium is further proposed, which stores a program or instruction, and the program or instruction enables a computer to execute any of the above methods for storing, transmitting and loading cloud-native machine learning model files.

[0119] Figure 3 It is a schematic block diagram of an electronic device provided by an embodiment of the present invention.

[0120] like Figure 3As shown, the electronic device includes: at least one processor 301, at least one memory 302 and at least one communication interface 303. The various components in the electronic device are coupled together through a bus system 304. The communication interface 303 is used to transmit information between external devices. It can be understood that the bus system 304 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 304 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 3 Various buses are labeled as bus system 304 .

[0121] It can be understood that the memory 302 in this embodiment can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories.

[0122] In some implementations, the memory 302 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system and application programs.

[0123] Among them, the operating system includes various system programs, such as the framework layer, the core library layer, the driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program that implements any of the methods for storing, transmitting, and loading cloud-native machine learning model files provided in the embodiments of the present invention can be included in the application program.

[0124] In an embodiment of the present invention, the processor 301 calls the program or instructions stored in the memory 302, specifically, the program or instructions stored in the application, and the processor 301 is used to execute the steps of each embodiment of the method for storing, transmitting and loading cloud-native machine learning model files provided in the embodiment of the present invention.

[0125] Package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image;

[0126] Push model weight images and model application images to the central Harbor image repository;

[0127] The central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country;

[0128] Deploy the model service, specifying the model weight image required by the model application;

[0129] Alluxio Worker calls Alluxio UFS Plugin, which calls the container runtime interface to download the model weight image and model application image.

[0130] Alluxio UFS Plugin mounts the model weight image and model application image to the specified directory of the cloud host;

[0131] Alluxio Runtime builds a virtual file system based on the specified directory as the data source;

[0132] The model application mounts the virtual file system, loads the model weight file and the environment and configuration required for the model application to run, and starts the model application service.

[0133] Any of the methods for storing, transmitting and loading cloud native machine learning model files provided in the embodiments of the present invention may be applied to the processor 301 or implemented by the processor 301. The processor 301 may be an integrated circuit chip having signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in the processor 301 or by instructions in the form of software. The above processor 301 may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0134] The steps of any method in the storage, transmission and loading method of the cloud native machine learning model file provided in the embodiment of the present invention can be directly reflected as the hardware decoding processor, or the hardware and software units in the decoding processor are combined and executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 302, and the processor 301 reads the information in the memory 302 and completes the steps of the method in combination with its hardware.

[0135] Those skilled in the art will appreciate that although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features from different embodiments is meant to be within the scope of the present invention and to form different embodiments.

[0136] Those skilled in the art will appreciate that the description of each embodiment has its own emphasis, and for parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0137] Although the embodiments of the present invention are described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims. The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed by the present invention, and these modifications or substitutions should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

[0138] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for storing, transmitting and loading cloud-native machine learning model files, characterized in that: include: Package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image; Push model weight images and model application images to the central Harbor image repository; The central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country; Deploy the model service, specifying the model weight image required by the model application; Alluxio Worker calls Alluxio UFS Plugin, which calls the container runtime interface to download the model weight image and model application image. Alluxio UFS Plugin mounts the model weight image and model application image to the specified directory of the cloud host; Alluxio Runtime builds a virtual file system based on the specified directory as the data source; The model application mounts the virtual file system, loads the model weight file and the environment and configuration required for the model application to run, and starts the model application service.

2. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: Package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image, including: Package the model weight file into a model weight image according to the container image specification, and package the environment and configuration required for the model application to run into a model application image according to the container image specification; Among them, each model file is saved as a layer of the image.

3. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: The central Harbor image warehouse distributes the model weight image and model application image to Harbor image warehouses across the country, including: The central Harbor image warehouse distributes the model weight image and the model application image to Harbor image warehouses across the country based on the distribution technology of the Harbor image warehouse and the distribution strategy.

4. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: The deployment model service includes: After receiving the user's operation to deploy the model service on the k8s in the computer room, the model service is deployed on the k8s in the computer room.

5. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: Before the Alluxio UFS Plugin calls the container runtime interface to download the model weight image and model application image, it also includes: The cloud host's container runtime configures mirror and configures the addresses of Harbor image repositories across the country in order of distance.

6. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: The Alluxio UFS Plugin calls the container runtime interface to download the model weight image and model application image, including: The container runtime downloads the model weight image and model application image from the nearest Harbor image repository.

7. The method for storing, transmitting and loading cloud native machine learning model files according to claim 1, characterized in that: The Alluxio UFS Plugin mounts the model weight image and model application image to the specified directory of the cloud host, including: The Alluxio UFS Plugin uses overlayfs technology to mount the model weight image and model application image to the specified directory of the cloud host.

8. A device for storing, transmitting and loading cloud-native machine learning model files, characterized in that: include: Packaging module: used to package the model weight file into a model weight image, and package the environment and configuration required for the model application to run into a model application image; Push module: used to push model weight images and model application images to the central Harbor image repository; Distribution module: used by the central Harbor image warehouse to distribute model weight images and model application images to Harbor image warehouses across the country; Deployment module: used to deploy model services and specify the model weight image required by the model application; Calling module: used by Alluxio Worker to call Alluxio UFS Plugin. Alluxio UFS Plugin calls the container runtime interface to download the model weight image and model application image. Mounting module: used by Alluxio UFS Plugin to mount the model weight image and model application image to the specified directory of the cloud host; Building module: used by Alluxio Runtime to build a virtual file system based on the specified directory as the data source; Loading module: used for model application to mount virtual file system, load model weight file and the environment and configuration required for model application to run, and start model application service.

9. An electronic device, characterized in that: include: Processor and memory; The processor is used to execute the storage, transmission and loading method of the cloud-native machine learning model file as described in any one of claims 1 to 7 by calling the program or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, which enables a computer to execute the method for storing, transmitting, and loading a cloud-native machine learning model file as described in any one of claims 1 to 7.

Images