Machine learning operation method and machine learning operation system in private network environment

Abstract

The machine learning operation method in a private network environment comprises the steps of: verifying, by a machine learning operation platform, a request when a user requests a machine learning job from the machine learning operation platform, and generating a job specification according to the request; performing, by a cluster agent, polling on the machine learning operation platform at a preset first period to check whether a machine learning job specification has been allocated from the machine learning operation platform; converting, by the cluster agent, the machine learning job specification received from the machine learning operation platform into a Kubernetes resource form; and dynamically scheduling, by a Kubernetes API server, container resources in response to a container resource generation request received from the cluster agent.

Description

Machine learning operation method and machine learning operation system in a private network environment

[0001] The following embodiments relate to a machine learning operation method and system capable of efficient resource management and log collection by securely integrating with user computing resources in a private network environment. More specifically, the following embodiments relate to a machine learning platform operation method and system that supports an on-premise environment.

[0002] With the advancement of Machine Learning (ML) and Artificial Intelligence (AI) technologies, the development of solutions utilizing them is actively taking place across various industrial sectors. Consequently, the importance of Machine Learning Operations (MLOps) platforms for efficiently managing the development, deployment, and operation of these ML / AI models is gradually increasing.

[0003] MLOps platforms provide an integrated environment that manages the entire lifecycle of ML / AI models, including data preparation, model training, evaluation, deployment, and monitoring. This allows data scientists and ML engineers to focus on model development, while enabling companies to improve the productivity and quality of ML / AI projects.

[0004] Conventional MLOps platforms have primarily been deployed as services provided in cloud environments. While this offers advantages in terms of scalability and flexibility, it has drawbacks regarding data security, regulatory compliance, and network bandwidth. Consequently, there is a growing demand for deploying MLOps platforms in on-premises environments.

[0005] Meanwhile, with the advancement of container technology and the widespread use of container orchestration platforms such as Kubernetes, these technologies are also being applied to ML / AI workload management. Container-based MLOps platforms provide environmental consistency, isolation, and portability, making the development and operation of ML / AI models more efficient.

[0006] The problem that the present disclosure aims to solve is to provide a method and system for operating a machine learning platform that supports an on-premise environment.

[0007] Conventional MLOps platforms suffer from limited accessibility in private network environments. Many companies operate systems in private network environments isolated from the outside world for security reasons, making it difficult to use existing cloud-based MLOps platforms in such settings. In particular, when external connections are restricted due to firewalls or security policies, seamless communication between the MLOps platform and the user's computing resources becomes challenging.

[0008] Furthermore, training and inference of ML / AI models require significant computational resources, but conventional MLOps platforms have limitations in efficiently utilizing users' local computational resources. In particular, they lack the capability to allocate and optimize resources in real-time according to dynamically changing workloads.

[0009] Furthermore, conventional MLOps platforms face difficulties in effectively collecting and transmitting data to a centralized repository, given that monitoring and improving the performance of ML / AI models requires collecting and analyzing various metrics and logs. In particular, continuous monitoring and analysis become challenging when network disconnections occur or direct data transmission is impossible due to security policies.

[0010] Although container-based ML / AI workloads have been increasing recently, conventional MLOps platforms struggle to provide granular resource management capabilities in container environments. In particular, they lack the ability to efficiently allocate and monitor specialized hardware resources, such as GPUs.

[0011] Conventional MLOps platforms are often dependent on specific cloud environments or infrastructure, making it difficult to deploy and operate flexible platforms across various environments such as on-premises, multi-cloud, and hybrid cloud. This hinders the rapid adaptation to MLOps environments in response to changes in a company's infrastructure strategy.

[0012] Due to the aforementioned problems, there were difficulties in building and operating an efficient and secure MLOps platform in a security-critical corporate environment.

[0013] Accordingly, the inventor(s) of the present disclosure have developed a new machine learning operation method and a machine learning operation system that solves the aforementioned limitations or difficulties and can effectively utilize the user's computational resources.

[0014] The problems that the present disclosure aims to solve are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

[0015] A machine learning operation method in a private network environment, performed by a machine learning operation system, may include: a step of verifying a request and generating a job specification according to the request when a user requests a machine learning job from the machine learning operation platform by a machine learning operation platform; a step of polling the machine learning operation platform by a cluster agent to check whether a machine learning job specification has been assigned from the machine learning operation platform according to a preset first cycle; a step of converting the machine learning job specification received from the machine learning operation platform into a Kubernetes resource form by the cluster agent; and a step of dynamically scheduling a container resource by a Kubernetes API server in response to a request to create a container resource received from the cluster agent.

[0016] The above method may further include the steps of: issuing a unique one-time token in response to a user's request to register a computation cluster by the machine learning operating platform; delivering the unique one-time token to the user by the machine learning operating platform; and installing the cluster agent in the computation cluster by the user using the unique one-time token.

[0017] The above method may further include the step of authenticating to the machine learning operation platform using the unique one-time token by the cluster agent; the step of issuing a unique certificate to the cluster agent by the machine learning operation platform if the authentication is successful; and the step of performing mutual Transport Layer Security (mTLS) communication by the machine learning operation platform and the cluster agent based on the unique certificate.

[0018] The above method may further include, after the step of dynamically scheduling the container resources, the step of confirming to the cluster agent that the container resources corresponding to the machine learning task have been created by the Kubernetes API server; and the step of reporting to the cluster agent that the machine learning task has been assigned.

[0019] The above method may further include the step of monitoring the monitoring data according to a preset second cycle when monitoring data is generated in a workload pod by the sidecar container; and the step of collecting the monitoring data and storing it in a temporary storage by the sidecar container.

[0020] The above method may further include the steps of: collecting the monitoring data collected and stored in the sidecar container by the aggregator according to a preset third period from the sidecar container; compressing and batching the monitoring data collected from the aggregator by the aggregator; receiving the compressed and batched monitoring data from the aggregator by the cluster agent according to a preset fourth period or when the size of the compressed and batched monitoring data is greater than or equal to a preset threshold; and transmitting the compressed and batched monitoring data to the machine learning operation platform by the cluster agent.

[0021] The above method may further include the steps of: receiving the monitoring data compressed and batched by the machine learning operation platform; verifying and parsing the monitoring data compressed and batched by the machine learning operation platform; storing the monitoring data compressed and batched by the machine learning operation platform in a central repository; and indexing the monitoring data compressed and batched by the machine learning operation platform.

[0022] A machine learning operation system according to one embodiment may include a machine learning operation platform and a computation cluster configured to validate a request and generate a job specification in accordance with the request when a user requests a machine learning job. The computation cluster may include: a cluster agent configured to poll the machine learning operation platform to check whether a machine learning job specification has been assigned from the machine learning operation platform according to a preset first cycle, and to convert the machine learning job specification received from the machine learning operation platform into a Kubernetes resource form; and a Kubernetes API server configured to dynamically schedule a container resource in response to a request to create a container resource received from the cluster agent. The computation cluster may be configured as a network that allows outbound traffic and does not allow inbound traffic.

[0023] A non-transient computer-readable recording medium according to one embodiment can record a computer program executed by a computer. The computer program may include: verifying a request and generating a job specification according to the request when a user requests a machine learning job from the machine learning operation platform by a machine learning operation platform; polling the machine learning operation platform by a cluster agent to check whether a machine learning job specification has been assigned from the machine learning operation platform according to a preset first cycle; converting the machine learning job specification received from the machine learning operation platform into a Kubernetes resource form by the cluster agent; and dynamically scheduling a container resource by a Kubernetes API server in response to a container resource creation request received from the cluster agent.

[0024] The means for solving the problem of the present disclosure are not limited to the means for solving the problem described above, and means for solving the problem not mentioned will be clearly understood by those skilled in the art to which the present disclosure belongs from the present specification and the attached drawings.

[0025] According to the embodiments, it is possible to flexibly deploy a cloud-level MLOps platform even in an on-premises environment for data security and regulatory compliance, and thereby improve the productivity and quality of container-based ML / AI projects through the efficient allocation and monitoring of specialized hardware resources such as GPUs.

[0026] According to the embodiments, security can be enhanced through automatically generated one-time tokens and mutual TLS authentication, and restrictions on network security policies can be overcome by using only outbound connections.

[0027] According to the embodiments, machine learning costs can be optimized through real-time resource monitoring and a dynamic allocation system in a container environment.

[0028] According to the embodiments, various metrics and logs can be effectively collected and transmitted to users by utilizing cluster agents and sidecar containers, thereby facilitating the monitoring and maintenance of machine learning model performance or metrics.

[0029] The effects according to the present disclosure are not limited to the effects described above, and unmentioned effects will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.

[0030] The accompanying drawings, included as part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and explain the technical features of the present specification together with the detailed description.

[0031] FIG. 1 is a conceptual diagram showing the architecture of a machine learning operating system according to one embodiment of the present disclosure.

[0032] FIG. 2 is a block diagram showing the configuration of a computing device for machine learning operation according to one embodiment of the present disclosure.

[0033] FIG. 3 is a sequence diagram illustrating a user computation cluster authentication process using a unique one-time token according to one embodiment of the present disclosure.

[0034] FIG. 4 is a sequence diagram illustrating the process of scheduling a user's task execution command to a computation cluster according to one embodiment of the present disclosure.

[0035] FIG. 5 is a sequence diagram illustrating a monitoring data collection process and a process in which monitoring data is transmitted to a machine learning operation platform according to one embodiment of the present disclosure.

[0036] FIG. 6 is a flowchart illustrating a machine learning operation method in a private network environment according to one embodiment of the present disclosure.

[0037] Specific structural or functional descriptions of embodiments according to the concept of the present disclosure disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present disclosure, and embodiments according to the concept of the present disclosure may be implemented in various forms and are not limited to the embodiments described herein.

[0038] Embodiments according to the concept of the present disclosure may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present disclosure to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present disclosure.

[0039] Terms such as "first" or "second" may be used to describe various components, but said components shall not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.

[0040] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.

[0041] The terms used herein are used merely to describe specific embodiments and are not intended to limit the disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0042] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0043] In this specification, the term "processor" may refer to hardware capable of performing functions and operations according to each name described in this specification, computer program code capable of performing specific functions and operations, or an electronic recording medium loaded with computer program code capable of performing specific functions and operations.

[0044] In other words, the term "processor" may mean a functional and / or structural combination of hardware for carrying out the technical concept of the present disclosure and / or software for driving said hardware.

[0045] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.

[0046] FIG. 1 is a conceptual diagram showing the architecture of a machine learning operating system according to one embodiment of the present disclosure.

[0047] Referring to FIG. 1, a machine learning operating system (1000) according to one embodiment of the present disclosure may include a user (100), a machine learning operating platform (200) corresponding to a control plane managed by the machine learning operating system (1000), and / or a computation cluster (300) corresponding to a data plane operated on the infrastructure of the user (100).

[0048] According to an embodiment of the present disclosure, the machine learning operations platform (200) may be a Machine Learning Operations (MLOps) platform. The machine learning operations platform (200) is a platform for automating and managing the entire lifecycle of a machine learning model from development to deployment and operation, and can directly provide core functions of machine learning, service, and computational resource allocation to the user (100). The machine learning operations platform (200) can perform functions such as user authentication, machine learning job scheduling, machine learning model version management, and experiment tracking. For example, the machine learning operations platform (200) can communicate with the user (100) and the computation cluster (300) via a RESTful API.

[0049] The machine learning operation platform (200) may include an API server (202). The API server (202) can execute business logic of the machine learning operation platform (200). It may provide an interface that communicates directly with the user (100) using methods such as the Web (World Wide Web), CLI (Command Line Interface), and / or SDK (Software Development Kit). In addition to the API server (202), the machine learning operation platform (200) may further include a metric server (204) that stores and retrieves metadata of logs, metrics, and / or files generated from the workload, a database, etc.

[0050] In one embodiment of the present disclosure, computational resources may be constructed in the form of a cluster. According to one embodiment of the present disclosure, the computational cluster (300) may be a component owned by a user (100) and constructed in a private network environment. The computational cluster (300) may be implemented with computational resources including hardware resources such as a CPU (Central Processing Unit), GPU (Graphic Processing Unit), memory, and storage required at each stage of the machine learning lifecycle. The computational cluster (300) may execute actual ML / AI workloads. According to an embodiment, the computational cluster (300) may be configured based on containers and may be managed using Kubernetes.

[0051] The computation cluster (300) may include a cluster agent (302). The computation cluster (300) may communicate with the machine learning operation platform (200) through the cluster agent (302).

[0052] The cluster agent (302) is a core component installed in the computation cluster (300) and is the entity that manages the computation cluster (300). The cluster agent (302) can perform the role of communicating with the machine learning operation platform (200). The cluster agent (302) can perform functions such as executing machine learning tasks, reporting status, and monitoring computation resources. Specifically, the cluster agent (302) can control and determine how to execute machine learning tasks requested by a user, how to report the status of machine learning tasks within the computation cluster (300) to the user (100), and how to monitor the allocation status of computation resources for machine learning tasks. According to an embodiment of the present disclosure, since most of the data required for machine learning contains sensitive information that should not be exposed externally, the computation cluster (300) is configured to use only outbound connections. In other words, the computation cluster (300) according to an embodiment of the present disclosure may be configured as a network that allows outbound traffic but does not allow inbound traffic. Through this, the security of the machine learning operating system (1000) according to one embodiment of the present disclosure can be enhanced. Due to the characteristics of this computation cluster (300), the user (100) communicates only through the machine learning operating platform (200), and the cluster agent (302) operates to schedule and control machine learning tasks on computation resources within the computation cluster (300) in response to the user's request.

[0053] A security mechanism of a machine learning operating system (1000) according to one embodiment of the present disclosure can be realized by encrypting all communications within the machine learning operating system (1000) via HTTPS (Hypertext Transfer Protocol Secure), using mutual Transport Layer Security (mTLS) communication, and verifying the validity of each request using a token-based authentication system. Details are described later in FIG. 3.

[0054] A security mechanism of a machine learning operating system (1000) according to one embodiment of the present disclosure can be implemented by having a cluster agent (302) periodically poll the machine learning operating platform (200) to check for new machine learning tasks or commands from a user (100).

[0055] Specifically, since the computation cluster (300) according to the embodiment of the present disclosure uses only outbound connections from the internal network to the outside, the machine learning operation platform (200) cannot directly issue commands for machine learning tasks to the computation cluster (300). At this time, according to the embodiment of the present disclosure, in order to perform machine learning tasks on the computation resources owned by the user (100), the cluster agent (302) included in the computation cluster (300) can perform polling on the machine learning operation platform (200) to check whether machine learning task specifications are assigned according to a preset period. Details are described later in FIG. 4.

[0056] A machine learning operating system (1000) according to one embodiment of the present disclosure can dynamically allocate and monitor computational resources. Specifically, a resource manager installed on each node of a computation cluster (300) can monitor resource usage, such as CPU, memory, and GPU, in real time. Monitoring data (e.g., logs and / or metrics) collected from the computation cluster (300) can be stored in real time through a Prometheus component and can be periodically reported to the machine learning operating platform (200). A scheduler of the machine learning operating platform (200) can make optimal resource allocation decisions based on the collected monitoring data periodically reported to the machine learning operating platform (200). A sidecar container may be deployed with each workload, and the sidecar container may collect monitoring data including loads or metrics. The collected monitoring data may be buffered in a local temporary storage, then compressed or distributed by an aggregator and transmitted to the machine learning operating platform (200). Detailed information is described later in Fig. 5.

[0057] A machine learning operating system (1000) according to one embodiment of the present disclosure can support the flexible deployment of a cloud-level machine learning operating platform even in an on-premises environment for data security and regulatory compliance. Accordingly, the machine learning operating system (1000) according to one embodiment of the present disclosure can improve the productivity and quality of container-based ML / AI projects through the efficient allocation and monitoring of specialized hardware resources such as GPUs. In addition, the machine learning operating system (1000) according to one embodiment of the present disclosure can enhance security through automatically generated one-time tokens and mutual TLS authentication, and can overcome restrictions on network security policies by using only outbound connections. Furthermore, the machine learning operating system (1000) according to one embodiment of the present disclosure can optimize machine learning costs through real-time resource monitoring and a dynamic allocation system in a container environment. In addition, the machine learning operating system (1000) according to one embodiment of the present disclosure can effectively collect various metrics and logs and transmit them to users by utilizing cluster agents and sidecar containers, thereby facilitating the monitoring and maintenance of machine learning model performance or metrics.

[0058] FIG. 2 is a block diagram showing the configuration of a machine learning operating platform server according to one embodiment of the present disclosure.

[0059] Referring to FIG. 2, the computing device (200) for machine learning operations may be a server that provides machine learning operations services in response to a request from a user (or user device), or a user device in which a web page, application, and / or program capable of performing machine learning operations is installed and executed.

[0060] A computing device (200) for machine learning operation may include a communication interface (110210), memory (120220), an I / O interface (130230) and / or a processor (140240), and each component may communicate with one or more communication buses or signal lines.

[0061] The communication interface (110210) can be connected to a user device (not shown) via a wired / wireless communication network to exchange data. For example, if the computing device (200) for machine learning operation is a server, the communication interface (110210) can receive machine learning task commands or requests for computation cluster registration from the user device. Additionally, the communication interface (110210) can transmit monitoring data including logs and / or metrics to the user. The monitoring data transmitted to the user may be data in which logs and / or metrics have been compressed, batched, and indexed.

[0062] Meanwhile, a communication interface (110210) that enables the transmission and reception of such data may include a wired communication port (111211) and a wireless circuit (112212). Here, the wired communication port (111211) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), IEEE1394 (e.g., FireWire, Apple; i.Link, Sony; Lynx, Texas Instruments), etc. Additionally, the wireless circuit (112212) may transmit and receive data with an external device via an RF (Radio Frequency) signal or an optical signal. Furthermore, wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, such as GSM (Global System for Mobile Communications), EDGE (Enhanced Data rates for GSM Evolution), CDMA (Code-Division Multiple Access), TDMA (Time Division Multiple Access), Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

[0063] The memory (120220) can store various data used in the computing device (200) for machine learning operations. For example, the memory (120220) can store logs, metrics, system metrics, and / or metadata of files generated from a workload. As another example, the memory (120220) can store data collected by a cluster agent (302) in a computing cluster (300) according to one embodiment of the present disclosure. Additionally, the memory (120220) can store data that the user has directly written collection code for or that has been stored in an automatic upload directory.

[0064] In various embodiments, the memory (120220) may include a volatile or non-volatile recording medium capable of storing various data, commands and / or information. For example, the memory (120220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), Random Access Memory (RAM), Static RAM (SRAM), Read-Only Memory (ROM), Programmable ROM (PROM), Electrically Erasable ROM (EEPROM), network storage, cloud, and blockchain database.

[0065] In various embodiments, the memory (120220) may store at least one configuration of an operating system (1212221), a communication module (122222), a user interface module (123223), and one or more applications (124224).

[0066] An operating system (121221) (e.g., embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers for controlling and managing general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.

[0067] The communication module (123223) can support communication with another device through the communication interface (110210). The communication module (123223) may include various software components for processing data received by the wired communication port (111211) or wireless circuit (112212) of the communication interface (110210).

[0068] The user interface module (123223) can receive a user's request or input from a keyboard, touch screen, keyboard, mouse, and / or microphone, etc., through the I / O interface (130230) and provide a user interface on the display.

[0069] The application (124224) may include a program or module configured to be executed by one or more processors (140240). Here, the application, which provides services for handling all processes necessary for machine learning research and development, including machine learning, machine learning model deployment, monitoring, and / or computational resource scheduling, etc., may be implemented on a server farm.

[0070] The I / O interface (130230) can connect at least one of an input / output device (not shown) of a computing device (200) for machine learning operation, such as a display, keyboard, touch screen, and microphone, to a user interface module (123223). The I / O interface (130230) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (123223) and process commands based on the received input.

[0071] The processor (140240) is connected to a communication interface (110210), a memory (120220), and an I / O interface (130230) to control the overall operation of a computing device (200) for machine learning operations, and can execute various commands for machine learning operations through applications and / or programs stored in the memory (120220).

[0072] The processor (140240) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). Additionally, the processor (140240) may be implemented in the form of an Integrated Chip (IC), such as a System on Chip (SoC) that integrates various computing devices. Alternatively, the processor (140240) may include a module for computing artificial neural network models, such as a Neural Processing Unit (NPU).

[0073] FIG. 3 is a sequence diagram illustrating a user computation cluster authentication process using a unique one-time token according to one embodiment of the present disclosure.

[0074] Referring to FIG. 3, the process illustrates a user (100) installing a cluster agent (302) on a computation cluster (300) through an MLOps platform (200) and authenticating and linking the computation cluster (300) using a unique one-time token generated by the MLOps platform (200). The MLOps platform (200) in FIG. 3 is an example of the machine learning operation platform of FIG. 1.

[0075] For security reasons, the MLOps platform (200) needs to authenticate only the computation cluster (300) that the user has explicitly commanded for the computational resources they wish to use. According to an embodiment of the present disclosure, once authentication is performed only once initially for the corresponding computation cluster (300) in response to the user's request, the corresponding computation cluster (300) is subsequently determined to be an authenticated cluster, and the user can continuously communicate with the corresponding computation cluster through the MLOps platform (200).

[0076] Referring to FIG. 3, the user (100) can request the registration of a new computation cluster to the MLOps platform (200) (S301).

[0077] The MLOps platform (200) can generate a unique one-time token in response to a user's request (S302) and deliver the generated unique one-time token to the user (100) (S303).

[0078] The user (100) can install a cluster agent (302) on the user's (100) computation cluster (300) using a received unique one-time token (S304). Here, the user (100) can access the computation cluster (300) only during the installation step (S304) of the entire service operation, declare that the computation cluster (300) is a cluster dependent on the MLOps platform (200), and deliver a token corresponding to the computation cluster (300).

[0079] At initial boot, the cluster agent (302) can authenticate itself to the MLOps platform (200) using a unique one-time token provided by the user (100) (S305).

[0080] When the MLOps platform (200) authenticates the cluster agent (302), that is, when the cluster agent (302) succeeds in authentication, the MLOps platform (200) can issue a unique certificate to the cluster agent (302) (S306).

[0081] The MLOps platform (200) and the cluster agent (302) can perform mutual TLS-based communication based on unique certificates.

[0082] FIG. 4 is a sequence diagram illustrating the process of scheduling a user's task execution command to a computation cluster according to one embodiment of the present disclosure.

[0083] Referring to FIG. 4, the process of a task execution command issued by a user (100) through an MLOps platform (200) being scheduled to the user's computation cluster (300) is illustrated. The MLOps platform (200) in FIG. 4 is an example of the machine learning operation platform of FIG. 1.

[0084] Referring to FIG. 4, a user (100) can request a machine learning job (e.g., training a machine learning model) using a web UI (User Interface) or CLI (Command-Line Interface) (S401).

[0085] The MLOps platform (200) can verify a request received from a user (100) and generate a job specification according to the request (S402). The generated job specification can be stored in the internal database of the MLOps platform (200).

[0086] At this time, the step of verifying the request received from the user may be, for example, a step of checking whether the request is to execute a process that the MLOps platform (200) can schedule, whether there are sufficient computational resources to perform the task, whether the user (100) is requesting a task using data that can actually be accessed from the computation cluster (300), and whether there are no errors in the command that the user (100) intends to execute. The step of verifying the request received from the user (100) may be a step of verifying it primarily at the MLOps platform (200) level for security purposes. Since the computation cluster (300) is configured to use only outbound connections, the MLOps platform (200) cannot directly request a machine learning task based on the task specification from the cluster agent (302). The stored task specification and the machine learning task request can be transmitted to the computation cluster (300) through the steps from step S403 onwards.

[0087] The cluster agent (302) can poll the MLOps platform (200) to check whether a task specification has been assigned according to a preset period (e.g., 15 seconds) (S403). By polling periodically, the cluster agent (302) can check if there is a task specification that has not yet been assigned to the MLOps platform (200).

[0088] When polling at a specific point in time, if there are work specifications that have not yet been assigned, the MLOps platform (200) can transmit the work specifications that have not yet been assigned to the cluster agent (302) (S404).

[0089] The cluster agent (302) can analyze the received task specification and convert the task specification into a Kubernetes resource form (S405). In other words, the cluster agent (302) can convert the task specification into a form required by Kubernetes (e.g., a container), which is a tool for managing the cluster, on the premise that the task specification is a verified task specification. Thus, the cluster agent (302) can define, search, and / or translate the task specification so that the user's (100) command can be executed by the Kubernetes API server (304) as a container.

[0090] The cluster agent (302) can create the necessary resources (e.g., workload pods, jobs) through the Kubernetes API server (304).

[0091] At this time, the cluster agent (302) can request the Kubernetes API server (304) to create a container resource (S406).

[0092] The Kubernetes API server (304) can schedule container resources to appropriate nodes within the compute cluster (300) (S407). The Kubernetes scheduler can allocate container resources to appropriate nodes within the compute cluster (300). At this time, container resources can be allocated in the form of Kubernetes Pods. Machine learning tasks can be packaged into Docker containers and executed. Machine learning-specific workloads (e.g., training tasks, inference services) can be defined and managed using Kubernetes Custom Resource Definitions (CRDs). For example, compute resources such as GPU resources are managed through the NVIDIA Device Plugin and can be dynamically allocated as needed.

[0093] The Kubernetes API server (304) can confirm to the cluster agent (302) that a container resource has been created (S408). For example, the Kubernetes API server (304) can send an Ack message, etc., to the cluster agent (302) confirming that a container resource has been created.

[0094] The cluster agent (302) can report to the MLOps platform (200) that a machine learning job requested by the user has been assigned (S409).

[0095] According to an embodiment of the present disclosure, the user cannot directly transmit work commands to the computation cluster (300) but issues work commands to the computation cluster (300) through an interface called an MLOps platform (200), and the commands issued by the user to the MLOps platform (200) can be transmitted to the computation cluster (300) through the process described above.

[0096] FIG. 5 is a sequence diagram illustrating a monitoring data collection process and a process in which monitoring data is transmitted to a machine learning operation platform according to one embodiment of the present disclosure.

[0097] The computation cluster (300) may include a workload pod (310), a sidecar container (308), an aggregator (306), and a cluster agent (302).

[0098] In this specification, monitoring data may include logs (log data) and / or metrics (metric data). Logs may include indicators representing the machine learning process in the form of a graph, indicators indicating where the workload is actually allocated or scheduled, the time spent on machine learning, and characteristics or types of output data of machine learning data. Metrics may be indicators expressed as scalars over time during the progress of machine learning (e.g., loss, accuracy).

[0099] According to an embodiment of the present disclosure, monitoring data generated in the workload pod (310) is not transmitted to the MLOps platform (200) in real time, taking into account the burden that the machine learning operation service places on the network, but rather the monitoring data is collected and stored and then periodically transmitted to the MLOps platform (200).

[0100] Referring to FIG. 5, a workload pod (310) can generate logs or metrics (S501). A workload pod (310) may be a learning process created by a user's request. A sidecar container (308) for collecting logs or metrics may be deployed in each workload pod (310) along with an application container.

[0101] The sidecar container (308) can collect logs or metrics and store the collected logs or metrics in a local temporary storage (e.g., emptyDir volume) (S502). The logs or metrics stored in the sidecar container (308), i.e., the monitoring data, must eventually be delivered to the MLOps platform (200).

[0102] The sidecar container (308) can continuously monitor the logs or metrics of the application container according to a preset period (S503).

[0103] The aggregator (306) can periodically collect data, i.e., monitoring data, collected from the sidecar container (308) (S504). The aggregator (306) may also be referred to as a central collector within the computation cluster (300).

[0104] The aggregator (306) can compress and batch the collected monitoring data (S505). The aggregator (306) can collect logs or metrics collected from multiple sidecar containers, including sidecar containers (308), and compress them in one place. By compressing and batching the collected monitoring data, the aggregator (306) can prepare for efficient transmission.

[0105] The aggregator (306) can transmit the compressed and batched monitoring data to the cluster agent (302) (S506). The aggregator (306) can retrieve the compressed and batched monitoring data (S507).

[0106] The cluster agent (302) can transmit compressed and batched monitoring data to the MLOps platform (200) (S508). In one embodiment, the cluster agent (302) can retrieve compressed and batched monitoring data from the aggregator (306) according to a preset period and then transmit it to the MLOps platform (200). In another embodiment, the cluster agent (302) can retrieve compressed and batched monitoring data from the aggregator (306) and then transmit it to the MLOps platform (200) when the size of the compressed and batched monitoring data is greater than or equal to a preset threshold.

[0107] An MLOps platform (200) or a log manager (not shown) of an MLOps platform (200) can verify and parse the received monitoring data (S509).

[0108] The MLOps platform (200) or the log manager of the MLOps platform (200) can store verified and parsed monitoring data in a central repository (S510).

[0109] The MLOps platform (200) can index monitoring data stored in a central repository (S511). By indexing the monitoring data stored in the central repository, the MLOps platform (200) can process the monitoring data so that it can be quickly searched and analyzed.

[0110] In the event that communication between the MLOps platform (200) and the computation cluster (300), or between the MLOps platform (200) and the cluster agent (302), is impossible (e.g., network disconnection), the cluster agent (302) can prevent data loss through a retry mechanism. Specifically, the aggregator (306) initiates the retry mechanism, and the cluster agent (302) can retrieve monitoring data from the aggregator (306).

[0111] FIG. 6 is a flowchart illustrating a machine learning operation method in a private network environment according to one embodiment of the present disclosure.

[0112] According to a machine learning operation method in a private network environment according to one embodiment of the present disclosure, a machine learning operation platform may issue a unique one-time token in response to a user's request to register a computation cluster (S600).

[0113] After the step (S600) of issuing the unique one-time token described above, the machine learning operation platform may deliver the unique one-time token to the user. Then, the user may install the cluster agent in the computation cluster using the unique one-time token.

[0114] According to a machine learning operation method in a private network environment according to one embodiment of the present disclosure, a cluster agent can authenticate to a machine learning operation platform using a unique one-time token (S602).

[0115] In the step (S602) of authenticating to the machine learning operation platform using the aforementioned unique one-time token, if authentication is successful, the machine learning operation platform may issue a unique certificate to the cluster agent. Subsequently, the machine learning operation platform and the cluster agent may perform mutual Transport Layer Security (TLS) communication based on the unique certificate.

[0116] According to a machine learning operation method in a private network environment according to one embodiment of the present disclosure, when a user requests a machine learning operation to the machine learning operation platform, the received machine learning operation request can be verified and a work specification according to the machine learning operation request can be generated (S604).

[0117] According to a method for operating machine learning in a private network environment according to one embodiment of the present disclosure, a cluster agent may perform polling on a machine learning operation platform to check whether a machine learning task specification has been assigned according to a preset first period (S606).

[0118] According to a machine learning operation method in a private network environment according to one embodiment of the present disclosure, a machine learning task specification received from a machine learning operation platform can be converted into a Kubernetes resource form by a cluster agent (S600).

[0119] According to a machine learning operation method in a private network environment according to one embodiment of the present disclosure, a container resource can be dynamically scheduled by a Kubernetes API server in response to a container resource creation request received from a cluster agent (S610).

[0120] After the step of dynamically scheduling the above-mentioned container resources (S610), the cluster agent can confirm that a container resource corresponding to a machine learning task has been created by the Kubernetes API server. Subsequently, the cluster agent can report that the machine learning task has been assigned.

[0121] In one embodiment, when monitoring data is generated in a workload pod, the monitoring data can be monitored by a sidecar container according to a preset second cycle. Then, the monitoring data can be collected and stored in a temporary storage by the sidecar container. Subsequently, the monitoring data collected and stored in the sidecar container can be collected from the sidecar container according to a preset third cycle by an aggregator. Then, the monitoring data collected by the aggregator can be compressed and batched by the aggregator.

[0122] The machine learning operation platform can receive the compressed and batched monitoring data. The machine learning operation platform can then verify and parse the compressed and batched monitoring data. Next, the machine learning operation platform can store the compressed and batched monitoring data in a central repository. Subsequently, the machine learning operation platform can index the compressed and batched monitoring data.

[0123] A machine learning operating system according to one embodiment may include a machine learning operating platform and a computation cluster configured to issue a unique one-time token in response to a user's request to register a computation cluster, and, when receiving a machine learning task request from a user, to verify the received machine learning task request and generate a task specification according to the machine learning task request.

[0124] The above-described computation cluster may include a cluster agent configured to authenticate to a machine learning operation platform using a unique one-time token, poll the machine learning operation platform to determine whether a machine learning job specification has been assigned according to a preset first cycle, and convert the machine learning job specification received from the machine learning operation platform into a Kubernetes resource, and a Kubernetes API server configured to dynamically schedule a container resource in response to a request to create a container resource received from the cluster agent.

[0125] The above-mentioned computation cluster may be configured as a network that allows outbound traffic and does not allow inbound traffic.

[0126] A machine learning operating system according to one embodiment may include a computer program stored on a computer-readable recording medium to perform steps S600 to S610 of FIG. 6 described above, combined with a computer or computing device which is hardware.

[0127] A machine learning operating system according to one embodiment may be implemented as a computing device comprising at least one processor that executes instructions of programs loaded in memory. A program comprising instructions described to execute steps S600 to S610 of FIG. 6 described above may be loaded in the memory of the computing device.

[0128] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0129] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0130] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

[0131] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

[0132] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.

[0133] The machine learning operation method and machine learning operation system in a private network environment as described above can be applied to service fields related to Machine Learning Operations (MLOps) platforms.

Claims

1. When a user requests a machine learning job from the machine learning operation platform, the machine learning operation platform verifies the request and generates a job specification according to the request; A step of performing polling on the machine learning operation platform by a cluster agent to determine whether a machine learning job specification has been assigned from the machine learning operation platform according to a preset first cycle; A step of converting the machine learning job specification received from the machine learning operation platform into a Kubernetes resource form by the cluster agent; and A step comprising: dynamically scheduling container resources by a Kubernetes API server in response to a request to create container resources received from the cluster agent; A machine learning operation method in a private network environment performed by a machine learning operation system.

2. In Paragraph 1, A step of issuing a unique one-time token in response to a user's request to register a computation cluster by the machine learning operating platform described above; The step of delivering the unique one-time token to the user by the machine learning operating platform; and The method further comprises the step of installing the cluster agent in the computation cluster using the unique one-time token by the user. A machine learning operation method in a private network environment performed by a machine learning operation system.

3. In Paragraph 1, A step of authenticating to the machine learning operation platform using the unique one-time token by the cluster agent; If the above authentication is successful, the step of issuing a unique certificate to the cluster agent by the machine learning operation platform; and The method further comprises the step of performing mutual Transport Layer Security (mTLS) communication based on the unique certificate by the machine learning operating platform and the cluster agent. A machine learning operation method in a private network environment performed by a machine learning operation system.

4. In Paragraph 1, After the step of dynamically scheduling the above container resources, A step of confirming with the cluster agent that the container resource corresponding to the machine learning task has been created by the above Kubernetes API server; and A step of reporting that the machine learning task has been assigned by the cluster agent; further comprising A machine learning operation method in a private network environment performed by a machine learning operation system.

5. In Paragraph 1, When monitoring data is generated in a workload pod by a sidecar container, the step of monitoring said monitoring data according to a preset second cycle; and The method further comprises the step of collecting the monitoring data and storing it in a temporary storage by the above-mentioned sidecar container. A machine learning operation method in a private network environment performed by a machine learning operation system.

6. In Paragraph 5, A step of collecting the monitoring data collected and stored in the sidecar container by an aggregator from the sidecar container according to a preset third cycle; A step of compressing and batching the monitoring data collected from the aggregator by the aggregator; A step of receiving the compressed and batched monitoring data from the aggregator by the cluster agent according to a preset fourth period or when the size of the compressed and batched monitoring data is greater than or equal to a preset threshold; and The method further comprises the step of transmitting the monitoring data, which is compressed and batched by the cluster agent, to the machine learning operation platform. A machine learning operation method in a private network environment performed by a machine learning operation system.

7. In Paragraph 6, A step of receiving the monitoring data compressed and batched by the machine learning operating platform; A step of verifying and parsing the monitoring data compressed and batched by the machine learning operation platform; A step of storing the monitoring data, compressed and batched by the machine learning operation platform, in a central repository; and The method further comprises the step of indexing the monitoring data compressed and batched by the machine learning operation platform. A machine learning operation method in a private network environment performed by a machine learning operation system.

8. As a machine learning operating system, A machine learning operation platform configured to validate a request and generate a job specification based on the request when a user requests a machine learning job; and Includes computation clusters, The above computation cluster is, A cluster agent configured to perform polling on the machine learning operation platform to determine whether a machine learning task specification has been assigned from the machine learning operation platform according to a preset first cycle, and to convert the machine learning task specification received from the machine learning operation platform into a Kubernetes resource form; and It includes a Kubernetes API server configured to dynamically schedule container resources in response to a container resource creation request received from the cluster agent; The above computation cluster is configured as a network that allows outbound traffic but does not allow inbound traffic, System.

9. A non-transient computer-readable recording medium that records a computer program executed by a computer, The above computer program is, When a user requests a machine learning job from the machine learning operation platform, the machine learning operation platform verifies the request and generates a job specification according to the request; Polling the machine learning operation platform by the cluster agent to determine whether a machine learning job specification has been assigned from the machine learning operation platform according to a preset first cycle; Converting the machine learning job specification received from the machine learning operation platform by the cluster agent into a Kubernetes resource form; and It includes dynamically scheduling container resources by the Kubernetes API server in response to a request to create container resources received from the cluster agent.

Images