Method for scheduling and verifying tasks for artificial intelligence model and computing device for performing same
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- VESSL AI KOREA INC
- Filing Date
- 2025-06-20
- Publication Date
- 2026-07-30
Smart Images
Figure KR2025008541_30072026_PF_FP_ABST
Abstract
Description
Method for task scheduling and verification for an artificial intelligence model and a computing device for performing the same
[0001] The following embodiments relate to a task scheduling and verification method for an artificial intelligence model and a computing device for performing the same. More specifically, the following embodiments relate to a task scheduling and verification method for automating the verification of an artificial intelligence model and implementing a pipeline system for the learning process of an artificial intelligence model, and a computing device for performing the same.
[0002] Machine learning (ML) and artificial intelligence (AI) technologies are advancing day by day and reaching a stage where they are commercially viable. Accordingly, the use of complex techniques such as fine-tuning and evaluation of AI models is required to commercialize AI models. After undergoing the tuning and evaluation stages of the AI model, tasks such as retuning the model based on the evaluation results or deploying the tuned model production environment on an inference server for user use are performed.
[0003] In order for users to ultimately utilize an artificial intelligence model, organically linked tasks are required. If a small error occurs during the execution of linked tasks, it can affect subsequent tasks. Furthermore, due to the nature of machine learning tasks, errors made in previous tasks significantly increase GPU usage costs and time costs. Since there is a high possibility of error when each task of such an AI model is performed manually by humans, the automation of the automatic execution of linked tasks and the transmission of data and task results (or Context, Context Value) between linked tasks have become essential elements in performing tasks for AI learning.
[0004] Meanwhile, the development of the cloud and data center industries has led to the emergence of auto-scaling techniques that allow computing resources to be increased or decreased as needed. Maximizing work efficiency by scaling up resources as required and processing them in parallel for tasks that are not dependent on computing resources has become the prevailing trend. Pipeline-type workloads have traditionally existed in a form where they are processed from start to finish on a single computing resource, emphasizing only the specificity of dependent relationships between workloads. This implies that meaningless time resources were being wasted even in situations where workloads could be executed independently due to non-dependent relationships. For example, while data preprocessing and model quantization can be performed independently, conventional methods do not execute them separately but instead process them on a single computing resource, sometimes resulting in situations where one task must wait until the other is completed.
[0005] The background art of this disclosure is provided to facilitate understanding of this disclosure. It should not be understood as an acknowledgment that the matters described in the background art of this disclosure exist as prior art.
[0006] The configuration of control flow, which enables the execution of different tasks based on the results of small-scale detailed operations, is an essential element in the automation of tasks for artificial intelligence models. The purpose of control flow is to allow different tasks to be performed depending on the outcome of previous operations. While existing pipeline systems can also be used by explicitly specifying conditions for creating control flow, they have the problem of being highly susceptible to semantic errors. Here, a 'semantic error' refers to an error where a system operates in a direction different from its intended purpose.
[0007] A representative example of a semantic error is the reference to an undefined variable. While this error can occur due to simple typos when specifying conditions, it can also arise from variable references that fail to properly consider inter-task dependencies. For instance, consider a scenario where Task A must be executed first, followed by Task B, due to inter-task dependencies. In this case, if Task A attempts to reference variable x exposed by Task B, the variable x cannot be referenced, making pipeline execution impossible. In such situations, conventional solutions would either halt execution midway or perform undefined behavior, leading to incorrect results.
[0008] This can significantly reduce the efficiency of AI model development, such as by derailing the workflow of high-cost AI models and requiring the workflow to be executed again from scratch, and can cause serious problems, such as the exposure of erroneous AI models to users.
[0009] Furthermore, tasks such as data preprocessing often involve sensitive information and are subject to constraints requiring them to be performed within a specific pool of computing resources; in such situations, inefficiency arises where all other tasks utilize those specific resources. In other words, pipelined workloads suffer from inefficiency issues caused by the serial processing of tasks.
[0010] Accordingly, the inventors of the present disclosure have developed a method for specifying the contents of several organically connected sub-unit detailed tasks required for the development of an artificial intelligence model, a method for automatically performing said detailed tasks through said specification, a method for specifying and performing the control flow of the tasks, a method for automatically verifying whether the control flow conditions specified by an engineer do not cause errors through said specification, and a system for performing such tasks.
[0011] Accordingly, the embodiments are intended to provide a method for scheduling and verifying tasks for an artificial intelligence model and an apparatus for performing the same.
[0012] The problems that the present disclosure aims to solve are not limited to those described above, and problems not mentioned will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.
[0013] A method for scheduling and verifying tasks for an artificial intelligence model, performed by a processor according to one embodiment, may include the step of generating a task graph for a first pipeline of the artificial intelligence model, wherein the task graph includes a plurality of nodes and at least one edge; the step of verifying the task graph based on the plurality of nodes and the at least one edge; the step of scheduling at least one first workload corresponding to a node without dependencies using a topological alignment algorithm when the task graph is verified; the step of executing the at least one first workload; the step of scheduling at least one second workload corresponding to a node whose dependency with the at least one first workload has been released; the step of executing the at least one second workload; and the step of distributing the artificial intelligence model to a user device when all of the plurality of workloads corresponding to the plurality of nodes included in the task graph have been executed.
[0014] According to the features of the present disclosure, the step of verifying the work graph may include determining whether the work graph is a Directed Acyclic Graph (DAG) using a breadth-first search algorithm.
[0015] According to another feature of the present disclosure, the step of verifying the work graph may include determining whether a variable referenced by one of the plurality of nodes is a referenceable variable based on the at least one edge.
[0016] According to another feature of the present disclosure, the at least one edge indicates a dependency relationship between workloads and may be represented as a directional line between nodes.
[0017] According to another feature of the present disclosure, the plurality of nodes may include information regarding at least one variable or workload execution specification used in the first pipeline, received from the user device.
[0018] According to another feature of the present disclosure, the method may further include the step of transmitting the result of at least one of the plurality of workloads and the corresponding Software Development Kit (SDK) code to a node dependent on at least one of the plurality of workloads.
[0019] According to another feature of the present disclosure, one of the plurality of nodes is a node that generates a control flow, and the node that generates the control flow can generate the control flow by using a work result received from a node on which the node that generates the control flow depends.
[0020] According to another feature of the present disclosure, one of the plurality of nodes is a node that performs a second pipeline, and the node that performs the second pipeline can transmit a work result generated at the node that performs the second pipeline to the second pipeline.
[0021] According to another feature of the present disclosure, the method may further include the step of providing a user interface on a user device in which the operation graph is displayed.
[0022] A computing device for scheduling and verifying tasks of an artificial intelligence model according to one embodiment may include: memory; a transceiver configured to receive information regarding at least one variable or workload execution specification used in a first pipeline from a user device; and at least one processor including an artificial intelligence model. The processor may be configured to generate a task graph for the first pipeline of the artificial intelligence model, wherein the task graph includes a plurality of nodes and at least one edge, and verify the task graph based on the plurality of nodes and the at least one edge; when the task graph is verified, schedule at least one first workload corresponding to a node without dependencies using a topological alignment algorithm, execute the at least one first workload, schedule at least one second workload corresponding to a node whose dependency with the at least one first workload has been released, execute the at least one second workload, and when all of the plurality of workloads corresponding to the plurality of nodes included in the task graph have been executed, distribute the artificial intelligence model to the user device.
[0023] A computer program according to one embodiment may be a computer program stored on a non-transient computer-readable recording medium to perform a task scheduling and verification method for the artificial intelligence model described above, combined with a computer which is hardware.
[0024] A non-transient computer-readable recording medium according to one embodiment may include a computer program executed by a computer. The computer program may include the step of generating a task graph for a first pipeline of an artificial intelligence model, wherein the task graph includes a plurality of nodes and at least one edge; the step of verifying the task graph based on the plurality of nodes and the at least one edge; the step of scheduling at least one first workload corresponding to a node without dependencies using a topological alignment algorithm when the task graph is verified; the step of executing the at least one first workload; the step of scheduling at least one second workload corresponding to a node whose dependency with the at least one first workload has been released; the step of executing the at least one second workload; and the step of distributing the artificial intelligence model to a user device when all of the plurality of workloads corresponding to the plurality of nodes included in the task graph have been executed.
[0025] The means for solving the problem of the present disclosure are not limited to the means for solving the problem described above, and means for solving the problem not mentioned will be clearly understood by those skilled in the art to which the present disclosure belongs from the present specification and the attached drawings.
[0026] According to the embodiments, by implementing the automation of verification of an artificial intelligence model, the possibility of errors that may occur while a person is manually performing tasks through static analysis can be eliminated as much as possible, and thereby the stability of the artificial intelligence model can be improved.
[0027] According to the embodiments, by performing the tasks of an artificial intelligence model that can be performed in parallel as much as possible, the time required for the entire task of developing an artificial intelligence model can be minimized and the efficiency of the entire process can be increased.
[0028] According to the embodiments, in performing tasks necessary for developing an artificial intelligence model, a web interface that is easy for a developer or user to use is provided, and version control can be enabled using YAML (YAML Not Markup Language) code.
[0029] The effects according to the present disclosure are not limited to the effects described above, and unmentioned effects will be clearly understood by those skilled in the art from the present specification and the accompanying drawings.
[0030] The accompanying drawings, included as part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and explain the technical features of the present specification together with the detailed description.
[0031] FIG. 1 is a conceptual diagram showing an AI system for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure.
[0032] FIG. 2 is a block diagram showing the configuration of a server system for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure.
[0033] FIG. 3 is a flowchart for illustrating a machine learning pipeline according to one embodiment of the present disclosure.
[0034] FIG. 4 is a flowchart illustrating the task scheduling and task execution process of an artificial intelligence model using a task graph according to one embodiment of the present disclosure.
[0035] FIG. 5 is a conceptual diagram illustrating the process of verifying variables that can be referenced in nodes included in a work graph according to one embodiment of the present disclosure.
[0036] FIG. 6 is a conceptual diagram illustrating a work scheduling and execution process according to one embodiment of the present disclosure.
[0037] FIG. 7 is a screen showing a user interface in which a work graph according to one embodiment of the present disclosure is displayed.
[0038] FIG. 8 is a flowchart illustrating a task scheduling and verification method for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure.
[0039] Specific structural or functional descriptions of embodiments according to the concept of the present disclosure disclosed herein are provided merely for the purpose of explaining embodiments according to the concept of the present disclosure, and embodiments according to the concept of the present disclosure may be implemented in various forms and are not limited to the embodiments described herein.
[0040] Embodiments according to the concept of the present disclosure may be subject to various modifications and may take various forms; therefore, embodiments are illustrated in the drawings and described in detail in this specification. However, this is not intended to limit the embodiments according to the concept of the present disclosure to specific disclosed forms, and includes modifications, equivalents, or substitutions that fall within the spirit and scope of the present disclosure.
[0041] Terms such as "first" or "second" may be used to describe various components, but said components shall not be limited by said terms. For the sole purpose of distinguishing one component from another, for example, without departing from the scope of rights according to the concept of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component.
[0042] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. Conversely, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Expressions describing the relationships between components, such as "between," "exactly between," or "directly adjacent to," should be interpreted in the same way.
[0043] The terms used herein are used merely to describe specific embodiments and are not intended to limit the disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “comprising” or “having” are intended to specify the existence of the described features, numbers, steps, actions, components, parts, or combinations thereof, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.
[0044] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which this disclosure pertains. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.
[0045] For clarity in the interpretation of this specification, the terms used in this specification are defined below.
[0046] As used in this specification, the term 'AI system' refers to an AI system for task scheduling and verification. The AI system includes a server system responsible for the creation of AI models, post-deployment management, such as collecting and processing data, training AI models, inferring based on user requests, managing and distributing versions of AI models, and establishing pipelines for evaluating and retraining AI models.
[0047] As used herein, the term "pipeline" refers to an AI pipeline for the development and operation of artificial intelligence models, and signifies a workflow that connects a series of work processes step by step, ranging from data processing to the training, evaluation, and deployment of artificial intelligence models. For example, any pipeline may be a Machine Learning Operations (MLOps) pipeline for model operation automation. An MLOps pipeline is a workflow that connects the tasks of data collection, preprocessing, model training, model validation, model storage and management, Continuous Integration (CI) / Continuous Delivery / Continuous Deployment (CD), deployment, and monitoring step by step. As used herein, the term "task" refers to a task included in any pipeline. "Task" may be used interchangeably with "workload" and corresponds to a "node" in a "task graph." For example, any task may be data preprocessing, artificial intelligence model quantization, or model deployment.
[0048] As used herein, the term 'task graph' refers to a graph representing an arbitrary pipeline for the task scheduling and verification of an artificial intelligence model using nodes and edges. Nodes of the task graph represent the workload execution specifications of each task. Edges of the task graph are located between nodes and represent the dependency relationships between tasks corresponding to two nodes.
[0049] As used herein, the term "AI agent" refers to a software system that automates business tasks by calling external application programming interfaces (APIs) or services. Specifically, an AI agent may include a model, a tool, and / or memory. A model as a component of an AI agent may be a large language model that actually generates text or commands. A tool as a component of an AI agent may be a set of APIs or services that connect to the model and can perform various functions, such as database queries, document creation, and / or responding to customer inquiries. Memory as a component of an AI agent stores the history or state of the model's use of the tool, and / or the history or state of the model's generation of responses.
[0050] As used herein, the term "pseudo-code" refers to a form of code written in a human-readable manner to explain algorithms or logic, while not adhering to the syntax of an actual programming language. As used herein, the term "executable code" refers to a program composed of instructions that a computer can interpret or compile and actually execute. For example, code written in languages such as Python, JavaScript, C++, and Java may be executable code.
[0051] As used in this specification, the term "processor" may mean hardware capable of performing functions and operations according to each name described in this specification, computer program code capable of performing specific functions and operations, or an electronic recording medium loaded with computer program code capable of performing specific functions and operations.
[0052] In other words, the term "processor" may mean a functional and / or structural combination of hardware for carrying out the technical concept of the present disclosure and / or software for driving said hardware.
[0053] Hereinafter, embodiments will be described in detail with reference to the attached drawings. However, the scope of the patent application is not limited or restricted by these embodiments. Identical reference numerals in each drawing indicate identical components.
[0054] FIG. 1 is a conceptual diagram showing an AI system for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure.
[0055] Referring to FIG. 1, an AI system (1000) for scheduling and verifying tasks of an artificial intelligence model according to one embodiment of the present disclosure may include a user device (100) and / or a server system (200). For example, the AI system (1000) may fine-tune the artificial intelligence model upon a user's request. The AI system (1000) may verify and / or evaluate whether the artificial intelligence model was properly generated upon a user's request.
[0056] The user device (100) may be implemented in a PC (personal computer), server, or portable device. The portable device may be implemented as a laptop computer, mobile phone, smartphone, tablet PC, mobile internet device (MID), PDA (personal digital assistant), EDA (enterprise digital assistant), digital still camera (DSC), digital video camera, PMP (portable multimedia player), PND (personal navigation device or portable navigation device), handheld game console, e-book, or smart device. The smart device may be implemented as a smart watch, smart band, or smart ring.
[0057] A user device (100) is a device of a user who wishes to request an artificial intelligence model adapted for use by a customer from a server system (200) and to request verification of the artificial intelligence model. The user device (100) may include a smartphone, a tablet PC (Personal Computer), a laptop, and a PC, etc. Here, the user of the user device (100) refers to a developer or administrator who develops an artificial intelligence model using the server system (200). A customer refers to a person who purchases or uses the artificial intelligence model developed by the user of the user device (100). The user device (100) may install or run one or more of the Web, Mobile Application, and Program provided by the server system (200) for task scheduling and verification operations for the artificial intelligence model. For example, the user device (100) may access a web page and output a user interface screen that can create and modify a task graph containing various components. A task graph refers to a graph that displays an arbitrary pipeline for task scheduling and verification of the artificial intelligence model as nodes and edges.
[0058] The server system (200) can be implemented as a printed circuit board (PCB), such as a motherboard, an integrated circuit (IC), or a system on chip (SoC). For example, the server system (200) can be implemented as an application processor (AP).
[0059] The server system (200) may be a server of a service provider for developing and operating an artificial intelligence model that performs arbitrary requests from customers. The server system (200) may include a general-purpose computer, a laptop and / or a data server, etc. The server system (200) may provide one or more of the Web, a mobile application, and a program to the user device (100) to provide the user with a job scheduling and verification service for the artificial intelligence model so that the customer can ultimately use the artificial intelligence model. Specific details of the server system (200) will be described later in FIG. 2.
[0060] An artificial intelligence model according to an embodiment of the present disclosure may be a machine learning model or a large language model (LLM). Artificial intelligence models include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), GAN (Generative Adversarial Network), LSM (Liquid State Machine), ELM (Extreme Learning Machine), ESN (Echo State) It may include Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and / or AN (Attention Network).However, a person skilled in the art will understand that it may include any artificial intelligence model, not limited to those exemplified.
[0061] Up to now, an AI system (1000) for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure has been described in general terms.
[0062] According to the embodiments of the present disclosure, by implementing the automation of verification of an artificial intelligence model, the possibility of errors that may occur while a person performs manual work through static analysis can be eliminated as much as possible, and the stability of the artificial intelligence model can be improved.
[0063] According to the embodiments of the present disclosure, by performing the tasks of an artificial intelligence model that can be performed in parallel as much as possible, the time required for the entire task of developing an artificial intelligence model can be minimized and the efficiency of the entire process can be increased.
[0064] According to embodiments of the present disclosure, a web interface that is easy for a developer or user to use is provided for performing tasks necessary to develop an artificial intelligence model, and version control can be enabled using YAML (YAML Not Markup Language) code.
[0065] FIG. 2 is a block diagram showing the configuration of a server system for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure.
[0066] Referring to FIG. 2, the server system (200) may be a server that provides a service for scheduling and verifying the work of an artificial intelligence model in response to a request from a user or user device (100), or a computing device in which one or more of a web page, application, and program capable of performing a specific process for the development or operation of an artificial intelligence model are installed and executed. The server system (200) may include various types of server(s) for developing and / or operating an artificial intelligence model, such as a data processing server, a training server, an inference server, a model management server, and / or a data pipeline server.
[0067] The server system (200) may include a communication interface (210210), memory (220), an I / O (Input / Output) interface (230) and / or a processor (240), and each component may communicate with one or more communication buses or signal lines.
[0068] The communication interface (210) can be connected to the user device (100) via a wired / wireless communication network to exchange data. For example, the communication interface (210) can receive requests from the user device (100) for inference using an artificial intelligence model, requests for verification of an artificial intelligence model, requests for evaluation of an artificial intelligence model, and requests for fine-tuning of an artificial intelligence model.
[0069] Meanwhile, a communication interface (210) that enables the transmission and reception of such data may include a wired communication port (211) and a wireless circuit (212). Here, the wired communication port (211) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), and IEEE1394 (e.g., Apple’s FireWire, Sony’s i.Link, Texas Instruments’ Lynx). Additionally, the wireless circuit (212) may transmit and receive data with an external device via an RF (Radio Frequency) signal or an optical signal. Furthermore, wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, such as GSM (Global System for Mobile Communications), EDGE (Enhanced Data rates for GSM Evolution), CDMA (Code-Division Multiple Access), TDMA (Time Division Multiple Access), Bluetooth, Wi-Fi, VoIP, Wi-MAX, and any other suitable communication protocol.
[0070] The memory (220) can store various data used in the server system (200). For example, the memory (220) can store an artificial intelligence model. As another example, the memory (220) can store information necessary for learning an artificial intelligence model and inferring results according to one embodiment of the present disclosure. Additionally, the memory (220) can store information necessary for verifying and / or evaluating an artificial intelligence model.
[0071] In various embodiments, the memory (220) may include a volatile or non-volatile recording medium capable of storing various data, commands and / or information. For example, the memory (220) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), Random Access Memory (RAM), Static RAM (SRAM), Read-Only Memory (ROM), Programmable ROM (PROM), Electrically Erasable ROM (EEPROM), network storage, cloud, and blockchain database.
[0072] In various embodiments, the memory (220) may store at least one configuration of an operating system (221), a communication module (222), a user interface module (223), and one or more applications (224).
[0073] An operating system (221) (e.g., embedded operating systems such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers for controlling and managing general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.
[0074] The communication module (222) can support communication with another device through the communication interface (210). The communication module (222) may include various software components for processing data received by the wired communication port (211) or wireless circuit (212) of the communication interface (210).
[0075] The user interface module (223) can receive user requests or inputs from a keyboard, touch screen, keyboard, mouse, and / or microphone, etc., through the I / O interface (230) and provide a user interface on the display.
[0076] The application (224) may include a program or module configured to be executed by one or more processors (240). Here, the application providing job scheduling and verification services for handling all processes required for training an artificial intelligence model, generating a job graph, scheduling a workload, and / or executing a workload may be implemented on a server farm.
[0077] The I / O interface (230) can connect at least one of an input / output device (not shown) of the server system (200), such as a display, keyboard, touch screen, and microphone, to the user interface module (223). The I / O interface (230) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (223) and process commands based on the received input.
[0078] The processor (240) is connected to a communication interface (210), memory (220), and I / O interface (230) to control the overall operation of the server system (200), and can perform various commands for job scheduling and verification through applications and / or programs stored in memory (220).
[0079] The processor (240) may correspond to a computing device such as a CPU (Central Processing Unit) or an AP (Application Processor). Additionally, the processor (240) may be implemented in the form of an integrated chip (IC), such as a System on Chip (SoC) that integrates various computing devices. Alternatively, the processor (240) may include a module for computing artificial neural network models, such as a Neural Processing Unit (NPU).
[0080] FIG. 3 is a flowchart for illustrating a machine learning pipeline according to one embodiment of the present disclosure.
[0081] In FIG. 3, each step represented by a rectangle or a diamond shape represents a single workload. Solid arrows indicate dependencies between tasks (or workloads), and dotted arrows indicate the transmission flow of task results.
[0082] In this specification, a 'node without dependencies' means a node where no arrow points toward the node. A 'node with dependencies' means a node where at least one arrow points toward the node. In FIG. 3, 'model loading' and 'model lightweighting' are nodes without dependencies, and the remaining nodes are nodes with dependencies.
[0083] In this specification, 'a node dependent on the first workload (or the first node)' refers to a node connected to the first workload by a solid arrow and dependent on the work result of the first workload. For example, a node dependent on a workload that loads a model is a workload that fine-tunes a model. Conversely, 'a node dependent on the second workload (or the second node)' refers to a node connected to the second workload by a solid arrow and transmitted work results to the second workload. The second workload refers to a node dependent on the work result of the 'node dependent on the second workload'. For example, a node dependent on a workload that fine-tunes a model is a workload that loads a model.
[0084] Referring to FIG. 3, a server system (200) according to one embodiment of the present disclosure can load an artificial intelligence model (S300). In other words, the server system (200) can load a stored artificial intelligence model file or data (e.g., learned weights and the architecture of the artificial intelligence model) into memory and prepare it in an executable state. As a result of the work of the workload corresponding to model loading, for example, the name of the artificial intelligence model (modelName=Llama-3.3-70B-Instruct) can be transmitted to a workload that uploads and distributes the artificial intelligence model.
[0085] A server system (200) according to one embodiment of the present disclosure can quantize an artificial intelligence model (S301). Step S301 has no dependency on Step S300, which loads the artificial intelligence model. Therefore, Step S301 can be performed in parallel with Step S300.
[0086] A server system (200) according to one embodiment of the present disclosure can fine-tune the artificial intelligence model when the artificial intelligence model loading and artificial intelligence model lightweighting are completed (S302).
[0087] A server system (200) according to one embodiment of the present disclosure can evaluate fine tuning when the fine tuning of an artificial intelligence model is completed (S303). For example, the server system (200) can evaluate whether a formula or criterion for evaluating the stability of the fine-tuned artificial intelligence model has been appropriately set, and can calculate the evaluation result as an evaluation score. An evaluation score (score=0.92), for example, 0.92 points, which is the result of a workload corresponding to the fine tuning evaluation, can be transmitted to a workload that determines the evaluation score.
[0088] A server system (200) according to one embodiment of the present disclosure can determine whether the evaluation score based on the evaluation result is 0.9 points or higher when the fine-tuning evaluation of the artificial intelligence model is completed (S304). Here, the 0.9 points used as the criterion for judgment is merely one embodiment, and the criterion score can be set in various ways depending on the case.
[0089] A server system (200) according to one embodiment of the present disclosure may notify a user device (100) of the failure of fine tuning if, as a result of the judgment in step S304, it is determined that the evaluation score is less than 0.9 points (S304, No). (S305).
[0090] A server system according to one embodiment of the present disclosure can upload and distribute an artificial intelligence model if, as a result of the judgment in step S304, it is determined that the evaluation score is 0.9 points or higher (S304, Yes) (S306).
[0091] As described above, organically linked tasks are required to enable customers to ultimately use an artificial intelligence model. According to embodiments of the present disclosure, linked tasks can be performed automatically, tasks that can be performed independently or in parallel can be allocated in parallel to available computing resources, and the transmission of data and task results between linked tasks can be automated.
[0092] FIG. 4 is a flowchart illustrating the task scheduling and task execution process of an artificial intelligence model using a task graph according to one embodiment of the present disclosure.
[0093] A server system (200) according to one embodiment of the present disclosure can generate a work graph for a specific pipeline (S400).
[0094] A specific pipeline may include at least one task. A specific pipeline refers to a pipeline whose execution is determined by a user's request. The user can specify the execution of the workload and define variables used throughout the specific pipeline using a pre-determined YAML (YAML Not Markup Language) structure or a web interface.
[0095] A node represents a workload corresponding to a single task for an artificial intelligence model and can be indicated as a rectangle in the task graph. An edge represents a dependency relationship between tasks (or workloads) and can be indicated as a solid arrow in the task graph. A specific example of a task graph is described later in Fig. 7.
[0096] The execution specifications of user-created workloads are represented as nodes in the task graph, and dependencies between workloads can be indicated by directed edges between nodes. Assuming an arrow starts at Task A and ends at Task B (A→B), it means that the workload corresponding to Task A must be completed before the workload for Task B can execute. The workload for Task B can receive the results of Task A in the form of environment variables or subdirectories, and the results of Task A can be specified in the execution specification of the workload for Task B.
[0097] In one embodiment, the first node may be a node that generates a control flow. The node that generates the control flow may generate the control flow using the work result received from the node on which the node that generates the control flow depends.
[0098] In one embodiment, if final user confirmation is required (Human-in-the-loop), the user can create an execution specification that enables final confirmation by configuring or adding a separate node for the corresponding task.
[0099] A server system (200) according to one embodiment of the present disclosure can analyze dependency relationships between nodes (S401).
[0100] Additionally, a server system (200) according to one embodiment of the present disclosure can analyze dependency relationships between pipelines (S402). In other words, the server system (200) can analyze dependency relationships between a specific pipeline determined to be executed by a user and other pipelines.
[0101] In one embodiment, the second node may be a node that causes a pipeline different from the specific pipeline determined to be executed by the user to be executed. Here, the specific pipeline determined to be executed by the user may be referred to as the 'first pipeline', and the pipeline different from the specific pipeline determined to be executed by the user may be referred to as the 'second pipeline'. That is, the second node may be a node that causes the second pipeline to be executed. The server system (200) may specify which variables to transmit as work results to the second pipeline. For example, the server system (200) may transmit results exposed by the second node, pipeline variables, and / or constants, etc., to the second pipeline. If the first pipeline and the second pipeline mutually call each other and are executed in an infinite loop, for example, if a cycle is created between pipelines in such a way that the calling second pipeline calls the original pipeline, the first pipeline, the server system (200) may reject the creation of the pipeline through static analysis.
[0102] The completed task graph must not be an infinite loop or an unmanageable graph. That is, the task graph may be a Directed Acyclic Graph (DAG) containing edges. The server system (200) can verify whether the task graph is a DAG by utilizing a Breadth-First Search (BFS) algorithm.
[0103] A server system (200) according to an embodiment of the present disclosure can verify whether a variable referenced by any node is a variable that can be referenced. Specifically, the server system (200) can verify whether a variable referenced by any node is a variable that can be referenced according to a causal relationship or dependency relationship between workloads.
[0104] In one embodiment, the server system (200) may verify whether a variable referenced by any node is a referenceable variable based on the scope of any node, the scope of arguments, and / or the definition of the task result. The scope of any node may refer to a set of strings or logical subdirectories exposed to other nodes as a result of the execution of that node. For example, if a variable named 'result' and a subdirectory named 'data' are exposed as a result of task A and other nodes intend to utilize the data, the scope of task A may be defined as {result, / data}. The scope of arguments may refer to a set of variables and logical subdirectories injected from the outside during the execution of tasks included in the pipeline. The task result (or context) of any node may be the union of the scopes of the nodes preceding that node and the scope of arguments. The nodes preceding that node refer to nodes that must necessarily be executed before that node. For example, if node X must necessarily be executed before node A due to a causal relationship, node X corresponds to the previous node of node A. For example, in the case of a causal relationship A->B->C, the result of the operation C can be defined as the union of the range of node A, the range of node B, and the range of the factor.
[0105] In another embodiment, a topological sort algorithm is performed on the task graph to simulate from a node without dependencies to the completion of the last task, thereby verifying whether a variable referenced by any node is a referenceable variable. For example, referring to FIG. 5, assume a case where a referenceable variable is verified at a node (501) corresponding to task A.
[0106] The server system (200) checks whether a variable or logical subdirectory referenced by Task A exists in the context of Task A, and if it does not exist, it may reject the variable or logical subdirectory. The node (502) corresponding to Task B may use the results of tasks that must be performed prior to Task B to drive the workload. If the node (502) corresponding to Task B references a variable of the node (503) corresponding to Task C, which has no causal or dependent relationship with Task B, the server system (200) may reject the variable. The server system (200) may generate a new task result that includes both the task result of Task A and the scope generated after the completion of Task A's execution. The newly generated task result for Task A may be defined as C'. For a node dependent on Task A, for example, the node (502) corresponding to Task B, the task result of Task B may be newly generated or updated by including both the task result (Ctx(B)) of the node (502) and the task result generated in the previous step of the node (502).
[0107] Referring again to FIG. 4, the specification of at least one task included in a specific pipeline can be generated as an executable specification through deployment. After deployment, the specification can be configured so that it cannot be edited by the user. This is to prevent the specification from being suddenly changed when there is a task being performed based on the specification.
[0108] A server system (200) according to one embodiment of the present disclosure can schedule tasks (S403) and can perform scheduled tasks (S404).
[0109] The server system (200) can schedule a workload corresponding to each node for all nodes that are not dependent on each other. Referring to FIG. 6, for example, the scheduling of a node (604) corresponding to task D can be performed by publishing a message (6_1) using a message queue (60). The server system (200) can retrieve and verify a schedule message (6_2) from the message queue (60) according to the priority of the message queue (60), and request the execution of a task from a designated computing resource pool according to the workload specifications specified in the schedule message (6_2). Upon the request for task execution, the task can be scheduled and executed on a computing resource through an execution agent within the computing resource pool.
[0110] The workload being performed can specify the result value of the workload to be exposed through the node range after the operation through the SDK (Software Development Kit) code. The SDK code is specified as key-value pairs and can be reflected in the node range configured in the form of a dictionary. When the operation result is exposed in the form of a subdirectory, the server system (200) can back up the operation result to the data storage (62) connected to the server system (200) as soon as the workload being performed is finished, and deliver the result by having the workload download the backup.
[0111] In one embodiment, the server system (200) can schedule tasks by performing a topological alignment algorithm. For example, the server system (200) can resolve all causal relationships starting from the node (604) corresponding to task D after the completion of the workload corresponding to task D. Thus, when a node (e.g., the node (601) corresponding to task A) is released from dependency with a specific node (e.g., the node (604) corresponding to task D), the workload corresponding to the node can be scheduled.
[0112] In one embodiment, in a task specified to require user confirmation or input value, the server system (200) may stop the control flow and switch the state of the corresponding node to the AWAITING_INPUT state. For example, if the task specified to require user confirmation or input value is a node (601) corresponding to task A, the user may input variables required for the task using an HTTP Callback API that can switch the state. Variables input via the HTTP Callback API are included in the task result of the node, and, as with other nodes, the server system (200) may perform a topological alignment algorithm on task A and continue the workload.
[0113] FIG. 7 is a screen showing a user interface in which a work graph according to one embodiment of the present disclosure is displayed.
[0114] FIG. 7 is a screen (700) showing a user interface that allows a user to generate a task graph in order to perform tasks necessary for developing an artificial intelligence model.
[0115] On the screen (700) representing the user interface, a plurality of nodes and edges connecting the plurality of nodes are displayed. Referring to FIG. 7, the first node (70) is a node without dependencies and is a node that fine-tunes using QLoRA (Quantized Low-Rank Adapter). The second node (71) and the third node (72), which are nodes dependent on the first node (70), are a node that inserts an additional adapter layer and a node that quantizes the model, respectively.
[0116] The screen (700) displaying the user interface also displays various tools (702) that the user needs to create a work graph.
[0117] According to the embodiments, a web interface that is easy for a developer or user to use is provided, thereby facilitating the work required to develop an artificial intelligence model.
[0118] FIG. 8 is a flowchart illustrating a task scheduling and verification method for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure.
[0119] According to a method for scheduling and verifying tasks for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can generate a task graph for a first pipeline of the artificial intelligence model (S800). The task graph may include a plurality of nodes and at least one edge.
[0120] At least one edge indicates the dependency relationship between workloads and can be represented as a directed line between nodes.
[0121] Multiple nodes may include information regarding at least one variable or workload execution specification used in the first pipeline received from the user device (100).
[0122] According to a method for scheduling and verifying an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can transmit the result of at least one of a plurality of workloads and the corresponding SDK (Software Development Kit) code to a node that depends on at least one of the plurality of workloads.
[0123] In one embodiment, one of the plurality of nodes is a node that generates a control flow. The node that generates the control flow can generate the control flow by using the work result received from the node on which the node that generates the control flow depends.
[0124] In another embodiment, one of the plurality of nodes is a node that performs a second pipeline. The node that performs the second pipeline can transmit the work result generated by the node that performs the second pipeline to the second pipeline.
[0125] According to a method for scheduling and verifying tasks for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can provide a user interface displaying a task graph to a user device (100).
[0126] According to a method for task scheduling and verification for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can verify a task graph based on a plurality of nodes and at least one edge (S802).
[0127] In one embodiment, the server system (200) can determine whether the work graph is a DAG containing edges by utilizing a breadth-first search algorithm based on the results of analyzing dependency relationships between nodes or dependency relationships between pipelines. Alternatively, the server system (200) can determine whether a variable referenced by any node is a variable that can be referenced according to causal or dependency relationships between workloads based on the results of analyzing dependency relationships between nodes or dependency relationships between pipelines.
[0128] According to a method for scheduling and verifying an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can schedule at least one first workload corresponding to a node without dependencies when the task graph is verified (S804).
[0129] According to a method for scheduling and verifying work for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can execute at least one first workload (S806).
[0130] According to a method for scheduling and verifying an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can schedule at least one first workload and at least one second workload corresponding to a node with released dependencies (S808).
[0131] According to a method for scheduling and verifying work for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can execute at least one second workload (S810).
[0132] According to a method for scheduling and verifying tasks for an artificial intelligence model performed by a processor according to one embodiment of the present disclosure, a server system (200) can distribute an artificial intelligence model to a user device (100) when all of the multiple workloads corresponding to multiple nodes included in a task graph have been performed (S812).
[0133] A computing device for scheduling and verifying tasks of an artificial intelligence model according to one embodiment of the present disclosure may include at least one processor comprising a memory, a transceiver configured to receive information regarding at least one variable or workload execution specification used in a first pipeline from a user device (100), and / or an artificial intelligence model. The processor may be configured to generate a task graph for the first pipeline of the artificial intelligence model—the task graph comprises a plurality of nodes and at least one edge—verify the task graph based on the plurality of nodes and the at least one edge, and when the task graph is verified, schedule at least one first workload corresponding to a node without dependencies using a topological alignment algorithm, execute the at least one first workload, schedule at least one second workload corresponding to a node that has released dependencies with the at least one first workload, execute the at least one second workload, and when all of the plurality of workloads corresponding to the plurality of nodes included in the task graph are executed, distribute the artificial intelligence model to the user device.
[0134] A computing device for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure may include a computer program stored on a computer-readable recording medium to perform steps S800 to S812 of FIG. 8 described above, combined with a computer (or computing device) which is hardware.
[0135] A computing device for task scheduling and verification of an artificial intelligence model according to one embodiment of the present disclosure may be implemented as a computing device comprising at least one processor that executes instructions of programs loaded in memory. A program comprising instructions described to execute steps S800 to S812 of FIG. 8 described above may be loaded in the memory of the computing device.
[0136] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include a plurality of processing elements and / or a plurality of types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.
[0137] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.
[0138] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the embodiment, or may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.
[0139] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.
[0140] Therefore, other implementations, other embodiments, and equivalents to the claims also fall within the scope of the claims set forth below.
[0141] The task scheduling and verification method for an artificial intelligence model as described above and the computing device for performing the same can be applied to fields utilizing machine learning models and artificial intelligence models.
Claims
1. A method for task scheduling and verification for an artificial intelligence model performed by a processor, A step of generating a work graph for the first pipeline of an artificial intelligence model, wherein the work graph includes a plurality of nodes and at least one edge; A step of verifying the work graph based on the plurality of nodes and the at least one edge; If the above task graph is verified, a step of scheduling at least one first workload corresponding to a node without dependencies using a topological alignment algorithm; Step of executing at least one first workload; A step of scheduling at least one second workload corresponding to a node whose dependency with the at least one first workload has been released; Step of executing at least one second workload; and A step of distributing the artificial intelligence model to a user device when all of the plurality of workloads corresponding to the plurality of nodes included in the above work graph have been executed; method.
2. In Paragraph 1, The step of verifying the above work graph is, A step of determining whether the above work graph is a Directed Acyclic Graph (DAG) using a breadth-first search algorithm; comprising method.
3. In Paragraph 1, The step of verifying the above work graph is, A step of determining whether a variable referenced by one of the plurality of nodes is a referenceable variable based on at least one edge; comprising method.
4. In Paragraph 1, The above at least one edge indicates a dependency relationship between workloads and is represented by a directional line between nodes, method.
5. In Paragraph 4, The plurality of nodes include information regarding at least one variable or workload execution specification used in the first pipeline, received from the user device. method.
6. In Paragraph 1, The method further comprises the step of transmitting the result of at least one of the plurality of workloads and the corresponding SDK (Software Development Kit) code to a node dependent on at least one of the plurality of workloads. method.
7. In Paragraph 1, One of the above plurality of nodes is a node that generates control flow, and The node generating the control flow generates the control flow by utilizing the work result received from the node on which the node generating the control flow depends. method.
8. In Paragraph 1, One of the above plurality of nodes is a node that performs the second pipeline, and A node that enables the execution of the second pipeline transmits a work result generated by the node that enables the execution of the second pipeline to the second pipeline. method.
9. In Paragraph 1, The method further comprises the step of providing a user interface on a user device in which the above-mentioned operation graph is displayed. method.
10. As a computing device for task scheduling and verification of an artificial intelligence model, Memory; A transceiver configured to receive information about at least one variable or workload execution specification used in the first pipeline from a user device; and at least one processor including an artificial intelligence model; and The above processor is, A task graph for the first pipeline of the above artificial intelligence model is generated, wherein the task graph includes a plurality of nodes and at least one edge, and Based on the plurality of nodes and at least one edge, the work graph is verified, and If the above task graph is verified, at least one first workload corresponding to a node without dependencies is scheduled using a topological alignment algorithm, and Execute at least one first workload, and schedule at least one second workload corresponding to a node whose dependency with the at least one first workload has been released, The above at least one second workload is executed, and when all of the plurality of workloads corresponding to the plurality of nodes included in the work graph are executed, the artificial intelligence model is configured to be deployed to the user device. Computing device.
11. A method for performing task scheduling and verification for the artificial intelligence model of claim 1, combined with a computer which is hardware, and stored on a non-transient computer-readable recording medium, Computer program.
12. A non-transient computer-readable recording medium on which a computer program executed by a computing device is recorded, The above computer program is, A step of generating a work graph for the first pipeline of an artificial intelligence model, wherein the work graph includes a plurality of nodes and at least one edge; A step of verifying the work graph based on the plurality of nodes and the at least one edge; If the above task graph is verified, a step of scheduling at least one first workload corresponding to a node without dependencies using a topological alignment algorithm; Step of executing at least one first workload; A step of scheduling at least one second workload corresponding to a node whose dependency with the at least one first workload has been released; Step of executing at least one second workload; and A step of distributing the artificial intelligence model to a user device when all of the plurality of workloads corresponding to the plurality of nodes included in the above work graph have been executed; Non-transient computer-readable recording medium.