Electronic device for scheduling execution of artificial intelligence model, method therefor, and recording medium recording program therefor

Abstract

An electronic device according to various embodiments may comprise at least one processor and a memory. The memory may store a computer program including a plurality of artificial intelligence (AI) models and computer-executable instructions. The instructions may be individually or collectively executed by the at least one processor and instruct the electronic device to: obtain a response time function obtained by performing machine learning on training data including a response time according to a result of executing an AI pipeline including at least one of the plurality of AI models; identify the AI pipeline included in a queue; determine an occupancy time to be allocated to at least one AI model included in the AI pipeline, on the basis of the response time function; and cause the at least one AI model to be executed within the occupancy time. Various implementations other than the above embodiment are possible.

Description

An electronic device for performing scheduling to execute an artificial intelligence model, a method thereof, and a recording medium recording the program thereof.

[0001] The present disclosure relates to an electronic device for performing scheduling to execute an artificial intelligence model, a method thereof, and a recording medium recording the program thereof.

[0002] Artificial intelligence (AI) technology is being applied to mobile devices. Functions utilizing AI models can be provided in real time on various mobile devices, such as smartphones, tablets, and wearable devices. For example, various AI-based services, such as image recognition, speech recognition, natural language processing, and recommendation systems, can be actively used in mobile environments. Since these AI models require high computational performance, processing units such as GPUs (graphics processing units) suitable for parallel processing or NPUs (neural processing units) optimized for neural network computations can be utilized.

[0003] GPUs in mobile devices can play a crucial role in efficiently processing computational tasks within AI pipelines. However, mobile device resources (e.g., GPU and NPU resources) are limited, and various constraints may exist. For instance, factors such as battery capacity, heat generation, and limited physical space can significantly impact resource allocation and utilization. Furthermore, since various applications and system services simultaneously share GPU resources while running AI models on mobile devices, efficient resource allocation is required.

[0004] In mobile environments, various tasks comprising an AI pipeline require GPU resources, and these tasks may have different requirements. For instance, the time constraints for providing a response may vary depending on the service level intended to be delivered through the AI ​​pipeline. For instance, tasks requiring high-capacity computation, such as image or video processing, may run simultaneously with tasks requiring low computational power, such as simple text processing. Allocating GPU resources in a fixed manner without considering the demands of these diverse tasks, or assigning resources without priority, can degrade overall system performance and significantly reduce the efficiency of mobile devices.

[0005] Resource consumption, such as that of GPUs, can vary depending on the size and complexity of the model and the amount of data to be processed. Therefore, resource allocation needs to be dynamically adjusted to meet the real-time requirements of the AI ​​pipeline. However, existing resource allocation technologies often rely on fixed allocation methods or fail to consider the resource constraints of mobile devices, which may result in a lack of solutions for efficient resource management.

[0006] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art in relation to the present disclosure.

[0007] An electronic device according to one embodiment may include at least one processor and a memory. The memory may store a computer program comprising a plurality of artificial intelligence (AI) models and computer-executable instructions. The instructions may be executed individually or collectively by the at least one processor to obtain a response time function obtained by performing machine learning on training data, which includes a response time resulting from the execution of an AI pipeline comprising at least one of the plurality of AI models. The instructions may be executed individually or collectively by the at least one processor to enable the electronic device to identify the AI ​​pipeline included in a queue. The instructions may be executed individually or collectively by the at least one processor to enable the electronic device to determine the occupancy time to be allocated to at least one AI model included in the AI ​​pipeline based on the response time function. The above instructions may be executed individually or collectively by the at least one processor, so that the electronic device can execute the at least one AI model within the said occupancy time.

[0008] In one embodiment, a method of operating an electronic device may include an operation of obtaining a response time function obtained by performing machine learning on training data including a response time resulting from executing an AI pipeline comprising at least one of a plurality of AI models. The method may include an operation of identifying the AI ​​pipeline included in a queue. The method may include an operation of determining a occupancy time to be allocated to the at least one AI model included in the AI ​​pipeline based on the response time function. The method may include an operation of executing the at least one AI model within the occupancy time.

[0009] In a computer-readable non-transient recording medium storing a computer program comprising computer-executable instructions, the instructions may enable the electronic device to perform an operation of obtaining a response time function obtained by performing machine learning on training data, which includes a response time resulting from the electronic device executing an AI pipeline comprising at least one of a plurality of AI models when the electronic device is executed. The instructions may enable the electronic device to perform an operation of identifying the AI ​​pipeline included in a queue when the electronic device is executed. The instructions may enable the electronic device to include an operation of determining, when the electronic device is executed, a occupancy time to be allocated to the at least one AI model included in the AI ​​pipeline based on the response time function. The instructions may enable the electronic device to perform an operation of executing the at least one AI model within the occupancy time when the electronic device is executed.

[0010] FIG. 1 is a block diagram illustrating the configuration of an electronic device according to one embodiment of the present disclosure.

[0011] FIG. 2 illustrates the concept of an AI pipeline according to one embodiment of the present disclosure.

[0012] FIG. 3 illustrates the concept of a unit interval and occupancy time according to one embodiment of the present disclosure.

[0013] FIG. 4 illustrates the configuration of a queue according to one embodiment of the present disclosure.

[0014] FIG. 5 is a flowchart illustrating a process in which an electronic device according to one embodiment of the present disclosure executes an AI pipeline.

[0015] FIG. 6 is a flowchart illustrating a process in which an electronic device according to one embodiment of the present disclosure obtains a response time function.

[0016] FIG. 7 is a flowchart illustrating an example of a process for determining whether an electronic device according to one embodiment of the present disclosure will perform machine learning.

[0017] FIG. 8 is a flowchart illustrating a process in which an electronic device according to one embodiment of the present disclosure acquires learning data and performs machine learning.

[0018] FIG. 9 illustrates an example of learning data acquired by an electronic device according to one embodiment of the present disclosure.

[0019] FIG. 10 is a flowchart illustrating a process in which an electronic device according to one embodiment of the present disclosure obtains a response time function through a server.

[0020] FIG. 11 is a flowchart illustrating a process in which an electronic device according to one embodiment of the present disclosure drives an AI pipeline.

[0021] FIG. 12 is a block diagram of an electronic device in a network environment according to various embodiments.

[0022] Hereinafter, embodiments are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the contents of this disclosure. However, the disclosed embodiments may be implemented in various different forms and are not limited to the embodiments described herein.

[0023] Technology is required for the efficient allocation and management of GPU resources for AI pipelines in mobile device environments. This enables the optimization of system performance by identifying the requirements of various AI tasks in real time and dynamically adjusting GPU resources. Furthermore, technical approaches are needed to minimize battery consumption, resolve heat generation issues, and reduce GPU resource waste.

[0024] One embodiment of the present disclosure may be intended to provide an electronic device and a method of operation thereof that allocates resources to satisfy response time requirements required to provide AI services while using minimal resources of the electronic device for an AI pipeline executed to provide AI services.

[0025] One embodiment of the present disclosure may be intended to minimize the sum of the processing times required to process each AI pipeline while satisfying the response time requirements of a service requested by a user, in the case where there are multiple AI pipelines including at least one AI model operating in an electronic device.

[0026] The technical problems to be solved in this document are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure pertains.

[0027] FIG. 1 is a block diagram (100) illustrating the configuration of an electronic device (101) according to one embodiment of the present disclosure.

[0028] In one embodiment, an electronic device (101) (e.g., the electronic device (1201) of FIG. 12) may include at least one processor (120) (e.g., the processor (1220) of FIG. 12) and a memory (130) (e.g., the memory (1230) of FIG. 12). The memory (130) may store a plurality of AI models (131, 132). In the present disclosure, an AI model may mean a model configured to machine learn information and infer a prediction result based on the learning result. At least one processor (120) may include a processing circuit (121) capable of processing operations to drive a plurality of AI models (131, 132). For example, the processing circuit (121) may include a GPU for processing computer graphics or an NPU configured to perform neural network operations.

[0029] In one embodiment, the memory (130) may store a computer program containing instructions that are executable on a computer. At least one processor (120) may execute the instructions stored in the memory (130) to perform operations or control the operation of a component of the electronic device (101). In the present disclosure, the operation of the electronic device (101) may be understood as being performed by at least one processor (120) executing the instructions stored in the memory (130).

[0030] In one embodiment, at least one processor (120) can configure a response time function learning module and a resource calculation module by executing computer-executable instructions stored in memory (130). The response time function learning module can learn a response time function to automatically discover resources that allow the workload of the AI ​​pipeline to satisfy the service requirement value of the service to be provided through the AI ​​pipeline, based on the number of user requests changing in real time. The service requirement value may refer to the maximum value of time that may be required to provide a response to provide the service through the AI ​​pipeline. The response time function learning module can collect a dataset by measuring response times based on the number of various user requests and the number of instances per AI model. The response time function learning module can determine the response time function by performing machine learning on training data that includes at least a portion of the dataset. The response time function can be used to calculate the response time distribution for the number of user requests and the number of instances of the AI ​​model. The response time function learning module can perform machine learning on the response time function whenever a new AI model is added or updated to the electronic device (101). The resource calculation module can determine the resources to be dynamically allocated per AI model according to a user request based on a response time function. The resource calculation module can determine the minimum resources that ensure the response time of the AI ​​pipeline is less than or equal to the service requirement value. The electronic device (101) can execute the requested AI model based on the determined resources. In the present disclosure, an AI pipeline may define a series of processes for executing at least one AI model to provide an AI service. The AI ​​pipeline may include tools and data delivery means for providing an AI service.

[0031] In one embodiment, the electronic device (101) may acquire training data including response times resulting from the execution of an AI pipeline comprising at least one of a plurality of AI models (131, 132). The electronic device (101) may set a occupancy time for a resource (e.g., processing circuit (121)) for each AI model constituting the AI ​​pipeline. For example, the electronic device (101) may determine the occupancy time for the AI ​​pipeline to occupy the resource within a unit time for performing scheduling. The electronic device (101) may determine the occupancy time based on the AI ​​models constituting the AI ​​pipeline. For example, if the AI ​​pipeline includes a first AI model and a second AI model, and the minimum occupancy time specified for the first AI model is 0.07 and the minimum occupancy time specified for the second AI model is 0.03, the electronic device (101) may determine the minimum occupancy time for the AI ​​pipeline to be 0.1. The electronic device (101) can drive an AI pipeline based on a determined occupancy time and obtain a response time taken to obtain a response from the driven AI pipeline. The electronic device (101) can perform the operation of driving the AI ​​pipeline to obtain a response time multiple times while gradually increasing the occupancy time from the minimum occupancy time. The electronic device (101) can obtain training data including pairs of occupancy time and response time. The training data may also include other information. For example, the electronic device (101) may further include information about other processes occupying resources or information about the state of the electronic device (101) (e.g., a running process, a temperature measured through a temperature sensor included in the electronic device (101)).

[0032] In one embodiment, the electronic device (101) can obtain a response time function by performing machine learning on training data. The response time function may include a tail value function of the response time distribution for the occupancy time of each AI model. Various types of artificial neural networks may be used for the response time function. For example, the electronic device (101) may perform machine learning using a fully-connected layer (FC) neural network. For example, the electronic device (101) may perform machine learning using a graph neural network (GNN) capable of effectively performing representation of a graph-shaped workload. However, the method of performing machine learning is not limited to the examples described above. The electronic device (101) may perform machine learning through supervised learning using the tail value of the response time distribution as a label.

[0033] In one embodiment, when a plurality of AI pipelines are configured in memory (130), the electronic device (101) can perform machine learning to obtain response time functions for each of the plurality of AI pipelines. The electronic device (101) can store the response time functions for the plurality of AI pipelines in memory (130) by assigning an index value set for each AI pipeline.

[0034] In one embodiment, the electronic device (101) may allocate a occupancy time for at least one AI pipeline based on a response time function for at least one AI pipeline as the operation of at least one AI pipeline is requested. For example, the electronic device may determine a occupancy time for an AI pipeline requested to execute an AI assistant application as user input (e.g., a voice signal received through an audio module (1270), text entered through an input module (1250), or an image received through a camera module (1280)) is received to execute an AI assistant application that sequentially calls multiple AI models, including a large language model (LLM) and an embedding model. The electronic device (101) may control a processing circuit (121) to execute the AI ​​pipeline within the allocated occupancy time.

[0035] FIG. 2 illustrates the concept of an AI pipeline (200) according to one embodiment of the present disclosure.

[0036] In one embodiment, the AI ​​pipeline (200) may include at least one AI model. Referring to FIG. 2, if the AI ​​pipeline (200) includes a plurality of AI models (131, 132), the AI ​​pipeline (200) may include an edge (210) connecting at least two of the plurality of AI models (131, 132). However, it is not limited thereto. For example, the AI ​​pipeline (200) may include a greater number of AI models and edges than shown in FIG. 2.

[0037] FIG. 3 illustrates the concept of a unit interval (300) and a occupancy time (310) according to one embodiment of the present disclosure.

[0038] In one embodiment, the electronic device (101) can drive an AI pipeline for an allocated occupancy time (310) within a unit interval (300). The electronic device (101) can drive at least one AI pipeline by repeating the unit interval (300). The occupancy time (310) may indicate the ratio of the interval during which the AI ​​pipeline is driven within the unit interval (300). For example, if the electronic device (101) repeats the operation of driving the AI ​​pipeline for 3 ms every 10 ms, the occupancy time (310) allocated to the AI ​​pipeline may be referred to as 0.3.

[0039] FIG. 4 illustrates the configuration of a queue (400) according to one embodiment of the present disclosure.

[0040] In one embodiment, the electronic device (101) may insert at least one requested AI pipeline (411, 412) into a queue (400) to provide AI services. The queue (400) may include a data structure having a FIFO (first in, first out) structure in which the data inserted first is output and deleted first. In one embodiment, the queue (400) may be replaced with a different type of data structure. For example, the queue (400) may be replaced with a stack having a LIFO (last in, first out) structure in which the data inserted last is output and deleted first.

[0041] In one embodiment, the electronic device (101) can identify at least one AI pipeline (411, 412) included in the queue (400). The electronic device (101) can determine the occupancy time to be allocated to each of the at least one AI pipeline (411, 412) based on a response time function to execute the at least one AI pipeline (411, 412) included in the queue (400). The execution time to be allocated to each AI pipeline can be dynamically re-determined based on the addition of AI pipelines to the queue (400) or the removal of at least some of the AI ​​pipelines to be executed.

[0042] FIG. 5 is a flowchart (500) illustrating a process in which an electronic device (101) according to one embodiment of the present disclosure executes an AI pipeline.

[0043] According to one embodiment, in operation 510, the electronic device (101) may obtain a response time function through machine learning on training data containing information about the response time of an AI pipeline according to the occupancy time. In one embodiment, the method by which the electronic device (101) obtains training data may be configured in various ways. For example, when the electronic device (101) executes an AI pipeline in response to a user request, it may obtain training data including the occupancy time allocated to execute the AI ​​pipeline and the response time of the executed AI pipeline. For example, the electronic device (101) may obtain training data including the response time by intentionally driving the AI ​​pipeline regardless of the user request. In one embodiment, the electronic device (101) may obtain a response time function regarding the number of user requests and the number of instances per AI model through machine learning. Operation 510 may be performed in advance before the electronic device (101) allocates occupancy time to the AI ​​pipeline to drive the AI ​​pipeline.

[0044] According to one embodiment, in operation 520, the electronic device (101) can identify an AI pipeline included in a queue. For example, when the electronic device (101) inputs a prompt to request an AI service based on an application executed by a user, the electronic device (101) can insert an AI pipeline for calling at least one AI model corresponding to the AI ​​service into a queue (e.g., queue (400) of FIG. 4). By identifying the AI ​​pipeline included in the queue, the electronic device (101) can determine the index of the AI ​​pipeline to which a resource should be allocated.

[0045] According to one embodiment, in operation 530, the electronic device (101) can determine the occupancy time to be allocated to the identified AI pipeline. The electronic device (101) can determine a response time function to be used to determine the occupancy time based on the index of the AI ​​pipeline. The electronic device (101) can determine a loss function using at least one response time function corresponding to at least one AI pipeline and determine values ​​that have an optimal solution to the loss function as the occupancy time for each of the at least one AI pipeline. For example, if the first AI pipeline and the second AI pipeline need to be executed, the electronic device (101) can determine a loss function based on the sum of a first response time function corresponding to the first AI pipeline and a second response time function corresponding to the second AI pipeline, and determine a first response time for the first AI pipeline and a second response time for the second AI pipeline that minimize the loss function.

[0046] According to one embodiment, in operation 530, the electronic device (101) may determine the occupancy time by considering the service demand value of the AI ​​service to be provided based on the AI ​​pipeline. For example, the electronic device (101) may add a penalty value to the loss function for the occupancy time at which the response time of the AI ​​pipeline is expected to exceed the service occupancy time. The electronic device (101) may determine the occupancy time such that the response time of the AI ​​pipeline becomes less than or equal to the service demand value.

[0047] According to one embodiment, in operation 540, the electronic device (101) can execute an AI pipeline based on a determined occupancy time. For example, within a unit interval (300) of FIG. 3, at least one AI model included in the AI ​​pipeline may be executed while occupying a resource for an occupancy time (310), and within the next unit interval of the unit interval (300), at least one AI model may again occupy a resource for an occupancy time (310). Based on the change in the AI ​​pipeline to be executed, the electronic device (101) may dynamically apply the occupancy time allocated to at least one AI pipeline by repeating operations 530 and 540.

[0048] FIG. 6 is a flowchart (600) illustrating a process in which an electronic device (101) according to one embodiment of the present disclosure obtains a response time function.

[0049] According to one embodiment, in operation 610, the electronic device (101) may execute the AI ​​pipeline multiple times based on different occupancy times. The electronic device (101) may perform operations 610 to 630 illustrated in the flowchart (600) for an AI pipeline among the AI ​​pipelines that can be executed in the electronic device (101) for which the response time function has not been determined. For example, the electronic device (101) may determine a minimum occupancy time based on at least one AI model included in the AI ​​pipeline and drive the AI ​​pipeline based on the minimum occupancy time. Subsequently, the electronic device (101) may drive the AI ​​pipeline based on an occupancy time that is increased from the minimum occupancy time.

[0050] According to one embodiment, in operation 620, the electronic device (101) can obtain response times resulting from driving an AI pipeline based on various occupancy times. The electronic device (101) can obtain training data including at least some of the obtained response times. The obtained training data may include pairs of occupancy times and response times applied to drive the AI ​​pipeline. For example, the electronic device (101) may classify some of the information regarding the obtained response times as training data and classify the remaining parts as validation data.

[0051] According to one embodiment, in operation 630, the electronic device (101) can obtain a response time function for an AI pipeline by performing machine learning on training data. The electronic device (101) can perform machine learning using the response time included in the training data as a label.

[0052] FIG. 7 is a flowchart (700) illustrating an example of a process for determining whether an electronic device (101) according to one embodiment of the present disclosure will perform machine learning.

[0053] In one embodiment, the resources of the electronic device (101) may be used to drive an AI pipeline and perform machine learning. Accordingly, the electronic device (101) may perform operations such as acquiring training data to obtain a response time function or performing machine learning (e.g., operation 510 of FIG. 5) in a section where resources are not used or are used infrequently.

[0054] According to one embodiment, in operation 710, the electronic device (101) can determine whether the current time is within a designated time zone. The designated time zone may include, for example, a predetermined time interval of the day (e.g., night, from 2 a.m. to 5 a.m.) based on the region where the electronic device (101) is located. The designated time zone may be, for example, a time interval set by a user. If the current time is not within the designated time zone, the electronic device (101) may suspend the performance of machine learning until it becomes the designated time zone.

[0055] According to one embodiment, if the current time is within a specified time zone, in operation 720, the electronic device (101) can identify the operating state of the electronic device (101). For example, the electronic device (101) can determine whether the battery of the electronic device (101) (e.g., battery (1289) of FIG. 12) is being charged based on power supplied from an external source. If power for charging the battery is not being supplied, the electronic device (101) can suspend the execution of machine learning. If the battery is being charged, the electronic device (101) can perform operations to obtain a response time function (e.g., operations 610 to 630 of FIG. 6).

[0056] However, the process illustrated in FIG. 7 is an example of determining the conditions for the electronic device (101) to perform machine learning, and in one embodiment, the conditions for determining whether the electronic device (101) will perform machine learning may be configured differently. For example, the electronic device (101) may determine whether to perform machine learning based on the state in which the electronic device (101) is using resources.

[0057] FIG. 8 is a flowchart (800) illustrating a process in which an electronic device (101) according to one embodiment of the present disclosure acquires learning data and performs machine learning.

[0058] According to one embodiment, in operation 810, the electronic device (101) may determine a minimum occupancy time for an AI pipeline to acquire a response time function. The electronic device (101) may determine the minimum occupancy time based on a minimum value specified for at least one AI model included in the AI ​​pipeline. For example, the electronic device (101) may determine a value between 0 and 1 as the occupancy time, and determine the occupancy time such that the sum of all occupancy times is 1 or less. The minimum occupancy time for the AI ​​pipeline may be determined automatically by the electronic device (101), but may also be determined by user input received by the electronic device (101).

[0059] According to one embodiment, in operation 820, the electronic device (101) can drive the AI ​​pipeline based on the minimum occupancy time determined in operation 810. The electronic device (101) can be configured so that the AI ​​pipeline can use resources within the determined minimum occupancy time.

[0060] According to one embodiment, in operation 830, the electronic device (101) may obtain response time data including information about the response time taken until a response is obtained by driving the AI ​​pipeline in operation 820. The response time may correspond to the time taken to perform an operation that executes the AI ​​pipeline using resources. The electronic device (101) may collect a data set including a combination of the response time and the scheduling time set for each AI model.

[0061] According to one embodiment, in operation 840, the electronic device (101) can determine whether the reduction in response time is less than a threshold. If there is no response time data obtained prior to operation 830, the electronic device can perform operation 850 because there is no subject for comparison. If the response time obtained in operation 830 is reduced by a threshold greater than the response time obtained based on a relatively shorter occupancy time in the previous step (i.e., the degree of reduction in response time (reduction in response time) is greater than the threshold), it is necessary to collect more data on the response time obtained based on a longer occupancy time. If the degree of reduction in response time is greater than the threshold, the electronic device (101) can increase the occupancy time in operation 850 and perform operation 820 based on the increased occupancy time.

[0062] According to one embodiment, if the degree of reduction in response time is less than a threshold, the electronic device (101) in operation 860 can classify the acquired data. For example, the electronic device (101) can classify a certain portion (e.g., 80%) of the data acquired through operations 810 to 850 as training data and another portion (e.g., the remaining 20%) as validation data. According to one embodiment, in operation 870, the electronic device can perform machine learning based on the classified training data. The electronic device can obtain a response time function as a result of performing machine learning.

[0063] According to one embodiment, in operation 880, the electronic device (101) can determine whether there is another AI pipeline that needs to perform machine learning on a response time function. If there is another AI pipeline, the electronic device (101) can perform operation 810 and subsequent operations on the other AI pipeline. If there is no other AI pipeline, the electronic device (101) can terminate the process.

[0064] The process included in the flowchart (800) of FIG. 8 can be performed in operation 510 of FIG. 5, provided, but not limited thereto. The process of obtaining the response time function can be configured in various ways within the scope of performing machine learning on the response time of the AI ​​pipeline.

[0065] FIG. 9 illustrates an example of learning data acquired by an electronic device (101) according to one embodiment of the present disclosure.

[0066] In one embodiment, the electronic device (101) can obtain a response time of 0.532 by executing the first AI pipeline (910) based on a first occupancy time (901), which is a minimum occupancy time determined for the first AI pipeline (910). For example, referring to FIG. 9, the first occupancy time (901) may be 0.1. While executing the first AI pipeline (910), the electronic device (101) may execute the first AI pipeline (910) in parallel by allocating occupancy times to other AI pipelines such that the sum of the allocated occupancy times is 1 or less, but is not limited thereto.

[0067] In one embodiment, the electronic device (101) may execute a first AI pipeline (910) based on a second occupancy time (902) that is increased stepwise from a first occupancy time (901). The electronic device (101) may determine whether the difference (e.g., 0.080) between the response time corresponding to the first occupancy time (901) (e.g., 0.532) and the response time corresponding to the second occupancy time (902) (e.g., 0.452) is smaller than a threshold (e.g., 0.002). Based on the result of the determination, the electronic device (101) may determine whether to execute the first AI pipeline (910) based on an occupancy time greater than or equal to a third occupancy time (903) and to obtain information about the response time. If the difference (e.g., 0.080) between the response time (e.g., 0.532) corresponding to the first occupancy time (901) and the response time (e.g., 0.452) corresponding to the second occupancy time (902) is greater than a threshold (e.g., 0.002), the electronic device (101) can drive the first AI pipeline (901) based on the third occupancy time (903) and obtain information about the response time.

[0068] In one embodiment, the electronic device (101) may perform a first pipeline (910) based on an n-1 occupancy time (904) and an n occupancy time (905). If the difference between the response time corresponding to the n-1 occupancy time (904) (e.g., 0.263) and the response time corresponding to the n occupancy time (904) (e.g., 0.262) is less than a threshold value (e.g., 0.002) and has a value (0.001), the electronic device (101) may terminate the repetition of the operation of performing the first pipeline (901) while increasing the occupancy time (e.g., operations 820 to 850 of FIG. 8).

[0069] FIG. 10 is a flowchart (1000) illustrating a process in which an electronic device (101) according to one embodiment of the present disclosure obtains a response time function through a server (1001).

[0070] In one embodiment, a system comprising an electronic device (101) and a server (1001) may perform machine learning using the server (1001). In operation 1010, the electronic device (101) may acquire training data by driving an AI pipeline. Operation 1010 may include operations 810 to 850 of FIG. 8. In operation 1020, the electronic device (101) may transmit the acquired training data to the server (1001).

[0071] According to one embodiment, in operation 1030, the server (1001) can perform machine learning on training data received from the electronic device (101). In operation 1040, the server (1001) can obtain a response time function for the AI ​​pipeline as a result of performing machine learning. In operation 1050, the server (1001) can transmit the obtained response time function to the electronic device (101).

[0072] FIG. 11 is a flowchart (1100) illustrating a process in which an electronic device (101) according to one embodiment of the present disclosure drives an AI pipeline.

[0073] According to one embodiment, in operation 1110, the electronic device can identify an AI pipeline included in a scheduling queue (e.g., queue (400) of FIG. 4). An AI pipeline constituting an AI service may be inserted into the scheduling queue. For example, if two AI services are running on the electronic device (101), two AI pipelines may be included in the scheduling queue. In operation 1110, if one or more AI pipelines are included in the scheduling queue, the electronic device (101) can perform operation 1120.

[0074] According to one embodiment, in operation 1120, the electronic device may determine the occupancy time to be allocated to the AI ​​model constituting the AI ​​pipeline for the identified AI pipeline based on the response time function for the AI ​​pipeline. The electronic device (101) may determine a loss function constructed based on the response time function and determine the optimal solution of the loss function as the occupancy time. For example, the electronic device (101) may determine the optimal solution for the function of Equation 1, which is a convex function, as the occupancy time for each AI pipeline.

[0075] [Mathematical Formula 1]

[0076]

[0077] In the above mathematical formula 1, is the loss function, and represents an index for the AI ​​pipeline. It can be determined based on the following mathematical formula 2.

[0078] [Mathematical Formula 2]

[0079]

[0080] In the above mathematical formula 2, is the index It refers to the response time function for the AI ​​pipeline corresponding to. represents the occupancy time allocated to run the AI ​​pipeline. represents the workload (number of calls to the AI ​​model) for the AI ​​pipeline. represents a service requirement value that indicates the service level objective for the AI ​​service to be provided through the AI ​​pipeline.

[0081] In the above mathematical formula 1, This may refer to a penalty applied to the response time function for occupancy time that does not satisfy service level objectives. It can be determined by the following mathematical formula 3.

[0082] [Mathematical Formula 3]

[0083]

[0084] In the above mathematical formula 3, and can be a constant. can mean the larger value between a and b. If the response time of the AI ​​pipeline exceeds the service requirement value, Since this has a value greater than 0, the value of the loss function can increase significantly. By reflecting Equations 2 and 3, Equation 1 can be expressed as Equation 4 below.

[0085] [Mathematical Formula 4]

[0086]

[0087] In operation 1120, the electronic device has at least one that minimizes the value of the loss function. The value can be determined as the occupancy time to be allocated to at least one AI pipeline. In operation 1130, the electronic device can execute at least one AI model included in the AI ​​pipeline based on the determined occupancy time.

[0088] According to one embodiment, if the operation of the electronic device (101) is not terminated in operation 1140, the electronic device (101) may repeat the process shown in the flowchart (1100) until the operation is terminated.

[0089] FIG. 12 is a block diagram of an electronic device (1201) in a network environment (1200) according to various embodiments. Referring to FIG. 12, in the network environment (1200), the electronic device (1201) may communicate with an electronic device (1202) through a first network (1298) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (1204) or a server (1208) through a second network (1299) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (1201) may communicate with the electronic device (1204) through a server (1208). According to one embodiment, the electronic device (1201) may include a processor (1220), memory (1230), input module (1250), sound output module (1255), display module (1260), audio module (1270), sensor module (1276), interface (1277), connection terminal (1278), haptic module (1279), camera module (1280), power management module (1288), battery (1289), communication module (1290), subscriber identification module (1296), or antenna module (1297). In some embodiments, at least one of these components (e.g., connection terminal (1278)) may be omitted from the electronic device (1201), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (1276), camera module (1280), or antenna module (1297)) may be integrated into a single component (e.g., display module (1260)).

[0090] The processor (1220) can, for example, execute software (e.g., program (1240)) to control at least one other component (e.g., hardware or software component) of the electronic device (1201) connected to the processor (1220) and perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (1220) can store commands or data received from other components (e.g., sensor module (1276) or communication module (1290)) in volatile memory (1232), process the commands or data stored in volatile memory (1232), and store the resulting data in non-volatile memory (1234). According to one embodiment, the processor (1220) may include a main processor (1221) (e.g., a central processing unit or an application processor) or an auxiliary processor (1223) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (1201) includes a main processor (1221) and an auxiliary processor (1223), the auxiliary processor (1223) may be configured to use less power than the main processor (1221) or to be specialized for a specified function. The auxiliary processor (1223) may be implemented separately from the main processor (1221) or as part thereof.

[0091] The auxiliary processor (1223) may control at least some of the functions or states associated with at least one component of the electronic device (1201) (e.g., display module (1260), sensor module (1276), or communication module (1290)) on behalf of the main processor (1221) while the main processor (1221) is in an inactive (e.g., sleep) state, or together with the main processor (1221) while the main processor (1221) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (1223) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (1280) or communication module (1290)). According to one embodiment, the auxiliary processor (1223) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (1201) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (1208)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0092] The memory (1230) can store various data used by at least one component of the electronic device (1201) (e.g., processor (1220) or sensor module (1276)). The data may include, for example, software (e.g., program (1240)) and input or output data for related commands. The memory (1230) may include volatile memory (1232) or non-volatile memory (1234).

[0093] The program (1240) may be stored as software in memory (1230) and may include, for example, an operating system (1242), middleware (1244), or an application (1246).

[0094] The input module (1250) can receive commands or data to be used for a component of the electronic device (1201) (e.g., processor (1220)) from outside the electronic device (1201) (e.g., user). The input module (1250) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0095] The sound output module (1255) can output a sound signal to the outside of the electronic device (1201). The sound output module (1255) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0096] The display module (1260) can visually provide information to an external (e.g., user) of the electronic device (1201). The display module (1260) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (1260) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0097] The audio module (1270) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (1270) can acquire sound through the input module (1250) or output sound through the sound output module (1255) or an external electronic device (e.g., electronic device (1202)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (1201).

[0098] The sensor module (1276) can detect the operating state of the electronic device (1201) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (1276) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0099] The interface (1277) may support one or more specified protocols that can be used for the electronic device (1201) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (1202)). According to one embodiment, the interface (1277) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0100] The connection terminal (1278) may include a connector through which the electronic device (1201) can be physically connected to an external electronic device (e.g., electronic device (1202)). According to one embodiment, the connection terminal (1278) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0101] The haptic module (1279) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (1279) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0102] The camera module (1280) can capture still images and video. According to one embodiment, the camera module (1280) may include one or more lenses, image sensors, image signal processors, or flashes.

[0103] The power management module (1288) can manage the power supplied to the electronic device (1201). According to one embodiment, the power management module (1288) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0104] The battery (1289) can supply power to at least one component of the electronic device (1201). According to one embodiment, the battery (1289) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0105] The communication module (1290) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (1201) and an external electronic device (e.g., electronic device (1202), electronic device (1204), or server (1208)), and the performance of communication through the established communication channel. The communication module (1290) may include one or more communication processors that operate independently of the processor (1220) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (1290) may include a wireless communication module (1292) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (1294) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (1204) through a first network (1298) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (1299) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (1292) can identify or authenticate the electronic device (1201) within a communication network such as the first network (1298) or the second network (1299) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (1296).

[0106] The wireless communication module (1292) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (1292) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (1292) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (1292) can support various requirements specified in the electronic device (1201), external electronic device (e.g., electronic device (1204)), or network system (e.g., second network (1299)). According to one embodiment, the wireless communication module (1292) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0107] An antenna module (1297) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (1297) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (1297) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (1298) or a second network (1299), may be selected from the plurality of antennas, for example, by a communication module (1290). A signal or power may be transmitted or received between the communication module (1290) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (1297).

[0108] According to various embodiments, the antenna module (1297) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0109] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0110] According to one embodiment, commands or data may be transmitted or received between the electronic device (1201) and an external electronic device (1204) through a server (1208) connected to a second network (1299). Each of the external electronic devices (1202, or 1204) may be the same or a different type of device as the electronic device (1201). According to one embodiment, all or part of the operations performed on the electronic device (1201) may be performed on one or more of the external electronic devices (1202, 1204, or 1208). For example, if the electronic device (1201) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (1201) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (1201). The electronic device (1201) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (1201) may provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In one embodiment, the external electronic device (1204) may include an Internet of Things (IoT) device. The server (1208) may be an intelligent server using machine learning and / or neural networks.According to one embodiment, an external electronic device (1204) or server (1208) may be included within the second network (1299). The electronic device (1201) may be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0111] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0112] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0113] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0114] Various embodiments of the present document may be implemented as software (e.g., program (1240)) comprising one or more instructions stored in a storage medium (e.g., internal memory (1236) or external memory (1238)) readable by a machine (e.g., electronic device (1201)). For example, a processor (e.g., processor (1220)) of the machine (e.g., electronic device (1201)) may call at least one of the one or more instructions stored from the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0115] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0116] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0117] In one embodiment, an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) may include at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12)) and memory (e.g., memory (130) of FIG. 1, memory (1230) of FIG. 12). The memory (e.g., memory (130) of FIG. 1, memory (1230) of FIG. 12)) may store a plurality of artificial intelligence (AI) models (e.g., AI models (131, 132) of FIG. 1). The memory (e.g., memory (130) of FIG. 1, memory (1230) of FIG. 12)) may store a computer program including computer-executable instructions. The above instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to obtain a response time function obtained by performing machine learning on training data including a response time resulting from the execution of an AI pipeline (e.g., AI pipeline (200) of FIG. 2) that includes at least one of the plurality of AI models. The above instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to identify the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) included in the queue.The above instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to allow the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to determine the occupancy time to be allocated to at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) included in the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2)) based on the response time function. The above instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) so that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) can execute the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) within the said occupancy time.

[0118] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) to acquire the training data by executing the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) multiple times based on different occupancy times. The instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to acquire the response time function by performing machine learning on the training data.

[0119] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to determine whether the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is in a specified time period and whether the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging the battery of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12). The above instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to perform machine learning based on the determination that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging within the specified time period.

[0120] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to execute the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on a first occupancy time allocated to the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2). The above instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) to obtain a first response time taken to execute the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on the first occupancy time. The above instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to execute the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on a second occupancy time allocated to the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2).The above instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) to execute the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on the second occupancy time to obtain a second response time. The above instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to obtain the response time function by machine learning the training data including at least the first response time and the second response time. The second occupancy time may be longer than the first occupancy time.

[0121] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to obtain the training data containing information about the response time by executing the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) multiple times while gradually increasing the occupancy time allocated to at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) until the degree to which the response time for the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) is reduced is below a threshold.

[0122] In one embodiment, the instructions may be executed individually or collectively by at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to enable an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to perform machine learning through supervised learning using tail values ​​as labels within a distribution of response times included in the learning data.

[0123] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to determine the occupancy time such that the response time of the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) becomes less than or equal to the service demand value corresponding to the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on the response time function and the workload for the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2).

[0124] In one embodiment, the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) may include at least one graphics processing unit. The instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) so that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) may cause the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2)) to occupy the at least one graphics processing unit during the occupancy time allocated to the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2).

[0125] In one embodiment, the instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) so that the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) may determine the value of the loss function based on the response time function and the service request value for the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2). The above instructions may be executed individually or collectively by the at least one processor (e.g., at least one processor (120) of FIG. 1, processor (1220) of FIG. 12) to cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) to determine the value that minimizes the value of the loss function as the occupancy time to be allocated to the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2).

[0126] In one embodiment, a method of operation of an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) may include an operation of obtaining a response time function obtained by performing machine learning on training data including a response time according to the result of executing an AI pipeline (e.g., AI pipeline (200) of FIG. 2) that includes at least one of a plurality of AI models. The method may include an operation of identifying the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) included in a queue. The method may include an operation of determining the occupancy time to be allocated to the at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) included in the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2)) based on the response time function. The above method may include the operation of executing at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) within the above occupancy time.

[0127] In one embodiment, the operation of obtaining the response time function may include the operation of obtaining the training data by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) multiple times based on different occupancy times. The operation of obtaining the response time function may include the operation of obtaining the response time function by performing machine learning on the training data.

[0128] In one embodiment, the operation of obtaining the response time function may include suspending the performance of the machine learning when the current time is not included in a specified time zone or when the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging the battery of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12), and performing the machine learning when the current time is included in a specified time zone and when the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging the battery of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12).

[0129] In one embodiment, the operation of obtaining the response time function may include the operation of executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on a first occupancy time allocated to the at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2). The operation of obtaining the response time function may include the operation of obtaining a first response time taken to obtain a response by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on the first occupancy time. The operation of obtaining the response time function may include the operation of executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on a second occupancy time allocated to the at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2). The operation of obtaining the response time function may include the operation of obtaining a second response time taken to obtain a response by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on the second occupancy time. The operation of obtaining the response time function may include the operation of obtaining the response time function by machine learning the training data including at least the first response time and the second response time. The second occupancy time may be longer than the first occupancy time.

[0130] In one embodiment, the operation of obtaining the response time function may be to obtain the training data containing information about the response time by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) multiple times while gradually increasing the occupancy time allocated to at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2) until the degree to which the response time for the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) decreases below a threshold.

[0131] In one embodiment, the operation of performing the machine learning may include the operation of performing the machine learning through supervised learning using tail values ​​as labels within the distribution of response times included in the training data.

[0132] In one embodiment, the operation of determining the occupancy time to be allocated to at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) may include the operation of determining the occupancy time such that the response time of the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) does not exceed a service demand value corresponding to the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2) based on the response time function and the workload for the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2).

[0133] In one embodiment, a computer-readable non-transient recording medium may store a computer program containing computer-executable instructions. The instructions may enable an electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) to perform an operation to obtain a response time function obtained by performing machine learning on training data including a response time according to the result of executing an AI pipeline (e.g., AI pipeline (200) of FIG. 2) which includes at least one of a plurality of AI models when the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) executes an AI pipeline (e.g., AI pipeline (200) of FIG. 2). The above commands may cause the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (1201) of FIG. 12) to perform an operation to identify the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) included in the queue when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (1201) of FIG. 12) is executed. The above commands may cause the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12) to perform an operation to determine the occupancy time to be allocated to at least one AI model (e.g., AI model (131, 132) of FIG. 1, AI model (131, 132) of FIG. 2) included in the AI ​​pipeline (e.g., AI pipeline (200) of FIG. 2)) based on the response time function when the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) of FIG. 2 is executed. The above commands may enable the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (1201) of FIG. 12) to perform a method including an operation of executing the at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2) within the said occupancy time when the electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (1201) of FIG. 1) is executed.

[0134] In one embodiment, the operation of obtaining the response time function may include the operation of obtaining the training data by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) multiple times based on different occupancy times. The operation of obtaining the response time function may include the operation of obtaining the response time function by performing machine learning on the training data.

[0135] In one embodiment, the operation of obtaining the response time function may include the operation of suspending the performance of the machine learning when the current time is not included in a specified time zone or when the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging the battery of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12). The operation of obtaining the response time function may include the operation of performing the machine learning when the current time is included in a specified time zone and the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12)) is charging the battery of the electronic device (e.g., electronic device (101) of FIG. 1, electronic device (1201) of FIG. 12).

[0136] In one embodiment, the operation of obtaining the response time function may include the operation of executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on a first occupancy time allocated to the at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2). The operation of obtaining the response time function may include the operation of obtaining a first response time taken to obtain a response by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on the first occupancy time. The operation of obtaining the response time function may include the operation of executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on a second occupancy time allocated to the at least one AI model (e.g., the AI ​​model (131, 132) of FIG. 1, the AI ​​model (131, 132) of FIG. 2). The operation of obtaining the response time function may include the operation of obtaining a second response time taken to obtain a response by executing the AI ​​pipeline (e.g., the AI ​​pipeline (200) of FIG. 2) based on the second occupancy time. The operation of obtaining the response time function may include the operation of obtaining the response time function by machine learning the training data including at least the first response time and the second response time. The second occupancy time may be longer than the first occupancy time.

[0137] The electronic device and the method of operation thereof according to various embodiments may be intended to optimize infrastructure costs and maintain service quality through the automated discovery and allocation of resources so that an AI pipeline executed to provide AI services can satisfy response time requirements.

[0138] The electronic device and the method of operation thereof according to various embodiments may be intended to minimize downtime by automatically allocating resources even in situations where requests by users surge, and to provide flexibility to adaptively respond even when response time requirements change.

[0139] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description of the present disclosure.

[0140] Methods according to the claims or embodiments described in the specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0141] When implemented in software, a computer-readable storage medium may be provided for storing one or more programs (software modules). One or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. One or more programs include instructions that cause the electronic device to execute methods according to the claims or embodiments described in the specification of this disclosure.

[0142] In the present disclosure, the function or operation performed by an electronic device may be performed by one or more processors executing one or more instructions stored in memory (e.g., memory (130) of FIG. 1, memory (1230) of FIG. 12). The function or operation of the electronic device mentioned in the present disclosure may be performed by a single processor executing one or more instructions, or by a combination of multiple processors executing one or more instructions. A processor mentioned in the present disclosure is understood to include a circuit for performing operations or controlling other components of the electronic device. For example, the one or more processors may include a central processing unit (CPU), a micro-processor unit (MPU), an application processor (AP), a communication processor (CP), a neural processing unit (NPU), a system on chip (SoC), or an integrated circuit (IC) configured to execute one or more instructions. The one or more processors may be configured to perform the operation of the electronic device described above. For example, the processor may be implemented with multiple cores, multiple chips, or multiple chipsets. For example, the processor may include one or more processing circuits. For example, the processor may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively.

[0143] In the present disclosure, a program (software module, software) comprising computer-executable instructions may be stored in random access memory, non-volatile memory including flash memory, ROM (read-only memory), electrically erasable programmable read-only memory (EEPROM), magnetic disc storage device, compact disc-ROM (CD-ROM), digital versatile discs (DVDs), or other forms of optical storage devices, magnetic cassettes. Alternatively, it may be stored in a memory composed of some or all of these. The memory may be composed of a single storage medium or a combination of multiple storage media. The one or more instructions may be stored in a single storage medium or distributed across multiple storage media.

[0144] Additionally, the above program may be stored on an attachable storage device that can be accessed via a communication network such as the Internet, Intranet, LAN (local area network), WLAN (wide LAN), or SAN (storage area network), or a combination thereof. Such a storage device may be connected to a device performing an embodiment of the present disclosure through an external port. Additionally, a separate storage device on a communication network may be connected to a device performing an embodiment of the present disclosure.

[0145] In the specific embodiments of the present disclosure described above, the components included in the disclosure are expressed in a singular or plural form according to the specific embodiments presented. However, the singular or plural expression is selected to suit the situation presented for convenience of explanation, and the present disclosure is not limited to singular or plural components; even if a component is expressed in the plural form, it may be composed of a singular form, and even if a component is expressed in the singular form, it may be composed of a plural form.

[0146] Additionally, in the present disclosure, terms such as “part,” “module,” etc. may be a hardware component, such as a processor or circuit, and / or a software component executed by a hardware component, such as a processor.

[0147] "Parts" and "modules" may be implemented by a program that is stored on an addressable storage medium and can be executed by a processor. For example, "parts" and "modules" may be implemented by components such as software components, object-oriented software components, class components, and task components, as well as by processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0148] The specific embodiments described in this disclosure are merely examples and do not limit the scope of this disclosure in any way. For the sake of brevity, descriptions of prior electronic configurations, control systems, software, and other functional aspects of said systems may be omitted.

[0149] Additionally, in the present disclosure, “comprising at least one of a, b, or c” may mean “comprising only a, comprising only b, comprising only c, or comprising a combination of two or more (comprising a and b, comprising b and c, comprising a and c, or comprising all of a, b, and c).”

[0150] Meanwhile, although specific embodiments have been described in the detailed description of the present disclosure, it is understood that various modifications are possible within the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, but should be defined by the claims set forth below as well as equivalents thereof.

[0151] In the present disclosure, the term “if” will be understood, depending on the context, to mean “when, upon,” “in response to a decision,” or “in response to a detection.” Similarly, “when decided to,” or “when [mentioned condition or event] is detected” will be understood, optionally, to mean “when decided,” or “in response to a decision,” “when [mentioned condition or event] is detected,” or “in response to a detection.”

[0152] The device described above may be implemented as a hardware component, a software component, and / or a combination of a hardware component and a software component. For example, the device and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit (or processing circuit) may execute an operating system (OS) and one or more software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0153] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or instruct the processing unit independently or collectively. Software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

[0154] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. In this case, the medium may continuously store a program executable by a computer, or temporarily store it for execution or download. Additionally, the medium may be various recording or storage means in the form of a single or several hardware combined, and may not be limited to a medium directly connected to a computer system but may exist distributed over a network. Examples of media may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to store program instructions, including ROM, RAM, and flash memory. Additionally, other examples of media may include recording or storage media managed by an app store that distributes applications or a site or server that supplies or distributes various other software.

[0155] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art can make various modifications and variations from the description above. For example, suitable results can be achieved even if the described techniques are performed in a different order than described, and / or the components of the described system, structure, device, circuit, etc. are combined or assembled in a form different from described, or replaced or substituted by other components or equivalents.

Claims

1. In an electronic device, At least one processor; and It includes memory for storing a computer program that includes multiple artificial intelligence (AI) models and computer-executable instructions, and The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: Obtaining a response time function obtained by performing machine learning on training data including a response time according to the result of executing an AI pipeline including at least one of the plurality of AI models; Identify the above AI pipelines included in the queue; Determining the occupancy time to be allocated to at least one AI model included in the AI ​​pipeline based on the above response time function; An electronic device that enables the at least one AI model to be executed within the above-mentioned occupancy time.

2. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: The training data is obtained by executing the above AI pipeline multiple times based on different occupancy times, and An electronic device that obtains the response time function by performing machine learning on the above training data.

3. In Claim 2, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: The current time falls within a specified time period, and it is determined whether the electronic device is charging the battery of the electronic device, and An electronic device that performs machine learning based on the determination that the electronic device is charging within the specified time period.

4. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: Execute the AI ​​pipeline based on a first occupancy time allocated to at least one AI model, and Based on the first occupancy time above, the first response time taken to obtain a response by executing the AI ​​pipeline above is obtained, and Execute the AI ​​pipeline based on the second occupancy time assigned to at least one AI model, and Based on the second occupancy time above, the second response time taken to obtain a response by executing the AI ​​pipeline above is obtained, and At least the response time function is obtained by machine learning the training data including the first response time and the second response time, and An electronic device in which the second occupancy time is longer than the first occupancy time.

5. In Claim 4, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: An electronic device for acquiring training data containing information on response time by executing the AI ​​pipeline multiple times while gradually increasing the occupancy time allocated to at least one AI model until the degree of reduction in response time for the AI ​​pipeline becomes below a threshold.

6. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: An electronic device that performs machine learning through supervised learning using tail values ​​as labels within the distribution of response times included in the above training data.

7. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: An electronic device for determining the occupancy time such that the response time of the AI ​​pipeline becomes less than or equal to a service requirement value corresponding to the AI ​​pipeline, based on the above response time function and the workload for the above AI pipeline.

8. In Claim 1, The above at least one processor includes at least one graphics processing unit, and The above instructions are executed individually or collectively by the at least one processor, so that the electronic device causes the at least one AI model to occupy the at least one graphics processing unit during the occupancy time allocated to the at least one AI model.

9. In Claim 1, The above instructions are executed individually or collectively by the at least one processor, so that the electronic device: Determine the value of the loss function based on the above response time function and the service requirement value for the above AI pipeline, and An electronic device that determines a value that minimizes the value of the above loss function as the occupancy time to be allocated to at least one AI model.

10. In a method of operating an electronic device, An operation to obtain a response time function obtained by performing machine learning on training data including a response time based on the result of executing an AI pipeline including at least one of a plurality of AI models; An action to identify the above AI pipeline included in the queue; An operation to determine the occupancy time to be allocated to the at least one AI model included in the AI ​​pipeline based on the above response time function; and A method comprising the operation of executing at least one AI model within the above-mentioned occupancy time.

11. In claim 10, the operation of obtaining the response time function is: The operation of acquiring the training data by executing the above AI pipeline multiple times based on different occupancy times, and A method comprising the operation of obtaining the response time function by performing machine learning on the above training data.

12. In claim 11, the operation of obtaining the response time function is: If the current time does not fall within the designated time zone, or if the electronic device is charging the battery of the electronic device, the execution of the machine learning is suspended, and A method of performing machine learning while the current time falls within a specified time zone and the electronic device is charging the battery of the electronic device.

13. In claim 10, the operation of obtaining the response time function is: The operation of executing the AI ​​pipeline based on the first occupancy time allocated to the above at least one AI model, The operation of obtaining the first response time taken to obtain a response by executing the AI ​​pipeline based on the first occupancy time, The operation of executing the AI ​​pipeline based on the second occupancy time assigned to at least one AI model, An operation to obtain a second response time taken to obtain a response by executing the AI ​​pipeline based on the second occupancy time, and The operation of obtaining the response time function by machine learning the training data including at least the first response time and the second response time, and A method in which the second possession time is longer than the first possession time.

14. A method according to claim 13, wherein the operation of obtaining the response time function is to obtain the training data containing information about the response time by executing the AI ​​pipeline multiple times while gradually increasing the occupancy time allocated to at least one AI model until the degree to which the response time for the AI ​​pipeline decreases is below a threshold.

15. A computer-readable, non-transient recording medium storing a computer program comprising computer-executable instructions, wherein the instructions are, when the electronic device executes: An operation to obtain a response time function obtained by performing machine learning on training data including a response time based on the result of executing an AI pipeline including at least one of a plurality of AI models; An action to identify the above AI pipeline included in the queue; An operation to determine the occupancy time to be allocated to the at least one AI model included in the AI ​​pipeline based on the above response time function; and A recording medium that enables a method including the operation of executing at least one AI model within the above-mentioned occupancy time.

Images