Device and method for monitoring deep learning compiler optimization performance

Abstract

A method for monitoring deep learning compiler optimization performance is provided. The method includes the steps of: calculating metric information for evaluating the performance of a compiler; calculating a score function value corresponding to resource optimization policy information set in an AI-based optimizer of the compiler based on the metric information; and providing a performance analysis result of the AI-based optimizer based on the score function value.

Description

Technology Field

[0001] The present invention relates to a device and method for monitoring deep learning compiler optimization performance. Background Technology

[0002] A deep learning compiler performs resource optimization tasks to run a deep learning model on a specific AI accelerator and generates instructions for the AI ​​accelerator. For example, when a deep learning compiler performs deep learning workload optimization on limited hardware resources, it performs scheduling optimization to determine the order in which hardware resources will be used to execute a given workload and generates instructions for execution on the actual hardware.

[0003] In this case, the optimization task involves solving the Combination Optimization problem, and good performance can be achieved in this field by using Reinforcement Learning.

[0004] However, the performance of reinforcement learning-based optimizers applied to deep learning compilers is determined by the training data, just like other AI technologies. In other words, there are adversarial inputs that yield incorrect results, and performance cannot be guaranteed for untrained data. Therefore, for the practical use of reinforcement learning or AI-based deep learning compiler technologies, it is necessary to continuously monitor and manage the performance of the compiler. Prior art literature

[0005] Published Patent Application No. 10-2022-0107621, August 2, 2022 The problem to be solved

[0006] The problem that the present invention aims to solve is to provide a deep learning compiler optimization performance monitoring device and method that can monitor optimization performance by setting various indicators to effectively monitor the performance of an AI-based optimizer, and provide the results of the optimizer's performance analysis as notification information.

[0007] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0008] A method performed by a deep learning compiler optimization performance monitoring device according to the first aspect of the present invention for solving the above-described problem comprises the steps of: calculating metric information for evaluating the performance of a compiler; calculating a score function value corresponding to resource optimization policy information set in an AI-based optimizer of the compiler based on the metric information; and providing a performance analysis result of the AI-based optimizer based on the score function value.

[0009] In some embodiments of the present invention, the step of calculating metric information for evaluating the performance of the compiler may include: generating hardware specification information and workload set information for evaluating the performance of the compiler; executing a simulation according to the hardware specification information and the workload set information; collecting logs corresponding to the simulation; and calculating metric information for evaluating the performance of the compiler based on the logs.

[0010] In some embodiments of the present invention, the step of calculating metric information for evaluating the performance of the compiler may include one or more of the following: a first performance indicator representing the total time required to execute a compiled deep learning model; a second performance indicator representing the computational blocking time due to DMA during the total time; a third performance indicator representing the time during which a processing element array is utilized during the total time; a fourth performance indicator representing the number of DMA operations per convolution operation; a fifth performance indicator representing the length of a graph generated as an optimization result under the same hardware conditions; and a sixth performance indicator representing the buffer usage efficiency during the total compilation execution time.

[0011] In some embodiments of the present invention, the step of calculating a score function value corresponding to resource optimization policy information set in the optimizer of the AI-based compiler may include the step of calculating the score function value by utilizing a score function that takes hardware specification information, a deep learning workload, and the resource optimization policy information as inputs.

[0012] In some embodiments of the present invention, the step of providing a performance analysis result of the AI-based optimizer based on the score function value may include: a step of comparing the calculated score function value with a preset first setting value; and, if the comparison result is less than the first setting value, a step of providing information on the possibility of performance improvement through reinforcement learning retraining of the AI-based optimizer as the performance analysis result.

[0013] In some embodiments of the present invention, the step of providing information on the possibility of performance improvement through reinforcement learning retraining of the AI-based optimizer as a performance analysis result may provide information on the possibility of performance improvement of the AI-based optimizer for a specific deep learning model operating under specific hardware conditions as a performance analysis result.

[0014] In some embodiments of the present invention, the step of providing a performance analysis result of the AI-based optimizer based on the score function value may include: a step of comparing the calculated score function value with a preset second setting value; and, if the comparison result is greater than or equal to the second setting value, a step of providing redesign information for expanding hardware resources and predicted performance information based on the redesign information as the performance analysis result.

[0015] In some embodiments of the present invention, the step of providing redesign information for resource expansion of the hardware and prediction performance information based on the redesign information as the performance analysis result may provide redesign information for resource expansion of the hardware and prediction performance information as the performance analysis result, wherein a plurality of various deep learning models can operate universally.

[0016] In some embodiments of the present invention, the step of providing notification information when the monitoring result of the AI-based compiler performance satisfies a predetermined notification rule may be further included.

[0017] In addition, a deep learning compiler optimization performance monitoring device according to the third aspect of the present invention includes a compiler that performs a resource optimization policy through an AI-based optimizer and generates and provides instructions for a deep learning model, a metric module that defines and calculates metric information for evaluating the performance of the compiler, a simulation module that calculates a score function value corresponding to resource optimization policy information set in the optimizer based on the metric information, and a monitoring module that provides a performance analysis result of the optimizer based on the score function value.

[0018] Other specific details of the present invention are included in the detailed description and drawings. Effects of the invention

[0019] According to one embodiment of the present invention described above, the overall performance can be evaluated through simulation before applying an AI-based optimizer to a compiler to determine whether to update an existing optimizer within the compiler with a newly trained optimizer or to continue retraining the trained optimizer.

[0020] In addition, by evaluating the current performance of the optimizer using simulation, providing notification information when issues such as bottlenecks occur, and simultaneously providing users with hardware redesign plans to resolve these issues as well as information on expected performance improvements upon redesign, it is possible to improve the performance of the AI ​​accelerator by resolving hardware bottlenecks even in situations where reinforcement learning of the AI-based optimizer is at its limit.

[0021] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0022] FIG. 1 is a block diagram of a deep learning compiler optimization performance monitoring device according to one embodiment of the present invention. FIG. 2 is a flowchart of a deep learning compiler optimization performance monitoring method according to an embodiment of the present invention. FIG. 3 is a diagram illustrating detailed steps for providing performance analysis results in one embodiment of the present invention. Figure 4 shows an algorithm for evaluating current optimization resource policy information using score function values. Figure 5 shows an algorithm that provides redesign information for expanding hardware resources and predictive performance information based on the redesign information. Specific details for implementing the invention

[0023] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0024] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0025] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0026] In describing the present invention, detailed descriptions of related prior art are omitted if they are deemed obvious to a person skilled in the art and could unnecessarily obscure the essence of the invention.

[0027] In this specification, "hardware" is used interchangeably with "AI accelerator," and "optimizer" is used interchangeably with "optimization algorithm" or "optimization model." Additionally, "resource" and "resources" may be used interchangeably depending on the context to convey a clearer meaning.

[0028] Hereinafter, to facilitate understanding by a person skilled in the art, some details related to the present invention are explained, and then embodiments of the present invention are described in detail with reference to the attached drawings.

[0029] Conventional technology does not continuously monitor and manage the performance of the compiler itself. In other words, since the optimization tasks of traditional compilers utilize heuristic and rule-based optimization policies, the performance of the compiler itself is not subject to management. Put another way, conventional technology did not have any means to monitor compiler performance at all.

[0030] However, in deep learning applications, depending on the purpose, it is necessary to compile a given deep learning model to achieve optimal performance dependent on specific hardware, or to compile various deep learning models to achieve high overall performance on specific hardware. In these two cases, there is a need for technology that can monitor and evaluate the optimization performance of a compiler in order to develop optimization algorithms for the compiler or to train AI-based optimizers that achieve this.

[0031] In this regard, the performance of AI accelerators has traditionally been evaluated by throughput, which indicates how many inputs (queries) are processed per unit of time; however, this does not represent the performance of the compiler itself.

[0032] In compiling code written in languages ​​such as C / C++, the performance of compilers like GCC (GNU Compiler Collection) can be evaluated by compiler throughput, which indicates the time taken to process the compilation target; code quality, which indicates the time taken to execute the generated instructions; and code size, which indicates the size of the generated instructions. However, with the exception of code quality, the others are not suitable for representing the optimization performance of deep learning compilers.

[0033] Furthermore, conventional technology did not provide information related to the compiler's optimization performance or provide notification information based thereon.

[0034] In other words, conventional technology does not provide notification information when an AI-based optimizer shows lower optimization performance than usual for a new deep learning model during the compilation process.

[0035] In addition, when retraining an AI-based optimizer, optimization performance may degrade even if the same deep learning model is compiled. However, conventional technology lacked a means to automatically provide notification information even when such performance degradation occurred.

[0036] Furthermore, in the case of AI-based optimizers, there was a possibility that the deep learning model intended for deployment could be trained to perform better under specific hardware conditions through overfitting. However, conventional technology did not provide notification information regarding this situation, so there was no way for the user to know.

[0037] Furthermore, when the expected performance improvement was minimal even after additionally training an AI-based optimizer, there was no way to identify the bottlenecks causing this or to determine what resources were lacking so that this could be referenced during future hardware design.

[0038] In order to resolve these various problems, one embodiment of the present invention can monitor optimization performance by setting various indicators to effectively monitor the performance of an AI-based optimizer, and provide the results of the optimizer's performance analysis as notification information.

[0039] FIG. 1 is a block diagram of a deep learning compiler optimization performance monitoring device (100, hereinafter referred to as the device) according to one embodiment of the present invention.

[0040] A device (100) according to one embodiment of the present invention includes an AI accelerator (110), a compiler (120), a metric module (130), a simulation module (140), and a monitoring module (150).

[0041] The AI ​​accelerator (110) is a hardware accelerator that executes a deep learning model. The AI ​​accelerator (110) executes instructions output by the compiler (120).

[0042] The AI ​​accelerator (110) may include a GPU (Graphics Processing Unit), an NPU (Neural Processing Unit), an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0043] The compiler (120) includes an AI-based optimizer (121) and performs a resource optimization policy through the optimizer (121). At this time, the resource optimization policy refers to a rule applied when generating a plan to execute a given workload (deep learning model to be executed) on a given resource (hardware resource). Here, the plan defines the order in which each module of the hardware will be executed when executing the deep learning model on the hardware. Since reinforcement learning is applied to the AI-based optimizer (121), the resource optimization policy itself becomes the subject of training.

[0044] Additionally, the compiler (120) generates instructions for a deep learning model and provides them to the AI ​​accelerator (110). At this time, one embodiment of the present invention may further include a compiler training module (170) for performing reinforcement learning of the optimizer (121).

[0045] The compiler training module (170) performs reinforcement learning of the AI-based optimizer and, based on the results of the performance evaluation through simulation, can determine whether to update the existing optimizer (121) in the compiler (120) with the newly trained optimizer or to continue retraining the optimizer that is not performing well.

[0046] The metric module (130) defines metric information for evaluating the performance of the compiler (120) and calculates metric information corresponding to the resource optimization policy information set in the optimizer (121). The metric module (130) calculates metric information based on logs collected by the logging module (160).

[0047] In one embodiment, the metric information includes first to sixth performance indicators, and the calculation of the metric information may be to calculate any one of the first to sixth performance indicators or to calculate multiple indicators.

[0048] The first performance metric (Makespan) is the total time required to execute a compiled deep learning model. The first performance metric indicates that a shorter time is required to execute the same deep learning model on given hardware, which is superior.

[0049] The second performance metric (DMA blocking ratio) represents the computational blocking time caused by DMA as a ratio of the total makespan. The second performance metric indicates that when running the same deep learning model on specific hardware, a smaller ratio value indicates superior performance.

[0050] [Equation 1]

[0051]

[0052] The third performance metric (PE-array packing ratio) represents the ratio of the time the processing element array (PE-array) is utilized in the entire makespan. The third performance metric indicates that when running the same deep learning model on given hardware, a higher ratio value indicates superior performance.

[0053] [Equation 2]

[0054]

[0055] The fourth performance metric (DMA-convolution ratio) represents the number of DMA operations per convolution operation of a deep learning model as a ratio. The fourth performance metric indicates that when the same deep learning model is executed on given hardware, a smaller ratio value indicates superior performance.

[0056] The fifth performance metric (Critical Path Length) represents the length of the graph generated as a result of optimization under identical hardware conditions; a shorter length indicates better performance because it traverses fewer hardware components.

[0057] Here, the graph generated as a result of optimization refers to a directed graph that represents the execution order of each module of the hardware, taking into account the dependencies between the modules, given a fixed hardware and a plan to execute each module constituting the hardware sequentially. In this case, the vertices of the graph structure correspond to each hardware module, and the directed edges represent the execution order of the hardware considering dependencies. The edges of the graph can have weights (e.g., the operation time of a module). Consequently, the shortest path length among those calculated by considering the weights of the paths connecting from the root node of the graph to the terminal node is defined as the fifth performance metric, 'Critical Path Length'.

[0058] The 6th performance metric (Buffer packing ratio) represents the efficiency of buffer usage during the total compilation execution time, and a higher ratio value indicates superiority when running the same deep learning model on specific hardware.

[0059] The simulation module (140) simulates the execution of a deep learning model on specified hardware for performance evaluation of the compiler (120). The simulation module (140) includes a workload generator (141) that generates a workload set for performance evaluation of the compiler (120), and a hardware simulator (142) that provides a simulation based on hardware specification information (e.g., memory, buffer, DMA, etc.). Here, the workload set refers to a set of deep learning models for simulation. The logging module (160) includes a log collector (161) that collects logs and stores a trace data set (162). At this time, the log collector (161) can collect logs generated by the execution of a deep learning model on hardware, i.e., the AI ​​accelerator (110), or by the simulation of the simulation module (140). The trace data set (160) may include hardware specification information, workload set information, resource optimization policy information, metric information, etc., used for the AI ​​accelerator (110) or simulation. The trace data set (162) can be used in the reinforcement learning process of the optimizer (121) by the compiler training module (170).

[0060] The monitoring module (150) monitors the performance of the optimization model of the compiler (120) based on logs collected by the logging module (160) and metric information calculated by the metric module (130). To this end, the monitoring module (150) calculates a score function value corresponding to the resource optimization policy information set in the optimizer (121) based on the metric information. Here, the score function value is intended to score the performance of the optimizer, that is, to evaluate whether the resource optimization policy information is appropriate for a given hardware and workload, and a reward function used in reinforcement learning may be used considering practical aspects.

[0061] Additionally, the monitoring module (150) can monitor the performance of the optimizer (121) by collecting logs of the AI ​​accelerator (110) that runs a specific deep learning model after actual compilation rather than simulation.

[0062] The monitoring module (150) provides performance analysis results of the optimizer (121) based on score function values. The monitoring module (150) may include rules for monitoring, rules for providing notification information, and components for monitoring.

[0063] Hereinafter, with reference to FIGS. 2 to 5, a method performed by a deep learning compiler optimization performance monitoring device (100) according to an embodiment of the present invention will be described.

[0064] FIG. 2 is a flowchart of a deep learning compiler optimization performance monitoring method according to an embodiment of the present invention. FIG. 3 is a diagram illustrating detailed steps for providing performance analysis results in an embodiment of the present invention.

[0065] First, the metric module (130) calculates metric information for evaluating the performance of the compiler (120) (S110). As described above, the metric information targets the first to sixth performance indicators and can be provided in the form of one or more of them individually or in combination.

[0066] Next, the monitoring module (150) calculates a score function value corresponding to the resource optimization information set in the optimizer (121) of the compiler (120) based on the metric information (S120).

[0067] The score function refers to a function for scoring the performance of the optimizer (120). The score function is intended to evaluate how good the performance is of the resource optimization policy information determined by the optimizer (121) for a given hardware and workload, and takes hardware specification information, deep learning workload, and resource optimization policy information as inputs.

[0068] Next, the monitoring module (150) provides the performance analysis results of the optimizer (121) based on the score function value (S130).

[0069] In one embodiment, the monitoring module (150) compares the calculated score function value with a preset first setting value (S131), and if the comparison result is less than the first setting value, it can provide information on the possibility of performance improvement through retraining of the optimizer (121) as a performance analysis result (S132).

[0070] That is, if the score function value calculated in accordance with the resource optimization policy information currently set in the optimizer (121) for a workload set prepared with a specific deep learning model is less than the first setting value, the monitoring module (150) may provide this possibility to the user as an analysis result, as there is room to improve performance by retraining the optimizer (121) for a specific deep learning model operating under specific hardware conditions.

[0071] In contrast, the simulation module (140) compares the calculated score function value with a preset second setting value (S133), and if the comparison result is greater than or equal to the second setting value, it can provide redesign information for increasing hardware resources and predicted performance information based on the redesign information as a performance analysis result (S134).

[0072] That is, for a workload set prepared for various deep learning models, if the score function value calculated in accordance with the resource optimization policy information currently set in the optimizer (121) is greater than or equal to the second setting value, it is difficult to expect general performance improvement of the optimizer (121) through additional learning. In this case, the monitoring module (150) can provide the user with a performance analysis result indicating that resource expansion is necessary through hardware-level redesign, and can predict and inform the user of how much performance improvement can be expected relative to the increased resources.

[0073] Meanwhile, the second setting value may be greater than the first setting value by a predetermined amount. In this case, the simulation module (140) can maintain the current state of the optimizer (121) as is when the score function value falls between the first setting value and the second setting value (S135).

[0074] Meanwhile, in the process of providing performance analysis results based on score function values, it is necessary to verify the maximum value of the score function to confirm the highest performance obtainable through learning (training). To this end, one embodiment of the present invention may utilize a workload generator (141) and reinforcement learning.

[0075] For example, when hardware specification information is h, workload data information is d, and resource optimization policy information is π, the score function value L can be expressed as Equation 3.

[0076] [Equation 3]

[0077]

[0078] And let us assume that a set of criteria (R) is given, which is metric information, based on the first to sixth performance indicators. In this case, the set of criteria (R) may include conditions where the first to sixth performance indicators are greater than or less than a specific threshold.

[0079] Referring to the algorithm illustrated in Fig. 4, the optimizer is trained using reinforcement learning with current hardware specifications and workload data, and the score function values ​​obtained during training are recorded. The largest value among the recorded score function values ​​is set as the maximum score function value obtainable with the current hardware specifications. Then, the score function value is calculated based on the current optimization resource policy information, and it is checked how close this value is to the maximum score function value. If it is determined that the current score function value has reached the maximum score function value, training can be stopped. The model that recorded the largest score function value obtained during the training of the optimizer via reinforcement learning can be applied when updating the optimizer.

[0080] In addition, Figure 5 shows an algorithm that provides redesign information for expanding the resources of the aforementioned hardware, as well as providing predicted performance information based on the redesign information.

[0081] Specifically, the algorithm of FIG. 5 can fix workload data and resource optimization policy information to given values, perform simulations while varying hardware resources to calculate and compare metrics to determine the optimal hardware resource value, and propose the determined hardware resource value as redesign information.

[0082] For example, the appropriate buffer size can be predicted through the algorithm of Fig. 5. As another example, redesign information can be provided to add a PE-array to the hardware or to expand the bandwidth of the DRAM.

[0083] In one embodiment of the present invention, the monitoring module (150) may evaluate the performance when the AI ​​accelerator (110) directly executes the instructions of the compilation result. At this time, the measured log and metric information, etc. may differ from the simulation. And the log and metric information, etc. collected at this time may be used for the purpose of obtaining a trace data set during the reinforcement learning process of the optimizer.

[0084] One embodiment of the present invention can evaluate the optimization performance of the compiler (120) and, simultaneously or separately, monitor the performance of the compiler (120) and provide the results as notification information.

[0085] The monitoring module (150) can provide notification information when the monitoring result regarding the performance of the compiler (120) satisfies a predetermined notification rule. That is, the monitoring module (150) can output the monitoring result directly to the dashboard, or output it to the dashboard when a predetermined performance change is detected.

[0086] Meanwhile, in the above description, steps S110 to S130 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0087] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0088] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. Explanation of the symbols

[0089] 100: Deep Learning Compiler Optimization Performance Monitoring Device 110: AI Accelerator 120: Compiler 121: Optimizer 130: Metric Module 140: Simulation Module 150: Monitoring Module 160: Logging Module 170: Compiler Training Module

Claims

Claim 1 A method performed by a deep learning compiler optimization performance monitoring device comprises: a step of calculating metric information for evaluating the performance of a compiler; and a step of calculating a score function value corresponding to resource optimization policy information set in an AI-based optimizer of the compiler based on the metric information. A method for monitoring deep learning compiler optimization performance, comprising the step of providing a performance analysis result of the AI-based optimizer based on the score function value, wherein the step of calculating metric information for evaluating the performance of the compiler comprises, as a performance indicator for evaluating the performance of the compiler when executing a given deep learning model under given hardware conditions, a first performance indicator representing the computational blocking time due to DMA during the total time required for the execution of the compiled deep learning model, a second performance indicator representing the time during which a processing element array is utilized during the total time, a third performance indicator representing the number of DMA operations per convolution operation, a fourth performance indicator representing the buffer usage efficiency during the total compilation execution time, a fifth performance indicator representing the total time, and a sixth performance indicator representing the length of the graph generated as an optimization result under the given hardware conditions, wherein the step of providing a performance analysis result of the AI-based optimizer based on the score function value comprises the step of comparing the calculated score function value with a preset first setting value, and, if the comparison result is less than the first setting value, the step of providing information on the possibility of performance improvement through reinforcement learning retraining of the AI-based optimizer as the performance analysis result. Claim 2 A deep learning compiler optimization performance monitoring method according to claim 1, wherein the step of calculating metric information for performance evaluation of the compiler comprises: generating hardware specification information and workload set information for performance evaluation of the compiler; executing a simulation according to the hardware specification information and the workload set information; collecting logs corresponding to the simulation; and calculating metric information for performance evaluation of the compiler based on the logs. Claim 3 delete Claim 4 A method for monitoring deep learning compiler optimization performance, wherein, in claim 1, the step of calculating a score function value corresponding to resource optimization policy information set in the AI-based optimizer of the compiler comprises the step of calculating the score function value by utilizing a score function that takes hardware specification information, a deep learning workload, and the resource optimization policy information as inputs. Claim 5 delete Claim 6 A deep learning compiler optimization performance monitoring method according to claim 1, wherein the step of providing information on the possibility of performance improvement through reinforcement learning retraining of the AI-based optimizer as the performance analysis result is to provide information on the possibility of performance improvement of the AI-based optimizer for a specific deep learning model operating under specific hardware conditions as the performance analysis result. Claim 7 A method for monitoring deep learning compiler optimization performance, wherein, in claim 4, the step of providing a performance analysis result of the AI-based optimizer based on the score function value comprises: a step of comparing the calculated score function value with a preset second setting value; and, if the comparison result is greater than or equal to the second setting value, a step of providing redesign information for expanding hardware resources and predictive performance information based on the redesign information as the performance analysis result. Claim 8 A deep learning compiler optimization performance monitoring method, wherein in claim 7, the step of providing redesign information for resource expansion of the hardware and predictive performance information based on the redesign information as the performance analysis result is to provide redesign information for resource expansion of the hardware and predictive performance information as the performance analysis result, wherein a plurality of various deep learning models can operate universally. Claim 9 A deep learning compiler optimization performance monitoring method according to claim 1, further comprising the step of providing notification information when the monitoring result of the AI-based compiler performance satisfies a predetermined notification rule. Claim 10 A compiler that performs resource optimization policies through an AI-based optimizer and generates and provides instructions for a deep learning model; a metric module that defines and calculates metric information for evaluating the performance of the compiler; a simulation module that calculates a score function value corresponding to resource optimization policy information set in the optimizer based on the metric information; and a monitoring module that provides performance analysis results of the AI-based optimizer based on the score function value. The metric module calculates the metric information as a performance indicator for evaluating the performance of the compiler when executing a given deep learning model under given hardware conditions, the metric information including a first performance indicator representing the computation blocking time due to DMA during the total time required for the execution of the compiled deep learning model, a second performance indicator representing the time during which a processing element array is utilized during the total time, a third performance indicator representing the number of DMA operations per convolution operation, a fourth performance indicator representing the buffer usage efficiency during the total compilation execution time, a fifth performance indicator representing the total time, and a sixth performance indicator representing the length of the graph generated as an optimization result under the given hardware conditions. The monitoring module [compiles] the calculated score function value with a preset first setting value and A deep learning compiler optimization performance monitoring device that compares and, if the comparison result is less than a first set value, provides information on potential performance improvement through reinforcement learning retraining of the AI-based optimizer as the performance analysis result.

Images