System and method for running a computational processing graph of an artificial neural network

Abstract

A system and method for running a computational processing graph of an artificial neural network are provided. A system for managing a computational processing graph of an artificial neural network running through multiple heterogeneous resources includes: a task manager configured to allocate the multiple heterogeneous resources to a first subgraph and a second subgraph to be run, the first subgraph and the second subgraph being included in the computational processing graph; a first compiler configured to compile the first subgraph to be executable on a first resource among the multiple heterogeneous resources; and a second compiler configured to compile the second subgraph to be executable on a second resource among the multiple heterogeneous resources, wherein the first subgraph and the second subgraph are managed separately through independent computation paths.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2018-0136801, filed on November 8, 2018, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. Technical Field

[0002] The inventive concept relates to a neural network system for managing a calculation processing graph of an artificial neural network and a method for managing a calculation processing graph by using a neural network system, and more particularly, to a neural network system for performing parallel compilation on a calculation processing graph and a method for managing a calculation processing graph by using a neural network system. Background Art

[0003] Artificial neural network (ANN) represents a computing architecture that models the biological brain. Deep learning, machine learning, etc. can be implemented based on artificial neural networks. Recently, as the calculations processed by using ANNs have increased rapidly, there is a need to efficiently perform calculation processing by using ANNs. Summary of the invention

[0004] The inventive concept provides a neural network system for parallel processing of a computation processing graph allocated to a plurality of heterogeneous resources and a method for managing the computation processing graph by using the neural network system.

[0005] According to one aspect of the inventive concept, a system for managing a computational processing graph of an ANN running through multiple heterogeneous resources is provided, the system comprising: a task manager, configured to allocate the multiple heterogeneous resources to a first subgraph and a second subgraph to be run, the first subgraph and the second subgraph being included in the computational processing graph; a first compiler, configured to compile the first subgraph to be able to run on a first resource among the multiple heterogeneous resources; and a second compiler, configured to compile the second subgraph to be able to run on a second resource among the multiple heterogeneous resources, wherein the first subgraph and the second subgraph are managed separately through independent computational paths.

[0006] According to another aspect of the inventive concept, a method for managing a computational processing graph of an ANN running through multiple heterogeneous resources is provided, the method comprising: assigning a first subgraph among multiple subgraphs constituting the computational processing graph to a first resource among the multiple heterogeneous resources; compiling the first subgraph to be able to run on the first resource; assigning a second subgraph among the multiple subgraphs to a second resource among the multiple heterogeneous resources; and compiling the second subgraph to be able to run on the second resource, wherein the first subgraph and the second subgraph are managed separately through independent computational paths.

[0007] According to another aspect of the inventive concept, there is provided an apparatus for performing deep learning by using a computational processing graph of an ANN, the apparatus comprising: a first compiler configured to compile at least a portion of the computational processing graph to be capable of running on a first resource; a second compiler configured to compile at least a portion of the computational processing graph to be capable of running on a second resource different from the first resource; and a task manager configured to output a first subgraph included in the computational processing graph to the first compiler, and output a second subgraph included in the computational processing graph to the second compiler, wherein the first compiler and the second compiler operate independently of each other.

[0008] In one aspect of the inventive concept, a system for managing a computational processing graph of an ANN running through multiple heterogeneous resources is provided, the system comprising: at least one processor configured to implement: a task manager configured to allocate the multiple heterogeneous resources to a first subgraph and a second subgraph, the first subgraph and the second subgraph belonging to the computational processing graph of the ANN; a first compiler configured to compile the first subgraph to be able to run on a first resource among the multiple heterogeneous resources; and a second compiler configured to compile the second subgraph to be able to run on a second resource among the multiple heterogeneous resources, wherein the first subgraph and the second subgraph are output to the first compiler and the second compiler respectively through independent computational paths.

[0009] In another aspect of the inventive concept, a method for managing a computational processing graph of an ANN running through multiple heterogeneous resources is provided, the method comprising: assigning a first subgraph among multiple subgraphs of the computational processing graph to a first resource among the multiple heterogeneous resources, the computational processing graph belonging to the ANN; compiling the first subgraph to be able to run on the first resource; assigning a second subgraph among the multiple subgraphs to a second resource among the multiple heterogeneous resources; and compiling the second subgraph to be able to run on the second resource, wherein the first subgraph and the second subgraph are respectively output to a first compiler and a second compiler through independent computational paths.

[0010] In another aspect of the inventive concept, a device for performing deep learning by using a computational processing graph of an ANN is provided, the device comprising: at least one processor configured to implement: a first compiler configured to compile at least a portion of the computational processing graph to be capable of running on a first resource, the computational processing graph belonging to the ANN; a second compiler configured to compile at least a portion of the computational processing graph to be capable of running on a second resource different from the first resource; and a task manager configured to output a first subgraph in the computational processing graph to the first compiler, and output a second subgraph in the computational processing graph to the second compiler, wherein the first compiler and the second compiler operate independently of each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Embodiments of the inventive concept will be more clearly understood from the following detailed description in conjunction with the accompanying drawings, in which:

[0012] Figure 1 is a block diagram illustrating a neural network system according to an example embodiment;

[0013] Figure 2 is a block diagram illustrating a model processor according to an example embodiment;

[0014] Figure 3 is a flow chart illustrating the operation of a model processor according to an example embodiment;

[0015] Figure 4 is a diagram illustrating a computational processing diagram according to an example embodiment;

[0016] Figure 5 is a flow chart illustrating the operation of a model analyzer according to an example embodiment;

[0017] Figure 6 is a diagram illustrating the operation of a model processor according to an example embodiment;

[0018] Figure 7 is a diagram illustrating the operation of a model processor according to an example embodiment;

[0019] Figure 8 is a block diagram illustrating a neural network system according to an example embodiment;

[0020] Fig. 9 is a flow chart illustrating the operation of a neural network system according to an example embodiment;

[0021] Fig.10 is a diagram illustrating the operation of a neural network system according to an example embodiment;

[0022] Fig.11 is a diagram illustrating the operation of a neural network system according to an example embodiment;

[0023] Fig.12 is a diagram illustrating the operation of a neural network system according to an example embodiment;

[0024] Fig.13 is a block diagram illustrating a neural network system according to an example embodiment;

[0025] Fig.14 is a diagram illustrating the operation of a neural network system according to an example embodiment; and

[0026] Fig.15 is a block diagram illustrating an electronic system according to example embodiments. DETAILED DESCRIPTION

[0027] Figure 1 is a block diagram illustrating a neural network system according to an example embodiment.

[0028] Reference Figure 1 , the neural network system 10 (e.g., an artificial neural network system) may include multiple application APPs, a model processor 100, and multiple resources including a first resource 210, a second resource 220, and an Nth resource 230. Multiple application APPs may be generated through various application program interfaces (APIs), and may perform various tasks by using an artificial neural network model. For example, multiple application APPs may request the model processor 100 to perform data processing using an ANN model by outputting an input model IM to the model processor 100.

[0029] The model processor 100 may include a model analyzer 110, a task manager 120, and a plurality of compilers 130. Each component included in the model processor 100 may be implemented using software, and may be implemented using a neural network platform such as an evolutionary deep network (EDEN). In another example, each component included in the model processor 100 may be implemented using hardware such as an electronic circuit. In one example, the model processor 100 may be implemented using a framework of an ANN. In another example, the model processor 100 may be implemented by at least one processor, such that a single processor implements the model processor 100 or a plurality of processors implement the model processor 100.

[0030] The model analyzer 110 may receive an input model IM including data using an ANN from a plurality of applications APP, and may generate a computational processing graph including a plurality of subgraphs by analyzing the input model IM. The term "computational processing graph" used herein may represent a data structure represented by a graph including a plurality of computing nodes. The model analyzer 110 may generate a plurality of subgraphs by dividing the computational processing graph into structures suitable for being processed by a plurality of resources including a first resource 210, a second resource 220, and an Nth resource 230, respectively. The model analyzer 110 may output the computational processing graph and the plurality of subgraphs to the task manager 120.

[0031] The task manager 120 may allocate multiple subgraphs including computing nodes to multiple resources including the first resource 210, the second resource 220, and the Nth resource 230, whereby additional resources may exist between the second resource 220 and the Nth resource 230. According to the inventive concept, the task manager 120 may allocate appropriate resources (e.g., the first resource 210, the second resource 220, or the Nth resource 230) to each of the multiple subgraphs based on the structural characteristics of each subgraph and the type of calculation included in each subgraph. The task manager 120 may output the multiple subgraphs to the multiple compilers 130 via separate computing paths, each compiler 130 corresponding to the allocated resources. According to one embodiment, the multiple subgraphs may be managed separately via separate computing paths (or threads), so as to be output to the multiple compilers 130 in parallel.

[0032] Each of the plurality of compilers 130 may compile the subgraph received from the task manager 120 so that the subgraph is converted into a programming language understandable by each of the plurality of resources including the first resource 210, the second resource 220, and the Nth resource. Each of the plurality of compilers 130 may correspond to one resource, and may compile the subgraph received from the task manager 120 into the programming language of the corresponding resource. In one example, a first compiler among the plurality of compilers 130 may correspond to the first resource 210, and may compile the received subgraph into the programming language corresponding to the first resource 210. In addition, a second compiler among the plurality of compilers 130 may correspond to the second resource 220, and may compile the received subgraph into the programming language corresponding to the second resource 220. Each of the plurality of compilers 130 may generate execution data ED by compiling the subgraph, and may output the generated execution data ED to the corresponding resource (e.g., the first resource 210, the second resource 220, or the Nth resource 230). In one example, the execution data ED may be implemented as a programming file converted into the programming language corresponding to the resource.

[0033] According to the inventive concept, the plurality of compilers 130 can perform compilation independently of each other, and thus, parallel compilation can be performed on the plurality of subgraphs. In one example, even when the first compiler among the plurality of compilers 130 compiles the first subgraph, the second compiler among the plurality of compilers 130 can compile the second subgraph. Therefore, the time required for compilation can be reduced, and the calculation process of the ANN can be efficiently performed.

[0034] The plurality of resources including the first resource 210, the second resource 220, and the Nth resource 230 may include computing resources and communication resources. For example, the computing resources may include various computing processing devices (such as a central processing unit (CPU), a graphics processing unit (GPU), an application processor (AP), a digital signal processor (DSP), a field programmable gate array (FPGA), a neural network processing unit (NPU), an electronic control unit (ECU), an image signal processor (ISP), etc.). The communication resources may include resources capable of supporting internal communication (such as a system bus, a peripheral component interconnect (PCI), a PCI Express (PCIe), a serial advanced technology attachment (SATA), a small computer system interface (SCSI), and a serial attached SCSI (SAS)) and / or resources capable of supporting external communication (such as a universal serial bus (USB), Ethernet, WiFi, Bluetooth, near field communication (NFC), radio frequency identification (RFID), and mobile telecommunications).

[0035] Each computing resource may include specific task-specific hardware. For example, an NPU may include specific task-specific hardware for processing computations using an ANN.

[0036] The plurality of resources including the first resource 210, the second resource 220, and the Nth resource 230 may include heterogeneous resources. In one example, the first resource 210 may include an NPU, and the second resource 220 may include a GPU. Each of the plurality of resources including the first resource 210, the second resource 220, and the Nth resource 230 may perform calculation of a subgraph by executing execution data ED received from a corresponding compiler.

[0037] According to the inventive concept, a plurality of resources including the first resource 210, the second resource 220, and the N-th resource 230 may load the execution data ED independently of each other. The time for each of the plurality of resources including the first resource 210, the second resource 220, and the N-th resource 230 to load the execution data ED from the model processor 100 may be required, and in one example, when the first resource 210 performs calculation by executing the first execution data corresponding thereto, the second resource 220 may load the second execution data corresponding thereto from the model processor 100, so that the loading time may be reduced and the total calculation processing time may be reduced.

[0038] Although not shown, the model processor 100 may further include an operating system for running the model analyzer 110, the task manager 120, and the plurality of compilers 130. The operating system may include various mobile operating systems, online operating systems, offline operating systems, etc. In one example, the operating system may be implemented using Linux, Linux-based Android, Windows, iOS, MacOS, Chromium OS, etc.

[0039] Figure 2 is a block diagram illustrating a model processor according to an example embodiment. Figure 1 Given the duplicate descriptions, they will be omitted.

[0040] Reference Figure 2 , the model processor 100 may include a model analyzer 110, a task manager 120, a first compiler 131 and a second compiler 132, and the model analyzer 110 may include a model parser 111, a model builder 112 and a model optimizer 113. Figure 2 An example is shown in which the model processor 100 includes two compilers (eg, 131 and 132 ), but this is merely an example, and the model processor 100 may include more than two compilers.

[0041] The model parser 111 may generate model information Info_M by parsing the input model IM received from the application APP. The input model IM may include request information for executing instructions programmed into the application APP, and the term "parsing" may refer to a process of analyzing a grammatical structure or syntax of the input model IM.

[0042] The model builder 112 may generate a computation processing graph CPG' from the input model IM based on the model information Info_M received from the model parser 111. The computation processing graph CPG' may represent a graph including a plurality of computation nodes.

[0043] The model optimizer 113 may perform an optimization operation on the computation processing graph CPG', and may output the computation processing graph CPG generated as a result of the optimization operation to the task manager 120. The optimization operation may represent an operation that minimizes and optimizes the computation processing amount of the computation processing graph CPG'. In one embodiment, the optimization operation may include a graph replacement operation that replaces at least a portion of the computation processing graph CPG' with another graph, a graph merging operation that merges at least a portion of the computation processing graph CPG', and a graph partitioning operation that partitions at least a portion of the computation processing graph CPG'.

[0044] The processing graph partitioner in the model analyzer 110 (not shown, and may be included in the model optimizer 113 or be independent of the model optimizer 113) may partition the optimized computation processing graph CPG into a plurality of subgraphs including a first subgraph SG1 and a second subgraph SG2, and may output the first subgraph SG1 and the second subgraph SG2 to the task manager 120. In one embodiment, the model optimizer 113 may be omitted, and the processing graph partitioner may directly partition the computation processing graph CPG' into a plurality of subgraphs, but in this case, the efficiency of the system may be reduced.

[0045] The task manager 120 may respectively allocate the plurality of subgraphs SG1 and SG2 of the computation processing graph CPG to the plurality of compilers including the first compiler 131 and the second compiler 132. According to one embodiment, each of the plurality of subgraphs SG1 and SG2 may correspond to a resource, and the task manager 120 may respectively allocate the plurality of subgraphs SG1 and SG2 to the plurality of compilers including the first compiler 131 and the second compiler 132, each compiler corresponding to an appropriate resource.

[0046] In one embodiment, the task manager 120 may receive information about the hardware capability of each of the plurality of resources and may determine a suitable resource for each of the plurality of subgraphs SG1 and SG2. The task manager 120 may respectively assign the plurality of subgraphs SG1 and SG2 to a plurality of compilers including a first compiler 131 and a second compiler 132, each compiler corresponding to the determined resource.

[0047] exist Figure 2 In the example of , the first compiler 131 may correspond to a first resource (not shown), and the second compiler 132 may correspond to a second resource (not shown). That is, the first compiler 131 may compile the received input into a programming language used in the first resource, and the second compiler 132 may compile the received input into a programming language used in the second resource.

[0048] The task manager 120 may allocate the first subgraph SG1 of the received computing process graph CPG to the first resource, and allocate the second subgraph SG2 of the received computing process graph CPG to the second resource, based on the information about the hardware capability of each resource. The task manager 120 may output the first subgraph SG1 allocated to the first resource to the first compiler 131 corresponding to the first resource, and output the second subgraph SG2 allocated to the second resource to the second compiler 132 corresponding to the second resource. The first compiler 131 may compile the first subgraph SG1 to be able to run on the first resource, and the second compiler 132 may compile the second subgraph SG2 to be able to run on the second resource. In one embodiment, compiling a subgraph to be able to run on a resource is to convert or transform the subgraph into a code or form that will be read and executed by the resource.

[0049] Figure 3 is a flow chart illustrating the operation of a model processor according to an example embodiment.

[0050] Reference Figure 2 and Figure 3, the model processor 100 may generate a first subgraph SG1 by analyzing the input model IM (S110). The model processor 100 may allocate the first subgraph SG1 to the first resource (S120). The model processor 100 may compile the first subgraph SG1 to be able to run on the first resource (S130). In one embodiment, the first subgraph SG1 may be compiled by a first compiler 131 corresponding to the first resource.

[0051] The model processor 100 may generate a second subgraph SG2 by analyzing the input model IM (S140). The model processor 100 may allocate the second subgraph SG2 to the second resource (S150). The model processor 100 may compile the second subgraph SG2 to be able to run on the second resource (S160). In one embodiment, the second subgraph SG2 may be compiled by a second compiler 132 corresponding to the second resource.

[0052] According to one embodiment, the model processor 100 may compile the first subgraph SG1 before generating the second subgraph SG2 by analyzing the input model IM. That is, the first subgraph SG1 may be compiled before the analysis of the input model IM is completed. Therefore, the time required to process the computation processing graph CPG may be reduced.

[0053] Figure 4 is a diagram illustrating a calculation process diagram according to an example embodiment.

[0054] Reference Figure 2 and Figure 4 The model analyzer 110 may generate a calculation process graph CPG including a first calculation OP00 to a fourteenth calculation OP13 by analyzing the input model IM. Each of the first calculation OP00 to the fourteenth calculation OP13 may represent various mathematical calculations (e.g., convolution calculations, rectified linear unit calculations, memory copy calculations, etc.), and the first calculation OP00 to the fourteenth calculation OP13 may be the same or different from each other. It should be understood that the number of calculations here is only exemplary and is not limited to fourteen.

[0055] The model analyzer 110 may divide the generated computation processing graph CPG into a plurality of subgraphs SG1, SG2, and SG3 based on the type of computation, computation preference, graph shape, etc. Figure 4 In the example of , the model analyzer 110 may classify the first calculation OP00 to the fourth calculation OP03 as the first subgraph SG1, classify the fifth calculation OP04 to the eleventh calculation OP10 as the second subgraph SG2, and classify the twelfth calculation OP11 to the fourteenth calculation OP13 as the third subgraph SG3. It should be understood that the number of the multiple subgraphs here is only exemplary and is not limited to three.

[0056] The task manager 120 may receive a computational processing graph CPG including first to third subgraphs SG1 to SG3, and may allocate the first to third subgraphs SG1 to SG3 to a plurality of resources, respectively. In one embodiment, the task manager 120 may allocate each of the first to third subgraphs SG1 to SG3 to a suitable resource based on the hardware capability of each of the plurality of resources.

[0057] In one example, the task manager 120 may allocate the first subgraph SG1 and the third subgraph SG3 to a first resource (e.g., NPU), and allocate the second subgraph SG2 to a second resource (e.g., GPU). In another example, the task manager 120 may allocate the first subgraph SG1 to a first resource (e.g., NPU), allocate the second subgraph SG2 to a second resource (e.g., GPU), and allocate the third subgraph SG3 to a third resource (e.g., CPU).

[0058] The task manager 120 may output each of the first to third subgraphs SG1 to SG3 to each of the compilers 131 and 132 corresponding to the allocated resources, and each of the compilers 131 and 132 may generate a plurality of execution data by compiling each of the received first to third subgraphs SG1 to SG3.

[0059] Figure 5 is a flow chart illustrating the operation of a model analyzer according to an example embodiment.

[0060] Reference Figure 2 and Figure 5, the model analyzer 110 may extract model information Info_M from the input model IM (S210). The model analyzer 110 may generate a computational processing graph CPG' (S220) by analyzing the input model IM based on the model information Info_M. The model analyzer 110 may optimize the computational processing graph CPG' (S230). In one example, the model analyzer 110 may generate an optimized computational processing graph CPG by performing at least one of a graph replacement operation of replacing at least a portion of the computational processing graph CPG' with another graph, a graph merging operation of merging at least a portion of the computational processing graph CPG', and a graph partitioning operation of partitioning at least a portion of the computational processing graph CPG'. Hereinafter, the expression "at least one of a, b, and c" shall be understood as including only a, only b, only c, both a and b, both a and c, both b and c, all of a, b, and c, or all variations thereof. The model analyzer 110 can sequentially generate multiple subgraphs SG1 and SG2 (S240) by dividing the optimized calculation processing graph CPG according to the graph features. For example, the model analyzer 110 can divide the optimized calculation processing graph CPG into multiple subgraphs based on the type of calculation, calculation preference, the shape of the graph, etc. Here, the merge operation is used to merge at least part of the calculation processing graph CPG' into smaller parts. For example, the merge operation can be performed on the first calculation OP00 to the third calculation OP02 as at least a part of the calculation processing graph CPG' to obtain a fifth calculation that is smaller than the first calculation OP00 to the third calculation OP02. Here, the partition operation is used to divide at least a part of the calculation processing graph CPG' into a new part of the calculation processing graph CPG'. For example, the partition operation can be performed on the first calculation OP00 as at least a part of the calculation processing graph CPG' to obtain the fifth calculation and the sixth calculation as the new part of the calculation processing graph CPG'.

[0061] Figure 6 is a diagram illustrating the operation of a model processor according to an example embodiment. Specifically, Figure 6 is a diagram illustrating the operation of a model processor over time.

[0062] Reference Figure 6 , the model analyzer 110 may analyze the input model (T110), and may output the first subgraph SG1 generated as a result of the analysis to the task manager 120 (T120). In one embodiment, the model analyzer 110 may generate a calculation processing graph based on the input model, and may generate the first subgraph SG1 by segmenting the calculation processing graph based on the calculated features. The first subgraph SG1 may be generated earlier than the second subgraph SG2 (i.e., generated before the second subgraph SG2).

[0063] The task manager 120 may allocate the first subgraph SG1 to the first resource RS1 based on the characteristics of the calculations included in the first subgraph SG1 and the hardware capabilities of the plurality of resources (T130). The task manager 120 may output the first subgraph SG1 to the first compiler 131 corresponding to the first resource RS1 (T140). The first compiler 131 may compile the first subgraph SG1 even before the second subgraph SG2 is generated.

[0064] The model analyzer 110 may output the second subgraph SG2 generated as a result of analyzing the input model to the task manager 120 (T160). The model analyzer 110 may generate the second subgraph SG2 by partitioning the calculation processing graph based on the characteristics of the calculation. The task manager 120 may allocate the second subgraph SG2 to the second resource RS2 based on the characteristics of the calculation included in the second subgraph SG2 and the hardware capabilities of the plurality of resources (T170). The task manager 120 may output the second subgraph SG2 to the second compiler 132 corresponding to the second resource RS2 (T180), and the second compiler 132 may compile the second subgraph SG2 (T190).

[0065] According to one embodiment, even before analysis of the input model is completed, the model analyzer 110 may output the first subgraph SG1 generated first to the task manager 120 , and the task manager 120 may output the first subgraph SG1 first to the first compiler 131 .

[0066] Figure 7 is a diagram illustrating the operation of a model processor according to an example embodiment. Specifically, Figure 7 is a diagram illustrating the operation of a model processor over time.

[0067] Reference Figure 7 , the model analyzer 110 may analyze the input model (T210), and may output the generated first subgraph SG1 and second subgraph SG2 to the task manager 120 (T220). The task manager 120 may allocate the first subgraph SG1 to the first resource RS1 and the second subgraph SG2 to the second resource RS2 based on the characteristics of the calculations included in the first subgraph SG1 and the second subgraph SG2 and the hardware capabilities of the plurality of resources (T230). The first compiler 131 may compile the first subgraph SG1 (T250), and the second compiler 132 may compile the second subgraph SG2 (T260).

[0068] According to one embodiment, a plurality of compilers including the first compiler 131 and the second compiler 132 may compile the plurality of subgraphs SG1 and SG2 in parallel, respectively, and the compilation of the plurality of subgraphs SG1 and SG2 may be completed quickly.

[0069] Figure 8 is a block diagram illustrating a neural network system according to an example embodiment. Figure 2 Given the duplicate descriptions, they will be omitted.

[0070] Reference Figure 8 , the neural network system 10 may include a model processor 100, a first resource 210 and a second resource 220, and the model processor 100 may include a task manager 120, a first compiler 131, a second compiler 132 and a model holder 140. Figure 2 The task manager 120 , the first compiler 131 , and the second compiler 132 are described, so descriptions thereof will be omitted.

[0071] The first compiler 131 may receive the first subgraph SG1 from the task manager 120 and may generate first execution data ED1 by compiling the first subgraph SG1 to be executable on the first resource 210. The first compiler 131 may output the first execution data ED1 to the model holder 140.

[0072] The second compiler 132 may receive the second subgraph SG2 from the task manager 120 and may generate second execution data ED2 by compiling the second subgraph SG2 to be executable on the second resource 220. The second compiler 132 may output the second execution data ED2 to the model holder 140.

[0073] The model holder 140 may store the first execution data ED1 and the second execution data ED2, output the first execution data ED1 to the first resource 210 in response to or based on a request of the first resource 210, and output the second execution data ED2 to the second resource 220 in response to or based on a request of the second resource 220. In one embodiment, when the first compiler completes compiling the first subgraph and the second compiler completes compiling the second subgraph, the model holder may simultaneously output the first execution data to the first resource and the second execution data to the second resource based on the timing. In another embodiment, when the first compiler completes compiling the first subgraph and the second compiler completes compiling the second subgraph, the model holder may sequentially output the first execution data to the first resource and the second execution data to the second resource based on the timing.

[0074] The model holder 140 may include at least one storage device. As an example, the model holder 140 may include at least one of a volatile memory and a non-volatile memory. The non-volatile memory may include a flash memory, a phase change random access memory (PRAM), a magnetic random access memory (MRAM), a resistive random access memory (RRAM), a ferroelectric random access memory (FRAM), etc., and the volatile memory may include a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous DRAM (SDRAM), a latch, a flip-flop, a register, etc.

[0075] Despite Figure 8 The model holder 140 is shown as a component inside the model processor 100, but in another example, the model holder 140 may be implemented as a component separate from the model processor 100. In one example, the model holder 140 may be implemented by using at least one of a hard disk drive (HDD), a solid state drive (SSD), a compact flash (CF), a secure digital (SD), a micro SD, a mini SD, an extreme digital (xD), and a memory stick. In addition, the model processor 100 may include direct memory access (DMA), memory DMA (MDMA), peripheral DMA (PDMA), remote DMA (RDMA), smart DMA (SDMA), etc. to control access to the model holder 140.

[0076] The first resource 210 may receive the first execution data ED1, and may perform a first calculation corresponding to the first subgraph SG1 based on the first execution data ED1. The second resource 220 may receive the second execution data ED2 independently of the first resource 210, and may load the second execution data ED2. The first resource 210 may generate a calculation result CR by performing the first calculation, and when the second resource 210 receives the calculation result CR, the second resource 210 may perform a second calculation corresponding to the second subgraph SG2 based on the loaded second execution data ED2 and the calculation result CR.

[0077] According to one embodiment, each of the plurality of resources 210 and 220 may reduce computation time by pre-loading execution data before receiving computation results based on previous computations from other resources, and thus, computation processing capabilities of the neural network system 10 may be improved.

[0078] Fig. 9 is a flow chart illustrating the operation of a neural network system according to an example embodiment.

[0079] Reference Figure 8 and Fig. 9, the neural network system 10 may generate first execution data ED1 by compiling the first subgraph SG1 (S310). The neural network system 10 may perform a first calculation corresponding to the first subgraph SG1 by using the first execution data ED1 (S320). The neural network system 10 may generate second execution data ED2 by compiling the second subgraph SG2 (S330). Even before the first calculation for the first execution data ED1 is completed, the neural network system 10 may load the second execution data ED2 (S340). The neural network system 10 may generate a calculation result CR by completing the first calculation (S350), and may perform a second calculation corresponding to the second subgraph SG2 by using the generated calculation result CR and the pre-loaded second execution data ED2 (S360).

[0080] Fig.10 is a diagram illustrating the operation of a neural network system according to an example embodiment. Specifically, Fig.10 is a diagram illustrating the operation of a neural network system over time after compiling multiple subgraphs.

[0081] Reference Fig.10 , the first compiler 131 corresponding to the first resource 210 may compile the first subgraph SG1 (T310), and may output the first execution data ED1 generated thereby to the model holder 140 (T320). The model holder 140 may output the received first execution data ED1 to the first resource 210 (T330), and the first resource 210 may perform a first calculation based on the first execution data ED1 (T340).

[0082] The second compiler 132 corresponding to the second resource 220 may compile the second subgraph SG2 (T350), and may output the second execution data ED2 generated thereby to the model holder 140 (T360). The model holder 140 may output the received second execution data ED2 to the second resource 220 (T370), and the second resource 220 may load the second execution data ED2 (T380).

[0083] The first resource 210 may output the calculation result CR generated by performing the first calculation to the second resource 220 ( T390 ), and the second resource 220 may perform the second calculation by using the calculation result CR and the loaded second execution data ED2 ( T395 ).

[0084] Fig.11 is a diagram illustrating the operation of a neural network system according to an example embodiment. Specifically, Fig.11 is a diagram illustrating the operation of a neural network system over time after compiling multiple subgraphs.

[0085] Reference Fig.11 , the first compiler 131 corresponding to the first resource 210 may compile the first subgraph SG1 (T410), and the second compiler 132 corresponding to the second resource 220 may compile the second subgraph SG2 (T420). The compilation of the second subgraph SG2 may be completed before the compilation of the first subgraph SG1 is completed, and therefore, the second compiler 132 may output the generated second execution data ED2 to the model holder 140 (T430).

[0086] The model holder 140 may output the second execution data ED2 to the second resource 220 ( T440 ), and the second resource 220 may load the second execution data ED2 ( S450 ) even before the compilation ( T410 ) of the first subgraph SG1 is completed.

[0087] The first compiler 131 may output the first execution data ED1 generated by the completion of the compilation of the first subgraph SG1 to the model holder 140 (T460). The model holder 140 may output the first execution data ED1 to the first resource 210 (T470), and the first resource 210 may perform the first calculation corresponding to the first subgraph SG1 (T480).

[0088] The first resource 210 may output the calculation result CR generated by performing the first calculation to the second resource 220 ( T490 ), and the second resource 220 may perform the second calculation by using the calculation result CR and the loaded second execution data ED2 ( T495 ).

[0089] Fig.12 is a diagram illustrating the operation of a neural network system according to an example embodiment. Specifically, Fig.12 is a diagram illustrating the operation of a neural network system over time after compiling multiple subgraphs.

[0090] Reference Fig.12 , the first compiler 131 corresponding to the first resource 210 may compile the first subgraph SG1 (T510), and may output the first execution data ED1 generated thereby to the model holder 140 (T520). The model holder 140 may output the received first execution data ED1 to the first resource 210 (T530), and the first resource 210 may perform a first calculation based on the first execution data ED1 (T540).

[0091] The second compiler 132 corresponding to the second resource 220 may compile the second subgraph SG2 (T550), and may output the second execution data ED2 generated thereby to the model holder 140 (T560). When the second resource 220 is able to load the second execution data ED2, the second resource 220 may output the second execution data request signal Req_ED2 to the model holder 140 (T570). In one example, the second resource 220 may output the second execution data request signal Req_ED2 to the model holder 140 after completing all current tasks. In another example, when the second resource 220 is available to execute a task, the second resource 220 may output the second execution data request signal Req_ED2 to the model holder 140.

[0092] Even in the process of performing the first calculation ( T540 ), the model holder 140 may output the second execution data ED2 to the second resource 220 ( T580 ) in response to or based on the second execution data request signal Req_ED2 , and the second resource 220 may load the second execution data ED2 ( T585 ).

[0093] The first resource 210 may output the calculation result CR generated by performing the first calculation to the second resource 220 ( T590 ), and the second resource 220 may perform the second calculation by using the calculation result CR and the loaded second execution data ED2 ( T595 ).

[0094] Fig.13 is a block diagram illustrating a neural network system according to an example embodiment. Figure 8 Given the duplicate descriptions, they will be omitted.

[0095] Reference Fig.13 , the neural network system 10a may include a model processor 100a, a first resource 210a and a second resource 220a, and the model processor 100a may include a task manager 120a, a first compiler 131a and a model holder 140a. Figure 8 The task manager 120 and the first compiler 131 are the same or similar, so their description will be omitted.

[0096] The first compiler 131a may receive the first subgraph SG1 from the task manager 120a, and may generate first execution data ED1 by compiling the first subgraph SG1 to be executable on the first resource 210a. The first compiler 131a may output the first execution data ED1 to the model holder 140a.

[0097] The model holder 140a may store the first execution data ED1 and the first pre-built data PD1. The first pre-built data PD1 may be pre-built data corresponding to the second subgraph. The term "pre-built data" used herein may refer to data having a pre-known compilation result of a specific subgraph. The neural network system 10a may store the data having a specific subgraph compiled therein in advance in the model holder 140a, and may perform calculations by using the pre-built data stored in the model holder 140a during the calculation process of the specific subgraph without additional compilation.

[0098] The model holder 140 a may output the first execution data ED1 to the first resource 210 a in response to or based on a request of the first resource 210 a , and may output the first pre-built data PD1 to the second resource 220 a in response to or based on a request of the second resource 220 a .

[0099] The first resource 210a may receive the first execution data ED1, and may perform a first calculation corresponding to the first subgraph SG1 based on the first execution data ED1. The second resource 220a may receive the first pre-built data PD1 independently of the first resource 210a, and may load the first pre-built data PD1. The first resource 210a may generate a calculation result CR by performing the first calculation, and when the second resource 220a receives the calculation result CR, the second resource 220a may perform a second calculation corresponding to the second subgraph based on the loaded first pre-built data PD1 and the calculation result CR.

[0100] Fig.14 is a diagram illustrating the operation of a neural network system according to an example embodiment. Specifically, Fig.14 is a diagram illustrating the operation of a neural network system over time after compiling multiple subgraphs.

[0101] Reference Fig.14 , the first compiler 131a corresponding to the first resource 210a may compile the first subgraph SG1 (T610), and may output the first execution data ED1 generated thereby to the model holder 140a (T620). The model holder 140a may output the received first execution data ED1 to the first resource 210a (T630), and the first resource 210a may perform a first calculation based on the first execution data ED1 (T640).

[0102] When the second resource 220a is able to load the first pre-built data PD1, the second resource 220a may output the first pre-built data request signal Req_PD1 to the model holder 140a (T650). In one example, the second resource 220a may output the first pre-built data request signal Req_PD1 to the model holder 140a after completing all current tasks. In another example, when the second resource 220a is available to perform a task, the second resource 220a may output the first pre-built data request signal Req_PD1 to the model holder 140a.

[0103] Even in the process of performing the first calculation ( T640 ), the model holder 140 a may output the first prebuilt data PD1 to the second resource 220 a ( T660 ) ​​in response to or based on the first prebuilt data request signal Req_PD1 , and the second resource 220 a may load the first prebuilt data PD1 ( T670 ).

[0104] The first resource 210a may output the calculation result CR generated by performing the first calculation to the second resource 220a (T680), and the second resource 220a may perform the second calculation by using the calculation result CR and the loaded first pre-built data PD1 (T595).

[0105] Fig.15 is a block diagram illustrating an electronic system according to example embodiments.

[0106] Reference Fig.15 , the electronic system 1000 can extract effective information by analyzing input data in real time based on a neural network, and can determine a situation based on the extracted information, or can control components of an electronic device in which the electronic system 1000 is installed. For example, the electronic system 1000 can be applied to a robot device (such as a drone or an advanced driver assistance system (ADAS)), a smart TV, a smart phone, a medical device, a mobile device, an image display device, a measuring device, an IoT device, etc., and besides, can also be installed in at least one of various electronic devices.

[0107] The electronic system 1000 may include a processor 1010, a RAM 1020, a model processor 1030, a memory 1040, a sensor module 1050, and a resource 1060. The components of the electronic system 1000 may be connected to each other in a communicable manner via a bus. The processor 1010 may control the overall operation of the electronic system 1000. For example, the processor 1010 may control the function of the model processor 1030 by executing a program stored in the RAM 1020. In addition, the processor 1010 may output an input model to be deep-learned based on a neural network to the model processor 1030 by controlling the RAM 1020, the memory 1040, etc. The RAM 1020 may temporarily store programs, data, applications, or instructions.

[0108] The model processor 1030 may perform calculations of a neural network by controlling the resource 1060, and may generate an information signal based on the result of performing the calculation. The memory 1040 is a storage unit for storing data, and may store, for example, various data generated in the process of performing calculations by the model processor 1030 and the resource 1060. The sensor module 1050 may sense or receive a signal (e.g., an image signal, an audio signal, a magnetic signal, a bio-signal, a touch signal, etc.) from the outside of the electronic system 1000, and may convert the signal into data. The resource 1060 may include a computing resource for performing a large amount of calculations based on a neural network, or a communication resource implemented using various wired or wireless interfaces that can communicate with devices outside the electronic system 1000.

[0109] According to the inventive concept, the model processor 1030 may perform parallel compilation to output execution data to the resource 1060 , so that the process of computing processing may be efficiently performed.

[0110] The above-described example embodiments may be written into a program executable on a computer, and may be implemented in a general-purpose digital computer that runs the program by using a computer-readable recording medium. In addition, the data structure used in the above-described example embodiments may be recorded in a computer-readable recording medium by various means. The computer-readable recording medium may include a storage medium, such as a magnetic storage medium (e.g., a read-only memory (ROM), a floppy disk, a hard disk, etc.) or an optical reading medium (e.g., a compact disk-ROM (CD-ROM), a digital versatile disk (DVD), etc.).

[0111] Although the inventive concept has been specifically shown and described with reference to the embodiments of the inventive concept, it will be understood by those skilled in the art that various changes in form and details may be made in the embodiments without departing from the spirit and scope of the claims. In addition, it should be understood that the specific terms used herein are only used for the purpose of describing the embodiments and are not intended to limit the inventive concept. Therefore, the scope of the inventive concept should be defined by the claims and their equivalents.

Claims

1. A system for running a computational processing graph of an artificial neural network through multiple heterogeneous resources, the system comprising: A model analyzer configured to generate a plurality of subgraphs including a first subgraph and a second subgraph by dividing a computation processing graph generated based on an input model; The task manager is configured to: allocate the plurality of heterogeneous resources to the first subgraph and the second subgraph; A first compiler is configured to: generate first execution data capable of running on a first resource among the plurality of heterogeneous resources by compiling a first subgraph; A second compiler is configured to generate second execution data that can be run on a second resource among the plurality of heterogeneous resources by compiling the second subgraph in parallel with compiling the first subgraph; A model holder is configured to: receive first execution data and second execution data; outputting the first execution data to the first resource; and outputting the second execution data to the second resource, The first subgraph and the second subgraph are output to the first compiler and the second compiler respectively through independent calculation paths. The second resource is configured to: receive second execution data; receive a result value of the first calculation; and perform a second calculation based on the received second execution data and the result value of the first calculation, and The first subgraph is allocated to the first resource before the second subgraph is generated.

2. The system according to claim 1, wherein: The task manager is further configured to: output the first subgraph to a first compiler corresponding to the first resource, and output the second subgraph to a second compiler corresponding to the second resource, The first compiler and the second compiler are configured to operate independently of each other.

3. The system according to claim 1, wherein: The model analyzer is configured to: Receive input model, The input model is analyzed to generate a computational processing graph.

4. The system according to claim 3, wherein: The model analyzer is further configured to: when the generation of each of the plurality of subgraphs is completed, output the plurality of subgraphs to the task manager simultaneously or sequentially based on a timing sequence, The task manager is further configured to: output the first subgraph to a first compiler corresponding to a first resource allocated to the first subgraph.

5. The system according to claim 3, wherein: Model Analyzers include: A model parser is configured to: receive an input model and extract model information from the input model; A model builder configured to: generate a computational processing graph by analyzing an input model based on model information; and The model optimizer is configured to perform optimization operations on the computation processing graph.

6. The system according to claim 5, wherein: The optimization operation includes at least one of: a graph replacement operation of replacing at least a portion of a computational processing graph with another graph, a graph merging operation of merging at least a portion of a computational processing graph, and a graph partitioning operation of partitioning at least a portion of a computational processing graph.

7. The system according to claim 1, wherein: The model holder is further configured to: when the first compiler completes compiling the first subgraph and the second compiler completes compiling the second subgraph, output the first execution data to the first resource and the second execution data to the second resource simultaneously or sequentially based on timing.

8. The system of claim 1, wherein: The first resource is configured to: perform a first calculation corresponding to the first subgraph by using the first execution data; The second resource is configured to: perform a second computation corresponding to the second subgraph by using the second execution data; and The first resource performs the first computation before the second compiler completes compiling the second subgraph.

9. The system according to claim 8, wherein: The second resource loads second execution data from the model holder before receiving the result value of the first calculation, and executes a second calculation corresponding to the second subgraph by using the second execution data and the result value of the first calculation.

10. The system according to claim 9, wherein: When the second resource is capable of performing the second computation, the second resource is configured to output a second execution data request signal requesting second execution data to the model holder, The model holder outputs the second execution data to the second resource based on the second execution data request signal.

11. The system according to claim 1, wherein: The first compiler is further configured to: compile the first subgraph to generate first execution data executable on the first resource, and The second compiler is further configured to: compile the second subgraph to generate second execution data executable on the second resource.

12. A method for running a computational processing graph of an artificial neural network through multiple heterogeneous resources, the method comprising: Generating a plurality of subgraphs including a first subgraph and a second subgraph by dividing the computation processing graph; allocating the first subgraph to a first resource among the plurality of heterogeneous resources before generating the second subgraph; Generate first execution data capable of running on a first resource by compiling the first subgraph; Allocating a second subgraph to a second resource among the plurality of heterogeneous resources; generating second execution data capable of being executed on a second resource by compiling the second subgraph in parallel with compiling the first subgraph; receiving first execution data and second execution data; outputting first execution data to a first resource; as well as outputting the second execution data to the second resource, The first subgraph and the second subgraph are output to the first compiler and the second compiler respectively through independent calculation paths, and The second resource is configured to: receive second execution data; receive a result value of the first calculation; and perform a second calculation based on the received second execution data and the result value of the first calculation.

13. The method according to claim 12, wherein: The compiling of the first subgraph and the compiling of the second subgraph are performed independently of each other.

14. The method according to claim 12, further comprising: generating a first subgraph by analyzing an input model; as well as The second subgraph is generated by analyzing the input model, Before the generation of the second subgraph is completed, the first subgraph is output to the first resource.

15. The method according to claim 14, wherein: The steps of generating the first subgraph include: extracting model information from an input model; Generate a computational processing graph by analyzing the input model based on the model information; Optimizing the computational processing graph; and The first subgraph is generated by splitting the computational processing graph.

16. The method according to claim 15, wherein: The step of optimizing the computational processing graph includes at least one of the following items: a graph replacement operation of replacing at least a portion of the computational processing graph with another graph, a graph merging operation of merging at least a portion of the computational processing graph, and a graph partitioning operation of partitioning at least a portion of the computational processing graph.

17. The method according to claim 12, further comprising: generating first execution data as a result of compiling the first subgraph; as well as Second execution data is generated as a result of compiling the second subgraph.

18. The method according to claim 17, further comprising: By using the first execution data, performing a first calculation corresponding to the first subgraph, Wherein, before the compilation of the second subgraph is completed, the step of performing the first calculation is performed.

19. An apparatus for performing deep learning by using a computational processing graph of an artificial neural network, the apparatus comprising: A model analyzer configured to: generate a plurality of subgraphs including a first portion of the computational processing graph and a second portion of the computational processing graph by partitioning the computational processing graph; A first compiler is configured to: generate first execution data capable of running on a first resource by compiling at least a first portion of a computation processing graph; a second compiler configured to generate second execution data executable on a second resource different from the first resource by compiling at least a second portion of the computation processing graph in parallel with compiling the first portion of the computation processing graph; The task manager is configured to: output a first portion of the computation processing graph to a first compiler, and output a second portion of the computation processing graph to a second compiler; as well as A model holder is configured to: receive first execution data and second execution data; outputting first execution data to a first resource; receiving a request for second execution data from a second resource; and outputting the second execution data to the second resource while the first resource is performing the first computation, wherein the first compiler and the second compiler operate independently of each other, The second resource is configured to: receive second execution data; receive a result value of the first calculation; and perform a second calculation based on the received second execution data and the result value of the first calculation, and Therein, the first portion is allocated to the first resource before the second portion is generated.

Images