Electronic device including neural processing unit and method of operating same

Abstract

An electronic device according to various embodiments may include a memory and a processor operably connected to the memory. In some embodiments, a processor may include: a central processing unit including a cache memory; a neural processing unit; and an interconnector connected to the neural processing unit. The central processing unit may include any one of an arbiter, processing circuitry for connecting the cache memory to the central processing unit, and an interconnector. The neural processing unit may exchange data with the cache memory through the arbiter. Other various embodiments are possible.

Description

Technical Field

[0001] The present disclosure relates to an electronic device including a neural processing unit and an operating method thereof. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence technology can be applied to various fields. Among them, deep learning related to artificial intelligence technology requires high computing performance, more memory occupancy and high power consumption of hardware devices. Therefore, when deep learning is applied to mobile terminal applications or data sensor services, a large load may be imposed.

[0003] Neural network model technology can reduce energy consumption and inference latency by using neural network chips such as neural network processing units (NPUs), tensor processing units (TPUs), and field programmable gate arrays (FPGAs).

[0004] An electronic device (e.g., a smart phone) may include an application processor (AP). The AP chip may have built-in subsystems (e.g., CPU, GPU, and NPU) connected via a bus. The AP chip or a subsystem inside the AP chip may be connected to and communicate with a memory (e.g., dynamic random access memory) outside the AP chip.

[0005] In order to run the neural network model, the central processing unit and the neural network chip can communicate with each other. For example, automatic speech recognition requires fast processing of speech recognition from a speech stream. However, since it takes time to initialize the NPU, the central processing unit can execute the first half of the process for executing automatic speech recognition during the time it takes to initialize the NPU. The central processing unit can transmit the results of the first half of the automatic speech recognition process to the NPU. The NPU can process the second half of the automatic speech recognition process based on the transmitted results. Summary of the invention

[0006] Solution to the problem

[0007] The electronic device of an embodiment may include a memory and a processor operably connected to the memory. The processor may include: a central processing unit including a cache memory, a neural network processing unit, and an interconnect connected to the neural network processing unit. The central processing unit may include an arbitrator for connecting the cache memory to any one of the processing circuits or interconnects of the central processing unit. The neural network processing unit may be configured to exchange data with the cache memory through the arbitrator.

[0008] The method of operating an electronic device of an embodiment may include running a neural network calculation by a neural network processing unit of the electronic device. The method may include controlling an arbiter included in a central processing unit of the electronic device to connect a cache memory of the central processing unit to an interconnect connected to the neural network processing unit. The method may include receiving, by the neural network processing unit, data for performing the neural network calculation being run from the cache memory through the arbiter, or sending data to the cache memory through the arbiter.

[0009] The processor of an embodiment may include: a central processing unit including a cache memory, a neural network processing unit, and an interconnect connected to the neural network processing unit. The central processing unit may include an arbitrator for connecting the cache memory to any one of the processing circuits or interconnects of the central processing unit. The neural network processing unit may be configured to exchange data with the cache memory through the arbitrator. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram of an electronic device in a network environment according to various embodiments.

[0011] Figure 2 is a block diagram showing a configuration of an electronic device according to an embodiment.

[0012] Figure 3 is a block diagram showing a configuration of a processor including a plurality of core processors according to an embodiment.

[0013] Figure 4 is a flowchart illustrating a process of operation of an electronic device according to an embodiment.

[0014] Figure 5 is a flowchart illustrating a process of performing operations including preprocessing for performing neural network calculations in an electronic device according to an embodiment.

[0015] Figure 6 is a flowchart illustrating a process for performing calculations on a calculation layer included in a neural network model by a core processor of a central processing unit of an electronic device according to an embodiment.

[0016] Figure 7 is a flowchart illustrating a process for performing calculations on a next calculation layer by a neural network processing unit based on results of calculations performed on a calculation layer by a core processor of a central processing unit of an electronic device according to an embodiment.

[0017] Figure 8 is a diagram illustrating the operation of a central processing unit and a neural network processing unit according to an embodiment when performing calculations for processing a neural network model.

[0018] Fig. 9 is a flowchart illustrating a process of compiling a file for running a neural network model by an electronic device according to an embodiment. DETAILED DESCRIPTION

[0019] When some computational layers in a neural network include operations that may not be performed by a neural network processing unit configured to process a neural network model, some computational layers may be performed by a central processing unit. In order for the central processing unit to process some computational layers while processing another process, interrupt operations should be performed by an interrupt controller. Therefore, overhead of an interrupt controller may occur.

[0020] During the communication between the central processing unit and the neural network processing unit, the memory access overhead may occur in the process of writing data to the memory (e.g., dynamic random access memory) and reading the data stored in the memory. When the memory access overhead occurs, it may take tens to hundreds of cycles due to at least one of the following: overhead due to the L1 cache, L2 cache, L3 cache, bus and last level cache (LLC), delay due to the collection and scheduling of DRAM access requests, the dynamic random access memory (DRAM) controller for off-chip access, delay due to the decoder inside the DRAM, and delay due to internal DRAM scheduling.

[0021] In order for the central processing unit to perform calculations on a calculation layer in a neural network model being executed by the neural network processing unit while executing another process, the central processing unit may need to switch the process being executed by the central processing unit to a process for processing the calculation layer. In order to switch the process, it may take time (e.g., about 100 cycles) to write the register value of the central processing unit to the memory and retrieve the register value of the other process written to the memory. The operation of switching the executing process to another process may be called a context switch.

[0022] Due to overhead occurring in an interrupt controller, overhead of memory access or context switching, a delay may occur in the process in which the central processing unit and the neural network processing unit process the neural network model in conjunction with each other.

[0023] The technical tasks to be implemented in this document are not limited to the above-mentioned technical tasks, and other technical tasks not mentioned can be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0024] Figure 1 1 is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments. Figure 1, the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one component (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added in the electronic device 101. In some embodiments, some components (eg, sensor module 176, camera module 180, or antenna module 197) may be implemented as a single component (eg, display module 160).

[0025] The processor 120 may execute, for example, software (e.g., program 140) to control at least one other component (e.g., hardware or software component) of the electronic device 101 coupled to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resulting data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that may operate independently of the main processor 121 or in conjunction with the main processor 121. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121 or be specific to a specified function. The auxiliary processor 123 may be implemented separately from the main processor 121 or as part of the main processor 121.

[0026] The auxiliary processor 123 may replace the main processor 121 when the main processor 121 is in an inactive (e.g., sleep) state, or control at least some of the functions or states related to at least one of the components of the electronic device 101 (e.g., display module 160, sensor module 176, or communication module 190) together with the main processor 121 when the main processor 121 is in an active state (e.g., executing an application). According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as a part of another component (e.g., camera module 180 or communication module 190) related to the auxiliary processor 123 function. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure specified for artificial intelligence model processing. The artificial intelligence model may be generated by machine learning. Such learning may be performed, for example, by the electronic device 101 that performs artificial intelligence or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network can be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q network, or a combination of two or more thereof, but is not limited thereto. The artificial intelligence model can additionally or alternatively include a software structure in addition to a hardware structure.

[0027] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, input data or output data of software (e.g., the program 140) and commands related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134.

[0028] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0029] The input module 150 may receive a command or data to be used by another component (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0030] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as playing multimedia or playing recordings. The receiver may be used to receive incoming calls. Depending on the embodiment, the receiver may be implemented as being separated from the speaker or as a part of the speaker.

[0031] The display module 160 can visually provide information to the outside of the electronic device 101 (e.g., a user). The display module 160 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling a corresponding one of the display, the hologram device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch, or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0032] The audio module 170 can convert sound into an electrical signal, and vice versa. According to an embodiment, the audio module 170 can obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly coupled to the electronic device 101.

[0033] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a user's state) outside the electronic device 101, and then generate an electrical signal or data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0034] The interface 177 may support one or more designated protocols for direct (e.g., wired) or wireless coupling of the electronic device 101 with an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0035] The connection terminal 178 may include a connector, via which the electronic device 101 may be physically connected to an external electronic device (e.g., the electronic device 102). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0036] The haptic module 179 may convert the electrical signal into mechanical stimulation (eg, vibration or movement) or electrical stimulation, which may be recognized by the user via his tactile sense or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0037] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0038] The power management module 188 may manage power supplied to the electronic device 101. According to one embodiment, the power management module 188 may be implemented as, for example, at least a portion of a power management integrated circuit (PMIC).

[0039] The battery 189 may supply power to at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0040] The communication module 190 can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors that can operate independently of the processor 120 (e.g., application processor (AP)) and support direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., via a first network 198 and a second network 198). g. A second network 199 (e.g., Bluetooth TM, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a communication network). For example, a remote communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN)). These various types of communication modules may be implemented as a single component (e.g., a single chip), or may be implemented as multiple components separated from each other (e.g., multiple chips). The wireless communication module 192 may use user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196 to identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199).

[0041] The wireless communication module 192 can support 5G networks and next-generation communication technologies after the 4G network, such as new radio (NR) access technology. NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC) or ultra-reliable and low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., mmWave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple input and multiple output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., an electronic device 104) or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 can support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less round trip).

[0042] The antenna module 197 may transmit a signal or power to or receive a signal or power from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiation element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network such as the first network 198 or the second network 199 may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). Then, a signal or power may be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiation element may be additionally formed as a part of the antenna module 197.

[0043] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., millimeter wave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving signals of the specified high frequency band.

[0044] At least some of the above components can be coupled to each other and transmit signals (e.g., commands or data) therebetween via an inter-peripheral communication scheme (e.g., a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)).

[0045] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via a server 108 coupled to the second network 199. Each of the electronic devices 102 or 104 may be a device of the same type or a different type as the electronic device 101. According to an embodiment, all or some operations to be performed at the electronic device 101 may be performed at one or more of the external electronic devices 102, 104 or 108. For example, if the electronic device 101 should automatically perform a function or service, or in response to a request from a user or another device, the electronic device 101 may request one or more external electronic devices to perform at least a portion of the function or service instead of or in addition to performing the function or service. The one or more external electronic devices receiving the request may perform at least a portion of the requested function or service, or an additional function or additional service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a part of the reply to the request with or without further processing the result. To this end, for example, cloud computing, distributed computing, mobile edge computing (MEC) or client-server computing technology may be used. The electronic device 101 can provide ultra-low latency services using, for example, distributed computing or mobile edge computing. In an embodiment, the external electronic device 104 may include an Internet of Things (IoT) device. The server 108 may be an intelligent server using machine learning and / or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199. The electronic device 101 can be applied to intelligent services (e.g., smart homes, smart cities, smart cars, or health care) based on 5G communication technology or IoT-related technologies.

[0046] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to the above-mentioned electronic devices.

[0047] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, and include various changes, equivalents or replacements of the corresponding embodiments. With respect to the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It should be understood that, unless otherwise clearly stated in the relevant context, the singular form of the noun corresponding to the project may include one or more things. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "at least one of A, B or C" may include any one or all possible combinations of the items listed together in the corresponding phrase in the phrase. As used herein, terms such as "first" and "second" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It should be understood that if an element (e.g., a first element) is referred to as being "coupled to" another element (e.g., a second element), "coupled to" another element (e.g., the second element), "connected to" another element, or "connected to" another element (e.g., the second element), with or without the term "operably" or "communicatively", it means that the element can be coupled to the other element directly (e.g., by wire), wirelessly, or via a third element.

[0048] As used in conjunction with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," or "circuit"). A module may be a single integrated component or its smallest unit or portion suitable for performing one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0049] Various embodiments as described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) readable by a machine (e.g., electronic device 101). For example, a processor (e.g., processor 120) of a machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute it with or without one or more other components under the control of the processor. This allows the machine to be operated to perform at least one function according to at least one instruction called. One or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" simply means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between where data is semi-permanently stored in the storage medium and where data is temporarily stored in the storage medium.

[0050] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or via an application store (e.g., a Play Store). TM ) online distribution (e.g., download or upload), or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium, such as a memory of a manufacturer's server, an application store's server, or a relay server.

[0051] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to various embodiments, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as performed by a corresponding one of the multiple components before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the operations may be performed or omitted in a different order, or one or more other operations may be added.

[0052] Figure 2is a block diagram showing a configuration of an electronic device 101 according to an embodiment.

[0053] The electronic device 101 of the embodiment may include a memory 130 and a processor 200 (eg, Figure 1 The processor 120 and Figure 1 The processor 200 may include, for example, an application processor, which is a system semiconductor that executes driving, graphic processing, and the like of various application programs in the electronic device 101. The processor 200 may include a neural network processing unit 210 and a central processing unit 220. The neural network processing unit 210 may include a device configured to efficiently process parallel calculations and process neural network models. Compared to the central processing unit 220, the neural network processing unit 210 may be configured to perform multiple simple calculations in parallel. Compared to the neural network processing unit 210, the central processing unit 220 may be configured to perform complex calculations serially.

[0054] The neural network processing unit 210 and the central processing unit 220 of the embodiment may be integrated into the processor 200. The processor 200 of the embodiment may be configured to enable the neural network processing unit 210 to communicate with the central processing unit 220. For example, the processor 200 may be configured such that the neural network processing unit 210 sends data to the central processing unit 220 or receives data from the central processing unit 220 without passing through the memory 130.

[0055] The neural network processing unit 210 may be connected to an interconnect 230. The interconnect 230 may be configured to connect the neural network processing unit 210 to an arbiter 221 of the central processing unit 220. For example, the interconnect 230 may be connected to the neural network processing unit 210 and the central processing unit 220 via a physical interface (e.g., a bus). The central processing unit 220 may include an arbiter 221 and a cache memory 223. The arbiter 221 may be configured to selectively connect the cache memory 223 to a processing circuit or the interconnect 230 for allowing the central processing unit 220 to process data. The arbiter 221 may include circuits configured to perform the functions presented in the present disclosure, or functional blocks implemented by the central processing unit 220.

[0056] The neural network processing unit 210 and the central processing unit 220 can store data in the cache memory 223 through the arbiter 221, or retrieve data stored in the cache memory 223. For example, the cache memory 223 may include a level 1 (L1) cache memory and a level 2 (L2) cache memory. For example, the L1 cache memory may have a faster input / output speed than the L2 cache memory. In particular, since the neural network processing unit 210 can access the L1 cache memory of the central processing unit 220 through the arbiter 221 and the interconnect 230, the delay incurred in accessing the memory 130 (e.g., DRAM) can be reduced. The neural network processing unit 210 or the central processing unit 220 can store data in the cache memory 223, or retrieve data from the cache memory 223 and exchange data with each other.

[0057] Figure 3 is a block diagram showing a configuration of a processor 200 including a plurality of core processors 320 according to an embodiment.

[0058] The processor 200 of the embodiment may include a neural network processing unit 210, a central processing unit 220, an interconnect 230, a dynamic shared unit (DSU) 330, a cache coherent interconnect (CCI) 340, a system cache / bus 350, and a memory controller 360. However, Figure 3 An example of the configuration of the processor 200 for explaining the embodiment, and some components may be replaced with other components or omitted. Figure 3 The names used in (eg, dynamic shared unit, cache coherent interconnect) are used to explain the components individually, and the names themselves do not limit the characteristics of the components.

[0059] The central processing unit 220 of the electronic device 101 may include a plurality of core processors 320. Among the plurality of core processors 320, at least one core processor 321 may be configured to perform neural network calculations for executing a neural network model in conjunction with the neural network processing unit 210.

[0060] At least one core processor 321 may include an arbiter 221, a cache memory 223, and a processing circuit 323. The cache memory 223 may include a memory that temporarily stores data for fast data access. For example, the cache memory 223 may include an L1 cache memory and an L2 cache memory. However, the structure of the cache memory 223 is not limited thereto. The processing circuit 323 may be configured to perform calculations based on input data. The arbiter 221 may connect the cache memory 223 to the processing circuit 323 or the interconnect 230.

[0061] The neural network processing unit 210 may include an interrupt controller 311, a calculation circuit 313, or a static random access memory 315. The interrupt controller 311 may control interrupts occurring during the calculation process of the neural network processing unit 210. The interrupt controller 311 may be connected to at least one core processor 321 of the central processing unit 220 and transmit an interrupt signal to the at least one core processor 321. For example, the interrupt controller 311 may be physically directly connected to the central processing unit 220. The calculation circuit 313 may perform calculations of the neural network processing unit 210. The static random access memory 315 may store data used for calculations performed by the neural network processing unit 210. For example, the static random access memory 315 may store information input to a calculation layer included in a neural network model. The neural network processing unit 210 may include a direct memory access (DMA) 317. The direct memory access 317 may be configured to send data to or receive data from the arbitrator 221 via the interconnect 230. The data transmitted between the neural network processing unit 210 and the central processing unit 220 may include feature information (e.g., feature map) input to or output from the neural network model. The direct memory access 317 may perform conversion between a first data format (e.g., INT8) used in the neural network processing unit 210 and a second data format (e.g., FP32) used in the central processing unit 220.

[0062] The dynamic sharing unit 330 may be connected to the central processing unit 220. The dynamic sharing unit 330 may include a cache consistency control circuit that maintains cache consistency between the plurality of core processors 320. In addition, the dynamic sharing unit 330 may further include an L3 cache memory provided to the central processing unit 220. The dynamic sharing unit 330 may allocate core processors for running processes among the plurality of core processors 320. When at least one core processor 321 is exclusively used by a process executed in conjunction with the neural network processing unit 210, the dynamic sharing unit 330 may allocate another process to another core processor other than the at least one core processor 321. According to an embodiment, even when the at least one core processor 321 is not exclusively used by a process executed in conjunction with the neural network processing unit 210, the at least one core processor 321 may be allocated to a process with a low priority.

[0063] Cache coherent interconnect 340 may be connected to multiple subsystems. For example, cache coherent interconnect 340 may be connected to interconnect 230 and dynamic shared unit 330. Cache coherent interconnect 340 may include a bus that maintains cache coherence between the multiple subsystems connected. Cache coherent interconnect 340 may be connected to memory controller 360 via system cache / bus 350. Memory controller 360 may access dynamic random access memory 370 (e.g., Figure 2 memory 130) and stores data in or retrieves data from the dynamic random access memory 370.

[0064] When the arbiter 221 connects the cache memory 223 to the processing circuit 323, and the processing circuit 323 transmits data to be stored in the shared memory area of ​​the DRAM 370, the data may remain in the cache memory 223 (e.g., L1 cache) while passing through the cache memory 223 during the process of transferring the data to the DRAM 370. When the arbiter 221 connects the cache memory 223 to the interconnect 230, and the direct memory access 317 loads data stored in the shared memory area of ​​the DRAM 370, the data stored in the cache memory 223 may be loaded. Therefore, data may be transferred from the central processing unit 220 to the neural network processing unit 210 without performing an operation of accessing the DRAM 370.

[0065] In contrast, when the arbiter 221 connects the cache memory 223 to the interconnect 230, and the direct memory access 317 sends data to be stored in the shared memory area of ​​the DRAM 370, the data can be retained in the cache memory 223 (e.g., L1 cache) while passing through the cache memory 223 during the process of transferring the data to the DRAM 370. When the arbiter 221 connects the cache memory 223 to the processing circuit 323, and the processing circuit 323 loads the data stored in the shared memory area of ​​the DRAM 370, the data stored in the cache memory 223 can be loaded. Therefore, data can be transferred from the neural network processing unit 210 to the central processing unit 220 without performing an operation of accessing the DRAM 370.

[0066] When at least one core processor 321 is not connected to the neural network processing unit 210 through the interconnect 230 and the arbiter 221, the central processing unit 220 may write data to the DRAM 370 via the DSU 330, the CCI 340, the system cache / bus 350, and the memory controller 360. The neural network processing unit 210 may need to receive data from the central processing unit 220 by reading the data from the DRAM 370 via the CCI 340, the system cache / bus 350, and the memory controller 360. Here, a delay may occur in sending and receiving data through the DRAM 370. However, according to an embodiment, by allowing the at least one core processor 321 and the neural network processing unit 210 to exchange data with each other without passing through the DRAM 370 (for example, by exchanging data through the cache memory 223), the delay occurring in the process of switching the body of the calculation layer included in the calculation neural network model can be reduced.

[0067] Figure 4 is a diagram showing an electronic device (eg, Figure 1 and Figure 2 Flowchart 400 of the process of operation of the electronic device 101).

[0068] In operation 410, an electronic device (eg, Figure 1 and Figure 2 The electronic device 101 of FIG. 101 may perform neural network calculations based on the neural network model. Figure 1 and Figure 2 The central processing unit (eg, Figure 2 and Figure 3 The central processing unit 220 of the embodiment of the present invention can perform preprocessing for performing neural network calculations and initialize the neural network processing unit (e.g., Figure 2 and Figure 3 neural network processing unit 210).

[0069] In operation 420, an electronic device (eg, Figure 1 and Figure 2 The electronic device 101) can control the arbitrator (for example, Figure 2 and Figure 3 The arbiter 221) will be the neural network processing unit (for example, Figure 2 and Figure 3 The neural network processing unit 210) is connected to the central processing unit (e.g., Figure 2 and Figure 3 of the central processing unit 220) of the cache memory (e.g., Figure 2 and Figure 3 cache memory 223).

[0070] In operation 430, an electronic device (eg, Figure 1 and Figure 2 The electronic device 101 may allow a neural network processing unit (e.g., Figure 2 and Figure 3 The neural network processing unit 210) is connected to the arbitrator (e.g., Figure 2 and Figure 3 The arbiter 221) performs communication with the CPU cache memory (e.g., Figure 2 and Figure 3 Here, the neural network processing unit (e.g., Figure 2 and Figure 3 The neural network processing unit 210) can be obtained from the cache memory of the central processing unit (e.g., Figure 2 and Figure 3 The neural network processing unit (e.g., Figure 2 and 3 The neural network processing unit 210) can read the information generated by the central processing unit (e.g., Figure 2 and 3 The CPU 220) writes to the cache memory (e.g., Figure 2 and 3 cache memory 223), thereby enabling data to be transferred from a central processing unit (e.g., Figure 2 and 3 The CPU 220) is transmitted to the neural network processing unit (e.g., Figure 2 and 3 neural network processing unit 210).

[0071] Figure 5 is a diagram showing a method for using an electronic device (eg, Figure 1 and Figure 2 Flowchart 500 of a process in which an electronic device 101) performs operations including preprocessing for performing neural network calculations.

[0072] In operation 501, at least one core processor 321 may run a process (NPU) for performing neural network calculations in conjunction with the neural network processing unit 210. According to an embodiment, at least one core processor 321 may be used exclusively by a process for performing in conjunction with the neural network processing unit 210. An electronic device (e.g., Figure 1 and Figure 2The electronic device 101) can determine whether a process for executing in conjunction with the neural network processing unit 210 will exclusively use the at least one core processor 321 based on a parameter that defines whether to exclusively use the at least one core processor 321. The parameter can be configured as a value included in a file for running a neural network model. When the at least one core processor 321 exclusively runs a process for executing in conjunction with the neural network processing unit 210, delays due to context switching can be reduced.

[0073] When the parameter is a first value, the process for executing in conjunction with the neural network processing unit 210 may exclusively use the at least one core processor 321. For example, when the value of the parameter AF_NPU is ON, the process for executing in conjunction with the neural network processing unit 210 may exclusively use the at least one core processor 321. When the parameter is a second value different from the first value, the at least one core processor 321 may further execute another process. For example, when the value of the parameter AF_NPU is OFF, the at least one core processor 321 may further execute another process.

[0074] In operation 503, at least one core processor 321 may perform a preprocessing operation on input data to be input to the neural network model. For example, when a sentence composed of a natural language is to be processed using a neural network model for performing natural language processing, since the sentence cannot be directly input to the neural network model, the core processor 321 may perform at least one of an operation of tokenizing the sentence in units of sentences, an operation of tokenizing the sentence in units of words, or an operation of tokenizing the sentence in units of characters. The preprocessing operation may be configured in various ways depending on the neural network model.

[0075] In operation 505, at least one core processor 321 may transmit the pre-processing result obtained in operation 503 to the DRAM 370. Figure 3 , the pre-processing results can be transmitted via the arbiter 221, the cache memory 223 and the central processing unit (e.g., Figure 2 and Figure 3 220) to DRAM 370 (e.g., Figure 3 The arbiter 221 may transmit the pre-processing result obtained by the processing circuit 323 to the cache memory 223. In the process of transmitting the pre-processing result to the DRAM 370 in operation 505, the pre-processing result may be written to the cache memory 223 (e.g., L1 cache).

[0076] In operation 507, at least one core processor 321 may initialize the neural network processing unit 210 and transmit a start command for neural network calculation to the neural network processing unit 210. After transmitting the start command to the neural network processing unit 210, the at least one core processor 321 may wait until an interrupt signal is received from the neural network processing unit 210. When the at least one core processor 321 is exclusively used by a process for executing in conjunction with the neural network processing unit 210, the at least one core processor 321 may not perform context switching.

[0077] In operation 509, the neural network processing unit 210 may load the preprocessing result. Figure 3 , direct memory access of the neural network processing unit 210 (e.g., Figure 3 317) may attempt to access the pre-processing results stored in the shared memory area of ​​DRAM 370. Here, when the neural network processing unit 210 executes an instruction that allows access to an address that writes data to DRAM 370, the pre-processing results may actually be accessed from the cache memory of the core processor 321 (e.g., Figure 2 or Figure 3 Cache memory 223) (e.g., L1 cache) is loaded into SRAM (e.g., Figure 3 SRAM 315). Arbiter (e.g., Figure 2 or 3) through the interconnection (eg, Figure 2 or 3 interconnect 230 will be from the cache memory (e.g., Figure 2 The preprocessing result read from the cache memory 223 of the SRAM (e.g., L1 cache) is transmitted to the neural network processing unit 210. Therefore, the preprocessing result can be transferred from the cache memory (e.g., L1 cache) to the SRAM (e.g., Figure 3 Direct path to SRAM 315) for fast loading without dynamic random access memory access overhead.

[0078] In operation 511, the neural network processing unit 210 may perform neural network calculations based on the loaded preprocessing results. For example, the neural network processing unit 210 may input the preprocessing results into the first calculation layer of the neural network model and perform calculations on the first calculation layer.

[0079] Figure 6 is a diagram illustrating a method for using an electronic device (eg, Figure 1 and Figure 2 The central processing unit (eg, Figure 2 and Figure 3 The core processor of the central processing unit 220 (e.g., Figure 3Flowchart 600 of a process in which at least one core processor 321 of the neural network performs computations on a computational layer included in a neural network model.

[0080] In operation 601, the neural network processing unit 210 may perform calculations on a second calculation layer. The second calculation layer is a layer before the first calculation layer, which is the N+1th calculation layer, among a plurality of calculation layers included in the neural network model, and is the Nth calculation layer in the neural network model. For example, when the second calculation layer is the first in the neural network model, the neural network processing unit 210 may perform calculations on the second calculation layer. Figure 5 The preprocessing result loaded in operation 509 of is input to the second computing layer, and calculation is performed on the second computing layer. For example, when the second computing layer is the second or later layer in the neural network model, the neural network processing unit 210 can input the value output from the N-1th computing layer to the second computing layer, and perform calculation on the second computing layer. The neural network processing unit 210 can obtain the first feature information (e.g., feature map) output by performing calculation on the second computing layer.

[0081] In operation 603, the neural network processing unit 210 may identify a subject for performing calculations on a first calculation layer, which is the N+1th layer in the next order. For example, a transformer as a neural network available in a language field or a visual field may include a plurality of softmax layers. Here, when the neural network processing unit 210 does not support softmax calculations, the neural network processing unit 210 may identify that the calculations on the first calculation layer should be performed by a central processing unit (e.g., Figure 2 or Figure 3 Executed by the central processing unit 220).

[0082] In operation 605, the neural network processing unit 210 may send an interrupt signal to at least one core processor 321 based on the subject of the computation on the first computation layer. In response to the computation subject identified in operation 603 being at least one core processor 321, the interrupt signal may be generated by an interrupt controller (e.g., Figure 3 The interrupt controller 311) sends.

[0083] The neural network processing unit 210 may transmit the first feature information obtained by performing calculations on the second calculation layer to the DRAM 370. Figure 3 , direct memory access (e.g., Figure 3 Direct memory access 317) can be obtained via a slave interconnect (e.g., Figure 2 and Figure 3 interconnect 230), an arbitrator (e.g., Figure 2 and Figure 3arbiter 221), cache memory (e.g., Figure 2 and Figure 3 cache memory 223) and a central processing unit (e.g., Figure 2 and Figure 3 220) to DRAM 370 (e.g., Figure 3 The DSU 330, CCI 340, system cache / bus 350, and memory controller 360 of the system 370) sends the first characteristic information to the DRAM 370. Direct memory access (e.g., Figure 3 Direct memory access 317) can be accessed through an interconnect (e.g., Figure 2 and Figure 3 The interconnect 230) transmits the first characteristic information to the arbitrator (eg, Figure 2 and Figure 3 Arbitrator 221). Arbitrator (e.g., Figure 2 and Figure 3 The arbiter 221) can connect the slaves to each other (for example, Figure 2 and Figure 3 The first characteristic information received by the interconnect 230) is transmitted to the cache memory (eg, Figure 2 and Figure 3 Cache memory 223). Figure 6 , the first feature information can be connected via an interconnection (for example, Figure 2 and Figure 3 interconnect 230) and an arbiter (e.g., Figure 2 and Figure 3 arbiter 221) from the neural network processing unit 210 to the cache memory of the core processor 321 (e.g., Figure 2 and Figure 3 The first feature information may be stored in the cache memory 223 of the core processor 321 (eg, Figure 2 and Figure 3 223) is transferred 609 to DRAM 370. After the cache memory (e.g., Figure 2 and Figure 3 In the process of transmitting the first feature information to the DRAM 370, the first feature information may be written into the cache memory (eg, Figure 2 and Figure 3 cache memory 223).

[0084] In operation 611, in order to perform calculation on the first calculation layer as the N+1th calculation layer, the core processor 321 may read the first feature information. Here, since the first feature information is written into the cache memory (e.g., Figure 2 and Figure 3 cache memory 223), so the core processor 321 can be obtained from the cache memory (e.g., Figure 2 and Figure 3 The core processor 321 can obtain the first feature information by using the cache memory 223 of the DRAM 370. Therefore, the core processor 321 can obtain the first feature information without incurring the overhead for accessing the DRAM 370. The core processor 321 can input the obtained first feature information to the first calculation layer and perform calculation on the first calculation layer.

[0085] Figure 7 is a diagram showing a method for using an electronic device (eg, Figure 1 and Figure 2 The central processing unit (eg, Figure 2 and Figure 3 The core processor of the central processing unit 220 (e.g., Figure 3 Flowchart 700 of the process of performing calculations on a next calculation layer by a neural network processing unit based on the results of calculations performed on a calculation layer by at least one core processor 321).

[0086] In operation 611, the core processor 321 may perform calculations on the first calculation layer which is the N+1th calculation layer in the neural network model. For example, the core processor 321 may obtain second feature information output by inputting the first feature information to the first calculation layer.

[0087] In operation 701, in response to completion of the calculation on the first calculation layer, the core processor 321 may send the obtained second feature information to the DRAM 370. Figure 3 , the processing circuitry of the core processor 321 (e.g., Figure 3 The processing circuit 323) can be transmitted via the slave arbitrator (e.g., Figure 2 and Figure 3 arbiter 221), cache memory (e.g., Figure 2 and Figure 3 cache memory 223) and a central processing unit (e.g., Figure 2 and Figure 3 220) to DRAM 370 (e.g., Figure 3 The DSU 330, CCI 340, system cache / bus 350, and memory controller 360 of FIG. 340 send the second characteristic information to DRAM 370. The arbitrator (e.g., Figure 2 and Figure 3 The arbiter 221) can convert the slave processing circuit (e.g., Figure 3The second feature information output by the processing circuit 323 is transmitted to the cache memory 223.

[0088] In operation 703, the core processor 321 may send the second feature information to the neural network processing unit 210. For example, direct memory access (e.g., Figure 3 Direct memory access 317) can be performed by an arbitrator (e.g., Figure 2 and Figure 3 arbiter 221) and interconnect (e.g., Figure 2 and Figure 3 interconnect 230) will write to the cache memory (e.g., Figure 2 and 3 The second feature information of the cache memory 223) is sent to the SRAM (for example, Figure 3 SRAM 315). Arbiter (e.g., Figure 2 and Figure 3 The arbiter 221 of the embodiment may transmit the second feature information read from the cache memory 223 to the neural network processing unit 210 through the interconnection 230. In operation 705, the core processor 321 may send an interrupt signal to the neural network processing unit 210 for allowing calculation to be performed on the third calculation layer which is the N+2th calculation layer in the neural network model.

[0089] In operation 707, in response to the received interrupt signal, the neural network processing unit 210 may perform calculations on the third calculation layer based on the second feature information. For example, the neural network processing unit 210 may input the second feature information to the third calculation layer and perform calculations on the third calculation layer.

[0090] Figure 8 is a diagram illustrating an operation of the neural network processing unit 210 and the central processing unit 220 communicating and performing neural network calculation without passing through a memory according to an embodiment.

[0091] The central processing unit may run a process for performing calculations based on a neural network model. The central processing unit may initialize the neural network processing unit and transmit a start command for neural network calculations to the neural network processing unit. While the neural network processing unit performs calculations on a calculation layer in the neural network model, the central processing unit may process another process.

[0092] The neural network processing unit may perform calculations on the Nth calculation layer in the neural network model. After performing the calculation operation on the Nth calculation layer, the neural network processing unit may recognize that the calculation on the N+1th calculation layer should be performed by the central processing unit. The neural network processing unit may perform an operation of storing feature information for performing calculations on the N+1th calculation layer in a shared memory area of ​​the memory. The feature information for performing calculations on the N+1th calculation layer may include information obtained as a result of the calculations on the Nth calculation layer. In this operation, memory access overhead may occur in order for the neural network processing unit to access the memory.

[0093] In response to identifying that the calculation on the N+1th calculation layer should be performed by the central processing unit, the neural network processing unit may send an interrupt signal to the central processing unit. The central processing unit receiving the interrupt signal may switch a process from another process to a process for performing calculations based on the neural network model. In the process of switching the process, a delay due to context switching may occur.

[0094] The central processing unit may perform an operation of reading feature information for performing calculations on the N+1th calculation layer from the memory based on the process. Here, memory access overhead may occur again so that the central processing unit accesses the memory. The central processing unit may perform a calculation operation on the N+1th calculation layer based on the read feature information. The central processing unit may perform an operation of storing feature information for performing calculations on the N+2th calculation layer in a shared memory area of ​​the memory. The feature information for performing calculations on the N+2th calculation layer may include information obtained as a result of performing calculations on the N+1th calculation layer. In this operation, memory access overhead may occur so that the central processing unit accesses the memory.

[0095] The neural network processing unit may perform an operation of reading feature information from a memory for performing calculations on the N+2th calculation layer. Here, memory access overhead may occur in order for the neural network processing unit to access the memory. The neural network processing unit may perform a calculation operation on the N+2th calculation layer based on the read feature information.

[0096] On the contrary, reference Figure 8 , the central processing unit 220 of the embodiment can be used exclusively by the process 801-3 for performing calculations based on the neural network model. For example, when a calculation is performed on a calculation layer where the corresponding parameter (a parameter indicating whether the core processor is used exclusively) is set to ON, at least one processor of the central processing unit 220 (e.g., Figure 3 At least one core processor 321 of the processor 321 may only execute the process 801-3. Therefore, when executing the process 801-3, the delay caused by the context switch may be prevented.

[0097] The neural network processing unit 210 of the embodiment may perform a computing operation 811-2 on the Nth computing layer included in the neural network model. When the computing operation 821-2 on the N+1th computing layer should be performed by the central processing unit 220, the neural network processing unit 210 may transmit feature information for performing the computing on the N+1th computing layer to a cache memory (e.g., an L1 cache memory) of the central processing unit 220. The central processing unit 220 may perform an operation 819-2 for reading feature information stored in the cache memory for performing the computing on the N+1th computing layer. Since the central processing unit 220 can read the feature information from the cache memory, the central processing unit 220 can quickly obtain the feature information without memory access overhead. The central processing unit 220 may perform a computing operation 821-2 on the N+1th computing layer based on the feature information.

[0098] The result of the calculation performed on the N+1th calculation layer may be written to the cache memory of the central processing unit 220. The neural network processing unit 210 may perform operation 825-2 of reading the feature information written to the cache memory of the central processing unit 220. The feature information may include information obtained as a result of the calculation performed on the N+1th calculation layer. Since the neural network processing unit 210 of the embodiment can read the feature information written to the cache memory, it is possible to prevent the processor from accessing the DRAM (e.g., Figure 3 The neural network processing unit 210 may perform computing operation 827-2 on the N+2th computing layer based on the feature information obtained from the cache memory.

[0099] Fig. 9 is a diagram showing a method of performing a process ... Figure 1 and Figure 2 Flowchart 900 of the process of compiling a file for running a neural network model using an electronic device 101).

[0100] Fig. 9 The process shown can be understood as being performed by an electronic device (e.g., Figure 1 and Figure 2 A processor (eg, Figure 1 The processor 120 or Figure 2 and Figure 3 Executed by the central processing unit 220).

[0101] In operation 910, an electronic device (eg, Figure 1 and Figure 2The electronic device 101 of the embodiment may check the compilation option as a setting value for performing compilation. Figure 1 and Figure 2 The electronic device 101 of FIG. 100 may check a value specified by a user related to the compilation of the neural network model. In operation 920, the electronic device (eg, Figure 1 and Figure 2 The electronic device 101) can determine whether a computing layer is specified in the checked compile options, the computing layer specifying a parameter value regarding whether to exclusively use at least one processing of the central processing unit.

[0102] When the computing layer is specified in the compilation option (operation 920-yes), in operation 931, the electronic device (eg, Figure 1 and Figure 2 The electronic device 101 may run compilation based on a compilation option in which a computing layer is specified. When running compilation based on a compilation option in which a computing layer is specified, in operation 941, the compiler may specify a parameter of the specified computing layer as a first value (e.g., on). In operation 941, the compiler may specify a parameter of the remaining computing layers as a second value (e.g., off) different from the first value. In operation 980, the electronic device (e.g., Figure 1 and Figure 2 The electronic device 101) can provide a compiled file based on specified parameters.

[0103] When the computing layer is not specified in the compilation option (operation 920-No), in operation 933, the electronic device (eg, Figure 1 and Figure 2 The electronic device 101 of FIG. 100 may run the compilation based on the compilation option that does not specify the computing layer. In operation 950, the electronic device (eg, Figure 1 and Figure 2 The electronic device 101) may determine a neural network processing unit (e.g., Figure 2 and Figure 3 A neural network processing unit 210) and a central processing unit (e.g., Figure 2 and Figure 3 Whether the central processing unit 220 of the electronic device (e.g., Figure 1 and Figure 2 The electronic device 101) can determine whether the calculations included in each neural layer included in the neural network model can be processed by the neural network processing unit (e.g., Figure 2 and Figure 3 When the neural network processing unit (e.g., Figure 2 and Figure 3When the neural network processing unit 210 of the embodiment of the present invention may not support the computing operations included in at least one neural layer, the electronic device (e.g., Figure 1 and Figure 2 The electronic device 101) may determine a neural network processing unit (e.g., Figure 2 and Figure 3 A neural network processing unit 210) and a central processing unit (e.g., Figure 2 and Figure 3 The central processing unit 220) should run the neural network model in conjunction with each other.

[0104] When determining the neural network processing unit (e.g., Figure 2 and Figure 3 A neural network processing unit 210) and a central processing unit (e.g., Figure 2 and Figure 3 When the central processing unit 220 of the electronic device (e.g., the central processing unit 220 of the electronic device) should run the neural network model in conjunction with each other (operation 950-yes), in operation 961, the electronic device (e.g., Figure 1 and Figure 2 The electronic device 101) can recognize by the compiler that the central processing unit (for example, Figure 2 and Figure 3 For example, when the neural network model includes ten computational layers and the neural network processing unit (e.g., Figure 2 and Figure 3 When the neural network processing unit 210 of the embodiment of the present invention does not support the calculations on the 1st, 3rd and 5th calculation layers, the electronic device (for example, Figure 1 and Figure 2 The electronic device 101) can identify the 5th computing layer as the last computing layer.

[0105] In operation 963, an electronic device (eg, Figure 1 and Figure 2 The electronic device 101 of the embodiment may set the value of the parameter corresponding to the first computing layer to the identified last computing layer to the first value (eg, on) by using a compiler. Figure 1 and Figure 2 The electronic device 101 may set the values ​​of the parameters corresponding to the remaining layers to a second value (e.g., off) different from the first value. For example, when the fifth calculation layer in the calculation order among the ten calculation layers is the last calculation layer, the parameter values ​​of the first, second, third, fourth, and fifth calculation layers in the calculation order may be set to on, and the parameter values ​​of the sixth, seventh, eighth, ninth, and tenth calculation layers in the calculation order may be set to off. In operation 980, the electronic device (e.g., Figure 1 and Figure 2The electronic device 101) can provide a compiled file based on the set parameter values.

[0106] Electronic devices (e.g. Figure 1 and Figure 2 The electronic device 101) may determine a neural network processing unit (e.g., Figure 2 and Figure 3 A neural network processing unit 210) and a central processing unit (e.g., Figure 2 and Figure 3 The central processing unit 220 of the neural network model does not need to be combined with each other to run the neural network model (operation 950-no). For example, when the calculations on all the calculation layers can be performed by the neural network processing unit (e.g., Figure 2 and Figure 3 When the neural network processing unit 210 of the electronic device (for example, Figure 1 and Figure 2 The electronic device 101) may determine a neural network processing unit (e.g., Figure 2 and Figure 3 A neural network processing unit 210) and a central processing unit (e.g., Figure 2 and Figure 3 The central processing unit 220 of the electronic device (e.g., the central processing unit 220 of the electronic device) does not need to be combined with each other to run the neural network model. Figure 1 and Figure 2 The electronic device 101 of the embodiment may set the parameter values ​​of all the computing layers to a second value (eg, off). In operation 980, the electronic device (eg, Figure 1 and Figure 2 The electronic device 101) can provide a compiled file based on the set parameter values.

[0107] Various embodiments may present an electronic device and an operating method thereof, which may reduce delays occurring in the process of performing communications between a central processing unit and a neural network processing unit.

[0108] Various embodiments may present an electronic device and an operating method thereof that may allow a neural network processing unit to flexibly respond to newly proposed operations by allowing at least one of a plurality of core processors of a composite core central processing unit (CPU) to transmit information to the neural network processing unit without accessing a memory.

[0109] Effects obtainable from the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood from the following description by those skilled in the art to which the present disclosure pertains.

[0110] The electronic device of an embodiment may include a memory and a processor operably connected to the memory. The processor may include: a central processing unit including a cache memory, a neural network processing unit, and an interconnect connected to the neural network processing unit. The central processing unit may include an arbitrator for connecting the cache memory to any one of the processing circuits or interconnects of the central processing unit. The neural network processing unit may be configured to exchange data with the cache memory through the arbitrator.

[0111] According to an embodiment, the neural network processing unit may include a direct memory access connected to the interconnect. The direct memory access may be configured to send or receive feature data through the interconnect with the arbitrator. The feature data may include data input to a computational layer of the neural network model or data output from a computational layer of the neural network model.

[0112] According to an embodiment, direct memory access may be configured to convert a first data format used in the neural network processing unit into a second data format used in the central processing unit, and to send feature data from the neural network processing unit to the central processing unit.

[0113] In an embodiment, the central processing unit may include a plurality of core processors. The neural network processing unit may be configured to be connected to at least one of the plurality of core processors via an interconnect.

[0114] The electronic device of the embodiment may further include a dynamic random access memory external to the processor. The interconnect may be configured to connect the dynamic random access memory to any one of the arbitrator and a cache coherent interconnect connectable to the dynamic random access memory.

[0115] In an embodiment, the neural network processing unit may include an interrupt controller. The interrupt controller may be configured to be connected to at least one core processor and send an interrupt signal to the at least one core processor.

[0116] In an embodiment, the neural network processing unit may determine whether the calculation on the first calculation layer included in the neural network calculation should be performed by the central processing unit. In response to determining that the calculation on the first calculation layer should be performed by the central processing unit, the neural network processing unit may send an interrupt signal to at least one core processor through an interrupt controller. The neural network processing unit may be configured to transmit the first feature information to be input to the first calculation layer to the at least one core processor through an interconnect.

[0117] In an embodiment, the first feature information may include information output from the second computing layer by performing, by the neural network processing unit, a second computing layer calculation included in the neural network calculation.

[0118] At least one core processor of the electronic device of the embodiment may be configured to perform calculations on the first calculation layer based on the first feature information and obtain second feature information. The neural network processing unit may be configured to receive an interrupt from the at least one core processor. The neural network processing unit may be configured to perform calculations on the third calculation layer included in the neural network calculation based on the second feature information obtained from the at least one core processor.

[0119] At least one core processor of the electronic device of the embodiment may be configured to run a neural network computing process for performing neural network computing. At least one core processor may be configured to perform preprocessing on input values ​​for performing neural network computing based on the neural network computing process. At least one core processor may be configured to initialize a neural network processing unit and transmit the preprocessing result to the neural network processing unit through an interconnect.

[0120] The central processing unit of the electronic device of the embodiment may be configured to store a parameter defining whether to exclusively use at least one core processor. The central processing unit may be configured to allow a neural network calculation process for performing neural network calculation to exclusively use at least one core processor when the parameter is a first value. The central processing unit may be configured to allow at least one core processor to further perform another process when the parameter is a second value different from the first value.

[0121] The central processing unit of the electronic device of the embodiment may be configured to compile a file containing parameters and providing a neural network model.

[0122] The central processing unit of the electronic device of the embodiment may be configured to, in the process of compiling a file of a neural network model, identify at least one computing layer on which the central processing unit should perform calculations among the multiple computing layers included in the neural network calculation. The central processing unit may be configured to specify a value of a parameter for each of the multiple computing layers based on the identified at least one computing layer during the compilation process.

[0123] The method of operating an electronic device of an embodiment may include running a neural network calculation by a neural network processing unit of the electronic device. The method may include controlling an arbiter included in a central processing unit of the electronic device to connect a cache memory of the central processing unit to an interconnect connected to the neural network processing unit. The method may include receiving data for executing the neural network calculation run by the neural network processing unit from the cache memory by the arbiter, or sending data to the cache memory by the arbiter.

[0124] In an embodiment, the operation method of the electronic device may further include: determining, by the neural network processing unit, whether the calculation on the first calculation layer included in the neural network calculation should be performed by the central processing unit. The method may include, in response to determining that the calculation on the first calculation layer should be performed by the central processing unit, sending an interrupt signal by the neural network processing unit to the central processing unit. The method may also include transmitting, by the neural network processing unit, the first feature information to be input into the first calculation layer to the central processing unit via an interconnect.

[0125] In an embodiment, the first feature information may include information output from the second calculation layer by performing calculations on the second calculation layer included in the neural network calculations by the neural network processing unit.

[0126] In an embodiment, the method may include performing, by the central processing unit, a calculation on a first calculation layer based on the first feature information, and obtaining the second feature information. The method may include receiving, by the neural network processing unit, an interrupt from the central processing unit. The method may include obtaining, by the neural network processing unit, the second feature information from the central processing unit. The method may also include performing, based on the second feature information, a calculation on a third calculation layer included in the neural network calculation.

[0127] The processor of an embodiment may include: a central processing unit including a cache memory, a neural network processing unit, and an interconnect connected to the neural network processing unit. The central processing unit may include an arbitrator for connecting the cache memory to any one of the processing circuits or interconnects of the central processing unit. The neural network processing unit may be configured to exchange data with the cache memory through the arbitrator.

[0128] In an embodiment, the neural network processing unit may include a direct memory access connected to the interconnect. The direct memory access may be configured to send or receive feature data through the interconnect with the arbitrator. The feature data may include data input to a computational layer of the neural network model or data output from a computational layer of the neural network model.

[0129] In an embodiment, the central processing unit may include a plurality of core processors. The neural network processing unit may be configured to be connected to at least one of the plurality of core processors via an interconnect.

[0130] The methods of the embodiments described in the claims or the specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0131] When the method is implemented by software, a computer-readable storage medium storing one or more programs (software modules) may be presented. One or more programs stored in the computer-readable storage medium are configured to be executable by one or more processors within an electronic device. One or more programs include instructions that allow the electronic device to perform the method of the embodiment described in the claims or specification of the present disclosure.

[0132] These programs (software modules and software) can be stored in random access memory, non-volatile memory including flash memory, read-only memory (ROM), electrically erasable programmable ROM (EEPROM), magnetic disk storage device, compact disk ROM (CD-ROM), digital versatile disk (DVD) or other types of optical storage devices, and magnetic tape cassettes. Alternatively, the program can be stored in a memory composed of a combination of some or all of them. In addition, each combined memory can also include multiple.

[0133] In addition, the program can be stored in an attachable storage device, which can be accessed via a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), a storage area network (SAN), or a combination thereof. The storage device can be connected to the device performing the embodiments of the present disclosure via an external port. In addition, a separate storage device on the communication network can also be connected to the device performing the embodiments of the present disclosure.

[0134] In the specific embodiments of the present disclosure described above, the components included in the present disclosure have been expressed in singular or plural form, depending on the specific embodiments presented. However, for ease of explanation, the singular or plural expression is appropriately selected for the presented situation, and the present disclosure is not limited to singular or plural components, and even components expressed as plural can be constructed as singular, or even components expressed as singular can be constructed as plural.

[0135] Furthermore, in the present disclosure, the terms “unit”, “module” and the like may include a hardware component such as a processor or a circuit, and / or a software component executed by a hardware component such as a processor.

[0136] "Units" and "modules" may also be implemented by programs that can be stored in an addressable storage medium and executed by a processor. For example, "units" and "modules" may be implemented by components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays, and variables.

[0137] The specific implementations described in this disclosure are merely embodiments and do not limit the scope of this disclosure in any way. For the sake of brevity, descriptions of conventional electronic components, control systems, software, and other functional aspects of the system may be omitted.

[0138] In addition, in the present disclosure, "including at least one of A, b or c" may mean "including only A", "including only b", "including only c", "including A and b", "including b and c", "including A and c" or "including all of A, b and c".

[0139] Meanwhile, in the detailed description of the present disclosure, specific embodiments have been described, but it is undoubted that various modifications are possible without departing from the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the described embodiments, and should be limited not only by the scope of the claims described later, but also by those equivalent to the scope of the claims.

Claims

1. An electronic device, comprising: Memory; as well as a processor operatively connected to the memory, Wherein, the processor comprises: a central processing unit including a cache memory; A neural network processing unit; and an interconnect connected to the neural network processing unit, The central processing unit comprises an arbiter for connecting the cache memory to any one of the processing circuitry of the central processing unit or the interconnect, and The neural network processing unit is configured to exchange data with the cache memory through the arbitrator.

2. The electronic device according to claim 1, wherein: The neural network processing unit includes a direct memory access connected to the interconnect, The direct memory access is configured to send or receive characteristic data with the arbitrator via the interconnect, and The feature data includes data input to the computing layer of the neural network model or data output from the computing layer of the neural network model.

3. The electronic device according to claim 2, wherein: The direct memory access is configured to convert a first data format used in the neural network processing unit into a second data format used in the central processing unit, and to send the feature data from the neural network processing unit to the central processing unit.

4. The electronic device according to claim 1, wherein: The central processing unit includes multiple core processors, and The neural network processing unit is configured to be connected to at least one core processor of the plurality of core processors through the interconnect.

5. The electronic device according to claim 4 further comprising a dynamic random access memory external to the processor, in, The interconnect is configured to connect the dynamic random access memory to any one of the arbiter and a cache coherent interconnect connectable to the dynamic random access memory.

6. The electronic device according to claim 4, wherein: The neural network processing unit includes an interrupt controller, and The interrupt controller is connected to the at least one core processor and sends an interrupt signal to the at least one core processor.

7. The electronic device according to claim 6, wherein: The neural network processing unit is configured to: determining whether a computation on a first computation layer included in a neural network computation should be performed by the central processing unit; In response to determining that the computation on the first computation layer should be performed by the central processing unit, sending an interrupt signal to the at least one core processor via the interrupt controller; as well as First feature information to be input to the first computing layer is transmitted to the at least one core processor through the interconnect.

8. The electronic device according to claim 7, wherein: The first feature information includes information output from the second calculation layer by performing calculation on the second calculation layer included in the neural network calculation by the neural network processing unit.

9. The electronic device according to claim 7, wherein: The at least one core processor is configured to perform the calculation on the first calculation layer based on the first feature information and obtain second feature information, and The neural network processing unit is configured to: receiving an interrupt from the at least one core processor; Obtaining the second feature information from the at least one core processor; as well as Based on the second feature information, calculations on a third calculation layer included in the neural network calculations are performed.

10. The electronic device according to claim 4, wherein: The at least one core processor is configured to: running a neural network computing process for performing neural network computing; Obtaining input values ​​for performing the neural network calculations; Performing preprocessing on the input value based on the neural network calculation process; Initializing the neural network processing unit; as well as The result of the preprocessing is transmitted to the neural network processing unit through the interconnection.

11. The electronic device according to claim 4, wherein: The central processing unit is configured to: storing a parameter defining whether the at least one core processor is used exclusively; Based on the parameter being the first value, allowing a neural network computation process for performing neural network computations to exclusively use the at least one core processor; as well as Based on the parameter being a second value different from the first value, the at least one core processor is allowed to further execute another process.

12. The electronic device according to claim 11, wherein: The central processing unit is configured to compile a file to include the parameters and provide a neural network model.

13. The electronic device according to claim 12, wherein: The central processing unit is configured to: during the process of compiling the file of the neural network model: identifying, among a plurality of computational layers included in the neural network computation, at least one computational layer on which computation should be performed by the central processing unit; and Based on the identified at least one computational layer, a value for a parameter is assigned to each computational layer in the plurality of computational layers.

14. A method for operating an electronic device, comprising: Running a neural network calculation by a neural network processing unit of the electronic device; controlling an arbiter included in a central processing unit of the electronic device to connect a cache memory of the central processing unit to an interconnect connected to the neural network processing unit; as well as Data for performing neural network calculations run by the neural network processing unit is received from the cache memory through the arbitrator, or the data is sent to the cache memory through the arbitrator.

15. The operating method according to claim 14, further comprising: Determining, by the neural network processing unit, whether a calculation on a first calculation layer included in the neural network calculation should be performed by the central processing unit; In response to determining that the computation on the first computation layer should be performed by the central processing unit, the neural network processing unit sends an interrupt signal to the central processing unit; as well as The first feature information to be input into the first computing layer is transmitted by the neural network processing unit to the central processing unit through the interconnection.

Images