Electronic device for performing operation using ai model and method for operating electronic device

Abstract

An electronic device according to various embodiments includes a processor and a memory, the memory storing instructions, and when executed by the processor, the electronic device may: load and compile an artificial intelligence model stored in the memory; determining whether the compiled artificial intelligence model includes an activation function of a first type; when the compiled artificial intelligence model comprises an activation function of a first type, if a specified value exists in the feature map, skipping an operation on the specified value, and executing an operation on a value to be operated immediately after the specified value; and when the compiled artificial intelligence model does not include the activation function of the first type, executing an operation on the input value of the feature map. Various other embodiments are possible.

Description

Technical Field

[0001] Certain embodiments of the present disclosure relate to an electronic device for performing calculations using an artificial intelligence model and an operating method of the electronic device. Background Art

[0002] Artificial neural networks (ANNs) refer to the computing architecture that models biological brains. Based on artificial neural networks, deep learning, machine learning, etc. can be realized. Deep neural networks or deep learning as examples of artificial neural networks may have a multi-layer structure including multiple layers.

[0003] AI models can be used in various ways to analyze vision and sound. Research and development of hardware related to AI models has been actively conducted to enable efficient operation of AI models in mobile terminals. For example, in addition to research for optimizing multiply-accumulate (MAC) calculations performed in deep learning AI models, hardware structure improvement techniques taking AI models into consideration are also being studied and applied.

[0004] Furthermore, the data processing system may include at least one processor referred to as a central processing unit (CPU). The data processing unit may include at least one other processor, eg, a neural processing unit (NPU) for various types of specialized processing. Summary of the invention

[0005] Technical issues

[0006] AI models can be operated in a neural processing unit (NPU), and research on hardware architectures has been actively ongoing to improve the performance of AI models and increase power efficiency.

[0007] In order to improve power efficiency, a zero skip function may be used to omit multiplication-accumulation calculations (MACs) on 0s included in input values ​​(e.g., result values ​​from a previous hidden layer and a feature map). The zero skip function may use an activation function ReLU, and may be effectively applied when the ratio of 0s is high. For example, when an activation function, i.e., a rectified linear unit (ReLU), is used, the ratio of 0s in a feature map is high, and thus the effect of the zero skip function may be maximized.

[0008] However, since various types of activation functions are used in artificial intelligence models, and the characteristics of these activation functions are also different, the efficiency of the zero-skip function may be reduced in the case where the ratio of 0 in the characteristic graph as the output value of the activation function is not high.

[0009] The embodiments disclosed herein propose a method for efficiently operating hardware by partitioning activation layers of an artificial intelligence model.

[0010] Embodiments of the present disclosure provide an electronic device and a method of operating an electronic device that activates or deactivates a zero-skip function in response to the fact that the activation function included in the artificial intelligence model is a function of a specified type.

[0011] Embodiments of the present disclosure may provide an electronic device and a method of operation of an electronic device that activates or deactivates a feature map compression function in response to the fact that the activation function included in the artificial intelligence model is a function of a specified type.

[0012] Problem Solution

[0013] According to certain embodiments, an electronic device includes: a processor; and a memory storing instructions, wherein the instructions are executed by the processor, and the electronic device is used to: load and compile an artificial intelligence model stored in the memory; determine whether the compiled artificial intelligence model includes a first type of activation function; when the compiled artificial intelligence model includes the first type of activation function, when a specified value exists in a feature map, skip calculations about the specified value and calculate a value to be calculated after the specified value; and when the compiled artificial intelligence model does not include the first type of activation function, perform calculations about input values ​​of the feature map.

[0014] According to certain embodiments, an operating method of an electronic device includes: loading and compiling an artificial intelligence model stored in a memory; determining whether the compiled artificial intelligence model includes a first type activation function; when the compiled artificial intelligence model includes the first type activation function, skipping calculations about a specified value when there is a specified value in a feature map, and calculating a value to be calculated after the specified value; and when the compiled artificial intelligence model does not include the first type activation function, performing calculations about input values ​​of the feature map.

[0015] Advantageous Effects of the Invention

[0016] Embodiments disclosed herein may provide an electronic device for reducing power consumption in the calculation of an artificial intelligence model.

[0017] The embodiments disclosed herein may provide an electronic device for effectively compressing a feature map corresponding to a result value in the calculation of an artificial intelligence model.

[0018] According to the embodiments disclosed herein, depending on the activation function included in the artificial intelligence model, power consumption can be effectively reduced by omitting calculations of specified values ​​included in the input of a hidden layer.

[0019] According to the embodiments disclosed herein, the feature map can be effectively compressed by compressing with values ​​other than the specified value included in the hidden layer output, depending on the activation function included in the artificial intelligence model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In conjunction with the description of the drawings, the same or similar reference numerals may be used for the same or similar elements.

[0021] Figure 1 is a block diagram illustrating electronic devices in a network environment according to an embodiment of the present disclosure.

[0022] Figure 2A is a view showing an example of a neural network (NN) structure according to an embodiment of the present disclosure.

[0023] Figure 2B is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0024] Figure 3 is a flowchart illustrating a method for performing calculation by determining, by a processor, whether to activate a first function and / or a second function in calculation using an artificial intelligence model according to an embodiment of the present disclosure.

[0025] Figure 4 is a view showing an artificial intelligence model configuration according to an embodiment of the present disclosure.

[0026] Figure 5 is a view depicting a graph of an activation function according to an embodiment of the present disclosure.

[0027] Figure 6 is a view showing a graph indicating a ratio of an output value of 0 in an artificial intelligence model using a first type function according to an embodiment of the present disclosure.

[0028] Figure 7 is a diagram illustrating a first function according to certain embodiments disclosed herein.

[0029] Figure 8 is a diagram illustrating a second function according to certain embodiments disclosed herein.

[0030] Fig. 9 is a view depicting a graph of a ReLU function and a PReLU function according to an embodiment of the present disclosure.

[0031] Fig.10 is a flowchart illustrating a method for performing calculation by determining, by a processor, whether to activate a first function in calculation using an artificial intelligence model according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Figure 1 Electronic devices that can be configured to use artificial intelligence are described.

[0033] Figure 1is a block diagram illustrating an electronic device 101 in a network environment 100 according to various embodiments.

[0034] Reference Figure 1 , the electronic device 101 in the network environment 100 may communicate with the electronic device 102 via the first network 198 (e.g., a short-range wireless communication network), or communicate with at least one of the electronic device 104 or the server 108 via the second network 199 (e.g., a long-range wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 via the server 108. According to an embodiment, the electronic device 101 may include a processor 120, a memory 130, an input module 150, a sound output module 155, a display module 160, an audio module 170, a sensor module 176, an interface 177, a connection terminal 178, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a user identification module (SIM) 196, or an antenna module 197. In some embodiments, at least one of the above components (e.g., the connection terminal 178) may be omitted from the electronic device 101, or one or more other components may be added to the electronic device 101. In some embodiments, some of the above-described components (eg, sensor module 176, camera module 180, or antenna module 197) may be implemented as a single integrated component (eg, display module 160).

[0035] The processor 120 may run, for example, software (e.g., program 140) to control at least one other component (e.g., hardware component or software component) of the electronic device 101 connected to the processor 120, and may perform various data processing or calculations. According to one embodiment, as at least part of the data processing or calculation, the processor 120 may store a command or data received from another component (e.g., sensor module 176 or communication module 190) in the volatile memory 132, process the command or data stored in the volatile memory 132, and store the resultant data in the non-volatile memory 134. According to an embodiment, the processor 120 may include a main processor 121 (e.g., a central processing unit (CPU) or an application processor (AP)) or an auxiliary processor 123 (e.g., a graphics processing unit (GPU), a neural processing unit (NPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)) that is independent of or combined with the main processor 121 in operation. For example, when the electronic device 101 includes a main processor 121 and an auxiliary processor 123, the auxiliary processor 123 may be adapted to consume less power than the main processor 121, or adapted to be dedicated to a specific function. The auxiliary processor 123 may be implemented separately from the main processor 121, or as part of the main processor 121.

[0036] When the main processor 121 is in an inactive (e.g., sleep) state, the auxiliary processor 123 (rather than the main processor 121) may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190), or when the main processor 121 is in an active state (e.g., running an application), the auxiliary processor 123 may control at least some of the functions or states related to at least one component among the components of the electronic device 101 (e.g., the display module 160, the sensor module 176, or the communication module 190) together with the main processor 121. According to an embodiment, the auxiliary processor 123 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., a camera module 180 or a communication module 190) that is functionally related to the auxiliary processor 123. According to an embodiment, the auxiliary processor 123 (e.g., a neural processing unit) may include a hardware structure dedicated to artificial intelligence model processing. The artificial intelligence model may be generated through machine learning. For example, such learning may be performed by the electronic device 101 where the artificial intelligence is executed or via a separate server (e.g., server 108). The learning algorithm may include, but is not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, for example. The artificial intelligence model may include multiple artificial neural network layers. The artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or a deep Q network or a combination of two or more thereof, but is not limited thereto. Additionally or optionally, the artificial intelligence model may include a software structure in addition to a hardware structure.

[0037] The memory 130 may store various data used by at least one component of the electronic device 101 (e.g., the processor 120 or the sensor module 176). The various data may include, for example, software (e.g., the program 140) and input data or output data for commands related thereto. The memory 130 may include a volatile memory 132 or a nonvolatile memory 134.

[0038] The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system (OS) 142 , middleware 144 , or applications 146 .

[0039] The input module 150 may receive commands or data to be used by other components (e.g., the processor 120) of the electronic device 101 from outside (e.g., a user) of the electronic device 101. The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus).

[0040] The sound output module 155 can output sound signals to the outside of the electronic device 101. The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as playing multimedia or playing records. The receiver can be used to receive incoming calls. Depending on the embodiment, the receiver can be implemented as a separate part from the speaker, or as part of the speaker.

[0041] The display module 160 may visually provide information to the outside of the electronic device 101 (e.g., a user). The display device 160 may include, for example, a display, a holographic device, or a projector, and a control circuit for controlling a corresponding one of the display, the holographic device, and the projector. According to an embodiment, the display module 160 may include a touch sensor adapted to detect a touch or a pressure sensor adapted to measure the strength of a force caused by a touch.

[0042] The audio module 170 may convert sound into an electrical signal, or vice versa. According to an embodiment, the audio module 170 may obtain sound via the input module 150, or output sound via the sound output module 155 or an earphone of an external electronic device (e.g., electronic device 102) directly (e.g., wired) or wirelessly connected to the electronic device 101.

[0043] The sensor module 176 may detect an operating state (e.g., power or temperature) of the electronic device 101 or an environmental state (e.g., a state of a user) outside the electronic device 101, and then generate an electrical signal or a data value corresponding to the detected state. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0044] The interface 177 may support one or more specific protocols to be used to connect the electronic device 101 directly (e.g., wired) or wirelessly with an external electronic device (e.g., the electronic device 102). According to an embodiment, the interface 177 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0045] The connection end 178 may include a connector, wherein the electronic device 101 can be physically connected to an external electronic device (e.g., the electronic device 102) via the connector. According to an embodiment, the connection end 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0046] The haptic module 179 may convert the electric signal into mechanical stimulation (eg, vibration or motion) or electric stimulation that can be recognized by the user via his sense of touch or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electric stimulator.

[0047] The camera module 180 may capture still images or moving images. According to an embodiment, the camera module 180 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0048] The power management module 188 may manage power supply to the electronic device 101. According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0049] The battery 189 may power at least one component of the electronic device 101. According to an embodiment, the battery 189 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0050] The communication module 190 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (e.g., electronic device 102, electronic device 104, or server 108), and perform communication via the established communication channel. The communication module 190 may include one or more communication processors capable of operating independently from the processor 120 (e.g., an application processor (AP)) and supporting direct (e.g., wired) communication or wireless communication. According to an embodiment, the communication module 190 may include a wireless communication module 192 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules can communicate with an external electronic device via a first network 198 (e.g., a short-range communication network such as Bluetooth, Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA)) or a second network 199 (e.g., a long-range communication network such as a traditional cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))). These various types of communication modules may be implemented as a single component (e.g., a single chip), or may be implemented as multiple components (e.g., multiple chips) separated from each other. The wireless communication module 192 may identify and authenticate the electronic device 101 in a communication network (such as the first network 198 or the second network 199) using user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 196.

[0051] The wireless communication module 192 can support 5G networks after 4G networks and next-generation communication technologies (e.g., new radio (NR) access technology). NR access technology can support enhanced mobile broadband (eMBB), massive machine type communication (mMTC), or ultra-reliable low-latency communication (URLLC). The wireless communication module 192 can support high-frequency bands (e.g., millimeter wave bands) to achieve, for example, high data transmission rates. The wireless communication module 192 can support various technologies for ensuring performance on high-frequency bands, such as, for example, beamforming, massive multiple-input multiple-output (massive MIMO), full-dimensional MIMO (FD-MIMO), array antennas, analog beamforming, or massive antennas. The wireless communication module 192 can support various requirements specified in the electronic device 101, an external electronic device (e.g., electronic device 104), or a network system (e.g., a second network 199). According to an embodiment, the wireless communication module 192 may support a peak data rate for implementing eMBB (e.g., 20 Gbps or greater), loss coverage for implementing mMTC (e.g., 164 dB or less), or U-plane delay for implementing URLLC (e.g., 0.5 ms or less for each of the downlink (DL) and uplink (UL), or 1 ms or less for a round trip).

[0052] The antenna module 197 may transmit or receive a signal or power to or from the outside of the electronic device 101 (e.g., an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a radiation element formed of a conductive material or a conductive pattern formed in or on a substrate (e.g., a printed circuit board (PCB)). According to an embodiment, the antenna module 197 may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication scheme used in a communication network (such as the first network 198 or the second network 199) may be selected from the plurality of antennas by, for example, the communication module 190 (e.g., the wireless communication module 192). A signal or power may then be transmitted or received between the communication module 190 and the external electronic device via the selected at least one antenna. According to an embodiment, another component (e.g., a radio frequency integrated circuit (RFIC)) other than the radiation element may be additionally formed as a part of the antenna module 197.

[0053] According to various embodiments, the antenna module 197 may form a millimeter wave antenna module. According to an embodiment, the millimeter wave antenna module may include a printed circuit board, a radio frequency integrated circuit (RFIC), and a plurality of antennas (e.g., array antennas), wherein the RFIC is disposed on a first surface (e.g., bottom surface) of the printed circuit board, or adjacent to the first surface and capable of supporting a designated high frequency band (e.g., millimeter wave band), and the plurality of antennas are disposed on a second surface (e.g., top surface or side surface) of the printed circuit board, or adjacent to the second surface and capable of transmitting or receiving signals of the designated high frequency band.

[0054] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and communicatively transfer signals (e.g., commands or data) therebetween.

[0055] According to an embodiment, a command or data may be sent or received between the electronic device 101 and the external electronic device 104 via the server 108 connected to the second network 199. Each of the electronic device 102 or the electronic device 104 may be a device of the same type as the electronic device 101, or a device of a different type from the electronic device 101. According to an embodiment, all or some operations to be executed in the electronic device 101 may be executed in one or more of the external electronic device 102, the external electronic device 104, or the server 108. For example, if the electronic device 101 should automatically execute a function or service or should execute a function or service in response to a request from a user or another device, the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service instead of executing the function or service, or the electronic device 101 may request the one or more external electronic devices to execute at least part of the function or service in addition to executing the function or service. The one or more external electronic devices that receive the request may execute the requested at least part of the function or service, or execute another function or another service related to the request, and transmit the result of the execution to the electronic device 101. The electronic device 101 may provide the result as at least a partial reply to the request, with or without further processing the result. To this end, for example, cloud computing technology, distributed computing technology, mobile edge computing (MEC) technology, or client-side computing technology may be used.

[0056] The auxiliary processor 123 may include a neural processing unit. The neural processing unit may perform various functions on the artificial intelligence model. The artificial intelligence model is shown in Figures 2a and 2b.

[0057] Figure 2Ais a view showing an example of a neural network (NN) structure according to an embodiment of the present disclosure.

[0058] refer to Figure 2A , the artificial neural network model may include multiple layers L1 to Ln. Each of the multiple layers L1 to Ln may correspond to a linear layer or a nonlinear layer. According to an embodiment, at least one linear layer and at least one nonlinear layer may be combined and referred to as a layer. For example, a linear layer may include a convolutional layer and a fully connected layer. A nonlinear layer may include a sampling layer, a pooling layer, and an activation layer.

[0059] AI-based computations may be performed in a central processing unit (CPU) and / or a neural processing unit (NPU).

[0060] exist Figure 2A In the example, the operations of a model or layer can be operations of a central processing unit (CPU) and / or a neural processing unit (NPU).

[0061] For example, the first layer L1 may correspond to a convolutional layer, and the second layer L2 may correspond to a sampling layer. The artificial neural network model may further include an activation layer, and may further include other layers for performing other types of calculations.

[0062] In an embodiment, each of the multiple layers L1, L2 may receive input image data or a feature map (FM). The feature map of layer LN may be generated from the previous layer L(N-1) as an input feature map. Layer LN may operate the input feature map to generate an output feature map. Here, the feature map may refer to data in which various characteristics of the input data are represented. Feature maps FM1, FM2, and FM3 may have the form of a two-dimensional matrix or a three-dimensional matrix. Feature maps FM1, FM2, and FM3 may have a width W (also referred to as a column), a height H (also referred to as a row), and a depth D. Each of the width, height, and depth may correspond to the x-axis, y-axis, and z-axis of the coordinate, respectively. Here, the depth D may be referred to as the number of channels.

[0063] In an embodiment, the first layer L1 may perform a convolution of the first feature map FM1 and the weight map WM to generate a second feature map FM2. The weight map WM may filter the first feature map FM1 and may therefore be referred to as a filter or a kernel. For example, the depth (ie, the number of channels) of the weight map WM may be the same as the depth (eg, the number of channels) of the first feature map FM1, and the convolution may be performed on the same channels of the weight map WM and the first feature map FM1. The weight map WM may be shifted in a crosswise manner by using the first input feature map FM1 as a sliding window. The shift amount may be represented by a "stride length" or "stride". During each shift, the weights included in the weight map WM may be multiplied by all feature values ​​of a portion overlapping the first feature map FM1 and added. When the convolution is performed on the first feature map FM1 and the weight map WM, one channel of the second feature map FM2 may be generated.

[0064] exist Figure 2A In the embodiment, according to some embodiments, a weight map WM is described, but in fact, convolution can be performed on multiple weight maps and the first feature map FM1, and thus multiple channels of the second feature map FM2 can be generated. In other words, the number of channels of the second feature map FM2 can correspond to the number of weight maps WM.

[0065] In an embodiment, the second layer L2 may generate a third feature map FM3 by changing the spatial size of the second feature map FM2.

[0066] For example, the second layer L2 may correspond to a sampling layer. The second layer L2 may perform upsampling or downsampling. The second layer L2 may select a portion of the data included in the second feature map FM2. For example, the two-dimensional window WD may be shifted on the second feature map FM2 in units of the size of the window WD (e.g., a 4×4 matrix). Here, the second layer L2 may select a value at a specific position (e.g., row 1 column 1) in an area overlapping with the window WD. The second layer L2 may output the selected data (e.g., the value at a specific position) as data of the third feature map FM3.

[0067] For another example, the second layer L2 may correspond to a pooling layer. In this case, the second layer L2 may select the maximum value of the feature value (or the average value of the feature value) of the area overlapping the window WD in the second feature map FM2. The second layer L2 may output the selected data (e.g., the maximum value of the selected feature value (or the average value of the feature value)) as the data of the third feature map FM3.

[0068] Therefore, a third feature map FM3 obtained by changing the spatial size of the second feature map FM2 may be generated. The number of channels of the third feature map FM3 may be equal to the number of channels of the second feature map FM2.

[0069] According to an embodiment of the present disclosure, the calculation speed of the sampling layer may be faster than that of the pooling layer, and the sampling layer may improve the quality of the output image (e.g., in terms of peak signal-to-noise ratio (PSNR)). For example, the calculation performed by the pooling layer may require the maximum value of the eigenvalue or the average value of the eigenvalue, and thus require more calculation time than the calculation of the sampling layer.

[0070] According to an embodiment, the second layer L2 may not be limited to a sampling layer or a pooling layer. For example, the second layer L2 may be a convolution layer similar to the first layer L1. The second layer L2 may perform convolution of the second feature map FM2 and the weight map WM to generate a third feature map FM3. In this case, the weight map WM on which the convolution calculation has been performed in the second layer L2 may be different from the weight map WM on which the convolution calculation has been performed in the first layer L1.

[0071] According to an embodiment, the Nth feature may be generated in the Nth layer via a plurality of layers including a first layer L1 and a second layer L2. The Nth feature map may be input to a reconstruction layer position at the back end of the artificial neural network for outputting output data. The reconstruction layer may generate an output image based on the Nth feature map. In addition to the Nth feature map, the reconstruction layer may also receive a plurality of feature maps, such as a first feature map FM1, a second feature map FM2, ..., an N-1th feature map, and may generate an output image based on the received plurality of feature maps.

[0072] For example, the reconstruction layer may correspond to a convolution layer or a deconvolution layer. According to an embodiment, the reconstruction layer may be implemented as another type of layer capable of reconstructing an image from a feature map.

[0073] In order to improve power efficiency, a zero skip function may be used to omit a multiplication-accumulation calculation (MAC) with respect to 0 included in an input value of an artificial intelligence model (e.g., a result value from a previous hidden layer and a feature map). The zero skip function may use an activation function ReLU and may be effectively applied when the ratio of 0 is high. For example, when an activation function, i.e., a rectified linear unit (ReLU), is used, the ratio of 0 in a feature map is high, and thus the effect of the zero skip function may be maximized.

[0074] To improve the efficiency of the zero skip function, the electronic device may activate or deactivate the feature map compression function when a specific type of activation function is included in the artificial intelligence model.

[0075] Figure 2B is a block diagram illustrating an electronic device according to an embodiment of the present disclosure.

[0076] The memory 230 may store artificial intelligence models such as Figure 2AThe processor 220 may load the artificial intelligence model from the memory 230 via the compiler 221 and compile it. The function activation determination unit 223 determines whether the artificial intelligence model includes a first type activation function. When the first type activation function is included, the calculation unit 224 may skip the calculation on the specified value when the specified value exists in the feature map, and the calculation unit 224 may calculate the value to be calculated after the specified value with the first function unit 225. When the first type activation function is not included in the artificial intelligence model, the first function unit 225 may be disabled to prevent the value from being calculated.

[0077] refer to Figure 2B , the electronic device 200 (eg, Figure 1 The electronic device 101 in the embodiment may include a processor 220 (eg, Figure 1 processor 120) and / or memory 230 (e.g., Figure 1 Memory 130 in the memory). Figure 2B The configuration components included in the electronic device 200 may correspond to a part of the configuration included in the electronic device 200, and the electronic device 200 may include Figure 1 Various configuration components are shown.

[0078] In an embodiment, the memory 230 may temporarily or non-temporarily store data associated with the artificial intelligence model. For example, the memory 230 may store data associated with the artificial model, such as layers, weight values, and / or operations of the artificial model, and update the data associated with the artificial intelligence model based on the output value of the artificial intelligence model. For example, the memory 230 may store the output value of the artificial intelligence model.

[0079] The artificial intelligence model according to the embodiment can be written in a specified language and can correspond to a neural network model including multiple layers and / or operations. The artificial model according to the embodiment can correspond to one of various types of networks, such as convolutional neural network (CNN), region with convolutional neural network (R-CNN), region proposal network (RPN), recurrent neural network (RNN), stacked deep neural network (S-DNN), state space dynamic neural network (S-SDNN), deconvolution network, deep belief network (DBN), restricted Boltzmann machine (RBM), full convolution network, long short-term memory (LSTM) network and classification network. The artificial model according to the embodiment can learn specified data, obtain input data and perform calculations based on the obtained input data to generate output data.

[0080] The artificial model according to the embodiment may include an input layer, a hidden layer, and an output layer. The input layer may be related to the input value input to the artificial model. The hidden layer may perform a multiplication-accumulation (MAC) calculation and an activation calculation on the input value to output a feature map. The output layer may be related to the result value of the calculation performed in the hidden layer.

[0081] According to an embodiment, the artificial intelligence model may be stored in the memory 230. In some embodiments, the memory 230 may also store a plurality of executable instructions executable by the processor 220.

[0082] According to an embodiment, artificial intelligence-based calculations may be performed in the processor 220 (eg, a central processing unit (CPU)) and / or a neural processing unit (NPU).

[0083] In an embodiment, processor 220 may perform calculations based on an artificial intelligence model stored in memory 230 .

[0084] The processor 220 according to an embodiment may include at least one processor. For example, the processor 220 may include a central processing unit (CPU) (eg, Figure 1 The main processor 121 in the embodiment of the present invention) and / or the neural processing unit (NPU) (e.g., Figure 1 Auxiliary processor 123 in the apparatus).

[0085] The processor 220 according to an embodiment may include a compiler 221 , an activation function identification unit 222 , a function activation determination unit 223 , a calculation unit 224 including a first function unit 225 , and / or a second function unit 226 .

[0086] The compiler 221 according to an embodiment can compile source code written in a specific language into target code that can be processed in a target program and / or target hardware. For example, the compiler 221 can call (or load) an artificial model from the memory 230 and compile the artificial intelligence model into a binary file that can be used by the processor 220.

[0087] The activation function identification unit 222 according to an embodiment may identify the activation function included in the compiled artificial intelligence model. For example, the activation function identification unit 222 may identify whether the first type function is included in the activation function of the compiled artificial intelligence model.

[0088] The function activation determination unit 223 according to an embodiment can determine whether to activate the first function and / or the second function based on whether the first type function is included in the activation function of the compiled artificial intelligence model identified by the activation function identification unit 222, and activate or deactivate the first function unit 225 and / or the first function unit 226.

[0089] For example, the function activation determination unit 223 may activate the first function and / or the second function in response to the first type function being included in the activation function of the compiled artificial intelligence model.

[0090] For example, the function activation determination unit 223 may deactivate the first function and the second function in response to the first type function not being included in the activation function of the compiled artificial intelligence model.

[0091] According to an embodiment, the first type function may be a Rectified Linear Unit (ReLU) function. The ReLU function is a function that outputs 0 when the input value is negative and outputs the input value as is when the input value is positive.

[0092] The calculation unit 224 according to the embodiment may cause the hidden layer to perform a multiply-accumulate (MAC) calculation and an activation calculation on the input value in order to output a feature map. The MAC calculation may be a calculation that multiplies the input value by the corresponding weight and sums the multiplied value. The activation calculation may be a calculation that inputs the result of the MAC calculation to the activation function and outputs a synthesized value.

[0093] The first function unit 225 according to an embodiment may perform a first function of omitting calculation of a designated value included in an input value with respect to a hidden layer.

[0094] For example, the first function may correspond to a zero skip function. The zero skip function may correspond to a function for omitting MAC calculations with respect to 0 included in an input value (eg, a result value from a previous hidden layer and a feature map).

[0095] The second function unit 226 according to an embodiment may perform a second function of compressing with values ​​included in the output value of the hidden layer excluding a designated value.

[0096] For example, the second function may be a feature map compression function. The feature map compression function may be a function in which the processor 220 extracts a value other than 0 of an output value (eg, feature map) regarding a hidden layer to compress the feature map.

[0097] The second function unit 226 according to an embodiment may store the compressed feature map in the memory 230 .

[0098] In with Figure 2B In the related description, at least part of the components described as included in the processor 220 may be software (SW) modules executed in the processor 220 , and in this case, operations performed by these components may be understood as operations of the processor 220 .

[0099] Figure 3is a flowchart illustrating a method for performing calculation by determining, by the processor 220, whether to activate a first function and / or a second function in a calculation using an artificial intelligence model according to an embodiment of the present disclosure.

[0100] After compiling the artificial intelligence model, the first function and the second function are activated or deactivated according to whether the first type of activation function exists in the compiled artificial intelligence model. When the first function and the second function are deactivated, a multiplication-accumulation calculation can be performed.

[0101] According to an embodiment, in operation 310, a processor (e.g., Figure 2B The processor 220 in the processor can load and compile the artificial intelligence model.

[0102] The processor 220 according to an embodiment may call (or load) a data file stored in the memory ( Figure 2B The artificial intelligence model in the memory 230 in the processor 220 is compiled into a binary file that can be used by the processor 220.

[0103] In operation 320, the processor 220 may identify whether the activation function included in the artificial intelligence model includes a first type function.

[0104] The first type of function may be a rectified linear unit (ReLU) function. The ReLU function may include a function that outputs 0 when the input value is negative and outputs the input value as is when the input value is positive. That is: when x<0, f(x)=0, and when x≥0, f(x)=x.

[0105] According to an embodiment, in operation 331 , the processor 220 may activate the first function and / or the second function in response to the activation function including the first type function (eg, yes in operation 320 ).

[0106] According to an embodiment, the first function may be a function that omits calculation of a designated value included in an input value with respect to the hidden layer.

[0107] For example, the first function may correspond to a zero skip function. The zero skip function may correspond to a function for omitting MAC calculations with respect to 0 included in an input value (eg, a result value from a previous hidden layer and a feature map).

[0108] According to an embodiment, the second function may be a function for compressing with values ​​included in the output value of the hidden layer excluding a designated value.

[0109] For example, the second function may be a feature map compression function. The feature map compression function may be a function in which the processor 220 extracts a value other than 0 of an output value (eg, feature map) regarding a hidden layer to compress the feature map.

[0110] According to an embodiment, the processor 220 may activate the first function and / or the second function in response to at least one first type function being included in the hidden layer.

[0111] According to an embodiment, the processor 220 may activate the first function and / or the second function with respect to the hidden layer including the first type function. For example, among the plurality of hidden layers, the processor 220 may activate the first function in the MAC calculation of the layer next to the hidden layer including the first type function.

[0112] According to an embodiment, in operation 341, the processor 220 may activate a first function to perform a MAC calculation and an activation calculation.

[0113] According to an embodiment, the processor 220 may perform MAC calculation and activation calculation by using a feature map which is a result value of an input layer and / or a previous hidden layer as an input value.

[0114] According to an embodiment, in operation 331, in response to activation of the first function, the processor 220 may omit MAC calculation with respect to 0 of the input value, and perform MAC calculation by subsequently calculating a value other than 0. The MAC calculation may be a calculation of multiplying the input value by the corresponding weight and summing the multiplied values.

[0115] According to an embodiment, the processor 220 may perform activation calculation by inputting the result of MAC calculation to the activation function.

[0116] According to an embodiment, the processor 220 may perform MAC calculation and activation calculation to generate a feature map corresponding to the result value.

[0117] According to an embodiment, the hidden layer may include at least one layer. In the case where the hidden layer includes a plurality of layers, a feature map corresponding to a result value of a MAC calculation and an activation calculation of a previous hidden layer may be used as an input value of a next hidden layer.

[0118] According to an embodiment, in a case where a previous hidden layer includes a first type function, the processor 220 may perform MAC calculation and activation calculation by activating the first function in the next hidden layer with respect to a feature map corresponding to an output value of the previous hidden layer.

[0119] According to an embodiment, in operation 351 , the processor 220 may execute a second function.

[0120] According to an embodiment, the processor 220 may perform the second function of operation 351 after completing the calculation of the hidden layer of operation 341 .

[0121] According to an embodiment, in operation 331, the processor 220 may perform feature map compression on the feature map corresponding to the output value of the hidden layer in response to the activation of the second function. For example, when the second function is activated, the processor 220 may compress the feature map with a value other than 0 of the feature map.

[0122] According to an embodiment, in operation 332 , the processor 220 may deactivate the first function and / or the second function in response to not including the first type function in the activation function (eg, No in operation 320 ).

[0123] According to an embodiment, in operation 342, the processor 220 may perform MAC calculation and activation calculation.

[0124] According to an embodiment, the processor 220 may perform MAC calculation and activation calculation by using a feature map which is a result value of an input layer and / or a previous hidden layer as an input value.

[0125] According to an embodiment, the processor 220 may perform MAC calculations on all input values ​​in response to deactivation of the first function.

[0126] According to an embodiment, the processor 220 may perform activation calculation by inputting the result of MAC calculation to the activation function.

[0127] According to an embodiment, the processor 220 may perform MAC calculation and activation calculation to generate a feature map corresponding to the result value.

[0128] According to an embodiment, the hidden layer may include at least one layer. In the case where the hidden layer includes a plurality of layers, a feature map corresponding to a result value of a MAC calculation and an activation calculation of a previous hidden layer may be used as an input value of a next hidden layer.

[0129] According to an embodiment, in operation 360 , the processor 220 may output a result and store the result in the memory 230 .

[0130] According to an embodiment, the processor 220 may store the feature map compressed by performing the second function of operation 351 in the memory 230 .

[0131] According to an embodiment, the processor 220 may store the feature map output by performing the MAC calculation and the activation calculation of operation 342 in the memory 230 .

[0132] Figure 4 is a diagram showing the configuration of an artificial intelligence model according to an embodiment of the present disclosure. The input layer includes an input x 1 ……x n The hidden layer can perform multiplication and accumulation calculations, where the value x 1 ……xn Multiply by the corresponding multiple weights w 1 ...w n . The output layer can provide output results.

[0133] The artificial model according to an embodiment may include an input layer, a hidden layer, and an output layer.

[0134] In an embodiment, the input layer is a layer related to input values ​​input to the artificial model.

[0135] In an embodiment, the hidden layer may perform a multiply-accumulate (MAC) calculation and an activation calculation on an input value to output a feature map.

[0136] In an embodiment, the MAC calculation may be a calculation that multiplies the input value by the corresponding weight and sums the multiplied values. For example, the MAC calculation may correspond to That is, multiple input x 1 ……x n Can be multiplied by the corresponding multiple weights w 1 ...w n .

[0137] In an embodiment, the activation calculation may be a calculation that inputs the result of the MAC calculation to an activation function and outputs a resultant value. The activation calculation may be of various types. For example, the activation function may include any one of a sigmoid function, a tangent function, a ReLU function, a Leaky ReLU function, a maxout function, and / or an exponential linear unit (ELU) function, but is not limited thereto.

[0138] In an embodiment, the hidden layer may include at least one layer. For example, in the case where the hidden layer includes a first hidden layer and a second hidden layer, the first hidden layer may perform MAC calculation and activation calculation based on the input value of the input layer, and output a feature map. The feature map corresponding to the result value of the first hidden layer may become the input value of the second hidden layer. The second hidden layer may perform MAC calculation and activation calculation based on the feature map corresponding to the result value of the first hidden layer.

[0139] In an embodiment, the output layer may be a layer related to the result value of the calculation performed in the hidden layer.

[0140] Figure 5 2 is a diagram depicting a graph of an activation function according to an embodiment of the present disclosure. The activation function may include any one of the following: (a) sigmoid function, (b) tanh function, (c) ReLU function, (d) Leaky ReLU function, (e) maxout function, and (f) ELU function.

[0141] The hidden layer of the artificial intelligence model according to the embodiment may include various types of activation calculations. For example, the activation calculation may include a sigmoid function, a tangent function, a ReLU function, a Leaky ReLU function, a maxout function, and / or an exponential linear unit (ELU) function. However, the present disclosure is not limited thereto.

[0142] The graph (a) according to the embodiment is a graph of a sigmoid function.

[0143] The sigmoid function is an S-shaped function that outputs a value between 0 and 1 depending on the value of x.

[0144] The sigmoid function according to an embodiment may be as shown in Equation 1 below.

[0145] [Equation 1]

[0146] The graph (b) according to the embodiment is a graph of a tangent function.

[0147] The tangent function according to an embodiment may be as shown in Equation 2 below.

[0148] [Equation 2]

[0149] Graph (c) according to an embodiment is a graph of a rectified linear unit (ReLU) function (gradient function).

[0150] The ReLU function is a function that outputs 0 when the input value is negative and outputs the input value as it is when the input value is positive. Therefore, the output value of the ReLU function may include 0.

[0151] The ReLU function according to an embodiment may be as shown in Equation 3 below.

[0152] [Equation 3]

[0153] The graph (d) according to the embodiment is a graph of a Leaky ReLU function.

[0154] The Leaky ReLU function according to an embodiment may be as shown in Equation 4 below.

[0155] [Equation 4]

[0156] Graph (e) according to an embodiment is a graph of a maxout function.

[0157] The maxout function according to an embodiment may be as shown in Equation 5 below.

[0158] [Equation 5]

[0159] The graph (f) according to an embodiment is a graph of an exponential linear unit (ReLU) function.

[0160] The ELU function according to an embodiment may be as shown in Equation 6 below.

[0161] [Equation 6]

[0162] Figure 6 is a view showing a graph indicating a ratio of an output value of 0 in an artificial intelligence model using a first type function according to an embodiment of the present disclosure.

[0163] The first type function according to an embodiment may be a rectified linear unit (ReLU) function.

[0164] Figure 6 is a graph showing a ratio of an output value of 0 in each of an inceptionv3 artificial intelligence model and a deeplabv3 artificial intelligence model using a first type function as an activation function.

[0165] Referring to the chart, in the inceptionv3 AI model, about 50.8% of the output values ​​can be 0, and in the deeplabv3 AI model, about 53% of the output values ​​can be 0.

[0166] For example, the ratio of 0 in the output value of an artificial intelligence model that uses the first type of function as an activation function can be as high as about 50%.

[0167] Thus, in an AI model that uses the first type of function, in a processor (e.g. Figure 2B In the case where the processor 220 in the processor activates the first function, unnecessary calculation operations can be omitted, and in the case where the second function is activated, the compression efficiency can be improved.

[0168] Figure 7 is a diagram illustrating a first function according to certain embodiments disclosed herein.

[0169] According to an embodiment, the first function may correspond to a zero skip function.

[0170] For example, the zero skip function may correspond to a function for omitting a multiplication calculation about 0 in a MAC calculation, if there is 0 in an input value (eg, a result value from a previous hidden layer and a feature map).

[0171] exist Figure 7 In , black cells mean 0, and cells of another color may mean values ​​other than 0.

[0172] In an embodiment, when the zero skip function is activated, the processor (e.g., processor 220 in FIG. 2b ) may omit MAC calculations regarding 0s in the input values ​​(e.g., black cells in the figure), and perform MAC calculations by loading values ​​to be calculated other than 0 (e.g., cells other than the black cells in the figure).

[0173] Therefore, the computation time of the AI ​​model can be reduced.

[0174] In the MAC calculation, the processor 220 may output a result by summing a value obtained by multiplying a feature map that is a result value of a previous hidden layer by a weight.

[0175] The processor 220 may multiply the fmVec value of the feature map by the wVec value of the corresponding weight.

[0176] In loop #0, processor 220 may multiply fmVec0 and wVec0.

[0177] In loop #1, since the third value of fmVec1 is 0, the processor 220 may next calculate a fourth value of fmVec4 which is a value other than 0, and multiply the fourth value by the value of wVec4 to perform calculation.

[0178] In loop #2, processor 220 does not perform calculations about 0 for fmVec2, fmVec3, and fmVec4, and may provide the second value of fmVec5, the second value of fmVec6, the fourth value of fmVec5, the fourth value of fmVec6, the sixth value of fmVec6, and the sixth value of fmVec5 as values ​​other than 0 to be calculated next, and multiply them by the corresponding second value of wVec5, the second value of wVec6, the fourth value of wVec5, the fourth value of wVec6, the sixth value of wVec5, and the sixth value of wVec6, respectively.

[0179] In loop #3, the processor 220 may perform calculation by multiplying the value of fmVec7 other than 0 to be calculated next by the value of wVec7.

[0180] Without executing the zero skip function, the processor 220 needs to perform a total of nine calculations from fmVec0 to fmVec8, while with executing the zero skip function, the MAC calculation can be completed with three calculations.

[0181] Therefore, with the zero skipping function activated, the number of computations can be reduced for feature maps containing many zeros.

[0182] Figure 8 is a view showing a second function which is a feature map compression function excluding 0 values.

[0183] According to an embodiment, the second function may correspond to a feature map compression function (feature map lossless compressor).

[0184] The feature map compression function may be a function of a processor (e.g., Figure 2B The processor 220 in the embodiment of the present invention extracts a value other than 0 about the output value (eg, feature map) of the hidden layer to compress the feature map.

[0185] In case the feature map compression function is activated, the processor 220 may compress the feature map with a value other than 0 of the feature map.

[0186] refer to Figure 8 , the processor 220 may compress the feature map with the values ​​“0110”, “1110”, “1111”, “1111”, “1111”, “1111”, “1110”, “1101”, “1011” and “0111” excluding the value “0000” in the feature map.

[0187] The processor 220 may store the compressed feature map in a memory (eg, Figure 2B In the memory 230).

[0188] Therefore, when the zero skipping function is activated, for a feature map containing many zeros, the capacity stored in the memory 230 can be reduced.

[0189] Fig. 9 is a view depicting a graph of a ReLU function and a PReLU function according to an embodiment of the present disclosure.

[0190] The hidden layer of the artificial intelligence model according to the embodiment may include various types of activation calculations. For example, the activation function may include a ReLU function and / or a PReLU function. However, the present disclosure is not limited thereto.

[0191] In an embodiment, the ReLU function may be a function that outputs 0 when the input value is negative and outputs the input value as it is when the input value is positive. Therefore, in the case where the hidden layer includes the ReLU function, the ratio of 0 included in the feature map may be high.

[0192] In an embodiment, the PReLU function may be a function that outputs 1 / 2 of the input value when the input value is negative and outputs the input value as it is when the input value is positive. Therefore, in the case where the hidden layer includes the PReLU function, the ratio of 0 included in the feature map may be low.

[0193] Therefore, in the case where the hidden layer includes an activation function whose output value includes a high ratio of many 0s (such as a ReLU function), activating the first function (e.g., a zero skip function) can improve the efficiency of calculation, and activating the second function (e.g., a feature map compression function) can improve the compression efficiency of data stored in the memory. On the other hand, in the case where the hidden layer includes an activation function whose output value includes a low ratio of many 0s (such as a PReLU function), activating the first function (e.g., a zero skip function) may reduce the efficiency of calculation due to the operation of identifying 0, and even if the second function (e.g., a feature map compression function) is activated, the compression efficiency may be low.

[0194] Table 1 may be a result of comparing current consumption when a first function (zero skip function) and a second function are activated according to types of functions included in a hidden layer.

[0195]

Table 1

[0196] The first operation mode, the second operation mode, and the third operation mode in Table 1 may be modes operated when measuring performance and consumed current of NPU hardware (HW).

[0197] Referring to Table 1, in terms of performance, when the first function and the second function are activated, the performance difference between the artificial intelligence model including ReLU and the artificial intelligence model including non-ReLU may not be significant.

[0198] On the other hand, in terms of current consumption, when the first function and the second function are activated, it can be identified that the current consumption in the artificial intelligence model including non-ReLU is higher than that in the artificial intelligence model including ReLU.

[0199] For example, even if the rate of 0 is low, the first function performs unnecessary calculations to identify 0, and thus a feature map of an artificial intelligence model including a non-ReLU may increase current consumption.

[0200] Furthermore, even if the ratio of 0s is low, the second function performs unnecessary operations to exclude 0s, and thus a feature map of an artificial intelligence model including non-ReLU may increase current consumption.

[0201] Therefore, as an embodiment of the present disclosure, since the processor 220 identifies whether to activate the first function and / or the second function and determines the activation of the first function and / or the second function according to the type of activation function included in the artificial intelligence model, not only can current consumption be reduced but also data can be compressed efficiently.

[0202] Fig.10 2 is a flowchart illustrating a method for performing calculation by determining, by the processor 220, whether to activate a first function in a calculation using an artificial intelligence model according to an embodiment of the present disclosure. The first function is activated or deactivated based on whether the compiled artificial intelligence model includes a first type activation function.

[0203] According to an embodiment, in operation 1010, a processor (eg, Figure 2B The processor 220 in the processor can load and compile the artificial intelligence model.

[0204] The processor 220 according to an embodiment may call (or load) a data file stored in the memory ( Figure 2B The artificial intelligence model in the memory 230 in the processor 220 is compiled into a binary file that can be used by the processor 220.

[0205] According to an embodiment, in operation 1020, the processor 220 may identify whether the activation function included in the compiled artificial intelligence model includes a first type function.

[0206] According to an embodiment, the first type function may be a rectified linear unit (ReLU) function. The ReLU function may include a function that outputs 0 when the input value is negative and outputs the input value as is when the input value is positive.

[0207] According to an embodiment, in operation 1030 , the processor 220 may activate the first function in response to including the first type function in the activation function (eg, YES in operation 1020 ).

[0208] According to an embodiment, the first function may be a function that omits calculation of a designated value included in an input value with respect to the hidden layer.

[0209] For example, the first function may correspond to a zero skip function. The zero skip function may correspond to a function for omitting MAC calculations with respect to 0 included in an input value (eg, a result value from a previous hidden layer and a feature map).

[0210] According to an embodiment, in operation 1040 , the processor 220 may deactivate the first function in response to not including the first type function in the activation function (eg, No in operation 1020 ).

[0211] According to an embodiment, the processor 220 may perform MAC calculations on all input values ​​in response to deactivation of the first function.

[0212] According to an embodiment, the processor 220 may perform activation calculation by inputting the result of MAC calculation to the activation function.

[0213] According to an embodiment, the processor 220 may perform MAC calculation and activation calculation to generate a feature map corresponding to the result value.

[0214] According to an embodiment, the hidden layer may include at least one layer. In the case where the hidden layer includes a plurality of layers, a feature map corresponding to a result value of a MAC calculation and an activation calculation of a previous hidden layer may be used as an input value of a next hidden layer.

[0215] The technical problems solved by the present disclosure are not limited to the above-mentioned technical problems, and those skilled in the art will understand other technical problems more clearly from the following description.

[0216] An electronic device according to an embodiment may include a processor and a memory, the memory storing instructions, wherein the instructions, when executed by the processor, enable the electronic device to: load and compile an artificial intelligence model stored in the memory; determine whether the compiled artificial intelligence model includes a first type of activation function; when the compiled artificial intelligence model includes the first type of activation function, skip calculations about a specified value when there is a specified value in a feature map, and calculate a value to be calculated after the specified value; and when the compiled artificial intelligence model does not include the first type of activation function, perform calculations about input values ​​of the feature map.

[0217] In the electronic device according to an embodiment, the first type activation function may be a rectified linear unit (ReLU) function.

[0218] In an embodiment, when the instruction is executed by the processor, the electronic device may: when a first type of activation function is included in the compiled artificial intelligence model, compress the feature map by excluding the specified value when the specified value exists in the feature map, and when the first type of activation function is not included in the compiled artificial intelligence model, disable compression of the feature map.

[0219] In the electronic device according to the embodiment, skipping calculation about the designated value and calculating a value to be calculated after the designated value includes executing a zero skip function that skips calculation for "0" if "0" exists in the characteristic map.

[0220] In the electronic device according to the embodiment, the compression feature map is a function of extracting values ​​excluding the value '0' from the feature map and compressing the feature map based on the extracted values.

[0221] In an embodiment, when the instruction is executed by the processor, the electronic device is configured to: in response to calculating the value, when a feature map corresponding to a result value of a previous hidden layer of the artificial intelligence model is used as an input value and there is "0" in the feature map, the calculation about "0" can be skipped and the calculation about the value other than "0" to be calculated next can be performed, and the calculation about the multiply-accumulate calculation (MAC) result can be performed.

[0222] In an embodiment, when the instruction is executed by the processor, the electronic device is configured to: in response to compressing the feature map, after completing the calculation of the hidden layer of the artificial intelligence model, compress the feature map with a value other than "0" corresponding to the output value of the hidden layer of the feature map; and store the compressed feature map in a memory.

[0223] According to the embodiment, the operating method of the electronic device may include: an operation of loading and compiling an artificial intelligence model stored in a memory; an operation of determining whether the compiled artificial intelligence model includes a first type of activation function; when the compiled artificial intelligence model includes the first type of activation function, when there is a specified value in the feature map, an operation of skipping calculations about the specified value and calculating a value to be calculated after the specified value; and when the compiled artificial intelligence model does not include the first type of activation function, an operation of performing calculations about the input value of the feature map.

[0224] In the operating method of the electronic device according to the embodiment, the first type activation function may be a rectified linear unit (ReLU) function.

[0225] According to the embodiment, the operating method of the electronic device may include the following operations: when the first type of activation function is included in the compiled artificial intelligence model, compressing the feature map by excluding the specified value when the specified value exists in the feature map, and when the first type of activation function is not included in the compiled artificial intelligence model, deactivating the compression of the feature map.

[0226] In the operating method of the electronic device according to the embodiment, skipping calculation about the designated value and calculating the value to be calculated after the designated value includes: if there is "0" in the characteristic map, executing a zero skip function that skips calculation for "0".

[0227] In the operating method of the electronic device according to the embodiment, compressing the feature map includes extracting a value excluding a value of '0' from the feature map and compressing the function of the feature map based on the extracted value.

[0228] According to an embodiment, the operating method of the electronic device may include: in response to activation of a first function, when a feature map corresponding to a result value of a previous hidden layer of an artificial intelligence model is used as an input value and there is "0" in the feature map, skipping calculations regarding "0" and performing calculations regarding a value other than "0" to be calculated next; and performing calculations regarding a multiply-accumulate calculation (MAC) result.

[0229] According to the embodiment, the operating method of the electronic device may include: in response to the activation of the second function, after completing the calculation of the hidden layer of the artificial intelligence model, an operation of compressing the feature map with a value other than 0 corresponding to the output value of the hidden layer of the feature map; and an operation of storing the compressed feature map in a memory.

[0230] According to certain embodiments, a non-transitory computer-readable medium stores a plurality of instructions executable by a processor, wherein execution of the plurality of instructions by the processor causes the processor to perform a plurality of operations, the plurality of operations comprising: loading and compiling an artificial intelligence model stored in a memory; determining whether the compiled artificial intelligence model includes a first type of activation function; when the compiled artificial intelligence model includes the first type of activation function, skipping calculations regarding a specified value when there is a specified value in a feature map, and calculating a value to be calculated after the specified value; and performing calculations regarding input values ​​of the feature map when the compiled artificial intelligence model does not include the first type of activation function.

[0231] According to some embodiments, the first type of activation function is a Rectified Linear Unit (ReLU) function.

[0232] According to certain embodiments, the plurality of operations further include: compressing the feature map by excluding a specified value when the specified value exists in the feature map when the compiled artificial intelligence model includes a first type of activation function, and disabling compression of the feature map when the first type of activation function is not included in the compiled artificial intelligence model.

[0233] According to some embodiments, skipping calculations on a designated value and calculating a value to be calculated after the designated value includes: if there is "0" in the feature map, executing a zero skip function that skips calculations for "0".

[0234] According to some embodiments, compressing the feature map is a function of extracting values ​​excluding the value "0" from the feature map and compressing the feature map based on the extracted values.

[0235] According to certain embodiments, the multiple operations also include: in response to compressing the feature map, after completing the calculation of the hidden layer of the artificial intelligence model, compressing the feature map with a value other than "0" of the feature map corresponding to the output value of the hidden layer; and storing the compressed feature map in a memory.

[0236] It will be understood by those skilled in the art that effects that can be achieved from the present disclosure are not limited to the above-described effects, and other effects not described above will be clearly understood from the following detailed description.

[0237] The electronic device according to various embodiments may be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to an embodiment of the present disclosure, the electronic device is not limited to those electronic devices described above.

[0238] It should be understood that the various embodiments of the present disclosure and the terms used therein are not intended to limit the technical features set forth herein to specific embodiments, but include various changes, equivalent forms or alternative forms for the corresponding embodiments. For the description of the accompanying drawings, similar reference numerals may be used to refer to similar or related elements. It will be understood that the nouns in the singular form corresponding to the term may include one or more things unless the relevant context clearly indicates otherwise. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include any one or all possible combinations of the items listed together with the corresponding one of the multiple phrases. As used herein, terms such as "1st" and "2nd" or "first" and "second" may be used to simply distinguish the corresponding component from another component, and do not limit the component in other aspects (e.g., importance or order). It will be understood that if an element (e.g., a first element) is referred to as being “coupled with another element (e.g., the second element)”, “coupled to another element (e.g., the second element)”, “connected with another element (e.g., the second element)”, or “connected to another element (e.g., the second element)” with or without the terms “operably” or “communicatively” being used, it means that the element may be directly (e.g., wired) connected to the other element, wirelessly connected to the other element, or connected to the other element via a third element.

[0239] As used in connection with various embodiments of the present disclosure, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "portion," or "circuit"). A module may be a single integrated component adapted to perform one or more functions or a minimum unit or portion of the single integrated component. For example, according to an embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0240] The various embodiments described herein may be implemented as software (e.g., program 140) including one or more instructions stored in a storage medium (e.g., internal memory 136 or external memory 138) that can be read by a machine (e.g., electronic device 101). For example, under the control of a processor, a processor (e.g., processor 120) of the machine (e.g., electronic device 101) may call at least one of the one or more instructions stored in the storage medium and execute the at least one instruction with or without the use of one or more other components. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Among them, the term "non-transitory" only means that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between data being semi-permanently stored in the storage medium and data being temporarily stored in the storage medium.

[0241] According to an embodiment, the method according to various embodiments of the present disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be released in the form of a machine-readable storage medium (e.g., a compact disk read-only memory (CD-ROM)), or may be downloaded via an application store (e.g., Play Store TM ) the computer program product may be published (e.g., downloaded or uploaded) online, or the computer program product may be distributed (e.g., downloaded or uploaded) directly between two user devices (e.g., smart phones). If published online, at least part of the computer program product may be temporarily generated, or at least part of the computer program product may be at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, a server of an application store, or a forwarding server).

[0242] According to various embodiments, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities, and some of the multiple entities may be separately arranged in different components. According to various embodiments, one or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., a module or a program) may be integrated into a single component. In this case, according to various embodiments, the integrated component may still perform the one or more functions of each of the multiple components in the same or similar manner as a corresponding one of the multiple components performing one or more functions before integration. According to various embodiments, the operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or in a heuristic manner, or one or more of the operations may be run or omitted in a different order, or one or more other operations may be added.

Claims

1. An electronic device, comprising: processor; as well as Memory, storage instructions, Wherein, when the instructions are executed by the processor, the electronic device: Loading and compiling an artificial intelligence model stored in the memory; determining whether the compiled artificial intelligence model includes a first type of activation function; When the first type of activation function is included in the compiled artificial intelligence model, skipping calculations on a specified value when the specified value exists in a feature map, and calculating a value to be calculated after the specified value; as well as When the first type of activation function is not included in the compiled artificial intelligence model, calculations on input values ​​of the feature map are performed.

2. The electronic device according to claim 1, wherein: The first type of activation function is a rectified linear unit (ReLU) function.

3. The electronic device according to claim 2, wherein: The instructions, when executed by the processor, further cause the electronic device to: When the first type of activation function is included in the compiled artificial intelligence model, compressing the feature map by excluding the specified value when the specified value is present in the feature map; as well as When the first type of activation function is not included in the compiled artificial intelligence model, compressing the feature map is disabled.

4. The electronic device according to claim 2, wherein: Skipping calculations about the designated value and calculating a value to be calculated after the designated value includes: if "0" exists in the feature map, executing a zero skip function, the zero skip function skipping calculations for "0".

5. The electronic device according to claim 3, wherein: Compressing the feature map is a function of extracting values ​​excluding the value "0" from the feature map and compressing the feature map based on the extracted values.

6. The electronic device according to claim 4, wherein: The instructions, when executed by the processor, further cause the electronic device to: In response to the calculated value, when a feature map corresponding to a result value of a previous hidden layer of the artificial intelligence model is used as an input value and "0" exists in the feature map, skipping calculations regarding "0" and performing calculations regarding a value other than "0" to be calculated next; as well as Perform calculations on the multiply-accumulate calculation MAC result.

7. The electronic device according to claim 5, wherein: The instructions, when executed by the processor, further cause the electronic device to: In response to compressing the feature map, after completing the calculation of the hidden layer of the artificial intelligence model, compressing the feature map with a value other than "0" of the feature map corresponding to the output value of the hidden layer; as well as The compressed feature map is stored in the memory.

8. A method for operating an electronic device, the method comprising: Loading and compiling AI models stored in memory; determining whether the compiled artificial intelligence model includes a first type of activation function; When the first type of activation function is included in the compiled artificial intelligence model, skipping calculations on a specified value when the specified value exists in a feature map, and calculating a value to be calculated after the specified value; as well as When the first type of activation function is not included in the compiled artificial intelligence model, calculations on input values ​​of the feature map are performed.

9. The method according to claim 8, wherein: The first type of activation function is a rectified linear unit (ReLU) function.

10. The method according to claim 9, further comprising: When the first type of activation function is included in the compiled artificial intelligence model, compressing the feature map by excluding the specified value when the specified value is present in the feature map; and When the first type of activation function is not included in the compiled artificial intelligence model, compressing the feature map is disabled.

11. The method according to claim 9, wherein: Skipping calculations about the designated value and calculating a value to be calculated after the designated value includes: if "0" exists in the feature map, executing a zero skip function, the zero skip function skipping calculations for "0".

12. The method according to claim 10, wherein: Compressing the feature map is a function of extracting values ​​excluding the value "0" from the feature map and compressing the feature map based on the extracted values.

13. The method according to claim 11, further comprising: In response to the calculated value, in a case where a feature map corresponding to a result value of a previous hidden layer of the artificial intelligence model is used as an input value and there is "0" in the feature map, skipping calculations on "0" and performing calculations on values ​​other than "0" to be calculated next; as well as Perform calculations on the multiply-accumulate MAC result.

14. The method according to claim 12, further comprising: In response to compressing the feature map, after completing the calculation of the hidden layer of the artificial intelligence model, compressing the feature map with a value other than "0" of the feature map corresponding to the output value of the hidden layer; as well as The compressed feature map is stored in the memory.

Images