Computing devices and methods for training artificial neural network models and memory systems

Abstract

A computing device and method for training an artificial neural network model and a memory system are disclosed. The computing device for training an artificial neural network model includes a model analyzer configured to receive a first artificial neural network model and divide the first artificial neural network model into a plurality of layers; a training logic configured to calculate first sensitivity data varying as the first artificial neural network model is pruned, calculate a target sensitivity corresponding to a target pruning rate based on the first sensitivity data, calculate second sensitivity data varying as each of the plurality of layers is pruned, and output an optimal pruning rate for each of the plurality of layers based on the second sensitivity data, the optimal pruning rate corresponding to the target pruning rate; and a model updater configured to prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2019-0053888, filed May 8, 2019, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference. TECHNICAL FIELD

[0002] The inventive concept relates to a computing device for training an artificial neural network model by adjusting a pruning rate based on sensitivity of an artificial neural network model, a method of training an artificial neural network model, and a memory system for storing an artificial neural network model. BACKGROUND

[0003] An artificial neural network (ANN) represents a computing architecture modeled after a biological brain. Deep learning or machine learning can be implemented based on an ANN. A deep neural network or deep learning, which is an example of an ANN, can have a multi-layer structure including a plurality of layers. Recently, the number of operations performed using an ANN has significantly increased, and thus, the amount of data for training an ANN has increased, and there is a demand for training an ANN with less data. SUMMARY

[0004] The inventive concept provides a computing device including a method of accurately training an artificial neural network model using a small amount of data by considering sensitivity of each layer of an artificial neural network, which varies according to a pruning rate, and a memory system.

[0005] According to an aspect of the inventive concept, there is provided a computing device for training an artificial neural network model, the computing device comprising: a model analyzer configured to receive a first artificial neural network model and divide the first artificial neural network model into a plurality of layers; a training logic configured to: calculate first sensitivity data varying as the artificial neural network model is pruned, calculate a target sensitivity corresponding to a target pruning rate based on the first sensitivity data, calculate second sensitivity data varying as each of the plurality of layers of the first artificial neural network model is pruned, and output a best pruning rate for each of the plurality of layers based on the second sensitivity data, the best pruning rate corresponding to the target pruning rate; and a model updater configured to: prune the first artificial neural network model based on the best pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model.

[0006] According to another aspect of the inventive concepts, there is provided a computing device for training an artificial neural network model, the computing device comprising: a model analyzer configured to receive a first artificial neural network model and partition the first artificial neural network model into a plurality of layers; a training logic configured to: receive a target pruning rate, compute a target sensitivity of the first artificial neural network model, and output an optimal pruning rate for each of the plurality of layers, the target sensitivity corresponding to the target pruning rate, the optimal pruning rate corresponding to the target sensitivity; and a model updater configured to: prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model.

[0007] According to another aspect of the inventive concepts, there is provided a method of training an artificial neural network model, the method comprising: receiving a target pruning rate; pruning a first artificial neural network model; computing first sensitivity data varying as a function of a pruning rate of the first artificial neural network model as the first artificial neural network model is pruned; computing a target sensitivity corresponding to the target pruning rate from the first sensitivity data; partitioning the first artificial neural network model into a plurality of layers; computing second sensitivity data varying as each of the plurality of layers is pruned; computing an optimal pruning rate for each of the plurality of layers corresponding to the target sensitivity based on the second sensitivity data; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model.

[0008] According to another aspect of the inventive concepts, there is provided a memory system for processing an artificial neural network model comprising a plurality of layers, the memory system comprising: a host processor; and a plurality of logical layers configured to communicate with the host processor, wherein each of the plurality of logical layers is electrically connected to a memory stack, and wherein the host processor is configured to: receive original weights for each of the plurality of layers, receive memory addresses, output the original weights to a first logical layer, and output a pruning rate corresponding to the original weights output to the first logical layer, the first logical layer comprising first memory stacks corresponding to at least some of the memory addresses, and wherein the first logical layer is configured to: compute a sensitivity based on at least one of the original weights and the pruning rate, and store at least one of the original weights, the pruning rate, and the sensitivity in the first memory stacks.

[0009] According to another aspect of the inventive concept, there is provided a non-transitory computer-readable recording medium having recorded thereon a program for pruning an artificial neural network model including a plurality of layers, the program comprising: receiving a target pruning rate; pruning a first artificial neural network model; calculating first sensitivity data varying according to a pruning rate of the first artificial neural network model; calculating a target sensitivity corresponding to the target pruning rate from the first sensitivity data; dividing the first artificial neural network model into a plurality of layers; calculating second sensitivity data varying according to a pruning rate of each of the plurality of layers; calculating an optimal pruning rate of each of the plurality of layers corresponding to the target sensitivity based on the second sensitivity data; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model.

[0010] According to another aspect of the inventive concept, there is provided a computing device for training an artificial neural network model for image processing, the computing device comprising: an image acquirer configured to acquire first training images and second training images; a model generator configured to train a neural network based on the first training images to generate a first artificial neural network model; a model analyzer configured to receive the first artificial neural network model and divide the first artificial neural network model into a plurality of layers; a training logic configured to calculate first sensitivity data varying as the first artificial neural network model is pruned based on the second training images, calculate a target sensitivity corresponding to a target pruning rate based on the first sensitivity data, calculate second sensitivity data varying as each of the plurality of layers of the first artificial neural network model is pruned based on the second training images, and output an optimal pruning rate of each of the plurality of layers based on the second sensitivity data, the optimal pruning rate corresponding to the target pruning rate; and a model updater configured to prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model.

[0011] According to another aspect of the inventive concept, there is provided a computing device for training an artificial neural network model for image processing, the computing device comprising: an image obtainer configured to obtain first training images and second training images; a model generator configured to train a neural network based on the first training images to generate a first artificial neural network model; a model analyzer configured to receive the first artificial neural network model and divide the first artificial neural network model into a plurality of layers; training logic configured to receive a target pruning rate, compute a target sensitivity of the first artificial neural network model based on the second training images, and output an optimal pruning rate for each of the plurality of layers, the target sensitivity corresponding to the target pruning rate, the optimal pruning rate corresponding to the target sensitivity; and a model updater configured to prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model.

[0012] According to another aspect of the inventive concept, there is provided a method of training an artificial neural network model for image processing, the method comprising: obtaining first training images and second training images; training a neural network based on the first training images to generate a first artificial neural network model; receiving a target pruning rate; pruning the first artificial neural network model; computing first sensitivity data varying according to a pruning rate of the first artificial neural network model based on the second training images; computing a target sensitivity corresponding to the target pruning rate from the first sensitivity data; dividing the first artificial neural network model into a plurality of layers; computing second sensitivity data varying according to a pruning rate of each of the plurality of layers based on the second training images; computing an optimal pruning rate for each of the plurality of layers corresponding to the target sensitivity based on the second sensitivity data; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model.

[0013] According to another aspect of the inventive concept, there is provided an electronic system for training an artificial neural network model for image processing, the electronic system comprising: a neural network device training a neural network based on a first training image to generate an artificial neural network model comprising a plurality of layers, and a memory system connected with the neural network device through a bus, the memory system comprising: a host processor; and a plurality of logical layers configured to communicate with the host processor, wherein each of the plurality of logical layers is electrically connected to a memory stack, and wherein the host processor is configured to: receive original weights of each of the plurality of layers, receive memory addresses, output the original weights to a first logical layer, and output a pruning rate corresponding to the original weights output to the first logical layer, include, in the first logical layer, a first memory stack corresponding to at least some of the memory addresses, and wherein the first logical layer is configured to: calculate a sensitivity and the pruning rate based on the original weights and a second training image, and store at least one of the original weights, the pruning rate, and the sensitivity in the first memory stack.

[0014] According to another aspect of the inventive concept, there is provided a non-transitory computer-readable recording medium having recorded thereon a program, which when executed by a processor, causes the processor to execute a method for training an artificial neural network model for image processing, the method comprising: obtaining a first training image and a second training image; training a neural network based on the first training image to generate a first artificial neural network model; receiving a target pruning rate; pruning the first artificial neural network model; calculating first sensitivity data varying according to a pruning rate of the first artificial neural network model based on the second training image; calculating a target sensitivity corresponding to the target pruning rate from the first sensitivity data; dividing the first artificial neural network model into a plurality of layers; calculating second sensitivity data varying according to a pruning rate of each of the plurality of layers based on the second training image; calculating an optimal pruning rate of each of the plurality of layers corresponding to the target sensitivity based on the second sensitivity data; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model. BRIEF DESCRIPTION OF DRAWINGS

[0015] Embodiments of the inventive concept will become more fully understood from the detailed description given herein below, and the accompanying drawings, wherein:

[0016] Figure 1 is a diagram for describing a computing device according to an example embodiment of the inventive concept;

[0017] Figure 2 shows an example of an artificial neural network;

[0018] Figure 3is a graph for describing a pruning method according to an example embodiment and a comparative example of the inventive concept;

[0019] Figure 4 is a block diagram of a computing device according to an example embodiment of the inventive concept;

[0020] Figure 5 is a graph showing first sensitivity data of an entire artificial neural network model according to an example embodiment of the inventive concept;

[0021] Figure 6 is a graph showing second sensitivity data including a sensitivity curve of each of a plurality of layers according to an example embodiment of the inventive concept;

[0022] Figure 7 is a graph showing third sensitivity data based on the second sensitivity data according to an example embodiment of the inventive concept;

[0023] Figure 8 is a graph showing a sensitivity value of each of pruning rates of a plurality of layers according to an example embodiment of the inventive concept;

[0024] Figure 9 is a graph showing accuracy of an artificial neural network model for a pruning rate according to an example embodiment and a comparative example of the inventive concept;

[0025] Figure 10 is a flowchart of a pruning operation of an artificial neural network model according to an example embodiment of the inventive concept;

[0026] Figure 11 is a flowchart of a method of obtaining first sensitivity data according to an example embodiment of the inventive concept;

[0027] Figure 12 is a flowchart of a method of obtaining second sensitivity data according to an example embodiment of the inventive concept;

[0028] Figure 13 is a graph for describing a memory system according to an example embodiment of the inventive concept;

[0029] Figure 14 is a block diagram of a logic layer according to an example embodiment of the inventive concept;

[0030] Figure 15 is a block diagram of a memory system for calculating a pruning rate by receiving an original weight according to an example embodiment of the inventive concept;

[0031] Figure 16 is a block diagram of a memory system for calculating a pruning rate by receiving a corrected weight according to an example embodiment of the inventive concept; and

[0032] Figure 17 is a block diagram of a memory system including a group ID manager according to example embodiments of the inventive concept. DETAILED DESCRIPTION

[0033] Hereinafter, the inventive concept will be described in detail by explaining example embodiments of the inventive concept with reference to the attached drawings.

[0034] Figure 1 is a diagram for describing a computing device according to example embodiments of the inventive concept.

[0035] The electronic system 10 according to example embodiments of the inventive concept can analyze input data in real time based on an artificial neural network model to extract effective information, and control components of an electronic device including the electronic system 10 based on the extracted information. For example, the electronic system 10 can be applied to a smart phone, a mobile device, an image display device, a measuring device, a smart TV, a drone, a robot device such as an advanced driver assistance system (ADAS), a medical device, an Internet of Things (IoT) device, etc., or can be implemented in one of various electronic devices other than these devices. Hereinafter, an artificial neural network can be referred to as a neural network.

[0036] For example, Figure 1 The electronic system 10 of FIG. 1 can be an application processor. Since the electronic system 10 performs an artificial neural network operation function, the electronic system 10 can be defined to include an artificial neural network model.

[0037] Referring to FIG. 2, Figure 1 The electronic system 10 can include a central processing unit (CPU) 11, a random access memory (RAM) 12, a neural network device 13, a storage device 14, and one or more input / output (IO) devices 15. The storage device 14 can include a memory of a conventional computer such as a hard disk drive (the memory of the conventional computer can also be a solid state drive, a DRAM, a NAND flash, etc.). The input / output device 15 can include a user interface of a conventional computer, and include a conventional input device such as a keyboard, a mouse, a touchpad, a touch screen, etc.

[0038] The neural network device 13 can include training logic 100. As will be described, the training logic 100 can train an artificial neural network model. In some embodiments, the training logic 100 can be included not only in the neural network device 13, but also in the CPU 11, the RAM 12, and the storage device 14, etc., or in various processors such as a digital signal processor (DSP). For ease of description, the training logic 100 is described as being included in the neural network device 13.

[0039] Although not shown, the electronic system 10 can further include a sensor module, a security module, a power control device, etc., and can further include various other types of operating units. For example, some or all of the components (CPU 11, RAM 12, neural network device 13, storage device 14, and input / output device 15) of the electronic system 10 can be implemented in a single semiconductor chip. For example, the electronic system 10 can be implemented as a system on chip (SoC). The components of the electronic system 10 can communicate with each other via a bus 16.

[0040] The CPU 11 controls the overall operation of the electronic system 10. The CPU 11 can include a single processor core (single core) or multiple processor cores (multi-core). The CPU 11 can process or execute programs and / or data stored in the storage device 14. For example, the CPU 11 can control the functions of the neural network device 13 by executing programs stored in the storage device 14.

[0041] The RAM 12 can temporarily store programs, data, and instructions. For example, programs and / or data stored in the storage device 14 can be temporarily stored in the RAM 12 according to the control or boot code of the CPU 11. The RAM 12 can include dynamic RAM (DRAM) or static RAM (SRAM).

[0042] The neural network device 13 performs operations based on an artificial neural network model and trains the artificial neural network model. The artificial neural network model can perform image processing. According to an example embodiment, the training logic 100 can perform post-training on an artificial neural network model that has been iteratively retrained. However, the inventive concept is not limited thereto, and according to another example embodiment, the training logic 100 can retrain the artificial neural network model. According to another example embodiment, the training logic 100 can perform both retraining and post-training.

[0043] According to an example embodiment of the inventive concept, the training logic 100 can perform post-training on an artificial neural network model that has been pre-trained (such as retrained) by using a data set including a relatively small amount of data rather than a complete data set.

[0044] For example, the training logic 100 can perform post-training on an artificial neural network model that has been pre-trained based on at least one first training image based on at least one second training image.

[0045] The artificial neural network model can include various types of artificial neural network models such as a convolutional neural network (CNN), a region convolutional neural network (R-CNN), a region proposal network (RPN), a recurrent neural network (RNN), a stacking-based deep neural network (S-DNN), a state space dynamic neural network (S-SDNN), a deconvolutional network, a deep belief network (DBN), a restricted Boltzmann machine (RBM), a fully convolutional network, a long short-term memory (LSTM) network, or a classification network, but is not limited thereto. Reference will be made to Figure 2 An example of an artificial neural network is described.

[0046] Figure 2 An example of an artificial neural network is shown.

[0047] Reference is made to Figure 2 The artificial neural network NN can include a plurality of layers L1 to Ln, where n is a positive integer. Each of the plurality of layers L1 to Ln can be a linear layer or a nonlinear layer, and according to an embodiment, at least one linear layer and at least one nonlinear layer can be combined and referred to as a single layer. For example, the linear layer can include a convolutional layer, a fully connected layer, etc., and the nonlinear layer can include a pooling layer and an activation layer.

[0048] For example, the first layer L1 can be a convolutional layer, the second layer L2 can be a pooling layer, and the nth layer Ln can be a fully connected layer as an output layer. The artificial neural network NN can further include an activation layer and a layer via which other types of operations are performed.

[0049] Each of the plurality of layers L1 to Ln can receive an input image frame or a feature map generated in a previous layer as an input feature map, and can generate an output feature map by computing the input feature map. For example, the layer L2 can receive an input feature map from the layer L1 and generate an output feature map by performing an operation on the input feature map received from the layer L1. The output of the layer L2 can be provided to the layer L3. The feature map refers to data representing various features of input data. The feature maps FM1 to FMn can have, for example, a two-dimensional matrix shape or a three-dimensional matrix shape. The feature maps FM1 to FMn can have a width W (also referred to as a column), a height H (also referred to as a row), and a depth D, which can respectively correspond to x-, y-, and z-axes on a coordinate. The depth D can be referred to as the number of channels.

[0050] The first layer L1 can convolve the first feature map FM1 with a weight map WM to generate a second feature map FM2. The weight map WM can filter the first feature map FM1, and can be referred to as a filter or a kernel. The depth of the weight map WM (e.g., the number of channels of the weight map WM) is equal to the depth of the first feature map FM1 (e.g., the number of channels of the first feature map), and a respective channel of the weight map WM can be convolved with a channel of the first feature map FM1. The weight map WM moves as a sliding window over the first feature map FM1. The amount of movement can be referred to as a "stride length" or "stride." During each movement, each weight included in the weight map WM can be multiplied by or added to each feature value in an area that overlaps the first feature map FM1. When the first feature map FM1 is convolved with the weight map WM, a channel of the second feature map FM2 can be generated. Although one weight map WM is shown in FIG. 1, in practice, multiple weight maps can be convolved with the first feature map FM1, and multiple channels of the second feature map FM2 can be generated. For example, the number of channels of the second feature map FM2 can correspond to the number of weight maps. Figure 2

[0051] The second layer L2 can modify the spatial size of the second feature map FM2 by pooling to generate a third feature map FM3. Pooling can be referred to as sampling or down-sampling. For example, in the second layer L2, the second feature map FM2 can be sampled (or down-sampled) to reduce the spatial size of the second feature map FM2 in the x-direction and the y-direction. A two-dimensional pooling window PW can move over the second feature map FM2 in units of the size of the pooling window PW, and a maximum feature value (or an average of feature values) of an area that overlaps the pooling window PW can be selected. For example, the two-dimensional pooling window PW can move in the x-direction and the y-direction in increments of the width and the height of the pooling window PW, respectively. Thus, the third feature map FM3 having a spatial size that is changed from the spatial size of the second feature map FM2 can be generated. The number of channels of the third feature map FM3 is equal to the number of channels of the second feature map FM2. For example, the depth of the third feature map FM3 can be the same as the depth of the second feature map FM2.

[0052] According to an example embodiment, the second layer L2 is not limited to a pooling layer. For example, the second layer L2 can be a pooling layer, or the second layer L2 can be a convolution layer similar to the first layer L1. The second layer L2 can convolve the second feature map FM2 with a weight map to generate a third feature map FM3. In this case, the weight map with which a convolution operation is performed in the second layer L2 can be different from the weight map WM with which a convolution operation is performed in the first layer L1.

[0053] ​After passing through the plurality of layers including the first layer L1 and the second layer L2, an Nth feature map FMn can be generated in the Nth layer. The Nth feature map FMn can be input to a reconstruction layer CL located at a back end of the artificial neural network NN, and output data is output from the reconstruction layer CL. The reconstruction layer CL can generate an output image REC by convolving the Nth feature map FMn with a weight map. The reconstruction layer CL can include a convolution layer, or can be implemented using other types of layers that can reconstruct an image from a feature map according to an example embodiment.

[0054] Referring to Figure 1 and Figure 2 , the training logic 100 can train the artificial neural network model so that the artificial neural network model can have an optimal pruning rate with respect to input data.

[0055] The training logic 100 can calculate first sensitivity data with respect to the entire artificial neural network model, and can calculate second sensitivity data of each of the plurality of layers L1 to Ln.

[0056] Hereinafter, the sensitivity data can include a degree to which data output according to an operation of the artificial neural network is affected when a particular layer is pruned or the entire artificial neural network model is pruned. The sensitivity data can include, for example, a sensitivity value that varies according to a pruning rate. The pruning rate can include a ratio of components corresponding to neurons and / or synapses included in each layer of the plurality of layers L1 to Ln that are removed. For example, the first sensitivity data can indicate a degree of a decrease in accuracy of a result output by the artificial neural network model according to an increase in neurons and / or synapses removed from the entire artificial neural network model. As another example, the second sensitivity data can indicate a degree of a decrease in accuracy of a result output by the artificial neural network model according to an increase in neurons and / or synapses removed from a particular layer.

[0057] The training logic 100 can calculate a target sensitivity based on the first sensitivity data. In this case, the training logic 100 can calculate the target sensitivity by applying a received target pruning rate to the first sensitivity data. For example, the target sensitivity can be a target value of sensitivity that each of the plurality of layers L1 to Ln is to have, and the target sensitivity can have a number of sensitivity data equal to a number of the plurality of layers L1 to Ln. For example, the target sensitivity can include a target sensitivity value of each of the plurality of layers L1 to Ln.

[0058] The training logic 100 can calculate the optimal pruning rate of each of the plurality of layers L1 to Ln based on the second sensitivity data and the target sensitivity. For example, the neural network device 13 can calculate an inverse function of the second sensitivity data. The inverse function can represent a pruning rate that varies according to the sensitivity. For example, the pruning rate can have an inverse function relationship with the second sensitivity data. The training logic 100 can calculate the optimal pruning rate of each of the plurality of layers L1 to Ln by inputting the target sensitivity of each of the plurality of layers L1 to Ln to the inverse function.

[0059] The training logic 100 can generate an artificial neural network that is trained by pruning the artificial neural network model according to the calculated optimal pruning rate. For example, the neural network device 13 can update the artificial neural network model.

[0060] The storage 14 is a storage space for storing data, and can store an operating system (OS), various programs, and various data. According to an embodiment, the storage 14 can include at least one processor that operates the training logic 100. For example, the storage 14 can be referred to as a memory-in-processor (PIM). The storage 14 can store at least one training image for training the artificial neural network model.

[0061] The storage 14 can be a DRAM, but is not limited thereto. The storage 14 can include at least one of a volatile memory and a non-volatile memory. The non-volatile memory can include, for example, a read only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), or a ferroelectric RAM (FRAM). The volatile memory can include, for example, a DRAM, an SRAM, a synchronous DRAM (SDRAM). According to an embodiment, the storage 14 can include at least one of a hard disk drive (HDD), a solid state drive (SSD), a compact flash (CF) card, a secure digital (SD) card, a micro secure digital (Micro-SD) card, a mini secure digital (Mini-SD) card, an extreme digital (xD) card, or a memory stick.

[0062] The input / output device 15 can be operable to output an operation result calculated by the electronic system 10 to the outside, or the input / output device 15 can receive data.

[0063] Figure 3 is a graph for describing a pruning method according to an embodiment of the inventive concept and a comparative example.

[0064] The first artificial neural network model NN1 can be an artificial neural network model before being pruned, and the second artificial neural network model NN2 can be a pruned artificial neural network model. The first artificial neural network model NN1 and the second artificial neural network model NN2 can include a plurality of layers LA, and each layer LA can include at least one neuron NR. The result of an operation output from a first neuron NR can be transmitted to a second neuron NR via a synapse SN. In the artificial neural network model NN2, the number of neurons NR in a layer LA can be less than or equal to the number of neurons NR in the same layer LA of the artificial neural network model NN1. Also, in the artificial neural network model NN2, the number of synapses SN between layers LA can be less than the number of synapses SN between the same layers LA of the artificial neural network model NN1. The number of layers LA in the artificial neural network models NN1 and NN2 can remain the same.

[0065] According to a comparative example, neurons NR and synapses SN can be pruned based on respective weights of the neurons NR and the synapses SN. Based on an assumption that when neurons NR and synapses SN having relatively low weight values are pruned, the overall network performance is hardly deteriorated. However, although the weight values are small, the weight values can greatly affect the accuracy of the entire artificial neural network. Accordingly, according to example embodiments of the inventive concept, the importance of a plurality of layers LA is calculated based on a sensitivity and a pruning rate, and a decrease in accuracy can be minimized. Also, according to example embodiments of the inventive concept, to prevent deterioration of the performance of the entire artificial neural network, an optimal pruning rate can be assigned to the plurality of layers LA based on a sensitivity and a pruning rate of the entire artificial neural network.

[0066] Figure 4 is a block diagram of a computing device 20 according to example embodiments of the inventive concept.

[0067] Referring to Figure 4 , the computing device 20 can include a training logic 100, a model analyzer 210, and a model updater 220. Figure 4 The training logic 100 can be the same as the training logic 100 described with respect to Figure 1 and can constitute a part of the electronic system 10 of Figure 1 . The training logic 100 can include a sensitivity analyzer 110 and a pruning controller 120. Also, the sensitivity analyzer 110 can include an artificial neural network sensitivity analyzer 111 and a layer sensitivity analyzer 112, and the pruning controller 120 can include a target sensitivity calculator 121 and an optimal pruning rate calculator 122.

[0068] The computing device 20 can be a computer (or some interconnected computers) and can include one or more processors (such as CPUs (central processing units), GPUs (graphics processing units), controllers, etc.) configured by, for example, software, and further include the training logic 100, the model analyzer 210, the model updater 220, etc., which form various functional modules of the computer. The computer can be a general-purpose computer, or can be a special-purpose hardware or firmware (e.g., an electronic circuit or an optical circuit such as a special-purpose hardware such as a digital signal processor (DSP) or a field-programmable gate array (FPGA)). A computer can be configured from some interconnected computers. Each functional module (or unit) described herein can include a separate computer, or some or all of the functional modules (or units) can be made up of and share the hardware of the same computer. The connections and interactions between the units described herein can be hardwired and / or in the form of data (such as a part of an application programming interface (API)) stored in and retrieved from the memory of the computer (such as a register, a buffer, a cache, a storage drive, etc.). The functional modules (or units) of the computing device 20 (e.g., the training logic 100, the model analyzer 210, and the model updater 220) can each correspond to a separate one or more segments of software (e.g., subroutines) that configure the computer of the computing device 20, and / or can correspond to one or more segments of software that also correspond to one or more other functional modules (or units) described herein (e.g., the functional modules (or units) can share particular one or more segments of software, or can be implemented by the same one or more segments of software). As understood, “software” denotes prescribed rules (such as code or scripts) for operating a computer.

[0069] The model analyzer 210 can receive the first artificial neural network model NN1, partition the first artificial neural network model NN1 into the plurality of layers L, and output information about the plurality of layers L. In addition, the model analyzer 210 can output information about the entire first artificial neural network model NN1. For example, the model analyzer 210 can replace, merge, and / or partition the data structure of the first artificial neural network model NN1 to output information about the plurality of layers L of the first artificial neural network model NN1. In some example embodiments, the model analyzer 210 can organize the nodes of the first artificial neural network model NN1 into the plurality of layers L, and transmit information defining each of the plurality of layers L to the sensitivity analyzer 110.

[0070] The sensitivity analyzer 110 can output first sensitivity data SD1 based on the first artificial neural network model NN1 and second sensitivity data SD2 based on the plurality of layers L. The sensitivity analyzer 110 can receive input data DIN required in the test. The input data DIN can be at least one training image. The sensitivity analyzer 110 can output the first sensitivity data SD1 and the second sensitivity data SD2 including sensitivity values varying according to pruning rates by applying various pruning rates to the first artificial neural network model NN1 and the plurality of layers L. For example, the artificial neural network sensitivity analyzer 111 can receive the first artificial neural network model NN1 output from the model analyzer 210, can calculate the first sensitivity data SD1 based on the received first artificial neural network model NN1 and the input data DIN, and can output the calculated first sensitivity data SD1. Also, the layer sensitivity analyzer 112 can receive information about the plurality of layers L output from the model analyzer 210, can calculate the second sensitivity data SD2 based on the received information about the plurality of layers L and the input data DIN, and can output the calculated second sensitivity data SD2.

[0071] According to an example embodiment of the inventive concept, to reduce the amount of computation, the sensitivity analyzer 110 can interpolate sensitivity values to output the first sensitivity data SD1 and the second sensitivity data SD2. For example, the sensitivity analyzer 110 can obtain pruning rates in units of, for example, 0.1 from among pruning rates having values from 0 to 1. The sensitivity analyzer 110 can apply pruning rates having predetermined unit intervals, such as 0, 0.1, …, 0.9, 1, to the first artificial neural network model NN1 and the plurality of layers L. The first artificial neural network model NN1 and the plurality of layers L can be pruned to correspond to the applied pruning rates. The first artificial neural network model NN1 and the plurality of layers L can have lower data accuracy due to neurons NR or synapses SN removed as the pruning rate increases. This can indicate that output data sensitively reacts due to the removed neurons NR or synapses SN, and sensitivity can increase as the pruning rate increases. The sensitivity analyzer 110 can obtain sensitivity values of sensitivity increasing as the pruning rate increases, and can output the first sensitivity data SD1 and the second sensitivity data SD2 representing the sensitivity values.

[0072] The pruning controller 120 can determine and output the optimal pruning rate OP based on the first sensitivity data SD1 and the second sensitivity data SD2. The pruning controller 120 can obtain a target sensitivity TS based on the first sensitivity data SD1 and the received target pruning rate TP. The pruning controller 120 can apply the target sensitivity TS to the sensitivity of each of the plurality of layers L included in the second sensitivity data SD2. The pruning controller 120 can output the optimal pruning rate OP, which corresponds to the target sensitivity TS and each of the plurality of layers L can have the optimal pruning rate OP.

[0073] According to an embodiment of the inventive concept, the target sensitivity calculator 121 can receive the target pruning rate TP and the first sensitivity data SD1 output from the artificial neural network sensitivity analyzer 111, and can output the target sensitivity TS. For example, the target sensitivity calculator 121 can calculate the target sensitivity TS corresponding to the target pruning rate TP from the first sensitivity data SD1. The target pruning rate TP can represent, for example, the degree to which the entire first artificial neural network model NN1 can be pruned. For example, the target pruning rate TP can vary according to a request from the electronic system 10. The optimal pruning rate calculator 122 can receive the target sensitivity TS output from the target sensitivity calculator 121 and the second sensitivity data SD2 output from the layer sensitivity analyzer 112. The optimal pruning rate calculator 122 can calculate the optimal pruning rate OP of each of the plurality of layers L according to the target sensitivity TS that the first artificial neural network model NN1 will have. For example, the optimal pruning rate calculator 122 can calculate the optimal pruning rate OP of each of the plurality of layers L based on the target sensitivity TS output from the target sensitivity calculator 121. The optimal pruning rate OP can vary, for example, among the plurality of layers L, or can vary only among at least some of the plurality of layers L. For example, the optimal pruning rate OP can include a plurality of pruning rate values. Meanwhile, the optimal pruning rate calculator 122 can output allocation information, whereby the optimal pruning rate OP is allocated to each of the plurality of layers L.

[0074] According to an example embodiment of the inventive concept, the model updater 220 can receive the first artificial neural network model NN1 and the optimal pruning rate OP output from the optimal pruning rate calculator 122. The model updater 220 can generate and output a second artificial neural network model NN2 updated by applying the optimal pruning rate OP to the first artificial neural network model NN1. For example, the model updater 220 can prune the neurons NR and / or synapses SN of one or more of the plurality of layers L based on the optimal pruning rate OP. In this case, different numbers of neurons NR and synapses SN can be pruned based on different pruning rates of each of the plurality of layers L.

[0075] Figure 5is a graph showing first sensitivity data of an entire artificial neural network model according to an example embodiment of the inventive concept.

[0076] Referring to Figure 5 , the horizontal axis indicates a pruning rate; when the pruning rate is 0, pruning is not performed; when the pruning rate is 1, all neurons NR and synapses SN are removed. The vertical axis indicates sensitivity and indicates a degree to which output data of an artificial neural network model does not accurately reflect according to a pruning rate. For example, the sensitivity can reflect a degree to which output data becomes inaccurate in response to different pruning rates.

[0077] Referring to Figure 5 , the first sensitivity data SD1 can include a change in sensitivity of the first artificial neural network model NN1 included in the training logic 100 according to a change in the pruning rate. The first sensitivity curve SD1_1 indicates an actual change in sensitivity of the first artificial neural network model NN1, the second sensitivity curve SD1_2 indicates an interpolation of the change in sensitivity of the first artificial neural network model NN1, and the third sensitivity curve SD1_3 indicates data obtained by shifting the second sensitivity curve SD1_2 about y=x symmetrically.

[0078] Referring to Figure 4 and Figure 5 , the artificial neural network sensitivity analyzer 111 can obtain output data by performing an operation on input data DIN according to the first artificial neural network model NN1, and can calculate sensitivity by analyzing a change in the output data or analyzing accuracy of the output data as the pruning rate changes. In this case, the artificial neural network sensitivity analyzer 111 can divide the pruning rate from 0 to 1 into a plurality of unit intervals, apply a certain number of pruning rates to the first artificial neural network model NN1, and obtain sensitivity values corresponding to the applied pruning rates. The artificial neural network sensitivity analyzer 111 can interpolate the obtained sensitivity values to obtain the second sensitivity curve SD1_2 and the third sensitivity curve SD1_3 shifted about y=x symmetrically. Next, the target sensitivity calculator 121 can substitute the received target pruning rate TP into the third sensitivity curve SD1_3 to obtain the target sensitivity TS.

[0079] Figure 6 is a graph showing second sensitivity data including a sensitivity curve of each of a plurality of layers L according to an example embodiment of the inventive concept, Figure 7 is a graph showing third sensitivity data based on the second sensitivity data according to an example embodiment of the inventive concept.

[0080] Referring to Figure 6The sensitivity values of the plurality of layers L included in the first artificial neural network model NN1 can vary according to a change in the pruning rate. For example, in a range of the pruning rate from about 0.2 to about 0.8, the sensitivity of the first layer LA1 can abruptly increase (e.g., with a larger slope), and the sensitivity of the second layer LA2 can slowly increase (e.g., with a smaller slope). When the first artificial neural network model NN1 is pruned by using the first pruning rate value P1, the first layer LA1 can have a relatively high first sensitivity value S1, but the second layer LA2 can have a relatively low second sensitivity value S2.

[0081] According to an example embodiment of the inventive concept, retaining the first pruning rate value P1 set to the first layer LA1 and increasing the first pruning rate value P1 set to the second layer LA2 can reduce a decrease in accuracy (or an increase in sensitivity) with respect to an increase in the pruning rate. For example, the pruning rate value of the second layer LA2 can increase to a third pruning rate value P3. When the pruning rate value of the second layer LA2 is the third pruning rate value P3, the second layer LA2 can have a sensitivity value S3 that is greater than the sensitivity value S2 and less than the sensitivity value S1.

[0082] Referring to Figure 6 and Figure 7 , the optimal pruning rate calculator 122 can interpolate the second sensitivity data SD2 and calculate pruning rate data PD obtained by performing an inverse function operation. In this case, the optimal pruning rate calculator 122 can calculate a sensitivity value for each specific portion of the pruning rate and interpolate the calculated sensitivity values.

[0083] The optimal pruning rate calculator 122 can receive a target sensitivity TS and apply the target sensitivity TS to the pruning rate data PD to calculate a first optimal pruning rate OPA of the first layer LA1 and a second optimal pruning rate OPB of the second layer LA2, respectively. For example, the first layer LA1 has a relatively high sensitivity in a specific pruning rate range, and thus the first optimal pruning rate OPA is assigned to the first layer LA1. The second layer LA2 has a relatively low sensitivity in a specific pruning rate range, and thus the second optimal pruning rate OPB, which is higher than the first optimal pruning rate OPA, is assigned to the second layer LA2.

[0084] Figure 8 is a graph showing a sensitivity value of each of pruning rates of a plurality of layers L according to an example embodiment of the inventive concept.

[0085] As described above with reference to Figure 7The optimal pruning rate calculator 122 can calculate a first optimal pruning rate OP A and a second optimal pruning rate OP B. The model updater 220 can prune the plurality of layers L of the first artificial neural network model NN1 based on the first optimal pruning rate OP A and the second optimal pruning rate OP B, and can output a second artificial neural network model NN2 updated after the pruning. The updated second artificial neural network model NN2 can include a first layer LA1 and a second layer LA2. The first layer LA1 can be pruned to have a first sensitivity SA according to the first optimal pruning rate OP A. The second layer LA2 can be pruned to have a second sensitivity SB according to the second optimal pruning rate OP B. The first sensitivity SA and the second sensitivity SB can be similar to the target sensitivity TS described above. As described above with reference to FIG. 1, the model updater 220 can output the updated second artificial neural network model NN2 as a final artificial neural network model. Figure 6 The second layer LA2 having a relatively low sensitivity can be pruned to a greater extent since the pruning rate value of the second layer LA2 having a relatively low sensitivity is set to be higher than the pruning rate value of the first layer LA1. Accordingly, the computing device 20 can efficiently prune the first artificial neural network model NN1.

[0086] Figure 9 is a graph showing accuracy of an artificial neural network model for a pruning rate according to an embodiment of the inventive concept and comparative examples. This embodiment will be described based on the reference numerals of the above-described drawings.

[0087] Referring to FIG. 2, Figure 9 the horizontal axis indicates a pruning rate for pruning an artificial neural network model, and the vertical axis indicates accuracy of data output by the artificial neural network model based on input data.

[0088] Referring to FIG. 2, Figure 9 Although the pruning rate increases, the first artificial neural network model NN1 pruned with the optimal pruning rate OP according to the above-described embodiment can have a relatively small decrease in accuracy AC1. This is because, for example, layers (e.g., the layer LA1 of the first artificial neural network model NN1) that greatly affect the sensitivity of the first artificial neural network model NN1 within a certain range (from about 0.5 to about 1) of the pruning rate are pruned with a relatively low pruning rate (e.g., the pruning rate OP A of the first artificial neural network model NN1). Figure 8 In addition, this is also because layers (e.g., the layer LA2 of the first artificial neural network model NN1) that rarely affect the sensitivity of the first artificial neural network model NN1 within the certain range of the pruning rate are pruned with a relatively high pruning rate (e.g., the pruning rate OP B of the first artificial neural network model NN1). Figure 8 Figure 8 Figure 8

[0089] ​​​On the other hand, the electronic device according to the comparative example can perform pruning in order starting from the lowest weight value included in the layers. When pruning only the L layers, synapses SN, or neurons NR having low weight values, the data output accuracy of the entire artificial neural network model is not considered. Accordingly, the accuracy AC2 for the pruning rate according to the comparative example can be lower than that of the embodiment of the inventive concept that considers the sensitivity for the pruning rate of the entire artificial neural network model and the sensitivity for the pruning rate of each layer.

[0090] Figure 10 is a flowchart of a pruning operation of an artificial neural network model according to an embodiment of the inventive concept.

[0091] According to an embodiment of the inventive concept, the computing device 20 can receive a target pruning rate TP (S110). The target pruning rate TP can be a value set by a user, and can indicate a degree to which the entire artificial neural network model can be pruned. For example, the target pruning rate TP can be input to the computing device 20 via an input / output device (not shown) of the computing device 20.

[0092] The computing device 20 can receive a first artificial neural network model NN1 (S120). The first artificial neural network model NN1 can perform image processing. The first artificial neural network model NN1 can be trained based on at least one first training image. The first artificial neural network model NN1 can be a model that is an object to be pruned, and can include a plurality of layers L.

[0093] The computing device 20 can calculate a first sensitivity of the entire first artificial neural network model NN1 and a second sensitivity of each of the plurality of layers L included in the first artificial neural network model NN1 (S130). Here, the first sensitivity and the second sensitivity can include inaccuracy of a result output by the first artificial neural network model NN1 according to input data. The input data can be at least one second training image. For example, the first sensitivity can include a plurality of sensitivity values of the first artificial neural network model NN1 pruned using a plurality of pruning rate values. For example, the second sensitivity can include sensitivity values with respect to output data of the first artificial neural network model NN1 according to the plurality of layers L, each of which is pruned at a different rate from each other.

[0094] The computing device 20 can determine an optimal pruning rate OP based on the first sensitivity obtained by performing a pruning operation on the entire first artificial neural network model NN1, the second sensitivity obtained by performing a pruning operation on each of the plurality of layers, and the target pruning rate TP (S140). For example, the computing device 20 can calculate the optimal pruning rate OP by considering both the first sensitivity of the entire first artificial neural network model NN1 according to the pruning rate and the second sensitivity of each of the plurality of layers L according to the pruning rate.

[0095] The computing device 20 can prune the artificial neural network model (S150). For example, the computing device 20 can prune the artificial neural network model by pruning the plurality of layers L with different optimal pruning rates OP from each other. The second artificial neural network model NN2 updated by pruning can be more optimized for input data compared to the first artificial neural network model NN1 before being pruned.

[0096] Figure 11 is a flowchart of a method of obtaining first sensitivity data according to an example embodiment of the inventive concept. This embodiment will be described based on the reference numerals of the above-described figures.

[0097] According to an embodiment of the inventive concept, the computing device 20 can receive a first artificial neural network model NN1 (S210).

[0098] The computing device 20 can prune the first artificial neural network model NN1 by using a plurality of pruning rate values (S220). For example, the sensitivity analyzer 110 can divide the pruning rate into a plurality of unit intervals, obtain sensitivity values by analyzing a change in output data of each unit interval, and interpolate the sensitivity values.

[0099] The computing device 20 can output first sensitivity data SD1 according to a pruning rate of the first artificial neural network model NN1 (S230). For example, the sensitivity analyzer 110 can apply the received input data DIN to the first artificial neural network model NN1. The sensitivity analyzer 110 can analyze a change in output data while increasing the pruning rate from 0 to 1 in unit intervals. The greater the change, the lower the accuracy can be, and the higher the sensitivity can be. The sensitivity analyzer 110 can digitize the sensitivity based on the change in output data. For example, when the pruning rate is 0.5, data output by the first artificial neural network model NN1 can differ from output data output with a pruning rate of 0 by a certain value. The sensitivity analyzer 110 can digitize the difference and output the first sensitivity data SD1 including sensitivity values from 0 to 1.

[0100] Figure 12 is a flowchart of a method of obtaining second sensitivity data according to an example embodiment of the inventive concept. This embodiment will be described based on the reference numerals of the above-described figures.

[0101] According to an embodiment of the inventive concept, the computing device 20 can receive a first artificial neural network model NN1 (S310).

[0102] The computing device 20 can divide the first artificial neural network model NN1 into a plurality of layers L (S320). For example, the model analyzer 210 can analyze the plurality of layers L included in the first artificial neural network model NN1 by using various well-known methods. In some embodiments, the model analyzer 210 can transmit each of the analyzed plurality of layers L to the training logic 100. In other embodiments, the training logic 100 can transmit information about the plurality of layers L without dividing the first artificial neural network model NN1.

[0103] The computing device 20 can determine the sensitivity by applying a plurality of pruning rates to the i-th layer (S330). For example, operation S330 can be repeatedly performed on all N layers included in the first artificial neural network model NN1, or in another example, on at least some layers, where N is a natural number greater than or equal to 2. For example, the sensitivity analyzer 110 can divide the pruning rate into a plurality of unit intervals. The sensitivity analyzer 110 can apply the pruning rate divided into a plurality of unit intervals to the plurality of layers, respectively. Next, the sensitivity analyzer 110 can apply the received input data DIN to the first artificial neural network model NN1. The sensitivity analyzer 110 can analyze the change in the output data according to the pruning rate respectively assigned to the plurality of layers while increasing the pruning rate from 0 to 1. The greater the change, the lower the accuracy can be, and the higher the sensitivity can be. The sensitivity analyzer 110 can obtain a sensitivity value based on the change in the output data.

[0104] The computing device 20 can output second sensitivity data SD2 according to the respective pruning rates of the plurality of layers L based on the obtained sensitivity value (S340). For example, when the first layer LA1 is pruned using a plurality of pruning rates, the artificial neural network model in which the first layer LA1 is pruned can output data and include a sensitivity value according to the output data. This process is applied to the second layer LA2.

[0105] According to embodiments of the inventive concept, pruning of a layer having a small weight value but a large influence on the performance of an artificial neural network can be prevented, thereby improving the accuracy of the artificial neural network.

[0106] According to embodiments of the inventive concept, the accuracy or sensitivity of the entire artificial neural network can be analyzed first, and then pruning can be performed on the entire artificial neural network using a post-training method, and thus, the artificial neural network can be trained quickly using a smaller amount of data than the amount of data of a retraining method of the related art.

[0107] Figure 13 is a diagram for describing a memory system according to an example embodiment of the inventive concept. This embodiment will be described based on the reference numerals of the above-described drawings.

[0108] Referring to Figure 13The electronic system can include the CPU 11, the neural network device 13, the memory system 17, and a bus 16 that performs communication in the electronic system.

[0109] According to embodiments of the inventive concept, when the pruning rate PR of each of the plurality of layers L is changed to the optimal pruning rate OP, the number of weights W included in the plurality of layers L can not be consistent. Accordingly, the electronic system 30 can update (or adjust) the weights W. In some embodiments, the training logic 100 can normalize the weights W of each of the plurality of layers L based on all the weights included in the first artificial neural network model NN1. For example, the normalization can be performed by setting the plurality of weights of the entire first artificial neural network model NN1 to 1. In some embodiments, the training logic 100 can adjust the weights W of each of the plurality of layers L and store the adjusted weights W of each of the plurality of layers L in the memory system 17. For example, the model analyzer 210 can output the first weights of each of the plurality of layers, and the training logic 100 can calculate second weights by updating the first weights based on the optimal pruning rate and store the second weights and the optimal pruning rate corresponding to the second weights in the memory. When the neural network device 13 continuously adjusts the weights W according to the change in the pruning rate PR, the weights W of each of the plurality of layers L can be repeatedly read and written via the memory system 17, and the processing speed can be accordingly reduced.

[0110] At least some of the processing of the neural network device 13 can be performed by using the memory system 17. For example, when the neural network device 13 outputs the pruning rate PR, the memory system 17 (e.g., the host processor 171 of the memory system 17) can update the weights W based on the received pruning rate PR and output the updated weights (e.g., updated weights W') to the neural network device 13. As another example, when the neural network device 13 outputs the updated weights (e.g., updated weights W'), the memory system 17 can update the pruning rate PR to correspond to the updated weights. For example, the memory system 17 can update at least one of the weights W, the pruning rate PR, and the sensitivity based on the values received from the neural network device 13. Figure 16 Figure 16 At least some of the processing of the neural network device 13 can be performed by using the memory system 17. For example, when the neural network device 13 outputs the pruning rate PR, the memory system 17 (e.g., the host processor 171 of the memory system 17) can update the weights W based on the received pruning rate PR and output the updated weights (e.g., updated weights W') to the neural network device 13. As another example, when the neural network device 13 outputs the updated weights (e.g., updated weights W'), the memory system 17 can update the pruning rate PR to correspond to the updated weights. For example, the memory system 17 can update at least one of the weights W, the pruning rate PR, and the sensitivity based on the values received from the neural network device 13.

[0111] The memory system 17 can include a host processor 171, a plurality of logic layers 172, and a plurality of memory stacks 173. In some embodiments, each logic layer 172 can implement a memory controller and / or other logic that interacts with the host processor 171 and the plurality of memory stacks 173. In some embodiments, each of the plurality of memory stacks 173 can include a plurality of memory devices (e.g., DRAM memory). Each logic layer 172 can be electrically connected to a respective memory stack 173. ​

[0112] The host processor 171 can receive the weights W included in the plurality of layers L from the neural network device 13 via the bus 16. Also, the host processor 171 can receive the address ADDR for storing a necessary value for an operation of the artificial neural network or training the artificial neural network from the CPU 11. For example, the address ADDR can include information on an address of the memory stack 173 corresponding to a specific weight.

[0113] According to an embodiment, the host processor 171 can calculate the pruning rate and the sensitivity based on the received weights W. The host processor 171 can store the calculated pruning rate PR and the sensitivity in the memory stack 173 corresponding to the address ADDR.

[0114] According to another embodiment, the host processor 171 can receive the weights W included in the plurality of layers L and the address ADDR. The host processor 171 can output the weights W assigned to each address ADDR to the logical layer 172 including the memory stack 173 corresponding to the address ADDR. The logical layer 172 can calculate the pruning rate PR and the sensitivity based on the received weights W. The logical layer 172 can store the calculated pruning rate PR and the sensitivity in the memory stack 173 corresponding to the address ADDR. In another example, the logical layer 172 can store at least one of the received weights W, the calculated pruning rate PR, and the sensitivity in the memory stack 173 corresponding to the address ADDR.

[0115] When a read request is received from the CPU 11, the memory system 17 can output information on the pruning rate PR, the sensitivity, and the weight W stored in the memory stack 173 corresponding to the read request.

[0116] Figure 14 is a block diagram of a logical layer according to an embodiment of the disclosure. This embodiment will be described based on the reference numerals of the above-described drawings.

[0117] Referring to Figure 14 , the logical layer 172 can include a pruning rate calculator 174, a sensitivity calculator 175, and an interpolation calculator 176. The logical layer 172 can perform an operation similar to that of the training logic 100 described above. For example, the pruning rate calculator 174 can perform an operation similar to that of the pruning rate calculator 104 described with reference to Figure 4The described pruning controller 120 performs similar operations. For example, the pruning rate calculator 174 can calculate the optimal pruning rate OP based on the received first sensitivity data SD1 and second sensitivity data SD2. The sensitivity calculator 175 can calculate the first sensitivity data SD1 and second sensitivity data SD2 based on the first artificial neural network model NN1 and the plurality of layers L. Further, the sensitivity calculator 175 can calculate the target sensitivity TS based on the target pruning rate TP. The interpolation calculator 176 can interpolate the plurality of sensitivity values to generate the first sensitivity data SD1 and second sensitivity data SD2. Further, the interpolation calculator 176 can interpolate the plurality of pruning rates to generate the pruning data PD.

[0118] According to embodiments, the host processor 171 can include the pruning rate calculator 174, the sensitivity calculator 175, and the interpolation calculator 176. For example, the host processor 171 can perform the above-described calculation of the pruning rate, the calculation of the sensitivity, and the interpolation operation in a similar manner.

[0119] Figure 15 is a block diagram of a memory system for calculating a pruning rate by receiving original weights according to example embodiments of the inventive concept.

[0120] According to embodiments of the inventive concept, the neural network device 13 can output the weights W. In some embodiments, the original weights W can be output to the logic layer 172. The weights W can be implemented, for example, in a matrix and include matrix values W11, W12, W21, and W22, and the matrix values can correspond to weight values, respectively. The CPU 11 can output the address ADDR. In some embodiments, the address ADDR can be a lookup table (LUT) including a plurality of addresses ADDR. The address ADDR can include, for example, a first address (e.g., Group 0 Addr) corresponding to the first weight value W11 and a second address (e.g., Group 1 Addr) corresponding to the second weight value W12. Further, the address ADDR can include a third address (e.g., Group 2 Addr) and a fourth address (e.g., Group 3 Addr). The weights W output by the neural network device 13 and the address ADDR output by the CPU 11 can be received by the host processor 171 of the memory system 17.

[0121] According to embodiments of the inventive concept, the host processor 171 can assign each memory stack to a plurality of groups. For example, the first memory stack 173_1 can be assigned to a first group, and the second memory stack 173_2 can be assigned to a second group. Further, the host processor 171 can assign at least one address ADDR value to a group. For example, a first address (e.g., Group 0 Addr) can be assigned to the first group, and a second address (e.g., Group 1 Addr) can be assigned to the second group.

[0122] According to an embodiment of the inventive concept, the host processor 171 can calculate pruning rates based on the received weights W. For example, a first pruning rate PR1 can be calculated based on the first weight value W11, and a second pruning rate PR2 can be calculated based on the second weight value W12. The host processor 171 can store the weight values and the pruning rate values corresponding to the weight values in the memory stack 173. For example, the first pruning rate PR1 can be calculated based on the first weight value W11 of the first group, and the first weight value W11 and the first pruning rate PR1 can be stored in the memory stack 173_1 of the first group. As another example, the second pruning rate PR2 can be calculated based on the second weight value W12 of the second group, and the second weight value W12 and the second pruning rate PR2 can be stored in the memory stack 173_2 of the second group.

[0123] Figure 16 is a block diagram of a memory system for calculating pruning rates by receiving corrected weights according to an example embodiment of the inventive concept. This embodiment will be described based on the reference numerals of the above-described figures.

[0124] Referring to Figure 16 , the neural network device 13 can output the corrected weights W'. For example, the neural network device 13 can perform training on the artificial neural network model to correct the original weights W and output the corrected weights W'.

[0125] The host processor 171 can calculate pruning rates based on the corrected weights W'. For example, the host processor 171 can output pruning rates PR1' and PR2' that are different from the pruning rates PR1 and PR2 output based on the original weights W, respectively. The host processor 171 can store the corrected weight values and the pruning rate values corresponding to the corrected weight values in the memory stack. For example, the first pruning rate PR1' can be calculated based on the first weight value W11' of the first group, and the first weight value W11' and the first pruning rate PR1' can be stored in the memory stack 173_1 of the first group. As another example, the second pruning rate PR2' can be calculated based on the second weight value W12' of the second group, and the second weight value W12' and the second pruning rate PR2' can be stored in the memory stack 173_2 of the second group.

[0126] Figure 17 is a block diagram of a memory system including a group ID manager according to an example embodiment of the inventive concept. This embodiment will be described based on the reference numerals of the above-described figures.

[0127] Referring to Figure 17 , the logic layer 172 can further include a group ID manager 177, and each memory stack 173 can be assigned a group ID.

[0128] According to embodiments of the inventive concept, the group ID manager 177 can analyze the address ADDR received from the CPU 11, and can store the pruning rate, the sensitivity, and the weight in the memory stack 173 corresponding to the group ID manager 177. For example, the group ID manager 177 can analyze the address ADDR to obtain an address value corresponding to the weight W received from the neural network device 13. The group ID manager 177 can identify the address value corresponding to a specific group ID received from the host processor 171. For example, the group ID can include a plurality of address values. The group ID manager 177 can identify the address value corresponding to the group ID of the memory stack 173 corresponding to the group ID manager 177, and a specific weight value can be stored in the memory stack 173. Meanwhile, the group ID manager 177 can store the pruning rate and the sensitivity corresponding to the specific weight value in the memory stack 173.

[0129] While the inventive concept has been particularly shown and described with reference to embodiments thereof, it will be understood that various changes in form and details can be made therein without departing from the spirit and scope of the claims.

Claims

1. A computing device for training an artificial neural network model for performing image processing, the computing device comprising: a model analyzer configured to receive a first artificial neural network model and divide the first artificial neural network model into a plurality of layers; training logic configured to: calculate first sensitivity data varying as the first artificial neural network model is pruned, calculate a target sensitivity corresponding to a target pruning rate based on the first sensitivity data, calculate second sensitivity data varying as each of the plurality of layers of the first artificial neural network model is pruned, and output an optimal pruning rate for each of the plurality of layers based on the second sensitivity data, the optimal pruning rate corresponding to the target sensitivity; and a model updater configured to prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model. 2.The computing device of claim 1, the plurality of layers including a first layer having a first sensitivity and a second layer having a second sensitivity, the second sensitivity being lower than the first sensitivity, wherein wherein the optimal pruning rate includes a first optimal pruning rate for the first layer and a second optimal pruning rate for the second layer, and wherein the second optimal pruning rate is higher than the first optimal pruning rate. The training logic is further configured to: apply a plurality of pruning rate values to the first artificial neural network model and the plurality of layers, obtain a plurality of sensitivity values, and interpolate the plurality of sensitivity values to calculate the first sensitivity data and the second sensitivity data.

3. The computing device of claim 1, wherein, The training logic is further configured to: obtain pruning rate data in an inverse function relationship with the second sensitivity data, calculate the optimal pruning rate from the pruning rate data based on the target sensitivity, and output the optimal pruning rate.

4. The computing device of claim 1, wherein, The training logic is further configured to: adjust weights of each of the plurality of layers according to the output of the optimal pruning rate.

5. The computing device of claim 1, wherein, The training logic is further configured to: normalize the weights of each of the plurality of layers based on all weights included in the first artificial neural network model.

6. The computing device of claim 5, wherein, 7.The computing device of claim 5, further comprising: a memory including a processor, wherein the processor is configured to adjust the weights to provide adjusted weights, and store the adjusted weights in the memory. 8.A computing device for training an artificial neural network model for performing image processing, the computing device comprising: a model analyzer configured to receive a first artificial neural network model and divide the first artificial neural network model into a plurality of layers; training logic configured to: receive a target pruning rate, calculate a target sensitivity of the first artificial neural network model, and output an optimal pruning rate for each of the plurality of layers, the target sensitivity corresponding to the target pruning rate, the optimal pruning rate corresponding to the target sensitivity; and a model updater configured to prune the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model, and output the second artificial neural network model. The training logic includes: ​ 9. The computing device of claim 8, wherein, ​ a sensitivity analyzer to analyze the first artificial neural network model to output first sensitivity data varying according to a pruning rate, and to analyze each of the plurality of layers to output second sensitivity data varying according to the pruning rate.

10. The computing device of claim 9, wherein the training logic is further configured to output the optimal pruning rate based on the target pruning rate, the first sensitivity data, and the second sensitivity data, and the optimal pruning rate is assigned to each of the plurality of layers.

11. The computing device of claim 9, wherein, the sensitivity analyzer is further configured to apply a plurality of pruning rates to the first artificial neural network model and each of the plurality of layers, obtain a plurality of sensitivity values, and interpolate the plurality of sensitivity values to calculate the first sensitivity data and the second sensitivity data.

12. The computing device of claim 9, wherein, the training logic further comprises: a pruning controller, wherein the pruning controller comprises: a target sensitivity calculator configured to output a target sensitivity based on the first sensitivity data and the target pruning rate; and an optimal pruning rate calculator configured to output the optimal pruning rate based on the second sensitivity data and the target sensitivity.

13. The computing device of claim 12, wherein, the optimal pruning rate calculator is further configured to obtain pruning rate data in an inverse function relationship with the second sensitivity data, the pruning rate data comprising a pruning rate varying according to the sensitivity data of each of the plurality of layers, and wherein the optimal pruning rate calculator is further configured to apply the target sensitivity to the pruning rate data and output the optimal pruning rate for each of the plurality of layers.

14. The computing device of claim 8, wherein the plurality of layers comprises a first layer and a second layer having a lower sensitivity than the first layer, wherein the optimal pruning rate comprises a first optimal pruning rate for the first layer and a second optimal pruning rate for the second layer, and wherein the second optimal pruning rate is higher than the first optimal pruning rate.

15. The computing device of claim 14, further comprising: a memory, wherein the model analyzer outputs first weights for each of the plurality of layers, and wherein the training logic is further configured to calculate second weights and store the second weights and the optimal pruning rate corresponding to the second weights in the memory, the second weights being obtained by updating the first weights based on the optimal pruning rate.

16. The computing device of claim 8, wherein, the training logic is further configured to adjust the weights of each of the plurality of layers according to the output of the optimal pruning rate.

17. A method of training an artificial neural network model for performing image processing, the method comprising: receiving a target pruning rate; pruning a first artificial neural network model; calculating first sensitivity data varying according to a pruning rate of the first artificial neural network model; calculating a target sensitivity corresponding to the target pruning rate from the first sensitivity data; dividing the first artificial neural network model into a plurality of layers; calculating second sensitivity data varying according to a pruning rate of each of the plurality of layers; calculating an optimal pruning rate for each of the plurality of layers corresponding to the target sensitivity based on the second sensitivity data; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model.

18. The method of claim 17, wherein, the plurality of layers includes a first layer having a first sensitivity and a second layer having a second sensitivity, the second sensitivity being lower than the first sensitivity, wherein the optimal pruning rates include a first optimal pruning rate for the first layer and a second optimal pruning rate for the second layer, wherein the second optimal pruning rate is higher than the first optimal pruning rate.

19. The method of claim 18, wherein, The step of calculating the first sensitivity data varying according to the pruning rate of the first artificial neural network model further comprises: dividing the pruning rate into a plurality of unit intervals; analyzing a change in data output from the first artificial neural network model in each of the plurality of unit intervals; obtaining a sensitivity value; interpolating the sensitivity values; and calculating the first sensitivity data based on the interpolated sensitivity values.

20. The method of claim 18, wherein, The step of calculating the second sensitivity data varying according to the pruning rate of each of the plurality of layers further comprises: inputting data to the first artificial neural network model; applying the pruning rate divided into a plurality of unit intervals to the plurality of layers, respectively; analyzing a change in data output from the first artificial neural network model; obtaining a sensitivity value; interpolating the sensitivity values; and calculating the second sensitivity data based on the interpolated sensitivity values.

21. A memory system for processing an artificial neural network model including a plurality of layers, the artificial neural network model being used to perform image processing, the memory system comprising: a host processor; and a plurality of logical layers configured to communicate with the host processor, wherein each of the plurality of logical layers is electrically connected to a memory stack, and wherein the host processor is configured to receive original weights of each of the plurality of layers, receive memory addresses, output the original weights to a first logical layer, and output a pruning rate corresponding to the original weights output to the first logical layer, and wherein the first logical layer is configured to include, in the first logical layer, first memory stacks corresponding to at least some of the memory addresses, and wherein the first logical layer is configured to calculate a sensitivity and a pruning rate based on the original weights, and store at least one of the original weights, the pruning rate, and the sensitivity in the first memory stacks.

22. A non-transitory computer-readable recording medium having recorded thereon a program, which, when executed by a processor, causes the processor to perform a method for pruning an artificial neural network model including a plurality of layers, the artificial neural network model being used to perform image processing, the method comprising: receiving a target pruning rate; pruning a first artificial neural network model; calculating first sensitivity data varying according to a pruning rate of the first artificial neural network model; calculating a target sensitivity corresponding to the target pruning rate from the first sensitivity data; dividing the first artificial neural network model into a plurality of layers; calculating second sensitivity data varying according to a pruning rate of each of the plurality of layers; based on the second sensitivity data, calculating an optimal pruning rate for each of the plurality of layers corresponding to the target sensitivity; pruning the first artificial neural network model based on the optimal pruning rate to obtain a second artificial neural network model; and outputting the second artificial neural network model.

Images