Artificial neural network and method of controlling fixed point therein

Abstract

An artificial neural network (ANN) system and a method for controlling a fixed-point format in the ANN system are provided. The ANN system includes a processor, a virtual overflow detection circuit, and a data format controller. The processor performs node operations on multiple nodes included in each layer of the ANN to obtain multiple result values ​​of the node operations, and performs quantization operations on the obtained multiple result values ​​based on the kth fixed-point format of the current quantization for each layer to obtain multiple quantized values. The virtual overflow detection circuit generates virtual overflow information indicating the distribution of the effective number of bits of the obtained multiple quantized values. The data format controller determines the (k+1)th fixed-point format for subsequent quantization of each layer based on the generated virtual overflow information. By using the virtual overflow to control the fixed-point format, overflow and / or underflow are effectively prevented.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the priority of Korean Patent Application No. 10-2018-0104403 filed on September 3, 2018 in the Korean Intellectual Property Office (KIPO), the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Apparatuses and methods according to one or more exemplary embodiments relate generally to neural networks, and more particularly to a neural network implementing a dynamic fixed-point strategy and a method of controlling fixed-point in an artificial neural network. Background Art

[0004] Fixed-point arithmetic and floating-point arithmetic are basic operations performed by digital circuits such as processors. Compared with fixed-point arithmetic, floating-point arithmetic provides better precision and a wider range of expression, but it is a longer operation time and higher power consumption. Therefore, relative to floating-point arithmetic, fixed-point arithmetic provides more economical or efficient digital signal processing. Fixed-point arithmetic is particularly important when training artificial neural networks (ANNs) that require a lot of calculations. However, the fixed-point arithmetic scheme has the risk of overflow due to the narrow range of real numbers. When the data size in the operation becomes larger than the register size of the processor, overflow occurs. The fraction length corresponding to the size of the fractional part can be reduced to prevent overflow. However, when the fraction length is unnecessarily reduced, underflow may occur and precision or value resolution may be unnecessarily sacrificed. Summary of the invention

[0005] Aspects of one or more exemplary embodiments provide an artificial neural network (ANN) system capable of efficiently executing a dynamic fixed-point strategy.

[0006] Aspects of one or more exemplary embodiments also provide a method for controlling fixed-point in an ANN system, which can effectively implement a dynamic fixed-point strategy.

[0007] According to an aspect of an exemplary embodiment, an artificial neural network (ANN) system is provided, comprising: a processor configured to perform node operations on a plurality of nodes included in each layer of the ANN to obtain a plurality of result values ​​of the node operations, and to perform quantization operations on the obtained plurality of result values ​​based on a k-th fixed-point format for current quantization of each layer to obtain a plurality of quantized values; a virtual overflow detection circuit configured to generate virtual overflow information indicating a distribution of effective bits of the obtained plurality of quantized values; and a data format controller configured to determine a (k+1)-th fixed-point format for subsequent quantization of each layer based on the generated virtual overflow information.

[0008] According to an aspect of another exemplary embodiment, an artificial neural network (ANN) system is provided, including: a virtual overflow detection circuit configured to: generate a plurality of thermometer codes corresponding to a plurality of quantization values, wherein the plurality of quantization values ​​are obtained by performing a quantization operation based on a fixed-point format; and generate virtual overflow information based on the generated plurality of thermometer codes, the virtual overflow information indicating a distribution of a number of significant bits of the plurality of quantization values; and a data format controller configured to update the fixed-point format based on the generated virtual overflow information.

[0009] According to an aspect of another exemplary embodiment, a method for controlling a fixed-point format in an artificial neural network (ANN) system is provided, the method comprising: performing node operations on a plurality of nodes included in each layer of the ANN to obtain a plurality of result values ​​of the node operations; performing quantization operations on the plurality of result values ​​obtained based on a kth fixed-point format of current quantization used for each layer to obtain a plurality of quantized values; generating virtual overflow information indicating a distribution of a number of effective bits of the plurality of quantized values ​​obtained; and determining a (k+1)th fixed-point format for subsequent quantization of each layer based on the generated virtual overflow information.

[0010] According to an aspect of another exemplary embodiment, a method for controlling a fixed-point format in an artificial neural network (ANN) system is provided, the method comprising: performing a quantization operation based on a k-th fixed-point format to obtain a plurality of quantized values; generating virtual overflow information indicating a distribution of a number of significant bits of the obtained plurality of quantized values; and updating a fixed-point format for subsequent quantization based on the generated virtual overflow information.

[0011] According to an aspect of another exemplary embodiment, there is provided a non-transitory computer-readable recording medium having recorded thereon a computer-executable program for executing any one of the above methods.

[0012] An ANN system and a method of controlling a fixed point in the ANN system according to one or more exemplary embodiments may effectively prevent overflow and / or underflow by controlling a fixed-point format using a virtual overflow, thereby enhancing performance of the ANN system and reducing power consumption of the ANN system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] One or more exemplary embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0014] Figure 1 is a flow chart illustrating a method of controlling a set point in an artificial neural network (ANN) system according to an exemplary embodiment;

[0015] Figure 2 is a block diagram illustrating an ANN system according to an exemplary embodiment;

[0016] Figure 3A and 3B is a diagram for describing an example of a deep learning neural network structure driven by an ANN system according to one or more exemplary embodiments;

[0017] Figure 4 is a block diagram illustrating a processor included in an ANN system according to an exemplary embodiment;

[0018] Figure 5A and 5B is a diagram for describing a quantization operation based on a fixed-point format according to one or more exemplary embodiments;

[0019] Figure 6 is a diagram illustrating an example of a plurality of quantization values ​​corresponding to a plurality of nodes included in one layer according to an exemplary embodiment;

[0020] Figure 7 It is shown Figure 6 A graph of the frequency distribution of the effective number of digits of a plurality of quantized values;

[0021] Figure 8 is a diagram illustrating a virtual overflow detection circuit according to an exemplary embodiment;

[0022] Fig. 9 It is shown Figure 8 FIG. 1 is a diagram of an exemplary embodiment of an accumulator included in a virtual overflow detection circuit of FIG. 1 ;

[0023] Fig.10 and 11 Is used to describe Figure 8 A diagram of the operation of a virtual overflow detection circuit;

[0024] Fig.12 It is shown Figure 8 FIG. 1 is a diagram of an exemplary embodiment of a zero data detection circuit included in a virtual overflow detection circuit of FIG. 1 ;

[0025] Fig.13 is a diagram illustrating a virtual overflow detection circuit according to an exemplary embodiment;

[0026] Fig.14 Is used to describe Fig.13 A diagram of the operation of a virtual overflow detection circuit;

[0027] Fig.15 is a diagram illustrating a virtual overflow detection circuit according to an exemplary embodiment;

[0028] Fig.16 is a flow chart illustrating a method of controlling a fixed-point format according to an exemplary embodiment;

[0029] Fig.17A and 17B is a diagram for describing a method of preventing overflow according to one or more exemplary embodiments;

[0030] Fig.18A and 18B is a diagram for describing a method of preventing underflow according to one or more exemplary embodiments;

[0031] Fig.19 is a diagram illustrating an example distribution of the number of significant digits of quantization values ​​according to an exemplary embodiment; and

[0032] Fig. 20 is a block diagram illustrating an ANN system according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Various example embodiments will be described more fully below with reference to the accompanying drawings, in which some example embodiments are shown. In the accompanying drawings, the same reference numerals always represent the same elements. Repeated descriptions may be omitted.

[0034] Hereinafter, it should be understood that expressions such as "at least one of a, b or c" and "a, b and / or c" mean only a, only b, only c, both a and b, both a and c, both b and c, all of a, b and c or variations thereof.

[0035] Figure 1 is a flowchart illustrating a method of controlling a set point in an artificial neural network (ANN) system according to an exemplary embodiment.

[0036] Reference Figure 1 , node operations are performed on multiple nodes included in each layer of the ANN to generate (e.g., obtain) multiple result values ​​of the node operations (S100). ANN is a machine learning scheme, which is an operation model of software and / or hardware that simulates a biological system (e.g., a human brain) using multiple artificial neurons connected by connecting lines. ANN uses artificial neurons with simplified functions of biological neurons, and the artificial neurons are connected by connecting lines with predetermined connection strengths to perform human recognition or learning. Recently, deep learning is being studied to overcome the limitations of ANN. The following will refer to Figure 3A and 3B A deep learning ANN structure is described to which one or more exemplary embodiments may be applied.

[0037] A quantization operation is performed on or for a plurality of result values ​​based on the kth fixed-point format of the current quantization for each layer of the ANN to generate (eg, obtain) a plurality of quantized values ​​(S200). Figure 5A and 5B Describes quantization based on a fixed-point strategy.

[0038] Generate virtual overflow information indicating the distribution of the effective number of digits of the plurality of quantized values ​​(S300). Here, the virtual overflow information is not information indicating whether an overflow actually occurs, but information indicating the probability of an overflow during a subsequent operation. In addition, the virtual overflow information may also include information indicating the probability of an underflow during a subsequent operation.

[0039] In one or more exemplary embodiments, a plurality of count signals indicating the frequency value of the effective bit number may be generated, and the plurality of count signals may be provided as virtual overflow information. The effective bit includes the first bit having a value of 1 and the lower bits of the first bit starting from the most significant bit of each quantized value. Figures 6 to 15 One or more exemplary embodiments of generating virtual overflow information are described.

[0040] Based on the virtual overflow information, the (k+1)th fixed-point format for subsequent quantization of each layer is determined (S400). ANN learning or ANN training may refer to or include updating the weight values ​​of the current ANN. The weight values ​​can be updated by repeatedly performing node operations according to a feedforward scheme and / or a backpropagation scheme. When performing repeated node operations, the fixed-point format can be updated each time an iteration of the node operation is completed for a layer. When the ANN includes multiple layers, a fixed-point format can be determined for each layer, and an update of the fixed-point format can be performed for each layer. The (k+1)th fixed-point format used for quantization of subsequent node operation results may be different from or the same as the kth fixed-point format used for quantization of the current node operation results. Reference will be made to Figures 16 to 19 One or more exemplary embodiments of updating a fixed-point format are described.

[0041] In this way, by using virtual overflow to control the fixed-point format, overflow and / or underflow can be effectively prevented. Therefore, the performance of the ANN system can be enhanced and the power consumption of the ANN system can be reduced.

[0042] In the related art, the distribution of the quantized value itself is obtained, and the average value and the statistical deviation (such as the variance of the quantized value) are calculated to manage the fixed-point strategy. In contrast, according to one or more exemplary embodiments, virtual overflow information (e.g., the distribution of the number of effective bits) of the quantized value can be obtained instead of the quantized value itself, and the fixed-point strategy can be managed based on the virtual overflow information. According to one or more exemplary embodiments, the hardware resources and the amount of calculation for dynamically managing the fixed point can be significantly reduced, and an effective dynamic fixed-point strategy can be implemented.

[0043] Figure 2 is a block diagram illustrating an ANN 100 system according to an exemplary embodiment.

[0044] Reference Figure 2, the ANN system 100 may include a plurality of processors 110, a task manager TKMNG 120, a data format controller DFCON 130, and a virtual overflow detection circuit VODET 140. The ANN system 100 may also include a special function register SFR 150 and a memory MEM 160.

[0045] The ANN system 100 may be driven by a plurality of processors 110. For example, the plurality of processors 110 may include Figure 2 According to an exemplary embodiment, the plurality of processors 110 may include at least two homogeneous processors. Various services (e.g., tasks TK or applications) such as image classification services, user authentication services, advanced driver assistance system (ADAS) services, and / or voice assistant services may be executed and processed by the plurality of processors 110.

[0046] The task TK may include at least one of a plurality of operations or arithmetic operations. For example, the task TK may represent an application such as an image classification service, a user authentication service based on biometric information, an ADAS service, a voice assistant service, etc. For example, the plurality of operations may include various operations such as a convolution operation, a rectified linear unit (RELU) operation, etc.

[0047] The plurality of processors 110 may include a central processing unit (CPU) 111, a graphics processing unit (GPU) 112, a neural processing unit (NPU) 113, a digital signal processor (DSP) 114, an image signal processor (ISP) 115, and dedicated hardware (DHW) 116. For example, the dedicated hardware 116 may include at least one of a visual processing unit (VPU), a visual intellectual property (VIP), etc. Each processor may be referred to as a processing element (PE).

[0048] Although Figure 2 Only computing resources are shown as an example of multiple processors 110, but multiple processors 110 may also include communication resources, such as a direct memory access unit (DMA) for controlling access to memory 160, connections for supporting various internal and / or external communications, etc.

[0049] The task manager 120 receives a task TK to be executed from an external device or a user, manages or schedules the received task TK, and distributes the task TK to a plurality of processors 110. For example, the task manager 120 may distribute operations included in the task TK to a plurality of processors 110, and generate path information indicating a computation path of the task TK. The computation path of the task TK may include a sequence of operations included in the task TK and a driving sequence of the processors for executing the operations included in the task TK.

[0050] At least one of the plurality of processors 110 may perform node operations on a plurality of nodes included in each layer of the ANN to generate a plurality of result values ​​of the node operations, and perform quantization operations on the plurality of result values ​​based on the k-th fixed-point format of the current quantization for each layer to generate a plurality of quantized values. In other words, at least one of the plurality of processors 110 may perform Figure 1 Processes S100 and S200 in .

[0051] The virtual overflow detection circuit 140 may generate virtual overflow information VOINF indicating the distribution of the number of significant bits of the plurality of quantized values. In other words, the virtual overflow detection circuit 140 may perform Figure 1 As will be referred to below, Figure 6 and 7 As described, the virtual overflow detection circuit 140 may generate a plurality of count signals indicating frequency values ​​of the number of effective bits, and provide the plurality of count signals as virtual overflow information.

[0052] The data format controller 130 may determine the (k+1)th fixed point format FPFMT for subsequent quantization of each layer based on the virtual overflow information VOINF. In other words, the data format controller 130 may perform Figure 1 Process S400 in .

[0053] In one or more exemplary embodiments, the data format controller 130 may provide the determined (k+1)th fixed-point format FPFMT to the special function register 150, and the special function register 150 may update the kth fixed-point format stored therein to the (k+1)th fixed-point format FPFMT. The plurality of processors 110 may execute the task TK using the dynamically updated fixed-point format provided from the special function register 150.

[0054] In one or more other exemplary embodiments, the data format controller 130 may directly provide the multiple processors 110 with a fixed-point format FPFMT, which is updated each time a node operation of a layer is repeated. In other words, when a current node operation is performed on a layer, the data format controller 130 may provide the multiple processors 110 with a k-th fixed-point format, and when a subsequent node operation is performed on a layer, the data format controller 130 may provide the multiple processors 110 with a (k+1)-th fixed-point format. In this case, the multiple processors 110 may execute the task TK based on the dynamically updated fixed-point format provided directly from the data format controller 130.

[0055] The memory 160 may store various data processed by the ANN system 100. The memory 160 may include at least one volatile memory, such as a dynamic random access memory (DRAM), a synchronous DRAM (SDRAM), a static random access memory (SRAM), etc., and / or include at least one non-volatile memory, such as an electrically erasable programmable read-only memory (EEPROM), a flash memory, a phase change random access memory (PRAM), a resistive random access memory (RRAM), a magnetic random access memory (MRAM), a ferroelectric random access memory (FRAM), a nano floating gate memory (NFGM), or a polymer random access memory (PoRAM), etc.

[0056] In one or more exemplary embodiments, at least a portion or all of the task manager 120, the data format controller 130, and / or the virtual overflow detection circuit 140 may be implemented as hardware, may be implemented as software (or program code) stored in a storage device (such as a non-transitory computer-readable medium), or may be implemented as a combination of hardware and software.

[0057] It should be understood that all elements in the ANN system 100 may be connected to each other via at least one bus, and thus all elements in the ANN system 100 may communicate with each other via at least one bus.

[0058] In addition, the ANN system 100 may also include software elements, such as a framework, a kernel or device driver, middleware, an application programming interface (API), an application program or application, etc. At least a portion of the software elements may be referred to as an operating system (OS).

[0059] Figure 3A and 3B is a diagram for describing an example of a deep learning neural network structure driven by an ANN system according to one or more exemplary embodiments.

[0060] refer to Figure 3A , a general neural network may include an input layer IL, a plurality of hidden layers HL1, HL2, ..., HLn and an output layer OL.

[0061] The input layer IL may include i input nodes x1, x2, ..., xi, where i is a natural number. Input data (e.g., vector input data) IDAT of length i may be input to the input nodes x1, x2, ..., xi, so that each element of the input data IDAT is input to a corresponding one of the input nodes x1, x2, ..., xi.

[0062] The plurality of hidden layers HL1, HL2, ..., HLn may include n hidden layers, where n is a natural number, and may include a plurality of hidden nodes h1 1 ,h 1 2 ,h 1 3 , ..., h 1 m ,h 2 1 ,h 2 2 ,h 2 3 , ..., h 2 m ,h n 1 ,h n 2 ,h n 3 , ..., h n m For example, the hidden layer HL1 may include m hidden nodes h 1 1 ,h 1 2 ,h 1 3 , ..., h 1 m , the hidden layer HL2 may include m hidden nodes h 2 1 ,h 2 2 ,h 2 3 , ..., h 2 m , and the hidden layer HLn may include m hidden nodes h n 1 ,h n 2 ,h n 3 , ..., h n m , where m is a natural number.

[0063] The output layer OL may include j output nodes y 1 ,y 2 , ..., y j , where j is a natural number. Output node y 1 ,y 2 , ..., y jEach of may correspond to a corresponding one of the classes to be classified. The output layer OL may output an output value (e.g., a class score or simply, a score) associated with the input data IDAT for each class. The output layer OL may be referred to as a fully connected layer and may indicate, for example, the probability that the input data IDAT corresponds to a sedan.

[0064] Figure 3A The structure of the neural network shown in can be represented by information about branches (or connections) between nodes (shown as lines) and weight values ​​assigned to each branch. Nodes within a layer may not be connected to each other, but nodes in different layers may be fully or partially connected to each other.

[0065] Each node (for example, node h 1 1 ) can receive the previous node (for example, node x 1 ), may perform a computational operation, operation, or calculation on the received output, and may output the result of the computational operation, operation, or calculation as a signal to a subsequent node (e.g., node h 2 1 ) output. Each node can calculate the value to be output by applying a specific function (e.g., a nonlinear function) to the input.

[0066] Generally, the structure of a neural network can be set in advance, and the weight values ​​of the connections between nodes are appropriately set using data, where the answer to which class the data belongs to is known. Data with known answers is called "training data", and the process of determining weight values ​​is called "training". The neural network "learns" during the training process. A set of independently trainable structures and weight values ​​is called a "model", and the process of predicting which class the input data belongs to through a model with determined weight values ​​and then outputting the predicted value is called the "testing" process.

[0067] Figure 3A The general neural network shown in may not be suitable for processing input image data (or input sound data) because each node (e.g., node h 1 1 ) is connected to all nodes in the previous layer (e.g., including node x in layer IL 1 , x 2 , ..., x i ), as the size of the input image data increases, the number of weighted values ​​increases dramatically. Therefore, a convolutional neural network (CNN) implemented by combining a filtering technique with a general neural network has been studied so that a two-dimensional image (e.g., input image data) can be effectively trained by a convolutional neural network.

[0068] refer to Figure 3B, a convolutional neural network may include multiple layers CONV1, RELU1, CONV2, RELU2, POOL1, CONV3, RELU3, CONV4, RELU4, POOL2, CONV5, RELU5, CONV6, RELU6, POOL3, and FC.

[0069] Unlike general neural networks, each layer of a convolutional neural network can have three dimensions: width, height, and depth. Therefore, the data input to each layer can be volume data with three dimensions: width, height, and depth. For example, if Figure 3B The input image in is of size 32 width units (e.g., 32 pixels) and 32 height units (e.g., 32 pixels) and three color channels R, G, and B, and the input data IDAT corresponding to the input image may have a size of 32×32×3. Figure 3B The input data IDAT in can be called input volume data or input activation volume.

[0070] Each of the convolution layers CONV1, CONV2, CONV3, CONV4, CONV5, and CONV6 can perform a convolution operation on the input volume data. In image processing, a convolution operation represents an operation of processing image data based on a mask having a weighted value, and an output value is obtained by multiplying the input value by the weighted value and adding the total product value. The mask can be called a filter, a window, or a kernel.

[0071] In more detail, the parameters of each convolutional layer may be composed of or may include a set of learnable filters. Each filter may be spatially small (along width and height) but may extend over the entire depth of the input volume. For example, during forward propagation, each filter may slide (more precisely, convolve) over the width and height of the input volume, and dot products may be calculated between entries of the filter and the input at any position. As the filter slides over the width and height of the input volume, a two-dimensional activation map may be generated that gives the response of the filter at each spatial position. As a result, an output volume may be generated by stacking these activation maps along the depth dimension. For example, if input volume data having a size of 32×32×3 passes through a convolutional layer CONV1 having four filters with zero padding, the output volume data of the convolutional layer CONV1 may have a size of 32×32×12 (e.g., the depth of the volume data increases).

[0072] Each of the RELU layers RELU1, RELU2, RELU3, RELU4, RELU5, and RELU6 may perform a rectified linear unit operation corresponding to an activation function defined by, for example, a function f(x)=max(0, x) (e.g., for all negative inputs x, the output is zero). For example, if input volume data of size 32×32×12 passes through the RELU layer RELU1 to perform the rectified linear unit operation, the output volume data of the RELU layer RELU1 may have a size of 32×32×12 (e.g., maintaining the size of the volume data).

[0073] Each of the pooling layers POOL1, POOL2, and POOL3 can perform a downsampling operation on the input volume data along the spatial dimensions of width and height. For example, four input values ​​arranged in a 2×2 matrix form can be converted into one output value based on a 2×2 filter. For example, the maximum value of the four input values ​​arranged in a 2×2 matrix form can be selected based on a 2×2 maximum pooling, or the average value of the four input values ​​arranged in a 2×2 matrix form can be obtained based on a 2×2 average pooling. For example, if input volume data having a size of 32×32×12 passes through a pooling layer POOL1 having a 2×2 filter, the output volume data of the pooling layer POOL1 can have a size of 16×16×12 (e.g., the width and height of the volume data are reduced, and the depth of the volume data is maintained).

[0074] Typically, a convolutional layer (e.g., CONV1) and a RELU layer (e.g., RELU1) can form a CONV / RELU layer pair in a convolutional neural network, the CONV / RELU layer pair can be repeatedly arranged in the convolutional neural network, and a pooling layer can be periodically inserted in the convolutional neural network to reduce the image spatial size and extract image features.

[0075] The output layer or fully connected layer FC may output a result (e.g., a class score) for the input volume data IDAT for each class. For example, when the convolution operation and the downsampling operation are repeated, the input volume data IDAT corresponding to the two-dimensional image may be converted into a one-dimensional matrix or vector. For example, the fully connected layer FC may represent the probability that the input volume data IDAT corresponds to a car, a truck, an airplane, a boat, and a horse.

[0076] It should be understood that the types and number of layers included in the convolutional neural network may not be limited to those described above. Figure 3B The examples described above may be changed or varied according to one or more other exemplary embodiments. In addition, it should be understood that the convolutional neural network may also include other layers, such as a softmax layer and a bias adding layer, wherein the softmax layer is used to convert the score value corresponding to the prediction result into a probability value, and the bias adding layer is used to add at least one bias.

[0077] In this way, the deep learning ANN may include multiple layers, and the fixed-point format may be determined independently for each of the multiple layers. The nodes in each layer may perform quantization based on the same fixed-point format. When the ANN includes multiple layers, the virtual overflow detection circuit 140 may generate multiple virtual overflow information corresponding to the multiple layers, respectively, and the data format controller 130 may determine the (k+1)th fixed-point format for subsequent quantization of each layer in the multiple layers based on the multiple virtual overflow information.

[0078] Figure 4 is a block diagram illustrating a processor 200 included in an ANN system according to an exemplary embodiment.

[0079] refer to Figure 4 , the processor 200 may include a control unit CTRL 210 (e.g., a controller), a logic unit LOG 220 (e.g., logic), an embedded memory eMEM 230, an input register REGI 240, a weight register REGW 250, and a result register REGR 260. It should be understood that the processor may include additional elements (e.g., general-purpose elements such as signal buses).

[0080] The control unit 210 controls the overall operation of the processor 200. The control unit 210 can control the flow of instructions and data in the implementation of the ANN. The embedded memory 230 can store instructions and data of the processor 200. The input data and weight values ​​of the node operation can be stored in the input register 240 and the weight register 250, respectively. The logic unit 220 can perform the node operation based on the input data and weight values ​​stored in the registers 240 and 250, and store the result value of the node operation in the result register 260. The input register 240 and the weight register 250 can be implemented with a memory with a higher operating speed than the embedded memory 230, so that the input data and weight values ​​can be loaded from the embedded memory 230 to the registers 240 and 250.

[0081] The control unit 210 may be based on Figure 2 The fixed-point format FPFMT provided by the special function register 150 or the data format controller 130 performs a quantization operation on the result value stored in the result register 260. The quantized value or the quantized value may be stored in the embedded memory 230. As described above, the fixed-point format may be dynamically updated.

[0082] Figure 5A and 5B is a diagram for describing a quantization operation based on a fixed-point format according to one or more exemplary embodiments.

[0083] refer to Figure 4 and5A , the result value RVL of the node operation can be stored in the result register 260, and the control unit 210 can perform a quantization operation on the result value RVL based on the fixed-point format FPFMT to generate a quantization value QVL.

[0084] For example, the fixed-point format FPFMT can be represented by Qm.n representation. In the Qm.n representation, m represents the number of bits of the integer part, and n represents the number of bits of the fractional part. In other words, Qm.n means that m bits are allocated before the decimal point position RPP and n bits are allocated after the decimal point position RPP. The bit width BW corresponding to the total number of bits allocated to the quantized value QVL is m+n, and the fraction length corresponding to the number of bits allocated to the fractional part is n.

[0085] Figure 5B An example of a quantization value QVL having a bit width of N is shown, that is, comprising N bits B1 to BN. Figure 5B In the example, the number of digits in the integer part is 5, and the fixed-point format can be represented as Q 5.N-5 . The three bits BN, BN-1 and BN-2 with 0 starting from the most significant bit BN correspond to invalid bits. The first bit BN-3 with 1 starting from the most significant bit BN and the lower bits B1 to BN-4 correspond to valid bits. Therefore, in Figure 5B In the example quantization value QVL in , the number of invalid bits is 3 and the number of valid bits is N-3.

[0086] According to one or more exemplary embodiments, the virtual overflow detection circuit 140 may detect frequency values ​​of effective digits of a plurality of quantization values ​​corresponding to a plurality of nodes in each layer to provide the frequency values ​​as virtual overflow information VOINF.

[0087] Figure 6 is a diagram showing an example of a plurality of quantization values ​​corresponding to a plurality of nodes included in one layer, Figure 7 It is shown Figure 6 A graph showing the frequency distribution of the number of significant digits for multiple quantized values.

[0088] In the following, for ease of illustration and description, it is assumed that one layer includes ten nodes, but it should be understood that this is only an example and other exemplary embodiments are not limited thereto. Each layer may include various numbers of nodes. As an example, each of the first to tenth quantization values ​​QVL1 to QVL10 includes first to Nth bits Q1 to QN.

[0089] refer to Figure 6, the first quantization value QVL1 includes two invalid bits QN and QN-1, and the effective bit number VBN of the first quantization value QVL1 is N-2, the second quantization value QVL2 includes six invalid bits QN, QN-1, QN-2, QN-3, QN-4 and QN-5 and the effective bit number VBN of the second quantization value QVL2 is N-6, and the third quantization value QVL3 includes five invalid bits QN, QN-1, QN-2, QN-3 and QN-4 and the effective bit number VBN of the third quantization value QVL3 is N-5. In this way, the effective bit number VBN can be determined for each of the first to tenth quantization values ​​QVL1~OVL10, respectively.

[0090] For ease of illustration and description, Figure 6 The case where the effective number of bits is distributed between N and N-6 is shown, but it should be understood that this is only an example and other exemplary embodiments are not limited thereto. Depending on the result of the node operation, the effective number of bits may be distributed between 0 and N.

[0091] Figure 6 The distribution of the quantized values ​​QVL1 to QVL10 can be expressed as follows Figure 7 The frequency distribution diagram shown in FIG. The frequency distribution diagram shows the mapping relationship between the effective number of bits VBN and the corresponding frequency value f(VBN). Figure 7 As shown in , f(N)=f(N-3)=f(N-4)=f(N-5)=f(N-6)=1, f(N-1)=2, f(N-2)=3, and f(N-7) to f(0)=0. Therefore, the total number NT of frequency values ​​becomes 10.

[0092] In this way, the frequency values ​​f(0) to f(N) can be determined by accumulating the effective number of bits 0 to N of the multiple quantization values ​​corresponding to the multiple nodes included in one layer. As described below, hardware such as a thermometer code generation circuit, an accumulation circuit, etc. can be used to generate multiple count signals SC(0) to SC(N), and multiple count signals SC(0) to SC(N) can be provided as the above-mentioned virtual overflow information VOINF.

[0093] Figure 8 is a diagram illustrating a virtual overflow detection circuit 300 according to an exemplary embodiment.

[0094] refer to Figure 8 The virtual overflow detection circuit 300 may include a thermometer code generation circuit 310 , an accumulation circuit 320 , and a subtraction circuit 330 .

[0095] The thermometer code generation circuit 310 may replace all significant bits of each quantization value QVLi to generate each thermometer code TCDi. The thermometer code generation circuit 310 may sequentially receive a plurality of quantization values ​​corresponding to a plurality of nodes included in each layer, and sequentially generate a plurality of thermometer codes corresponding to the plurality of quantization values.

[0096] In one or more exemplary embodiments, for example Figure 8 As shown, the thermometer code generation circuit 310 may include a plurality of OR logic gates 311 to 315. The plurality of OR logic gates 311 to 315 may perform an OR logic operation on each two adjacent bits of each quantization value QVLi to generate each thermometer code TCDi. For example, the OR logic gate 311 performs an OR logic operation on the Nth bit QN and the (N-1)th bit QN-1 of each quantization value QVLi to generate the (N-1)th bit TN-1 of each thermometer code TCDi, and the OR logic gate 312 performs an OR logic operation on the (N-1)th bit QN-1 and the (N-2)th bit QN-2 of each quantization value QVLi to generate the (N-2)th bit TN-2 of each thermometer code TCDi. The Nth bit QN corresponding to the highest bit of each quantization value QVLi may be provided as the Nth bit TN corresponding to the highest bit of each thermometer code TCDi. In this way, the plurality of OR logic gates 311 ˜ 315 may generate the first to Nth bits T1 ˜TN of the thermometer code TCDi based on the first to Nth bits Q1 ˜QN of the quantization value QVLi.

[0097] The accumulation circuit 320 may accumulate the plurality of thermometer codes bit by bit to generate a plurality of accumulated values ​​ACC(1) to ACC(N). The accumulation circuit 320 may include a plurality of accumulators AU 321 to 326 that receive the bits T1 to TN of the thermometer code TCDi, respectively. After completing the accumulation for the plurality of thermometer codes corresponding to the plurality of nodes included in each layer, the plurality of accumulators 321 to 326 may output the stored values ​​as the plurality of accumulated values ​​ACC(1) to ACC(N). Fig. 9 An example of describing the accumulator AU.

[0098] The subtraction circuit 330 may subtract two adjacent accumulated values ​​from the plurality of accumulated values ​​ACC(1)-ACC(N) to generate a plurality of counting signals SC(1)-SC(N). The plurality of counting signals SC(1)-SC(N) may be provided as virtual overflow information VOINF to the Figure 2 The data format controller 130 in.

[0099] The subtraction circuit 330 may include a plurality of subtractors 331 to 335. For example, the subtractor 331 may subtract the Nth accumulated value ACC(N) from the (N-1)th accumulated value ACC(N-1) to generate the (N-1)th count signal SC(N-1), and the subtractor 332 may subtract the (N-1)th accumulated value ACC(N-1) from the (N-2)th accumulated value ACC(N-2) to generate the (N-2)th count signal SC(N-2). The Nth accumulated value ACC(N) may be provided as the Nth count signal SC(N). In this way, the plurality of subtractors 331 to 335 may generate the first to Nth count signals SC(1) to SC(N) based on the first to Nth accumulated values ​​ACC(1) to ACC(N).

[0100] Fig. 9 It is shown Figure 8 FIG. 3 is a diagram of an exemplary embodiment of an accumulator AU included in a virtual overflow detection circuit 300 .

[0101] refer to Fig. 9 , including Figure 8 Each accumulator AU in the accumulation circuit 320 in may include an adder and a buffer. The accumulator AU may accumulate each bit Tj of the thermometer code to provide each accumulated value ACC(j). The buffer may be initialized in response to a reset signal RST, and the stored value may be output as the accumulated value ACC(j) in response to an output control signal OUTEN, wherein the output control signal OUTEN is activated after the accumulation is completed for a plurality of thermometer codes corresponding to a plurality of nodes included in each layer.

[0102] Fig.10 and 11 is used to describe according to one or more exemplary embodiments Figure 8 FIG. 3 is a diagram of the operation of the virtual overflow detection circuit 300 .

[0103] Fig.10 Shows the corresponding Figure 6 Multiple thermometer codes TCD1~TCD10 of multiple quantization values ​​QVL1~OVL10.

[0104] refer to Figure 8 and 10 The thermometer code generation circuit 310 may sequentially receive a plurality of quantization values ​​QVL1 to QVL10, and sequentially generate a plurality of thermometer codes TCD1 to TCD10 corresponding to the plurality of quantization values ​​QVL1 to QVL10, respectively. Fig.10 As shown, the invalid bits of the thermometer code have a value of 0 and the valid bits have a value of 1.

[0105] Fig.111 shows a plurality of accumulated values ​​ACC(1) to ACC(10), which are bitwise accumulated values ​​of a plurality of thermometer codes TCD1 to TCD10. As described above, the subtraction circuit 330 can subtract two adjacent accumulated values ​​from the plurality of accumulated values ​​ACC(1) to ACC(N) to generate a plurality of counting signals SC(1) to SC(N). As a result, Figure 8 The plurality of counting signals SC(1) to SC(N) generated by the virtual overflow detection circuit 300 can be Figure 7 The frequency values ​​f(1) to f(N) shown in FIG.

[0106] Fig.12 It is shown Figure 8 FIG. 1 is a diagram of an exemplary embodiment of a zero data detection circuit included in a virtual overflow detection circuit 300 .

[0107] refer to Fig.12 The zero data detection circuit can be implemented by a subtractor. The subtractor can subtract the first accumulated value ACC(1) from the total number NT of the plurality of nodes included in each layer to generate a zero data count signal SC(0). The total number NT of the plurality of nodes is equal to the total number of quantized values ​​and the sum NT of the frequency values ​​of the effective digits (as shown in reference Figure 7 described above).

[0108] Zero data indicates quantization in which all bits have a value of 0. The thermometer code corresponding to the zero data has a value of 0 for all bits, and thus the zero data does not contribute to the accumulated values ​​ACC(1) to ACC(N) of the accumulation circuit 320. The first accumulated value ACC(1) indicates the number of quantization values ​​in which at least one bit is 1. As a result, a zero data count signal SC(0) may be generated by subtracting the first accumulated value ACC(1) from the total number NT. The zero data count signal SC(0) may be included in the virtual overflow information VOINF and provided to the data format controller 130.

[0109] As reference Figures 8 to 12 As shown, the virtual overflow detection circuit 300 can generate a plurality of thermometer codes corresponding to a plurality of quantization values, and generate virtual overflow information VOINF based on the plurality of thermometer codes. The virtual overflow detection circuit 300 can generate a plurality of count signals SC(0)-SC(N) indicating the frequency value of the effective number of bits for each layer, and provide the plurality of count signals SC(0)-SC(N) as the virtual overflow information VOINF.

[0110] In this way, the ANN system and the method for controlling fixed point in the ANN system according to one or more exemplary embodiments can effectively prevent overflow and / or underflow by controlling the fixed point format using virtual overflow. As a result, the performance of the ANN system is enhanced and the power consumption of the ANN system is reduced, thereby improving the function of the ANN system.

[0111] Fig.13 is a diagram illustrating a virtual overflow detection circuit 400 according to an exemplary embodiment.

[0112] refer to Fig.13 The virtual overflow detection circuit 400 may include a thermometer code generation circuit 410 , a counting code generation circuit 420 , and an accumulation circuit 430 .

[0113] The thermometer code generation circuit 410 may replace all significant bits of each quantization value QVLi to generate each thermometer code TCDi. The thermometer code generation circuit 410 may sequentially receive a plurality of quantization values ​​corresponding to a plurality of nodes included in each layer, and sequentially generate a plurality of thermometer codes corresponding to the plurality of quantization values.

[0114] In one or more exemplary embodiments, for example Fig.13 As shown, the thermometer code generation circuit 410 may include a plurality of OR (or) logic gates 411 to 415. The plurality of OR logic gates 411 to 415 may perform an OR logic operation on each two adjacent bits of each quantization value QVLi to generate each thermometer code TCDi. For example, the OR logic gate 411 performs an OR logic operation on the Nth bit QN and the (N-1)th bit QN-1 to generate the (N-1)th bit TN-1 of the thermometer code TCDi, and the OR logic gate 412 performs an OR logic operation on the (N-1)th bit QN-1 and the (N-2)th bit QN-2 to generate the (N-2)th bit TN-2 of the thermometer code TCDi. The Nth bit QN corresponding to the most significant bit of the quantization value QVLi may be provided as the Nth bit TN corresponding to the most significant bit of the thermometer code TCDi. In this way, the plurality of OR logic gates 411 to 415 may generate the first to Nth bits T1 to TN of the thermometer code TCDi based on the first to Nth bits Q1 to QN of the quantization value QVLi.

[0115] The counting code generation circuit 420 may keep the most significant bit as the significant bit 1 with respect to each thermometer code TCDi and replace the other significant bits from 1 to 0 to generate each counting code CCDi. The counting code generation circuit 420 may sequentially receive a plurality of thermometer codes corresponding to a plurality of nodes included in each layer, and sequentially generate a plurality of counting codes corresponding to the plurality of thermometer codes.

[0116] In one or more exemplary embodiments, reference Fig.13 , the counting code generating circuit 420 may include a plurality of exclusive OR (XOR) logic gates 421 ˜ 425 .

[0117] The plurality of XOR logic gates 421 to 425 may perform an XOR logic operation on each two adjacent bits of each thermometer code TCDi to generate each counting code CCDi corresponding to each thermometer code TCDi. For example, the XOR logic gate 421 performs an XOR logic operation on the Nth bit TN and the (N-1)th bit TN-1 of each thermometer code TCDi to generate the (N-1)th bit CN-1 of the counting code CCDi, and the XOR logic gate 422 performs an XOR logic operation on the (N-1)th bit TN-1 and the (N-2)th bit TN-2 of each counting code TCDi to generate the (N-2)th bit CN-2 of each counting code CCDi. The Nth bit TN corresponding to the highest bit of each thermometer code TCDi may be provided as the Nth bit CN corresponding to the highest bit of each counting code CCDi. In this way, the plurality of XOR logic gates 421 to 425 may generate the first to Nth bits C1 to CN of the counting code CCDi based on the first to Nth bits T1 to TN of the thermometer code TCDi.

[0118] The accumulation circuit 430 may accumulate the plurality of counting codes bit by bit to generate a plurality of counting signals SC(1) to SC(N). The accumulation circuit 430 may include a plurality of accumulators AU 431 to 436 that receive bits C1 to CN of the counting code CCDi, respectively. After completing the accumulation for the plurality of counting codes corresponding to the plurality of nodes included in each layer, the plurality of accumulators 431 to 436 may output the stored values ​​as the plurality of counting signals SC(1) to SC(N). Examples and References of the Accumulator AU Fig. 9 Same (or similar) as described.

[0119] Fig.14 is used to describe the Fig.13 FIG. 4 is a diagram of the operation of the virtual overflow detection circuit 400 .

[0120] Fig.14 Shows the corresponding Figure 6 Multiple quantization values ​​QVL1 to QVL10 and Fig.10 Multiple thermometer codes TCD1~TCD10 and multiple counting codes CCD1~CCD10.

[0121] refer to Fig.13 and 14 , the counting code generating circuit 420 can sequentially receive a plurality of thermometer codes TCD1-TCD10, and sequentially generate a plurality of counting codes CCD1-CCD10 corresponding to the plurality of thermometer codes TCD1-TCD10, respectively. Fig.14 As shown, the insignificant bits of the counting code have a value of 0, the most significant bit has a value of 1, and the other significant bits have a value of 0.

[0122] exist Fig.14The bottom of the figure shows a plurality of counting signals SC(1) to SC(N), which can be generated by bitwise accumulation of a plurality of counting codes CCD1 to CCD10. Fig.13 The plurality of counting signals SC(1) to SC(N) generated by the virtual overflow detection circuit 400 can be Figure 7 The frequency values ​​f(1) to f(N) shown in are the same.

[0123] As reference Fig.13 and 14 As described, the virtual overflow detection circuit 400 can generate a plurality of thermometer codes corresponding to a plurality of quantization values, and generate virtual overflow information VOINF based on the plurality of thermometer codes. The virtual overflow detection circuit 400 can generate a plurality of count signals SC(0)-SC(N) indicating the frequency value of the effective number of bits for each layer, and provide the plurality of count signals SC(0)-SC(N) as the virtual overflow information VOINF.

[0124] In this way, the ANN system and the method for controlling fixed point in the ANN system according to the exemplary embodiment can effectively prevent overflow and / or underflow by controlling the fixed point format using virtual overflow. Therefore, the performance of the ANN system is enhanced and the power consumption of the ANN system is reduced, thereby improving the function of the ANN system.

[0125] Fig.15 is a diagram illustrating a virtual overflow detection circuit 500 according to an exemplary embodiment.

[0126] refer to Fig.15 , the virtual overflow detection circuit 500 may include a multiplexer MUX 510 , a first detection circuit DET1 520 , and a second detection circuit DET2 530 .

[0127] The multiplexer 510 may sequentially receive a plurality of quantization values ​​included in each layer, and divide the plurality of quantization values ​​into positive quantization values ​​PQVL and negative quantization values ​​NQVL based on a sign bit QS included in each quantization value QVLi.

[0128] The first detection circuit 520 may generate positive count signals PSC(1)-PSC(N) indicating the frequency value of the effective number of bits of the positive quantization value PQVL. The second detection circuit 530 may generate negative count signals NSC(1)-NSC(N) indicating the frequency value of the effective number of bits of the negative quantization value NQVL. In one or more exemplary embodiments, each of the first detection circuit 520 and the second detection circuit 530 may be implemented as shown in FIG. Figures 8 to 14 The virtual overflow detection circuit.

[0129] result, Fig.15The virtual overflow detection circuit 500 can generate a plurality of positive count signals PSC(1) to PSC(N) and a plurality of negative count signals NSC(1) to NSC(N), wherein the plurality of positive count signals indicate the frequency values ​​of the effective number of bits of the plurality of positive quantization values ​​PQVL, and the plurality of negative count signals indicate the frequency values ​​of the effective number of bits of the plurality of negative quantization values ​​NQVL. The plurality of positive count signals PSC(1) to PSC(N) and the plurality of negative count signals NSC(1) to NSC(N) can be provided as the virtual overflow information VOINF. As shown in FIG. Fig.12 The described zero data count signal SC(0) may also be included in the virtual overflow information VOINF.

[0130] According to one or more exemplary embodiments, it can be determined whether the plurality of quantized values ​​are in an overflow close state or in an underflow close state based on the virtual overflow information VOINF. In one or more exemplary embodiments, as will be referred to in Fig.16 As described, the overflow near state and the underflow near state may be determined based on the frequency value f(N) of the maximum number of significant digits N of a plurality of quantization values ​​corresponding to a plurality of nodes included in each layer.

[0131] Fig.16 is a flowchart illustrating a method of controlling a fixed-point format according to an exemplary embodiment.

[0132] refer to Fig.16 , obtain a frequency value f(N) of the maximum number of significant digits N of a plurality of quantized values ​​corresponding to a plurality of nodes included in each layer (S510). The maximum number of significant digits N corresponds to the bit width or the number of digits of the plurality of quantized values.

[0133] The overflow approach state and the underflow approach state may be determined based on the frequency value f(N) of the maximum significant digit N. In one or more exemplary embodiments, a ratio f(N) / NT of the frequency value f(N) of the maximum significant digit N relative to the sum NT of the frequency values ​​f(0) to f(N) of the significant digits 0 to N may be compared with an overflow threshold ratio RH or an underflow threshold ratio RL to determine whether the plurality of quantized values ​​are in an overflow approach state or in an underflow approach state.

[0134] When the ratio f(N) / NT is higher than the overflow threshold ratio RH (S520: Yes), the fraction length FL of the fixed-point format used for quantization is reduced (S540). Fig.17A and 17B Describes the reduction of the fraction length FL.

[0135] When the ratio f(N) / NT is lower than the underflow threshold ratio RL (S530: Yes), the fraction length FL of the fixed-point format used for quantization is increased (S550). Fig.18A and 18BDescribes the increase in fraction length FL.

[0136] When the ratio f(N) / NT is not higher than the overflow threshold ratio RH (S520: NO) and is not lower than the underflow threshold ratio RL (S530: NO), the fraction length FL of the fixed-point format used for quantization is maintained (S560).

[0137] Fig.17A and 17B is a diagram for describing a method of preventing overflow according to one or more exemplary embodiments.

[0138] refer to Figure 2 and 17A , when (or based on) the ratio f(N) / NT of the frequency value f(N) of the maximum significant digit N of the significant digits 0 to N of the multiple quantized values ​​of the multiple nodes in each layer relative to the sum NT of the frequency values ​​f(0) to f(N) of the significant digits 0 to N is higher than the overflow threshold ratio RH, the data format controller 130 may determine that the multiple quantized values ​​are in an overflow proximity state. Here, the overflow proximity state indicates that the probability of an overflow occurring during a subsequent node operation of the associated layer is very high.

[0139] Fig. 17B A quantization value QVL(K) based on the kth fixed point format Qm.n for current quantization of a layer and a corresponding quantization value QVL(K+1) based on the (k+1)th fixed point format Qm+1.n-1 for subsequent quantization of the layer are shown.

[0140] refer to Fig. 17B , the kth fixed-point format Qm.n indicates the integer part IPRT of m bits before the decimal point position RPP(K) and the fractional part FPRT of n bits after the decimal point position RPP(K). The (k+1)th fixed-point format Qm+1.n-1 indicates the integer part IPRT of m+1 bits before the decimal point position RPP(K+1) and the fractional part FPRT of n-1 bits after the decimal point position RPP(K+1). In other words, the fractional length FL(K) of the kth fixed-point format Qm.n is n, and the fractional length FL(K+1) of the (k+1)th fixed-point format Qm+1.n-1 is n-1.

[0141] Thus, when the data format controller determines that a plurality of quantized values ​​are in an overflow-prone state (as described above with reference to Fig.17A As described above, the data format controller 130 may reduce the fractional length FL(K+1) of the (k+1)th fixed-point format Qm+1.n-1 to be smaller than the fractional length FL(K) of the kth fixed-point format Qm.n. The accuracy (or value resolution) may be reduced, but overflow can be prevented by reducing the fractional length of subsequent node operations.

[0142] Fig.18A and 18B is a diagram for describing a method of preventing underflow according to one or more exemplary embodiments.

[0143] refer to Figure 2 and 18A , when the ratio f(N) / NT of the frequency value f(N) of the maximum significant digit N of the significant digits 0 to N of the multiple quantized values ​​of the multiple nodes in each layer relative to the sum NT of the frequency values ​​f(0) to f(N) of the significant digits 0 to N is lower than the underflow threshold ratio RL, the data format controller 130 may determine that the multiple quantized values ​​are in an underflow approaching state. Here, the underflow approaching state indicates that the probability of underflow occurring during the subsequent node operation of the associated layer is very high.

[0144] Fig.18B A quantization value QVL(K) based on the kth fixed point format Qm.n for current quantization of a layer and a corresponding quantization value QVL(K+1) based on the (k+1)th fixed point format Qm-1.n+1 for subsequent quantization of the layer are shown.

[0145] refer to Fig.18B The kth fixed-point format Qm.n represents the integer part IPRT of m bits before the decimal point position RPP(K) and the fractional part FPRT of n bits after the decimal point position RPP(K). The (k+1)th fixed-point format Qm-1.n+1 represents the integer part IPRT of m-1 bits before the decimal point position RPP(K+1) and the fractional part FPRT of n+1 bits after the decimal point position RPP(K+1). In other words, the fractional length FL(K) of the kth fixed-point format Qm.n is n, and the fractional length FL(K+1) of the (k+1)th fixed-point format Qm-1.n+1 is n+1.

[0146] Thus, when the data format controller determines that a plurality of quantized values ​​are in a state close to underflow (such as reference Fig.18A As described above, the data format controller 130 may increase the fractional length FL(K+1) of the (k+1)th fixed-point format Qm-1.n+1 to be longer than the fractional length FL(K) of the kth fixed-point format Qm.n. The precision (or value resolution) may be increased by increasing the fractional length of subsequent node operations and underflow may be prevented.

[0147] Fig.19 is a diagram illustrating an example distribution of the number of significant bits of quantization values ​​according to an exemplary embodiment.

[0148] refer to Fig.19 , the frequency value f(VBN) of the effective number of quantized values ​​VBN can be approximated as a normal distribution. In this case, Figure 2 The data format controller 130 in the example can calculate the mean value M and variance σ of the effective number of bits VBN. 2 , and based on the mean M and variance σ 2 Determines the fractional length of the (k+1)th fixed-point format used for subsequent quantization.

[0149] Fig.19 An example in which the frequency value f(VBN) is close to a normal distribution is shown, but it should be understood that this is only an example, and one or more other exemplary embodiments are not limited thereto. For example, according to another exemplary embodiment, in the case of a convolutional neural network (CNN), the frequency value f(VBN) of the effective number VBN may be approximated to a single-ended gamma distribution. In this case, the step length may be calculated based on the mean and variance of the gamma distribution, and the fraction length may be determined based on the step length.

[0150] Fig. 20 is a block diagram illustrating an ANN system according to an exemplary embodiment.

[0151] refer to Fig. 20 , the ANN system may include a first electronic device 1101 and a second electronic device 1201. In this exemplary embodiment, the deep learning system may be driven by the first electronic device 1101 and the second electronic device 1201.

[0152] The first electronic device 1101 may include a plurality of heterogeneous processors 1110, a task manager TKMNG 1120, a data format controller DFCON 1130, a virtual overflow detection circuit VODET 1140, a special function register SFR 1150, and a memory MEM 1160. The first electronic device 1201 may include a plurality of heterogeneous processors 1210, a task manager 1220, a data format controller 1230, a virtual overflow detection circuit 1240, a special function register 1250, and a memory 1260. The plurality of heterogeneous processors 1110 in the first electronic device 1101 and the plurality of heterogeneous processors 1210 in the second electronic device 1201 may include CPUs 1111 and 1211, GPUs 1112 and 1212, NPUs 1113 and 1213, DSPs 1114 and 1214, ISPs 1115 and 1215, and dedicated hardware 1116 and 1216, respectively.

[0153] A plurality of heterogeneous processors 1110 and 1210, task managers 1120 and 1220, data format controllers 1130 and 1230, virtual overflow detection circuits 1140 and 1240, special function registers 1150 and 1250, and Fig. 20 The memories 1160 and 1260 in the above reference Figure 1, 2, 3A-3B, 4, 5A-5B and 6 to 19 are substantially the same or similar to those described.

[0154] In one or more exemplary embodiments, some heterogeneous processors (e.g., 1111, 1112, 1113, 1114, 1115, and 1116) may be included in the first electronic device 1101, and other heterogeneous processors (e.g., 1211, 1212, 1213, 1214, 1215, and 1216) may be included in the second electronic device 1201. The first electronic device 1101 may be an electronic device that directly interacts with a user (e.g., is directly controlled by a user). The second electronic device 1201 may be physically separated from the first electronic device 1101 and may interoperate with the first electronic device 1101.

[0155] In one or more exemplary embodiments, the first electronic device 1101 can be any computing device and / or mobile device, such as a personal computer (PC), a laptop computer, a mobile phone, a smart phone, a tablet computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a digital camera, a portable camera, a music player, a video player, a portable game console, a navigation device, a wearable device, an Internet of Things (IoT) device, an Internet of Everything (IoE) device, a virtual reality (VR) device, an augmented reality (AR) device, a server, a workstation, an autonomous control device, etc.

[0156] In one or more exemplary embodiments, the second electronic device 1201 may be any computing device and / or mobile device that can interoperate with the first electronic device 1101. For example, the second electronic device 1201 may be a companion device that depends on the first electronic device 1101, such as a wearable device (e.g., a smart watch). Alternatively, the second electronic device 1201 may be an internal server (e.g., a home gateway) that controls IoT devices and / or IoE devices, or an external server (e.g., a cloud server).

[0157] The exemplary embodiments can be applied to various devices and systems including deep learning, ANN and / or machine learning systems. For example, the exemplary embodiments can be applied to mobile phones, smart phones, tablet computers, laptop computers, PDAs, PMPs, digital cameras, portable game consoles, wearable systems, IoT systems, VR systems, AR systems, etc.

[0158] The foregoing is illustrative of exemplary embodiments and should not be construed as limiting thereof. Although a few exemplary embodiments have been described, those skilled in the art will readily appreciate that many modifications may be made without materially departing from the inventive concept.

Claims

1. An artificial neural network (ANN) system, comprising: A processor configured to perform a node operation on input data including image data or sound data for a plurality of nodes included in each layer of the ANN to obtain a plurality of result values ​​of the node operation, and to perform a quantization operation on the obtained plurality of result values ​​based on a k-th fixed-point format of current quantization for each layer to obtain a plurality of quantized values ​​of the input data; a virtual overflow detection circuit configured to generate virtual overflow information indicating a distribution of the number of significant bits of the obtained plurality of quantized values; as well as a data format controller configured to determine a (k+1)th fixed-point format for subsequent quantization of each layer based on the generated virtual overflow information, Wherein, the virtual overflow detection circuit is further configured as: generating a plurality of thermometer codes corresponding to the obtained plurality of quantized values; generating a plurality of count signals indicating frequency values ​​of the effective number of bits based on the generated plurality of thermometer codes; as well as The generated plurality of count signals are provided as the virtual overflow information.

2. The ANN system according to claim 1, wherein: The data format controller is further configured to: Calculating the mean and variance of the significant digits; and Determine the fraction length of the (k + 1)th fixed-point format based on the computed mean and the computed variance.

3. The ANN system according to claim 1, wherein: The data format controller is further configured to determine whether the plurality of quantized values ​​are in an overflow approach state or an underflow approach state based on the generated virtual overflow information.

4. The ANN system according to claim 3, wherein: Based on the data format controller determining that the plurality of quantized values ​​are in an overflow proximity state, the data format controller is further configured to reduce the fractional length of the (k+1)th fixed-point format to be shorter than the fractional length of the kth fixed-point format.

5. The ANN system according to claim 3, wherein: Based on the data format controller determining that the plurality of quantized values ​​are in an underflow approaching state, the data format controller is further configured to increase the fractional length of the (k+1)th fixed-point format to be longer than the fractional length of the kth fixed-point format.

6. The ANN system according to claim 3, wherein: The data format controller is further configured to determine that the plurality of quantized values ​​are in an overflow proximity state based on a ratio of a frequency value of a maximum of the effective digits relative to a sum of frequency values ​​of the effective digits being higher than an overflow threshold ratio.

7. The ANN system according to claim 3, wherein: The data format controller is further configured to determine that the plurality of quantized values ​​are in an underflow approaching state based on a ratio of a frequency value of a maximum of the effective digits relative to a sum of frequency values ​​of the effective digits being lower than an underflow threshold ratio.

8. The ANN system according to claim 1, wherein: The virtual overflow detection circuit comprises: The thermometer code generating circuit is configured to sequentially receive the obtained multiple quantization values ​​and replace the significant bit of each quantization value to sequentially generate a plurality of thermometer codes corresponding to the obtained multiple quantization values.

9. The ANN system according to claim 8, wherein: The thermometer code generating circuit comprises: A plurality of OR logic gates are configured to perform an OR logic operation on every two adjacent bits of each quantized value to generate each thermometer code.

10. The ANN system according to claim 8, wherein: The virtual overflow detection circuit also includes: an accumulation circuit configured to accumulate the generated plurality of thermometer codes bit by bit to generate a plurality of accumulation values; and The subtraction circuit is configured to subtract two adjacent accumulated values ​​of the generated plurality of accumulated values ​​to generate a plurality of count signals, and provide the generated plurality of count signals as the virtual overflow information.

11. The ANN system according to claim 8, wherein: The virtual overflow detection circuit also includes: a counting code generating circuit configured to sequentially receive the generated plurality of thermometer codes, keep the most significant bit as a significant bit 1 for each thermometer code and replace the other significant bits from 1 to 0 to sequentially generate a plurality of counting codes corresponding to the generated plurality of thermometer codes; and The accumulation circuit is configured to accumulate the generated multiple counting codes bit by bit to generate multiple counting signals, and provide the generated multiple counting signals as the virtual overflow information.

12. The ANN system according to claim 11, wherein: The counting code generating circuit comprises: A plurality of XOR logic gates are configured to perform an XOR logic operation on every two adjacent bits of each thermometer code to generate each counting code corresponding to each thermometer code.

13. The ANN system of claim 1, wherein: The virtual overflow detection circuit comprises: a multiplexer configured to sequentially receive the obtained plurality of quantized values ​​and divide the received plurality of quantized values ​​into positive quantized values ​​and negative quantized values ​​based on a sign bit included in each quantized value; a first detection circuit configured to generate a positive count signal indicative of a frequency value of a significant digit of a positive quantized value; and The second detection circuit is configured to generate a negative count signal indicating a frequency value of a significant digit of a negative quantized value.

14. The ANN system of claim 1, wherein: The virtual overflow detection circuit is further configured to generate a plurality of virtual overflow information corresponding to a plurality of layers of the ANN, and The data format controller is further configured to determine a (k+1)th fixed-point format for each layer based on the generated plurality of virtual overflow information.

15. An artificial neural network (ANN) system, comprising: Virtual overflow detection circuit, configured as: generating a plurality of thermometer codes corresponding to a plurality of quantized values ​​of input data including image data or sound data, wherein the plurality of quantized values ​​are obtained by performing a quantization operation based on a fixed-point format, and generating virtual overflow information based on the generated plurality of thermometer codes, the virtual overflow information indicating a distribution of the number of significant bits of the plurality of quantized values; and A data format controller is configured to update the fixed-point format based on the generated virtual overflow information.

16. A method for controlling a fixed-point format in an artificial neural network (ANN) system, the method comprising: For a plurality of nodes included in each layer of the ANN, performing node operations on input data including image data or sound data to obtain a plurality of result values ​​of the node operations; Performing a quantization operation on the obtained multiple result values ​​based on the k-th fixed-point format of the current quantization for each layer to obtain multiple quantized values ​​of the input data; generating virtual overflow information indicating a distribution of the number of significant digits of the obtained plurality of quantized values; as well as determining the (k+1)th fixed-point format for subsequent quantization of each layer based on the generated virtual overflow information, The generating of virtual overflow information includes: generating a plurality of thermometer codes corresponding to the obtained plurality of quantized values; generating a plurality of count signals indicating the frequency value of the effective number of bits based on the generated plurality of thermometer codes; and The generated plurality of count signals are provided as the virtual overflow information.

17. The method of claim 16, wherein: Determining the (k+1)th fixed-point format for subsequent quantization includes: It is determined whether the obtained plurality of quantized values ​​are in an overflow-proximate state or an underflow-proximate state based on the generated virtual overflow information.

Images