Neural network device performing floating point operation and operating method thereof

Abstract

A neural network device performing floating point operations and an operating method thereof are provided. The neural network device performs a multiply-accumulate (MAC) operation for a product of a fraction of a weight and an input activation in a block floating point format by using an analog crossbar array, performs an addition operation for a shared exponent of the weight and the input activation in the block floating point format by using a digital computing circuit, and outputs a partial sum of a floating point output activation by combining a result of the MAC operation with a result of the addition operation.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2020-0071589, filed on June 12, 2020, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to neural network devices that perform floating-point operations and their operation methods. Background Technology

[0003] Interest in neuromorphic processors that perform neural network operations has increased. For example, research has been conducted on implementing neuromorphic processors that include neuronal and synaptic circuits. Neuromorphic processors can be used in neural network devices to drive various neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and feedforward neural networks (FNNs), for applications in fields including data classification or image recognition. Summary of the Invention

[0004] The present invention is provided in a simplified form to describe the choice of concepts further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0005] In one general aspect, a method of operating a neural network device for performing floating-point operations includes: determining, for each weight kernel in a weight kernel, a first shared index representing the weights included in the corresponding weight kernel, and obtaining weights in a first block of floating-point format including a first fraction adjusted based on the first shared index; determining, for each of a plurality of input strips included in an input feature map, a second shared index representing the input activation included in the corresponding input strip, and obtaining input activations in a second block of floating-point format including a second fraction adjusted based on the second shared index; performing a multiply-accumulate (MAC) operation on the first and second fractions using an analog crossbar switch array, and converting the result of the MAC operation into a digital signal using an analog-to-digital converter (ADC); and performing an addition operation on the first and second shared indices using digital computing circuitry, and outputting a partial sum of floating-point output activations included in a channel of an output feature map by combining the result of the MAC operation with the result of the addition operation.

[0006] The operation method may include: storing a first score corresponding to each of the weights included in the corresponding weight kernel in a memory cell arranged along a plurality of column lines of the analog crossbar array corresponding to the column lines of the corresponding weight kernel; and inputting a second score corresponding to each of the input activations included in the corresponding input strip to a plurality of row lines of the analog crossbar array.

[0007] The operation method may include: when the analog crossbar switch array supports signed input, inputting the sign bit activated by the second floating-point input along with the second fraction into the multiple row lines.

[0008] The operation method may include: when the analog cross switch array supports unsigned input, obtaining a first current sum along each of the plurality of column lines by activating only the row lines where the sign bit of the second block of floating-point format input is a first value; and obtaining a second current sum along each of the plurality of column lines by activating only the row lines where the sign bit of the second block of floating-point format input is a second value.

[0009] The operation method may include: using an ADC to convert a first current sum into a first digital signal; using an ADC to convert a second current sum into a second digital signal; and using a digital accumulator to combine the first digital signal and the second digital signal to output a digital signal corresponding to the result of the MAC operation.

[0010] The operation method may include: combining the first current sum and the second current sum using an analog accumulator to output a final current sum; and using an ADC to convert the final current sum into a digital signal corresponding to the result of the MAC operation.

[0011] In the case where the analog crossbar switch array supports signed weights, the step of storing the first fraction may include storing the sign bit of the first block of floating-point format weights together with the first fraction in a memory cell.

[0012] The operation method may include: storing, in the case that the analog crossbar array supports unsigned weights, a first fraction corresponding to each of the weights in which the sign bit of the first fraction is a first value in a memory cell arranged along a first column line of the analog crossbar array; storing, in the case that the first fraction corresponding to each of the weights in which the sign bit of the first fraction is a second value in a memory cell arranged along a second column line of the analog crossbar array; and outputting a final current sum by combining a first current sum of each output along the first column line with a second current sum of each output along the second column line.

[0013] The steps for outputting a partial sum may include: obtaining a third fraction by performing a shift operation on the result of a MAC operation to make the most significant bit 1; obtaining a third exponent by performing a conversion operation in which the number of shift operations performed is added to or subtracted from the result of the addition operation; and outputting a floating-point output activated partial sum including the third fraction and the third exponent.

[0014] The operation method may include: determining whether the floating-point output activation is negative based on the sign bit included in the result of the MAC operation; and in response to the floating-point output activation being determined to be negative, omitting the shift operation and the conversion operation, and outputting the floating-point output activation as 0.

[0015] In another general aspect, a neural network device for performing floating-point operations includes: at least one control circuit configured to: determine a first shared index representing weights included in the weight kernel for each weight kernel in the weight kernels, obtain weights in a first block floating-point format including a first fraction adjusted based on the first shared index, determine a second shared index representing input activations included in the input strips for each of the plurality of input strips included in an input feature map, and obtain input activations in a second block floating-point format including a second fraction adjusted based on the second shared index; in-memory computing circuitry including an analog crossbar array and an analog-to-digital converter (ADC), the analog crossbar array being configured to perform a multiply-accumulate (MAC) operation for the first and second fractions, the ADC being configured to convert the result of the MAC operation into a digital signal; and digital computing circuitry configured to: perform an addition operation for the first and second shared indexes, and output a partial sum of floating-point output activations included in a channel of an output feature map by combining the result of the MAC operation with the result of the addition operation.

[0016] The analog crossbar switch array may include: multiple row lines; multiple column lines intersecting the multiple row lines; and multiple memory cells disposed at the intersections of the multiple row lines and the multiple column lines. The at least one control circuit may be configured to: store a first score corresponding to each of the weights included in the weight kernel in a memory cell arranged along a column line corresponding to the corresponding weight kernel among the multiple column lines, and input a second score corresponding to each of the input activations included in the corresponding input strip to the multiple row lines.

[0017] When the analog crossbar switch array supports signed input, the at least one control circuit can be configured to input the sign bit activated by the second block of floating-point format input along with the second fraction to the plurality of row lines.

[0018] When the analog crossbar switch array supports unsigned inputs, the at least one control circuit can be configured to: obtain a first current sum along each of the plurality of column lines by activating only the row lines where the sign bit activated by the second block of floating-point format inputs is a first value, and obtain a second current sum along each of the plurality of column lines by activating only the row lines where the sign bit activated by the second block of floating-point format inputs is a second value.

[0019] The ADC can be configured to convert a first current sum into a first digital signal and a second current sum into a second digital signal, and the neural network device may include a digital accumulator configured to output a digital signal corresponding to the result of a MAC operation by combining the first digital signal and the second digital signal.

[0020] The neural network device may further include: an analog accumulator configured to output a final current sum by combining a first current sum with a second current sum; and an ADC configured to convert the final current sum into a digital signal corresponding to the result of a MAC operation.

[0021] In the case where the analog crossbar switch array supports signed weights, the at least one control circuit can be configured to store the sign bit of the first block of floating-point format weights together with the first fraction in a memory cell.

[0022] In the case where the analog crossbar array supports unsigned weights, the at least one control circuit can be configured to: store a first fraction corresponding to each of the weights in which the sign bit is a first value in a memory cell arranged along a first column line of the analog crossbar array, and store a first fraction corresponding to each of the weights in which the sign bit is a second value in a memory cell arranged along a second column line of the analog crossbar array; and the in-memory computing circuit can be configured to: output a final current sum by combining a first current sum of each output along the first column line with a second current sum of each output along the second column line.

[0023] The digital computing circuit can be configured to: obtain a third fraction by performing a shift operation on the result of a MAC operation to make the most significant bit 1, obtain a third exponent by performing a conversion operation, and obtain a partial sum of a floating-point output activation including the third fraction and the third exponent, wherein the number of times the shift operation is performed is added to or subtracted from the addition operation.

[0024] The digital computing circuit can be configured to: determine whether the floating-point output activation is negative based on the sign bit included in the result of the MAC operation, and in response to the floating-point output activation being determined to be negative, omit the shift operation and the conversion operation, and output the floating-point output activation as 0.

[0025] In another general aspect, a neural network device includes: at least one control circuit configured to: output a first shared index of weights included in a plurality of weight kernels and a first fraction of the weights adjusted by a shift operation, and output a second shared index of input strips included in an input feature map and a second fraction corresponding to input activations included in the input strips; an in-memory computing circuit configured to: perform a multiply-accumulate (MAC) operation on the first and second fractions, and convert the result of the MAC operation into a digital signal; and a digital computing circuit configured to: perform an addition operation on the first and second shared indices, and output a partial sum of floating-point output activations included in a channel of an output feature map by combining the digital signal with the result of the addition operation.

[0026] The first shared index can be the maximum value among the existing indices of the weights.

[0027] The in-memory computing circuitry may include an analog crossbar switch array and an analog-to-digital converter (ADC), wherein the analog crossbar switch array is configured to perform MAC operations and the ADC is configured to convert the result of the MAC operations into a digital signal.

[0028] The neural network device may include an accumulator configured to combine a first digital signal corresponding to positive input activation and a second digital signal corresponding to negative input activation.

[0029] An accumulator can be an analog accumulator included in the computing circuitry within memory.

[0030] An accumulator can be a digital accumulator included in a digital computing circuit.

[0031] Other features and aspects will become clear from the following detailed description, drawings, and claims. Attached Figure Description

[0032] Figure 1 It is a diagram used to illustrate the architecture of a neural network based on some examples.

[0033] Figure 2 These are diagrams used to illustrate operations performed in a neural network based on some examples.

[0034] Figure 3 This is a diagram illustrating an in-memory computing circuit based on some examples.

[0035] Figure 4 This is a schematic diagram illustrating the overall processing of floating-point operations performed by a neural network device, based on some examples.

[0036] Figure 5This is a diagram illustrating the process of converting weights into block floating-point values ​​using a neural network device, based on some examples.

[0037] Figure 6A This is a diagram illustrating how weights converted to block floating-point are stored according to some examples.

[0038] Figure 6B This is a diagram illustrating a method for storing weights converted to block floating-point according to another example.

[0039] Figure 7 This is a diagram used to illustrate the process of converting input activations into block floating-point numbers using a neural network device, based on some examples.

[0040] Figure 8 This is a diagram used to illustrate the processing of floating-point operations supported by signed weights and signed inputs in the execution of a simulated crossbar switch array, based on some examples.

[0041] Figure 9A and Figure 9B This is a diagram used to illustrate the processing of a simulated crossbar switch array that supports signed weights and floating-point operations with unsigned inputs, based on some examples.

[0042] Figure 10 This is a diagram used to illustrate the processing of floating-point operations with unsigned inputs and unsigned weights performed by an analog crossbar switch array, based on some examples.

[0043] Figure 11 This is a diagram illustrating how the final output of floating-point operations, based on some examples, is processed by combining the results of operations performed on an analog crossbar array with the results of operations performed on a digital computing circuit.

[0044] Figure 12 This is a flowchart illustrating how a neural network device operates, based on some examples.

[0045] Figure 13 This is a block diagram illustrating the configuration of an electronic system based on some examples.

[0046] Throughout the accompanying drawings and detailed embodiments, unless otherwise described or provided, the same reference numerals will be understood to denote the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative dimensions, scale, and depiction of elements in the drawings may be exaggerated. Detailed Implementation

[0047] The following detailed embodiments are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known upon understanding this disclosure may be omitted.

[0048] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein have been provided merely to illustrate some of the many feasible ways of implementing the methods, apparatus, and / or systems described herein that will be clear upon understanding the disclosure of this application.

[0049] Throughout this specification, when a component is described as "connected to" or "attached to" another element, that component may be directly "connected to" or "attached to" said other component, or there may be one or more other components in between. Conversely, when an element is described as "directly connected to" or "directly attached to" another element, there may be no other components in between. Similarly, similar expressions (e.g., "between" and "immediately between," and "adjacent to" and "closely adjacent to") may be interpreted in the same manner. As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more of the associated listed items.

[0050] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, assemblies, regions, layers, or parts, these components, assemblies, regions, layers, or parts should not be limited by these terms. Rather, these terms are used only to distinguish one component, assembly, region, layer, or part from another. Thus, without departing from the teaching of the examples described herein, the first component, first assembly, first region, first layer, or first part referred to as the first component, first assembly, first region, first layer, or first part may also be referred to as the second component, second assembly, second region, second layer, or second part.

[0051] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the features, quantities, operations, components, elements, and / or combinations thereof stated therein, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0052] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains and as understood based on the disclosure of this application. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art and in the disclosure of this application, and shall not be interpreted in an idealized or overly formal sense. The use of the term “may” (e.g., what an example or embodiment may include or implement) with respect to an example or embodiment indicates the existence of at least one example or embodiment that includes or implements such a feature, but not all examples are limited thereto.

[0053] Figure 1 This is a diagram used to illustrate the architecture of neural network 1 based on some examples.

[0054] Reference Figure 1 Neural Network 1 can be represented by a mathematical model using nodes and edges. Neural Network 1 may include a deep neural network (DNN) or an n-layer neural network architecture. A DNN or an n-layer neural network may correspond to a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network, a restricted Boltzmann machine, etc. For example, Neural Network 1 may be implemented as a CNN, but is not limited to this. Figure 1 Neural network 1 can correspond to some layers of a CNN. Therefore, neural network 1 can correspond to convolutional layers, pooling layers, or fully connected layers of a CNN. However, for convenience, in the following description, it is assumed that neural network 1 corresponds to a convolutional layer of a CNN.

[0055] In a convolutional layer, the first feature map FM1 corresponds to the input feature map, and the second feature map FM2 corresponds to the output feature map. Feature maps can represent datasets showcasing various characteristics of the input data. The first feature map FM1 and the second feature map FM2 can be two-dimensional or higher-dimensional high-dimensional matrices, each with its own activation parameters. When the first feature map FM1 and the second feature map FM2 correspond to, for example, three-dimensional feature maps, they have a width W (or columns), a height H (or rows), and a depth C. In this case, the depth C can correspond to the number of channels.

[0056] The first feature map FM1 may include multiple input stripes. For example, the first feature map FM1 may include H×W input stripes. Each input strip is channel-oriented input data for one space of the input feature map and may have a size of 1×1×C. For example, each input strip may include C input activations.

[0057] In the convolutional layer, a convolution operation is performed on the first feature map FM1 and the weight map WM, resulting in the generation of the second feature map FM2. The weight map WM can filter the first feature map FM1 and is referred to as a filter or kernel. In one example, the depth (i.e., the number of channels) of the weight map WM is the same as the depth C (i.e., the number of channels) of the first feature map FM1. The weight map WM is shifted by traversing the first feature map FM1 as a sliding window. In each shift, the weights included in the weight map WM are multiplied and summed with all the feature values ​​in the region overlapping with the first feature map FM1. As the first feature map FM1 is convolved with the weight map WM, one channel of the second feature map FM2 is generated.

[0058] exist Figure 1 In this context, although a weight map WM is depicted, multiple channels of the second feature map FM2 can essentially be generated by convolving multiple weight maps with the first feature map FM1. The second feature map FM2 of the convolutional layer can be the input feature map of the next layer. For example, the second feature map FM2 can be the input feature map of a pooling layer.

[0059] Figure 2 This is a diagram used to illustrate the operations performed in a neural network 2 based on some examples.

[0060] Reference Figure 2 The neural network 2 has a structure including an input layer, a hidden layer and an output layer, performs operations based on received input data (e.g., I1 and I2), and can generate output data (e.g., O1 and O2) based on the results of the operations.

[0061] As mentioned above, neural network 2 can be an n-layer neural network or a DNN comprising two or more hidden layers. For example, such as Figure 2 As shown, neural network 2 can be a DNN comprising an input layer (layer 1), two hidden layers (layers 2 and 3), and an output layer (layer 4). When neural network 2 is implemented as a DNN architecture, it includes a larger number of layers capable of processing effective information; therefore, neural network 2 can handle a larger number of complex datasets than a neural network with a single layer. However, although neural network 2 is shown as comprising four layers, this is merely an example, and neural network 2 can include a small or large number of layers, or a small or large number of channels. That is, neural network 2 can include... Figure 2 The diagram shows layers with various structures.

[0062] Each of the multiple layers included in neural network 2 may include multiple channels. A channel may correspond to multiple artificial nodes, which are referred to as neurons, processing elements (PEs), units, or similar terms. For example, as... Figure 2As shown, layer 1 may include two channels (nodes), and each of layers 2 and 3 may include three channels (e.g., first channel CH1 to third channel CH3). However, this is just an example, and each of the multiple layers included in neural network 2 may include a variety of numbers of channels (nodes).

[0063] In a neural network 2, the channels in each of its multiple layers can be connected to each other to process data. For example, one channel can receive data for operation from other channels and output the result of the operation to other channels.

[0064] Each input and output of each of the multiple channels can be referred to as input activation and output activation. That is, activation can be the output of a channel and can be a parameter corresponding to the input of a channel included in the next layer. Meanwhile, each of the multiple channels can determine its own activation based on the activations and weights received from the channels included in previous layers. Weights are parameters used to manipulate the output activation in each channel and can be values ​​assigned to the connection relationships between channels.

[0065] Each of the multiple channels can be processed by a computing unit or processing element that is activated by receiving input and outputting output, and the input-output of each of the multiple channels can be mapped. For example, when σ is an activation function, It represents the weights from the k-th channel included in layer (i-1) to the j-th channel included in layer i. It is the bias of the j-th channel in the i-th layer. When the j-th channel in the i-th layer is activated, the activation can be calculated using Equation 1 below, where i is a positive integer greater than 1, and j and k are positive integers.

[0066] Equation 1:

[0067]

[0068] like Figure 2 As shown, the activation of the first channel CH1 in the second layer (layer 2) can be represented as Furthermore, according to Equation 1, It can have a value The activation function σ can be a linear rectified unit (ReLU), but its configuration is not limited to this. For example, the activation function σ can be sigmoid, hyperbolic tangent, Maxout, etc.

[0069] As described above, in Neural Network 2, a large dataset is exchanged between multiple interconnect channels, and many computational processes are performed through multiple layers. Within this operation, numerous MAC (multiply-accumulate-add) operations are performed, and numerous memory access operations must be executed to load the activations and weights, which are operands of the MAC operations, at the appropriate time.

[0070] On the other hand, typical digital computers use a von Neumann architecture with separate computing units and memory, and include a common data bus for data transfer between the two separate blocks. Therefore, in the processing of neural networks2 where data movement and operations are continuously repeated, a significant amount of time is required for data transfer, and excessive power is consumed. To overcome this problem, in-memory computing circuits have been proposed as an architecture for integrating computing units and memory used to perform MAC operations into a single unit. In the following text, reference will be made to… Figure 3 A more detailed description of the computing circuitry within the memory.

[0071] Figure 3 This is a diagram illustrating an in-memory computing circuit 3 according to some examples.

[0072] Reference Figure 3 The in-memory computing circuitry 3 may include an analog crossbar array 30 and an analog-to-digital converter (ADC) 40. However, in Figure 3 The in-memory computing circuit 3 shown in the diagram depicts only components relevant to this example. Therefore, it will be clear to those skilled in the art that, in addition to... Figure 3 In addition to the components shown, the in-memory computing circuit 3 may also include other general-purpose components.

[0073] The analog crossbar switch array 30 may include multiple row lines 310, multiple column lines 320, and multiple memory cells 330. The multiple row lines 310 can be used to receive input data. For example, when there are N row lines 310 (N is any positive integer), the voltages V1, V2, ..., V corresponding to the input activation are... N It can be applied to N row lines. Multiple column lines 320 can intersect with multiple row lines 310. For example, when there are M column lines 320 (M is any positive integer), the multiple column lines 320 and multiple row lines 310 can intersect at N×M intersection points.

[0074] Meanwhile, multiple memory cells 330 can be arranged at the intersections of multiple row lines 310 and multiple column lines 320. Each of the multiple memory cells 330 can be implemented as a non-volatile memory (such as resistive RAM (ReRAM) or eFlash (embedded flash memory)) for storing weights, but is not limited thereto. Each of the multiple memory cells 330 can be a volatile memory (such as static random access memory (SRAM)).

[0075] exist Figure 3 In the analog cross switch array 30 shown, multiple memory cells 330 may include conductance G corresponding to weights. 11 ... G NM When the voltage corresponding to the input activation is applied to each of the multiple row lines 310, according to Ohm's law, a current of magnitude I = V × G can be output through each memory cell 330. Since the currents output from the memory cells arranged along a column line are summed, the currents I1, ..., I... M It can be output along multiple column lines 320. Current and I1, ..., I M This can correspond to the result of a MAC operation performed using a simulation method.

[0076] ADC 40 can convert the results of analog MAC operations (i.e., current and I1, ..., I2) output from analog cross switch array 30. M The signal is converted into a digital signal. The result of the MAC operation, which converts the signal into a digital signal, is output from the ADC 40 and can be used in subsequent neural network processing.

[0077] On the other hand, such as Figure 3 As shown, the in-memory computing circuit 3 has the advantages of low core computing unit complexity, low power consumption and small circuit size compared with digital computers, but it can only perform fixed-point based operations and may have difficulty performing floating-point based operations that support a large dynamic range.

[0078] Therefore, in related technologies, after training a floating-point based neural network, the trained neural network is quantized and converted to a fixed-point format, and the in-memory computing circuit 3 is only used in the processing of the quantized neural network. However, according to related technologies, precision loss may occur during the processing of the quantized neural network, or retraining may be required to minimize the precision loss. Furthermore, neural networks implementing specific applications have very large parameter dynamic ranges, making it impossible to achieve quantization while minimizing precision loss.

[0079] According to this example, a neural network device capable of performing floating-point operations while utilizing in-memory computing circuitry 3, which has various advantages, can be provided. Hereinafter, methods for performing floating-point operations by a neural network device according to some examples will be described in detail with reference to the accompanying drawings.

[0080] Figure 4 This is a schematic diagram illustrating the overall processing of floating-point operations performed by a neural network device 4 according to some examples.

[0081] Reference Figure 4 The neural network device 4 may include at least one control circuit 410, an in-memory computing circuit 420, and a digital computing circuit 430. However, in Figure 4 The neural network device 4 shown only depicts components relevant to this example. Therefore, it will be clear to those skilled in the art that, in addition to... Figure 4 In addition to the components shown, the neural network device 4 may also include other general-purpose components.

[0082] At least one control circuit 410 performs the overall function of controlling the neural network device 4. For example, at least one control circuit 410 can control the operation of the in-memory computing circuit 420 and the digital computing circuit 430. Simultaneously, at least one control circuit 410 can be implemented as an array of multiple logic gates, or as a combination of a general-purpose microprocessor and memory, wherein the program executed in the microprocessor can be stored in the memory.

[0083] At least one control circuit 410 is capable of determining, for each weight kernel in the weight kernel, a first shared index representing the weights included in the weight kernel, and is capable of obtaining weights in a first block floating-point format including a first fraction adjusted based on the first shared index. Referring below... Figure 5 , Figure 6A and Figure 6B The method by which at least one control circuit 410 obtains the block floating-point weights is described in more detail.

[0084] Figure 5 This is a diagram illustrating the process of converting weights into block floating-point values ​​using a neural network device, based on some examples.

[0085] Reference Figure 5 When the height of the weight kernel is R, the width of the weight kernel is Q, the depth of the weight kernel (i.e., the number of channels) is C, and the number of weight kernels is K, the process of converting the C×R×Q×K (hereinafter abbreviated as CRQK) weights included in the weight kernel into block floating-point format is shown.

[0086] Because, according to some examples, neural network devices are intended to utilize in-memory computing circuitry that can only perform fixed-point-based operations (e.g., Figure 4The in-memory computing circuit 420 performs floating-point operations at low power, so the portion of the floating-point weights that can be computed by the in-memory computing circuit 420 needs to be separated.

[0087] The weights included in the weight kernel are floating-point data and can have the same or different exponents depending on the size of each value. For efficient operation, a block floating-point format with a shared exponent for each block of a specific size can be used. For example, as... Figure 5 As shown, a shared index can be extracted for each weight kernel, and the weights included in a weight kernel can share the same index. Furthermore, to represent all weights included in a weight kernel, the shared index can be determined as the maximum value among the existing indices of the weights, but is not limited to this. When the index of a weight changes from an existing index to a shared index, the weight's score can also be adjusted through a corresponding shift operation.

[0088] like Figure 5 As shown, when CRQK weights are converted to block floating-point format in units of weight kernels, CRQK weights can be represented simply as CRQK sign bits, K shared exponents, and CRQK fractions. The K shared exponents can be input to digital computing circuit 430 and used for digital calculations, while the CRQK fractions can be input to in-memory computing circuit 420 for analog calculations. In the following text, reference will be made to… Figure 6A and Figure 6B Describe a method for storing shared exponents and fractions used for numerical and analog operations, respectively.

[0089] Figure 6A This is a diagram illustrating a method for storing weights converted to block floating-point values ​​based on some examples, and Figure 6B This is a diagram illustrating a method for storing weights converted to block floating-point according to another example.

[0090] Figure 6A Analog cross switch array 610 and Figure 6B Each of the analog cross switch array 615 corresponds to Figure 3 The analog cross switch array 30, and Figure 6A Digital computing circuit 620 and Figure 6B Each of the digital computing circuits 625 corresponding to Figure 4 The digital computing circuit 430 is used, therefore the description already given will be omitted.

[0091] Reference Figure 6A , as reference Figure 5 As described, a method for storing the shared index and score of weights for a specific layer is shown when the weights are converted to have a shared index for each weight kernel.

[0092] In one example, W represents the weights included in the nth layer, the i-th input channel, and the k-th weight kernel. n，i，k It can be represented as Equation 2 below, where n, i, and k are positive integers.

[0093] Equation 2:

[0094]

[0095] In equation 2 above, S W The sign bit representing the weight, E W(n，k) The exponent representing the weight, F W(n，i，k) The fraction represents the weight. In the example where the weight is a 16-bit floating point, the sign bit includes 1 bit, the shared exponent includes e bits, and the fraction includes f bits (e and f are positive integers and 1 + e + f = 16). In Equation 2, for ease of explanation, the exponent bias is not included, but when the exponent bias b... W When included, E W(n，k) Can be modified to E W(n，k) -b W On the other hand, input activation may also include exponential bias, and in this case, Equation 3, which will be described below, can be modified in the same way as when exponential bias is included in the weights. Furthermore, when Equations 2 and 3 are modified to take exponential bias into account, those skilled in the art will readily understand that Equation 4, which will be described below, can also be modified to correspond to the modifications of Equations 2 and 3.

[0096] The weighted score F of the block floating-point format W(n，i，k) These can be stored in the analog crossbar switch array 610. For example, assuming that when the analog crossbar switch array 610 includes M column lines, each of the M column lines can correspond to each of the M weight kernels, and the weights included in the weight kernels (e.g., W0 to W...) N-1 These can be stored in memory cells arranged along the corresponding column lines. The weights composed of each free f bits are stored in the number of cores (i.e., M) in units of stripes of size N.

[0097] Meanwhile, the shared index E extracted from the weights W(n，k) It can be stored separately in the digital computing circuit 620. The digital computing circuit 620 may include a register or memory for storing the digital value corresponding to the shared exponent. The shared exponent, composed of each free e bits, is stored as many times as the number of cores (i.e., M), therefore, a total of e×M bits are stored.

[0098] When the analog cross switch array 610 supports signed weights, only E W(n，k) It is stored separately in the memory space of the digital computing circuit 620, and the sign bit S of the weight.W It can be stored in the analog cross switch array 610. However, when the analog cross switch array 610 supports unsigned weights, S W Together with E W(n，k) It can be stored in the additional storage space of the digital computing circuitry 620, outside of the analog cross switch array 610.

[0099] The above has been referenced Figure 6A This describes the case where weights are transformed into a shared exponent with respect to each weight kernel, but this is only an example. Weights can be transformed into a shared exponent with respect to each group rather than each weight kernel. For example, as... Figure 6B As shown, a specific number of columns (i.e., weight kernels) included in the analog crossbar array 615 can be grouped into a set, and a shared index can be extracted for each set. In this case, the number of shared indices used to represent all weights can be reduced, and the computational cost in the process of converting the output of the analog crossbar array 615 into the final output can be reduced.

[0100] However, when too many columns are grouped into a single group, the loss of accuracy may increase. Therefore, the number of groups can be appropriately set, taking into account a trade-off between the loss of accuracy and the computational cost. In one example, since the weights correspond to data known in advance to the neural network device, the appropriate number of groups can be determined beforehand through simulation. However, the configuration is not limited to this, and the number of groups can be determined in real time. Furthermore, the type of columns included in a group can also be determined along with the number of groups.

[0101] Simultaneously, the shared index extracted from the weights can be stored separately in the digital computing circuit 625. The digital computing circuit 625 may include registers or memory for storing the digital values ​​corresponding to the shared indexes. The shared indexes can be stored in as many groups (i.e., G).

[0102] According to some examples, when the analog cross switch array 615 amplifies the sensed current or input voltage by a preset amplification factor, the amplification factor corresponding to the difference between the shared indexes of specific groups can be assigned to the amplification factor of the analog cross switch array 615. Therefore, the shared indexes between corresponding groups can be unified. For example, when the analog cross switch array 615 can amplify the sensed current by two times and the difference between the shared index to be assigned to the first group and the shared index to be assigned to the second group is 1, instead of assigning different shared indices to the first and second groups, the same shared index is assigned to both groups. However, when the current sum corresponding to the first group is obtained, the sensed current can be doubled. Therefore, the number of shared indices used to represent the total weight can be further reduced.

[0103] On the other hand, the configuration is not limited to the previous example, and those skilled in the art will readily understand that the input voltage or both the sensed current and the input voltage can be amplified instead of the sensed current, and the amplification factor can be set differently, such as 4, 8, etc., instead of 2. If each of the sensed current and the input voltage can be amplified, and the amplification factor can also be one of several amplification factors, the number of shared exponents used to represent the total weight can be further reduced.

[0104] return Figure 4 At least one control circuit 410 can determine a second shared index representing the input activation included in the input strip for each of a plurality of input strips included in the input feature map, and can obtain a second block floating-point format input activation including an adjusted second fraction based on the second shared index. Referring below... Figure 7 The method for at least one control circuit 410 to obtain block floating-point input activation is described in more detail.

[0105] Figure 7 This is a diagram used to illustrate the process of converting input activations into block floating-point numbers using a neural network device, based on some examples.

[0106] Reference Figure 7 When the height of the input feature map IFM is H, the width of the input feature map IFM is W, and the depth (i.e., the number of channels) of the input feature map IFM is C, the process of converting the H×W×C (hereinafter abbreviated as HWC) input activations included in the input feature map IFM into block floating-point format is shown.

[0107] The input activations included in the input feature map IFM can be converted to block floating-point format on a strip-by-strip basis. For example, as Figure 7 As shown, when input activations are converted to block floating-point format in units of input stripes, the input activations included in an input stripe can be represented by only one shared exponent. A shared exponent corresponding to an input stripe can be input to digital computing circuit 430 for digital calculations, and C fractions corresponding to an input stripe can be input to in-memory computing circuit 420 for analog calculations. Depending on the characteristics of the analog crossbar array (e.g., supporting signed or unsigned inputs), C sign bits corresponding to an input stripe can be input to digital computing circuit 430 for digital operations, or can be input to in-memory computing circuit 420 for analog calculations.

[0108] The conversion of input activations to block floating-point can be performed in real time as the floating-point input feature map (IFM) is input. The shared index representing an input strip can be determined as, but is not limited to, the maximum value among the existing indices of the input activations included in the input strip. The shared index can be appropriately determined according to predefined rules.

[0109] In addition, when the input activation exponent changes from the existing exponent to the shared exponent, the input activation fraction can also be adjusted by a corresponding shift operation. In one example, for the nth layer and the ith input channel, the input activation X converted to block floating point n，i can be expressed as Equation 3 below, where n and i are positive integers.

[0110] Equation 3:

[0111]

[0112] In Equation 3 above, S X represents the sign bit of the input activation, represents the exponent of the input activation, and F X(n，i) represents the fraction of the input activation.

[0113] Figure 7 shows an example where the input activation is converted to block floating point format in units of input strips. For example, if C, which is the number of channels corresponding to one input strip, is greater than ROW, which is the number of multiple row lines of the analog crossbar array included in the in-memory computing circuit 420, one input strip can be divided into strip parts having lengths C0 and C1 (C = C0 + C1, where C0 < ROW and C1 < ROW). However, the example is not limited thereto, and one input strip can be divided into two or more strip parts according to C and ROW.

[0114] In this case, the input activation can be converted to block floating point format in units of strip parts. However, the configuration is not limited thereto, and the input activation can be converted to block floating point format in units of input strips and can be input into the in-memory computing circuit 420 only in units of strip parts.

[0115] Returning again Figure 4 , at least one control circuit 410 can perform neural network operations using the weights and input activations in block floating point format. For example, at least one control circuit 410 can perform neural network operations by using the in-memory computing circuit 420 and the digital computing circuit 430.

[0116] As described for the in-memory computing circuit 3 referred to Figure 3 , the in-memory computing circuit 420 can include an analog crossbar array and an ADC. The analog crossbar array can include multiple row lines, multiple column lines that cross the multiple row lines, and multiple memory cells provided at the intersections of the multiple row lines and the multiple column lines.

[0117] At least one control circuit 410 can store a first score corresponding to each of the weights included in the weight kernel in a memory cell arranged along a plurality of column lines corresponding to the weight kernel, and can input a second score corresponding to each of the input activations included in the input strip to the plurality of row lines. Therefore, the analog crossbar array can perform a MAC operation in analog mode for the first score (corresponding to the weight) and the second score (corresponding to the input activation), and the result of the MAC operation can be output along the plurality of column lines. The ADC can convert the result of the MAC operation into a digital signal, such that digital operations on the result of the MAC operation are later performed by the digital computing circuit 430.

[0118] The digital computing circuit 430 can perform addition on the first shared exponent and the second shared exponent, combine the result of the MAC operation with the result of the addition operation, and output the partial sum of floating-point output activations included in the channels of the output feature map. For example, the digital computing circuit 430 can obtain the partial sum for calculating floating-point output activations by combining the result of the MAC operation corresponding to an input strip with the result of the addition operation. When the partial sums corresponding to all input strips included in the input feature map IFM are obtained, the digital computing circuit 430 can use the obtained partial sums to calculate the floating-point output activations included in the channels of the output feature map OFM.

[0119] On the other hand, when the weights are represented as Equation 2 above and the input activations are represented as Equation 3 above, the partial sum PS calculated for the k-th weight kernel... k It can be represented as Equation 4 below.

[0120] Equation 4:

[0121]

[0122] In equation 4, Indicates the index of input activation. The exponent representing the weight, S x(i) S represents the sign bit that activates the input. W(i) The sign bit indicating the weight. It can be easily calculated using the addition operation of the digital computing circuit 430. However, in In the calculation, each weight and the sign bit of the input activation should be considered, therefore, The calculation can be performed in different ways depending on whether the analog crossbar switch array included in the memory-in-memory computing circuit 420 supports signed weights and signed inputs. In the following text, reference will be made to... Figures 8 to 10 Detailed description of whether the analog cross switch array supports signed weights and signed input operation processing.

[0123] Figure 8 This is a diagram illustrating the processing of floating-point operations based on some examples of signed weights and signed inputs supported by an analog crossbar switch array.

[0124] According to some examples, when the analog crossbar switch array supports signed inputs, at least one control circuit 410 can input the sign bit (e.g., IFM sign) of the second block of floating-point format inputs together with the second fraction (e.g., IFM fraction) to multiple row lines. Furthermore, when the analog crossbar switch array supports signed weights, at least one control circuit 410 can store the sign bit of the first block of floating-point format weights together with the first fraction in a memory cell.

[0125] Since analog crossbar arrays support both signed weights and signed inputs, MAC operations considering both the input activation sign and weight sign can be performed by simply inputting the input activation sign bit and the weight sign bit together with the fraction into the analog crossbar array. For example, the output CO of the analog crossbar array corresponding to the k-th weight kernel... k It can be calculated as shown in Equation 5 below.

[0126] Equation 5:

[0127]

[0128] Figure 9A and 9B This is a diagram illustrating the processing of floating-point operations performed based on some examples of analog crossbar arrays that support signed weights but also support unsigned inputs.

[0129] Based on some examples, as referenced above Figure 8 The described analog crossbar array supports signed weights, so at least one control circuit 410 can store the sign bit of the first block of floating-point format weights together with the first fraction in a memory cell. Therefore, in the analog crossbar array, both positive and negative sign weights can be stored.

[0130] On the other hand, when the analog crossbar array supports unsigned inputs, at least one control circuit 410 can obtain a first current sum along each output in the column lines by preferentially activating only the row lines where the sign bit of the second block of floating-point format inputs is a first value. Subsequently, at least one control circuit 410 can obtain a second current sum along each output in multiple column lines by activating only the row lines where the sign bit of the second block of floating-point format inputs is a second value. The first and second values ​​can be 0 or 1, and can have values ​​different from each other.

[0131] As described above, at least one control circuit 410 can obtain a first current sum corresponding to positive input activation and a second current sum corresponding to negative input activation separately through at least two cycles (e.g., cycle 0 and cycle 1). Meanwhile, the method of combining the first current sum and the second current sum can vary depending on the configuration of the computing circuit 420 within the memory.

[0132] In one example, such as Figure 9A As described, digital conversion can be performed before the first current sum and the second current sum are combined. For example, an ADC can convert the first current sum into a first digital signal and the second current sum into a second digital signal. The timing of the conversion of the first current sum into the first digital signal and the timing of the conversion of the second current sum into the second digital signal may differ from each other, but are not limited thereto.

[0133] In the corresponding example, the neural network device may also include a digital accumulator 910. The digital accumulator 910 can output a digital signal corresponding to the result of the MAC operation by combining a first digital signal and a second digital signal. For example, the digital accumulator 910 can combine the first digital signal and the second digital signal by adding the first digital signal corresponding to positive input activation and subtracting the second digital signal corresponding to negative input activation.

[0134] In another example, such as Figure 9B As depicted, the combination of the first current sum and the second current sum can be performed before digital conversion. In a corresponding example, the neural network device may also include an analog accumulator 920. The analog accumulator 920 can output a final current sum by combining the first current sum and the second current sum. For example, the analog accumulator 920 outputs a final current sum by adding the first current sum corresponding to positive input activation and subtracting the second current sum corresponding to negative input activation. The ADC can convert the final current sum output from the analog accumulator 920 into a digital signal corresponding to the result of the MAC operation.

[0135] At the same time, Figure 9A The text depicts a digital accumulator 910 located outside the memory-based computing circuitry 420. Figure 9B In this configuration, the analog accumulator 920 is located within the in-memory computing circuitry 420, but the configuration is not limited thereto. Each of the digital accumulator 910 and the analog accumulator 920 may be located at any suitable location, either inside or outside the in-memory computing circuitry 420. For example, in one example, the digital accumulator 910 may be included in the digital computing circuitry 430.

[0136] Figure 10 This is a diagram illustrating the processing of floating-point operations based on some examples of supporting unsigned inputs and unsigned weights in an analog crossbar switch array.

[0137] When the analog crossbar array supports unsigned weights, at least one control circuit 410 can store the first score corresponding to each weight in which the sign bit of the first score is a first value in a memory cell arranged along the first column line of the analog crossbar array, and store the first score corresponding to each weight in which the sign bit of the first score is a second value in a memory cell arranged along the second column line of the analog crossbar array.

[0138] The in-memory computing circuit 420 can output a final current sum by combining a first current sum of each output along the first column line with a second current sum of each output along the second column line. In this way, the analog crossbar switch array included in the in-memory computing circuit 420 can operate as if it included two crossbar switch arrays (i.e., a first crossbar switch array 1010 storing positive weights and a second crossbar switch array 1020 storing negative weights).

[0139] The in-memory computing circuit 420 can output a final current sum by combining the first current sum of each output along the first column line with the second current sum of each output along the second column line using the add / subtract ADD / SUB module 1030. For example, the add / subtract ADD / SUB module 1030 can output a final current sum by adding the first current sum output from the first crossbar switch array 1010 and subtracting the second current sum output from the second crossbar switch array 1020.

[0140] On the other hand, as mentioned above... Figure 9A and 9B The described analog crossbar array supports unsigned inputs, so at least one control circuit 410 can individually input positive and negative input activations to the memory-based computing circuit 420 in two cycles (i.e., cycle 0 and cycle 1). Therefore, the final current sum corresponding to positive input activation can be output from the analog crossbar array in cycle 0, and the final current sum corresponding to negative input activation can be output from the analog crossbar array in cycle 1.

[0141] Accumulator 1040 can obtain a fractional sum by combining the final current sum corresponding to positive input activation with the final current sum corresponding to negative input activation. For example, accumulator 1040 can obtain a fractional sum by adding the final current sum corresponding to positive input activation and subtracting the final current sum corresponding to negative input activation.

[0142] Already referred to Figure 10An example of an analog crossbar array supporting unsigned weights and unsigned inputs is described; however, an analog crossbar array can support unsigned weights and also signed inputs. In this case, since the sign of the input activation is taken into account and two cycles of input are not required, all input activations can be input in one cycle regardless of the sign, and the configuration of accumulator 1040 can be omitted.

[0143] Already referred to Figures 8 to 10 A method for calculating the partial sum of fractions based on whether the analog crossbar switch array included in the memory-in-memory computing circuit 420 supports signed weights or signed inputs is described in detail. The partial sum of fractions output from the analog crossbar switch array can be input to the digital computing circuit 430 and used to calculate the final output. Referring below... Figure 11 Describe in detail the method by which the digital computing circuit 430 calculates the total output.

[0144] Figure 11 This is a diagram illustrating how the final output of floating-point operations, based on some examples, is processed by combining the results of operations performed on an analog crossbar array with the results of operations performed on a digital computing circuit.

[0145] The digital computing circuit 430 can obtain a third fraction by performing a shift operation, such that the most significant bit of the result of the MAC operation output from the analog crossbar switch array is 1. The digital computing circuit 430 may include a shift operator 1110 for performing the shift operation. The third fraction, to which the shift operation is performed, may correspond to the fraction of the sum of the activated portions of the output.

[0146] The digital computing circuit 430 can obtain a third exponent by performing a conversion operation in which the number of shift operations performed in advance is added to or subtracted from the addition result of the first shared exponent (i.e., the shared exponent of the weights) and the second shared exponent (i.e., the shared exponent of the input activation). The third exponent obtained by the conversion operation can correspond to the exponent of the partial sum of the output activation. Therefore, the digital computing circuit 430 can output a floating-point partial sum of the output activation that includes the third fraction and the third exponent.

[0147] After calculating the partial sum of the output activations corresponding to one input strip included in the input feature map, the digital computing circuit 430 can sequentially calculate the partial sums of the output activations corresponding to the remaining input strips. When the partial sums corresponding to all input strips included in the input feature map have been calculated, the digital computing circuit 430 can obtain the final output activation based on the calculated partial sums.

[0148] Simultaneously, the neural network device can selectively apply the activation function during or before the process of obtaining the floating-point output activation by combining the result of the MAC operation with the result of the addition operation in the digital computing circuit 430. In one example, when ReLU (where the output is 0 when the input is negative) is applied as the activation function, the neural network device can determine whether the output activation is negative based on the sign bit included in the result of the MAC operation, and when the output activation is determined to be negative, the neural network device can omit the shift and conversion operations and output the output activation as 0. Therefore, unnecessary calculations and operations can be omitted.

[0149] The floating-point output activation from the final output of the digital computing circuit 430 can be used as the input activation for the next layer. The above processing can be repeated in the next layer, and forward or backward passes can be performed along the layers of the neural network implemented by the neural network device. Furthermore, while forward or backward passes are being performed, training of the neural network implemented by the neural network device can be performed, or inference using the neural network can be performed.

[0150] After converting the input activations and weights into block floating-point format, the neural network device in this example performs operations on exponential numbers where precision is important, and uses an existing fixed-point based analog cross-switch array for operations on fractions that require a large number of operations, thus enabling floating-point operations to be performed with minimal loss of power and precision.

[0151] Figure 12 This is a flowchart illustrating methods for operating a neural network device according to some examples.

[0152] Reference Figure 12 Methods for operating neural network devices include Figure 4 The operation of time-series processing in neural network device 4 is shown in the diagram. Therefore, it can be seen that, for... Figures 4 to 11 The given description, even if omitted below, can still be applied to the operation. Figure 12 Methods for neural network devices.

[0153] In operation 1210, the neural network device may determine a first shared index representing the weights included in the weight kernel for each of the weight kernels, and obtain a first block of weights in floating-point format including a first fraction adjusted based on the first shared index.

[0154] In operation 1220, the neural network device may determine a second sharing index representing the input activation included in the input strip for each of the plurality of input strips included in the input feature map, and obtain a second block of floating-point format input activation including a second fraction adjusted based on the second sharing index.

[0155] In operation 1230, the neural network device can perform MAC operations on the first and second fractions using an analog cross switch array, and convert the result of the MAC operation into a digital signal using an ADC.

[0156] The neural network device can store a first score corresponding to each of the weights included in the weight kernel in memory cells arranged along multiple column lines of an analog crossbar array, corresponding to the column lines of the weight kernel. Furthermore, the neural network device can input a second score corresponding to each of the input activations included in the input strip to multiple row lines of the analog crossbar array. Therefore, MAC operations on the first and second scores can be performed in an analog manner.

[0157] According to some examples, when the analog crossbar array supports signed inputs, the neural network device can input the sign bit of the second block of floating-point formatted input along with a second fraction into multiple row lines. Therefore, by taking into account the sign bit of the input activation, the result of the MAC operation can be output from the analog crossbar array.

[0158] However, according to another example, when the analog crossbar array supports unsigned inputs, the neural network device can obtain the first current sum along each output in the column lines by preferentially activating only the row lines where the sign bit of the second block of floating-point formatted input is the first value. Subsequently, the neural network device can obtain the second current sum along each output in multiple column lines by activating only the row lines where the sign bit of the second block of floating-point formatted input is the second value. The first and second values ​​can be 0 or 1, and can have values ​​different from each other.

[0159] As described above, a neural network device can obtain a first current sum corresponding to positive input activation and a second current sum corresponding to negative input activation separately over at least two cycles. Furthermore, the method of combining the first and second current sums can vary depending on the configuration of the in-memory computing circuitry, including an analog crossbar switch array.

[0160] In one example, the neural network device may perform a digital conversion before combining the first and second current sums. The neural network device may convert the first current sum into a first digital signal using an ADC, and the second current sum into a second digital signal using an ADC. The neural network device may then combine the first and second digital signals using a digital accumulator to output a digital signal corresponding to the result of the MAC operation. In an example where the first value is 0 and the second value is 1, the neural network device may output the digital signal corresponding to the result of the MAC operation by subtracting the second digital signal from the first digital signal.

[0161] In another example, the neural network device can perform a combination of a first current sum and a second current sum before digital conversion. The neural network device can output a final current sum by combining the first and second current sums using an analog accumulator. In an example where the first value is 0 and the second value is 1, the neural network device can output the final current sum by subtracting the second current sum from the first current sum. The neural network device can then convert the final current sum into a digital signal corresponding to the result of a MAC operation using an ADC.

[0162] According to some examples, when the crossbar array supports signed weights, the neural network device can store the sign bit of the first block of weights in floating-point format along with a first fraction in a memory cell. Therefore, the result of the MAC operation can be output from the analog crossbar array by taking into account the sign bit of the weights.

[0163] According to another example, when the analog crossbar array supports unsigned weights, the neural network device can store a first fraction corresponding to each weight, where the sign bit of the first fraction is a first value, in memory cells arranged along the first column line of the analog crossbar array, and store a first fraction corresponding to each weight, where the sign bit of the first fraction is a second value, in memory cells arranged along the second column line of the analog crossbar array. The first and second values ​​can be 0 or 1, and can have different values.

[0164] The neural network device can output a final current sum by combining a first current sum of each output along the first column with a second current sum of each output along the second column. As described above, the analog cross-switch array included in the neural network device can operate as if it included two cross-switch arrays (i.e., a first cross-switch array where the sign bit of the weight is a first value and a second cross-switch array where the sign bit of the weight is a second value).

[0165] In operation 1240, the neural network device can perform addition operations on the first shared exponent and the second shared exponent using digital computing circuitry, and output the floating-point output activation portion included in the channel of the output feature map by combining the result of the MAC operation with the result of the addition operation.

[0166] For example, a neural network device can obtain a third fraction by performing a shift operation, such that the most significant bit of the result of the MAC operation is 1. Alternatively, the neural network device can obtain a third fraction by performing a transformation operation, in which the number of shift operations is added to or subtracted from the result of the addition operation. Therefore, the neural network device can output a partial sum of floating-point output activations that includes the third fraction and the third exponent.

[0167] After calculating the partial sum of the output activations corresponding to one input strip included in the input feature map, the neural network device can sequentially calculate the partial sums of the output activations corresponding to the remaining input strips. When the partial sums corresponding to all input strips included in the input feature map have been calculated, the neural network device can obtain the final output activations based on the calculated partial sums.

[0168] Neural network devices can selectively apply activation functions during or before the process of obtaining floating-point output activation by combining the result of a MAC operation with the result of an addition operation. In one example, when ReLU (where the output is 0 when the input is negative) is applied as the activation function, the neural network device can determine whether the output activation is negative based on the sign bit included in the result of the MAC operation. And when the output activation is determined to be negative, the neural network device can omit the shift and conversion operations and output the activation as 0. Therefore, unnecessary calculations and operations can be omitted.

[0169] In the operation method of the neural network device according to the example, after the input activations and weights are converted into block floating-point format, the operation of the exponent, which is important for precision, is performed digitally, and in the operation of fractions, which requires a lot of computation, an existing fixed-point based analog cross switch array is used, so that floating-point operations can be performed with minimal loss of power and precision.

[0170] at the same time, Figure 12 The operations described are only sequentially to explain the overall flow of the method of operating the neural network device; therefore, they do not necessarily need to be performed in the order described. For example, after performing operation 1210 for one input strip included in the input feature map and then performing operations 1220 to 1240, operation 1210 may not be performed again. Instead of performing operation 1210 again, operations 1220 to 1240 may be repeatedly performed for the remaining input strips included in the input feature map.

[0171] Furthermore, as operations 1220 to 1240 are performed on all input strips included in an input feature map, operation 1210 can be omitted even when floating-point operations begin to be performed on the output corresponding to the output activation of another input feature map, after the output has been activated for the corresponding input feature map. For example, in the process of performing floating-point operations on all input feature maps, operation 1210 may only be performed once at the first moment. However, this is only an example, and operation 1210 can be performed again at any time when changes to the weights used for floating-point operations are needed. In addition, each of operations 1220 to 1240 may be performed at an appropriate time as appropriate.

[0172] Figure 13 This is a block diagram illustrating the configuration of an electronic system based on some examples.

[0173] Reference Figure 13 The electronic system 1300 can extract useful information by analyzing input data in real time based on a neural network, and determine the status of the device on which the electronic system 1300 is installed or control the configuration of the device based on the extracted information. For example, the electronic system 1300 can be applied to robotic devices (such as drones or advanced driver assistance systems (ADAS)), smart TVs, smartphones, medical devices, mobile devices, image display devices, measuring devices, IoT (Internet of Things) devices, and can be installed on at least one of various types of electronic devices.

[0174] Electronic system 1300 may include processor 1310, RAM 1320, neural network device 1330, memory 1340, sensor module 1350, and communication module 1360. Electronic system 1300 may also include input / output module, security module, and power control device. Some hardware components of electronic system 1300 may be mounted on at least one semiconductor chip.

[0175] Processor 1310 controls the overall operation of electronic system 1300. Processor 1310 may include a single processor core (single-core) or multiple processor cores (multi-core). Processor 1310 can process or execute programs and / or data stored in memory 1340. In some examples, processor 1310 can control the function of neural network device 1330 by executing programs stored in memory 1340. Processor 1310 may be implemented by CPU, GPU, AP, etc.

[0176] RAM 1320 may temporarily store programs, data, or instructions. For example, programs and / or data stored in memory 1340 may be temporarily stored in RAM 1320 according to the control or startup code of processor 1310. RAM 1320 may be implemented as memory (such as dynamic RAM (DRAM) or static RAM (SRAM)).

[0177] The neural network device 1330 can perform neural network operations based on received input data and generate information signals based on the execution results. The neural network may include, but is not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), deep belief networks, restricted Boltzmann machines, etc. The neural network device 1330 may be a hardware accelerator dedicated to neural networks or a device including a hardware accelerator, and may correspond to the aforementioned neural network devices (e.g., Figure 4 4).

[0178] The neural network device 1330 can perform floating-point operations while utilizing in-memory computing circuitry that can significantly reduce power consumption. Furthermore, the neural network device 1330 can minimize the loss of precision by digitally processing operations concerning error-sensitive exponents.

[0179] The information signal may include one of various types of recognition signals (such as voice recognition signals, object recognition signals, image recognition signals, and biometric information recognition signals). For example, the neural network device 1330 may receive frame data included in a video stream as input data and generate recognition signals for objects included in an image displayed by the frame data based on the frame data. However, the configuration is not limited to this; the neural network device 1330 may receive various types of input data and generate recognition signals based on the input data, depending on the type or function of the electronic device on which the electronic system 1300 is installed.

[0180] The memory 1340 is a storage device for storing data, and may store an operating system (OS), various programs, and various types of data. In one embodiment, the memory 1340 may store intermediate results generated during processing that performs the operation of the neural network device 1330.

[0181] The memory 1340 may be DRAM, but is not limited thereto. The memory 1340 may include at least one of volatile memory and non-volatile memory. Non-volatile memory includes ROM, PROM, EPROM, EEPROM, flash memory, PRAM, MRAM, RRAM, FRAM, etc. Volatile memory includes DRAM, SRAM, SDRAM, PRAM, MRAM, RRAM, FeRAM, etc. In one embodiment, the memory 1340 may include at least one of HDD, SSD, CF, SD, micro SD, mini SD, xD, and Memory Stick.

[0182] Sensor module 1350 can collect information about the vicinity of the electronic device on which electronic system 1300 is installed. Sensor module 1350 can sense or receive signals (e.g., image signals, voice signals, magnetic signals, biosignals, touch signals, etc.) from outside the electronic device and convert the sensed or received signals into data. For this purpose, sensor module 1250 may include at least one of various types of sensing devices (e.g., microphone, imaging device, image sensor, LiDAR sensor, ultrasonic sensor, infrared sensor, biosensor, and touch sensor).

[0183] Sensor module 1350 can provide the converted data as input data to neural network device 1330. For example, sensor module 1350 may include an image sensor that generates a video stream by capturing images of the external environment of the electronic device and sequentially provides the continuous data frames of the video stream to neural network device 1330 as input data. However, the configuration is not limited to this, and sensor module 1350 can provide various types of data to neural network device 1330.

[0184] The communication module 1360 may include various wired or wireless interfaces capable of communicating with external devices. For example, the communication module 1360 may include a local area network (LAN), a wireless LAN (WLAN) (such as Wi-Fi), a wireless personal area network (WPAN) (such as Bluetooth), a wireless universal serial bus (USB), ZigBee, near field communication (NFC), radio frequency identification (RFID), power line communication (PLC), or a communication interface capable of connecting to a mobile cellular network (such as third-generation (3G), fourth-generation (4G), or long-term evolution (LTE)).

[0185] The example can be implemented as a computer-readable program and can be implemented in a general-purpose computer that executes the program using a non-transitory computer-readable recording medium. Furthermore, the structure of the data used in the embodiments of the inventive concept can be recorded on a non-transitory computer-readable recording medium by various means. The non-transitory computer-readable medium can be a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.) and an optical recording medium (e.g., CD-ROM or DVD).

[0186] While this disclosure includes specific examples, it will be clear upon understanding this disclosure that various changes in form and detail may be made to these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein are to be considered descriptive only and not for limiting purposes. The description of features or aspects in each example is to be considered applicable to similar features or aspects in other examples. Suitable results may be achieved if the described techniques are performed in a different order, and / or if components in the described system, architecture, apparatus, or circuit are combined in a different manner and / or replaced or supplemented by other components or their equivalents. Therefore, the scope of the disclosure is not limited by the specific embodiments but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents shall be construed as included in the disclosure.

Claims

1. A method of operating a neural network device that performs floating-point operations, wherein the floating-point value is specified by a sign bit, a fraction, and an exponent, the method comprising: For each of the multiple weight kernels, a first shared index is determined to represent a set of weight kernels and the weights included in that set of weight kernels, and a first block of converted weights in floating-point format is obtained, each converted weight including a first fraction adjusted based on the first shared index, wherein the number of sets of weight kernels is determined in advance through simulation. For each of the multiple input strips included in the input feature map, a second shared index is determined to represent the input activations included in the input strip, and a transformed input activation in a second block of floating-point format is obtained, each transformed input activation including a second score adjusted based on the second shared index; An analog crossbar switch array is used to perform a multiply-accumulate operation on a first fraction for each transformed weight and a second fraction for each transformed input activation, and an analog-to-digital converter is used to convert the result of the multiply-accumulate operation into a digital signal; and A digital computing circuit is used to perform an addition operation on the first shared exponent and the second shared exponent, and the digital signal from the result of the multiplication-accumulation operation is combined with the result of the addition operation to output the sum of the floating-point output activation portions included in the channel of the output feature map. The step of performing a multiplication and accumulation operation of a first score for each transformed weight and a second score for each transformed input activation using an analog crossbar array includes storing the first score corresponding to each of the transformed weights included in each weight kernel in a memory cell arranged along a plurality of column lines of the analog crossbar array corresponding to each weight kernel.

2. The operating method according to claim 1, wherein, The steps of performing the multiplicative summation of the first score for each transformed weight and the second score for each transformed input activation using an analog crossbar array also include: The second fraction corresponding to each of the transformed input activations included in each input strip is input into multiple rows of the analog crossbar array.

3. The operating method according to claim 2 further includes: When the analog crossbar switch array supports signed input, the sign bit of the second block of floating-point format input is activated and input together with the second fraction to the multiple row lines.

4. The operating method according to claim 2 further includes: When the analog crossbar switch array supports unsigned inputs The first current and sum of the outputs along each of the plurality of column lines are obtained by activating only the row lines in which the sign bit of the second block of floating-point format input is activated with a first value; and The second current sum along each of the plurality of column lines is obtained by activating only the row line where the sign bit of the second block of floating-point format input is activated to a second value.

5. The operating method according to claim 4 further includes: The first current is converted into a first digital signal using an analog-to-digital converter; The second current is converted into a second digital signal using an analog-to-digital converter; and By using a digital accumulator to combine the first digital signal and the second digital signal, a digital signal corresponding to the result of the multiplication-accumulation operation is output.

6. The operating method according to claim 4 further includes: The final current sum is output by combining the first current sum with the second current sum using an analog accumulator; and The final current is converted into a digital signal corresponding to the result of the multiplication and accumulation operation using an analog-to-digital converter.

7. The operating method according to claim 2, In the case that the analog crossbar array supports signed weights, the steps for storing the first score include: The sign bit of the weight in the first floating-point format is stored together with the first fraction in the memory cell.

8. The operating method according to claim 2, further comprising: When the analog cross switch array supports unsigned weights The first score corresponding to each of the weights in which the sign bit of the first score is the first value is stored in a memory cell arranged along the first column line of the analog crossbar switch array; The first score corresponding to each of the weights in which the sign bit of the first score is the second value is stored in a memory cell arranged along the second column line of the analog crossbar switch array; and The final current sum is output by combining the first current sum of each output along the first column with the second current sum of each output along the second column.

9. The operating method according to any one of claims 1 to 8, wherein, The steps for outputting the partial sum include: The third fraction is obtained by performing a shift operation on the result of the multiplication-accumulation operation to make the most significant bit 1; A third exponent is obtained by performing a conversion operation in which the number of shift operations performed is added to the result of the addition operation or subtracted from the result of the addition operation; and The output includes the sum of the floating-point outputs of the third fraction and the third exponent.

10. The operating method according to claim 9, further comprising: The sign bit included in the result of the multiplication-accumulation operation is used to determine whether the floating-point output activation is negative; and In response to the floating-point output activation being determined to be negative, the shift operation and the conversion operation are omitted, and the floating-point output activation is output as 0.

11. A neural network device for performing floating-point operations, wherein the floating-point value is specified by a sign bit, a fraction, and an exponent, the neural network device comprising: At least one control circuit is configured to: for each of a plurality of weight kernels, determine a first shared index representing a set of weight kernels and the weights included in that set of weight kernels; obtain converted weights in a first block of floating-point format, each converted weight including a first fraction adjusted based on the first shared index; wherein the number of sets of weight kernels is determined in advance through simulation; and for each of a plurality of input strips included in an input feature map, determine a second shared index representing the input activations included in the input strips; and obtain converted input activations in a second block of floating-point format, each converted input activation including a second fraction adjusted based on the second shared index. The in-memory computing circuitry includes an analog crossbar switch array and an analog-to-digital converter (ADC). The analog crossbar switch array is configured to perform a multiplicative-accumulator operation on a first fraction for each converted weight and a second fraction for each converted input activation. The ADC is configured to convert the result of the multiplicative-accumulator operation into a digital signal. A digital computing circuit is configured to perform an addition operation on a first shared exponent and a second shared exponent, and to output a portion of the floating-point output activation included in the channel for calculating the output feature map by combining a digital signal from the result of the multiplication-accumulation operation with the result of the addition operation. The analog crossbar switch array includes multiple row lines, multiple column lines intersecting the row lines, and multiple memory cells located at the intersections of the row lines and column lines. The at least one control circuit is configured to store a first score corresponding to each of the transformed weights included in each weight kernel in a memory cell arranged along the column lines corresponding to each weight kernel among the plurality of column lines.

12. The neural network device according to claim 11, in, The at least one control circuit is configured to: Each corresponding second fraction in the transformed input activation included in each input strip is input into the plurality of line lines.

13. The neural network device according to claim 12, wherein, In the case where the analog crossbar switch array supports signed input, the at least one control circuit is configured to input the sign bit of the second block of floating-point format input along with the second fraction to the plurality of row lines.

14. The neural network device according to claim 12, wherein, In the case where the analog crossbar switch array supports unsigned inputs, the at least one control circuit is configured to: obtain a first current sum along each of the plurality of column lines by activating only the row lines where the sign bit activated by the second block of floating-point format inputs is a first value, and obtain a second current sum along each of the plurality of column lines by activating only the row lines where the sign bit activated by the second block of floating-point format inputs is a second value.

15. The neural network device according to claim 14, wherein, The analog-to-digital converter is configured as follows: The first current is converted into a first digital signal, and the second current is converted into a second digital signal. The neural network device further includes a digital accumulator configured to output a digital signal corresponding to the result of the multiplication-accumulation operation by combining a first digital signal and a second digital signal.

16. The neural network device according to claim 14, wherein, The neural network device also includes: An analog accumulator is configured to output a final current sum by combining a first current sum with a second current sum; and An analog-to-digital converter is configured to convert the final current sum into a digital signal corresponding to the result of the multiplication-accumulation operation.

17. The neural network device according to claim 12, wherein, In the case where the analog crossbar switch array supports signed weights, the at least one control circuit is configured as follows: The sign bit of the weight in the first floating-point format is stored together with the first fraction in the memory cell.

18. The neural network device according to claim 12, wherein, When the analog crossbar switch array supports unsigned weights, the at least one control circuit is configured as follows: The first fraction corresponding to each of the weights in which the sign bit is a first value is stored in memory cells arranged along the first column line of the analog crossbar switch array, and the first fraction corresponding to each of the weights in which the sign bit is a second value is stored in memory cells arranged along the second column line of the analog crossbar switch array; and The in-memory computing circuitry is configured as follows: The final current sum is output by combining the first current sum of each output along the first column with the second current sum of each output along the second column.

19. The neural network device according to any one of claims 11 to 18, wherein, The digital computing circuit is configured as follows: The third fraction is obtained by performing a shift operation on the result of the multiplication-accumulation operation to make the most significant bit 1, the third exponent is obtained by performing a conversion operation, and a partial sum of floating-point output activation including the third fraction and the third exponent is obtained, in which the number of shift operations performed is added to the result of the addition operation or subtracted from the result of the addition operation.

20. The neural network device according to claim 19, wherein, The digital computing circuit is configured as follows: The sign bit included in the result of the multiplication-accumulation operation is used to determine whether the floating-point output activation is negative, and in response to the floating-point output activation being determined to be negative, the shift operation and the conversion operation are omitted, and the floating-point output activation is output as 0.

21. A neural network device, comprising: At least one control circuit is configured to: for each of a plurality of weight kernels, output a first shared index representing a set of weight kernels and the weights included in the set of weight kernels, and a first fraction of the weights that has been adjusted by a shift operation; and for each of a plurality of input strips included in an input feature map, output a second shared index representing the input activations included in the input strip and a second fraction corresponding to the input activations included in the input strip. The in-memory computing circuitry is configured to perform a multiplication-accumulation operation on a first fraction and a second fraction, and to convert the result of the multiplication-accumulation operation into a digital signal. and A digital computing circuit is configured to perform an addition operation on a first shared exponent and a second shared exponent, and to output a portion of the floating-point output activation included in the channel for calculating the output feature map by combining a digital signal with the result of the addition operation. The in-memory computing circuitry includes an array of analog cross switches configured to perform the multiply-accumulate-add operation, and The at least one control circuit is configured to store a first score corresponding to each of the weights included in each weight kernel in a memory cell arranged along multiple column lines of an analog crossbar array, corresponding to the column lines of each weight kernel.

22. The neural network device according to claim 21, wherein, The first shared index is the largest among the existing indices of the weights.

23. The neural network device according to claim 21, wherein, The in-memory computing circuitry also includes an analog-to-digital converter configured to convert the result of the multiplication-accumulation operation into a digital signal.

24. The neural network device according to any one of claims 21 to 23, further comprising: The accumulator is configured to combine a first signal corresponding to positive input activation and a second signal corresponding to negative input activation.

25. The neural network device according to claim 24, wherein, An accumulator is an analog accumulator included in the computing circuitry within memory.

26. The neural network device according to claim 24, wherein, An accumulator is a digital accumulator included in digital computing circuits.

Images