Computing device and method

Abstract

At least one embodiment provides a computing device, comprising: a controller, which receives first input data of a first data type and second input data of a second data type different from the first data type, and outputs a first signal representing the first data type, a second signal representing the second data type, and a clock signal based on the number of bits of the first input data and the second input data; and a computing circuit, which performs a multiplication calculation of the first input data and the second input data based on the first signal, the second signal, and the clock signal, and generates output data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefits of Korean Patent Application No. 10-2023-0133410 filed on October 6, 2023, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a computing device and method. Background Art

[0004] Deep learning, a field of artificial intelligence (AI), identifies patterns in complex data and enables refined predictions. Generally, deep learning includes a training phase in which a neural network model is trained using training data and an inference phase in which new data is input into the trained neural network model to obtain output. Although the performance of deep learning can be improved by achieving more refined predictions as the neural network is built deeper, such improvements generally require more power and slow down the model due to the huge amount of computation.

[0005] At the same time, neural networks can include many layers for refined predictions, which can further increase computational complexity. Therefore, to reduce computational complexity, algorithm lightweighting techniques are being used, such as mapping floating point data to integers. Summary of the invention

[0006] At least one embodiment provides a computing device and method configured to perform multiplication-accumulation calculations on input data having different data types.

[0007] At least one embodiment provides a computing device and method configured to perform multiplication-accumulation calculations on input data with different precisions.

[0008] At least one embodiment provides a computing device, comprising: a processing circuit, configured to: receive first input data of a first data type and second input data of a second data type different from the first data type; output a first signal representing the first data type, a second signal representing the second data type, and a clock signal based on the number of bits of the first input data and the second input data; and perform a multiplication calculation based on the first signal, the second signal, and the clock signal, and generate output data, so that a result of the multiplication calculation is based on the first input data and the second input data.

[0009] Another embodiment provides a computing device configured to perform a multiplication calculation on first data of a first digit and second data of a second digit different from the first digit, the computing device comprising: a first multiplexer including a first selection node and a plurality of input nodes, the first selection node being configured to receive an output signal of a counter, the counter being configured to count the number of cycles of a clock signal based on the first digit, and the plurality of input nodes being configured to receive respective bit values ​​of the first data; a second multiplexer including a second selection node, a first input node, and a second input node, the second selection node being configured to receive the output data of the first multiplexer, the first input node being configured to receive the first value, and the second input node being configured to receive one of the second data or data obtained by shifting the second data by one (1) bit from a shifter; and a first adder configured to add the output data of the second multiplexer based on the clock signal.

[0010] Another embodiment provides a neural processing system, comprising: a non-temporary memory; and a processing circuit configured to receive an input matrix and a weight matrix corresponding to input data from the memory, perform a matrix dot product calculation on the input matrix and the weight matrix, and output a result of performing the matrix dot product calculation as an output matrix, wherein the input matrix includes elements of a first data type and the weight matrix includes elements of a second data type different from the first data type, and the processing circuit includes a computing device, the computing device being configured to output one of exponential data of the elements of the input matrix or exponential data of the elements of the weight matrix as exponential data of the elements of the output matrix based on the first data type and the second data type. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A block diagram of a neural processing system is shown in accordance with at least one embodiment.

[0012] Figure 2 is a diagram for explaining neural network calculations according to at least one embodiment.

[0013] 3(A) and 3(B) are diagrams for explaining the input data format.

[0014] Figure 4 A calculation circuit according to a comparative example is shown.

[0015] Figure 5 is a flowchart for explaining a method of operating a neural processing unit according to at least one embodiment.

[0016] Figure 6 A schematic block diagram of a computing device according to at least one embodiment is shown.

[0017] Figure 7A schematic block diagram of a computing circuit according to at least one embodiment is shown.

[0018] Figure 8 An index calculation section according to at least one embodiment is shown.

[0019] Fig. 9 A mantissa calculation unit according to at least one embodiment is shown.

[0020] Fig.10 Is used to explain the Fig. 9 A diagram showing the calculation method of the decimal calculation part.

[0021] Fig.11 A timing diagram illustrating calculations performed in a mantissa calculation section according to at least one embodiment is shown.

[0022] Fig.12 A timing chart showing calculations performed in a fraction calculation section according to another embodiment is shown.

[0023] Fig.13 is an example diagram for explaining multi-head attention according to at least one embodiment.

[0024] Fig.14 It is used to explain Fig.13 Example graph of scaled dot product attention. DETAILED DESCRIPTION

[0025] The present disclosure will be described more fully below with reference to the accompanying drawings, in which embodiments of the present disclosure are shown. As will be appreciated by those skilled in the art, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present disclosure. The functional elements and blocks shown in the accompanying drawings may be implemented in processing circuits such as software, hardware, or a combination thereof. For example, more specifically, the processing circuit may include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a system-on-chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), and the like. For example, software may be machine code, firmware, embedded code, application software, and the like. For example, hardware may include electrical circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, passive components, combinations thereof, and the like.

[0026] Accordingly, the drawings and description are to be considered illustrative rather than restrictive in nature. Throughout the specification, the same reference numerals designate the same elements. In the flow charts described with reference to the drawings in this specification, the order of operations may be changed, various operations may be combined, certain operations may be divided, and certain operations may not be performed.

[0027] In addition, singular forms may be intended to include plural forms as well, unless explicit expressions such as "one" or "single" are used. Terms including ordinal numbers such as first and second will only be used to describe various constituent elements and should not be interpreted as limiting these constituent elements. These terms may be used for the purpose of distinguishing one constituent element from other constituent elements.

[0028] Figure 1 A block diagram of a neural processing system according to at least one embodiment is shown. The neural processing system 100 can be configured to train (or learn) a neural network and / or infer information included in the input data by analyzing the input data using the neural network. For example, the neural processing system 100 can use the trained neural network to determine a condition based on the inferred information, and / or control components of an electronic device on which the neural processing system 100 is installed based on the determination. For example, the neural processing system 100 can be applied to devices that perform speech recognition, image recognition, and image classification using neural networks, smart phones, tablet devices, smart TVs, augmented reality (AR) devices, Internet of Things (IoT) devices, autonomous vehicles, robots, medical devices, drones, advanced drivers assistance systems (ADAS), image display devices, measuring devices, etc. In addition, the neural processing system 100 can be installed on one of various types of electronic devices.

[0029] In at least one embodiment, the neural processing system 100 may include a central processing unit (CPU) 110, a neural processing unit (NPU) 120, a volatile memory device 130, a non-volatile memory device 140, and a user interface 150. In at least one embodiment, some or all components of the neural processing system 100 (e.g., the central processing unit 110, the neural processing unit 120, the volatile memory device 130, and the non-volatile memory device 140) may be formed on one semiconductor chip. For example, the neural processing system 100 may be implemented as a system on chip (SoC). The central processing unit 110, the neural processing unit 120, the volatile memory device 130, the non-volatile memory device 140, and the user interface 150 may be connected to each other via a bus (BUS).

[0030] The central processing unit 110 is configured to control the overall operation of the neural processing system 100. The central processing unit 110 may include one processor core (single core) or multiple processor cores (multi-core). The central processing unit 110 may process or run programs and / or data stored in storage areas such as the volatile memory device 130 and the non-volatile memory device 140. The central processing unit 110 may also be referred to as a central processor.

[0031] The central processing unit 110 can run the application and control the neural processing unit 120 to perform tasks based on neural networks required according to the operation of the application. The neural network may include at least one of various types of neural network models, such as convolution neural network (CNN), region with convolution neural network (R-CNN), region proposal network (RPN), recurrent neural network (RNN), stacking-based deep neural network (S-DNN), state-space dynamic neural network (S-SDNN), deconvolution network, deep belief network (DBN), restricted Boltzmann machine (RBM), fully convolutional network, long short-term memory network (LSTM), classification network, transformer, etc.

[0032] The neural network model may include multiple layers. Each of the multiple layers may be configured to receive input data of the layer and generate output data of the layer. In these cases, except for the final layer (also referred to as the output layer), the output data generated by the layer may be used as input data of another layer. Each of the multiple layers may transform the input data into output data through a neural network calculation. For example, each of the multiple layers may generate an output matrix corresponding to the output data of the layer by performing a dot product calculation on a weight matrix corresponding to the input data of the layer and an input matrix. However, the scope of the present disclosure is not limited thereto, and each of the multiple layers may generate output data by transforming an input matrix corresponding to the input data of the layer. For example, each of the multiple layers may also be implemented to generate output data of the layer by multiplying an input matrix corresponding to the input data of the layer by multiple weight matrices in sequence, or to transform the input matrix based on transformation parameters to generate output data of the layer. Reference will be made to Figure 2 A more detailed description of neural networks and layers.

[0033] Neural processing unit 120 may be configured to access data (e.g., data stored in volatile memory device 130) to run neural network calculations. For example, neural processing unit 120 may read parameters stored in volatile memory device 130 to perform calculations on layer input data, and / or temporarily store intermediate data generated during calculations in volatile memory device 130. Neural processing unit 120 may also be referred to as a neural processor.

[0034] The neural processing unit 120 can access the non-volatile memory device 140 to run the neural network calculation. For example, the neural processing unit 120 can perform the neural network calculation by reading the calculation parameters (e.g., weight values ​​and bias values) and input data (e.g., input feature maps) of the neural network stored in the non-volatile memory device 140.

[0035] In at least one embodiment, the neural processing unit 120 may include a computing device 121 for performing neural network calculations. For example, the computing device 121 may perform a dot product calculation on the input matrix and the weight matrix of each layer. The computing device 121 may include, for example, a multiplication and accumulation (MAC) operator for performing a dot product calculation on the input matrix and the weight matrix of each layer. The computing device 121 may perform a multiplication and accumulation calculation on the input data. The neural processing unit 120 may be a data-driven parallel computing device, so that the neural processing unit 120 may be configured to perform at least multiple dot product calculations in parallel.

[0036] Meanwhile, the neural network may be implemented with a complex architecture, and the computing device 121 of the neural processing unit 120 that performs the neural network calculations performs hundreds of millions to tens of billions of calculations. Therefore, lightweight techniques such as mapping floating point data to integer data are being used to reduce the burden of the neural network algorithm. In at least one embodiment, the computing device 121 may receive a plurality of data having different data types as input data. For example, the computing device 121 may receive floating point data and integer data as input data, but is not limited thereto.

[0037] In at least one embodiment, computing device 121 may receive multiple data with different precisions (eg, different bits) as input data. For example, computing device 121 may receive data bits within a predetermined number of bits that computing device 121 may calculate as input data.

[0038] In at least one embodiment, the volatile memory device 130 may be used as a buffer memory, an operating memory, and / or a cache memory of the central processing unit 110. The volatile memory device 130 may include, for example, dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), phase-change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM), ferroelectric RAM (FeRAM), etc. However, the scope of the present disclosure is not limited to this.

[0039] The non-volatile memory device 140 may store data for the operation of the neural processing system 100. For example, the non-volatile memory device 140 may store an operating system (OS) of the neural processing system 100, calculation parameters of a neural network (e.g., weight values, bias values, etc.), parameters for quantization of the neural network (e.g., scale factors, bias values, etc.), input data (e.g., input feature maps), and output data (e.g., output feature maps). The calculation parameters, quantization parameters, input data, and output data may be floating point data and / or integer data. The non-volatile memory device 140 includes a read only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM), a flash memory, a phase change RAM (PRAM), a magnetic RAM (MRAM), a resistive RAM (RRAM), a ferroelectric RAM (FeRAM), and the like. However, the scope of the present disclosure is not limited thereto.

[0040] In at least some embodiments, volatile memory 130 and non-volatile memory 140 may also be referred to as non-transitory memory and / or non-transitory storage media. The term "non-transitory" as used herein is a description of the media itself (e.g., tangible rather than a signal), rather than a limitation on data storage persistence (e.g., RAM versus ROM).

[0041] The central processing unit 110 may communicate with the user through the user interface 150. The central processing unit 110 may provide the input data provided by the user to the volatile memory device 130 or the neural processing unit 120 through the user interface 150. The user interface 150 may include, for example, a direct user interface, such as a touch screen, a touch pad, a keyboard, a microphone, etc., and / or an indirect user interface, such as a wireless interface including, for example, an antenna and / or a wireless transceiver and / or a communication interface protocol of a third generation communication system (such as code division multiple access (CDMA), global system for mobile communications (GSM), North American digital cellular (NADC), extended-time division multiple access (E-TDMA) and / or wide band code division multiple access (WCDMA), etc.) to communicate (directly and / or indirectly) with a device having a direct user interface. The central processing unit 110 may return output data generated by the artificial intelligence model based on the input data to the user through the user interface 150.

[0042] Figure 2 is a diagram for explaining neural network calculations according to at least one embodiment.

[0043] refer to Figure 2 , the neural network 200 may include a plurality of layers. For example, the neural network 200 may include an input layer 210, one or more hidden layers 220, and an output layer 230. The input layer 210 may correspond to a vector and / or a matrix representing the input data 201 of the neural network 200. For example, the vector representing the input data may have elements corresponding to the number of nodes included in the input layer 210. For example, the elements included in the matrix representing the input data may correspond to each node included in the input layer 210. The signal generated by the input data 201 at each node in the input layer 210 may be transmitted from the input layer 210 to the hidden layer 220. The output layer 230 may generate the output data 203 of the neural network 200 based on one or more signals received from the hidden layer 220. For example, the output data 203 may correspond to a vector and / or a matrix having elements corresponding to the number of nodes included in the output layer 230.

[0044] The data generated in each layer before the output layer 230 may be the output of one layer and the input parameters of the nodes included in the next layer. At the same time, each node in the layer may determine the output data based on the data and weights received from the nodes included in the previous layer. The weight is a parameter for calculating the output data at each node and may be a value assigned to the connection relationship between the nodes. The output data generated from the nodes included in the layer may be data generated by calculating the dot product of the data input to the node and the weight.

[0045] However, if Figure 1 As described, the data input to each layer can be of multiple data types. The electronic device 121 according to at least one embodiment can receive multiple data with different data types as input data, and perform dot product calculations on the input data. For example, the electronic device 121 according to at least one embodiment can perform dot product calculations on an input matrix of floating point data and a weight matrix converted into integers for lightweight quantization.

[0046] like Figure 1 As described above, the data input to each layer may be different bits. The electronic device 121 according to at least one embodiment may receive a plurality of data with different bits as input data, and perform dot product calculation on the input data. For example, the electronic device 121 may perform dot product calculation on 3-bit data and 2-bit data.

[0047] To compensate for different precisions in different data types, for example, the electronic device 121 according to at least one embodiment may generate a clock signal based on the input data bits, and perform a dot product calculation based on the corresponding clock signal. For example, the multiplication result for the input data may be derived by shifting and adding the data based on the clock signal.

[0048] Therefore, the electronic device 121 according to at least one embodiment can improve the performance of the computing device and can implement lightweighting of the neural network model algorithm based on reducing input data while maintaining the accuracy of the data.

[0049] 3(A) and 3(B) are diagrams for explaining input data formats. Specifically, FIG3(A) is a diagram for explaining a floating point data format processed by a computing device, and FIG3(B) is a diagram for explaining integer data.

[0050] 3(A), floating point data may have various forms depending on the precision, but the most commonly used method is the Institute of Electrical and Electronics Engineering (IEEE) 754 standard method. The IEEE 754 standard method divides a real number into a sign, an exponent, and a mantissa. For example, a floating point number may be expressed as Equation 1.

[0051] (Equation 1)

[0052]

[0053] In single precision, floating point data uses a total of 32 bits. Specifically, the sign is 1 bit, which indicates the sign of the data, when the data is positive, the sign is 0, and when the data is negative, the sign is 1. The exponent is 8 bits and indicates the exponent, and the mantissa is 23 bits and indicates the mantissa or significant digits.

[0054] For example, -314.625 can be expressed in IEEE 754 floating point format as follows.

[0055] First, s becomes 1 because of its negative sign.

[0056] Regarding the mantissa, when the absolute value of the number 314.625 is expressed as a binary number, it becomes In this case, when the decimal point is moved to the left so that only 1 is left to the left of the decimal point, it can be expressed as follows, which is called a normalized expression method.

[0057]

[0058] Here, the mantissa becomes the part to the right of the decimal point, i.e. 00111010101.

[0059] Regarding the exponent, in the normalized expression of -314.625, the exponent is 8, and adding the bias value of 127 to the exponent 8 becomes 135. In this case, when 135 is converted to binary, it becomes , and that value becomes the exponential.

[0060] Here, single precision is described as an example, and in half precision or double precision, the bits and bias values ​​of the exponent and mantissa may be different.

[0061] Meanwhile, referring to FIG3(B), integer data may be expressed in various types depending on the presence or absence of a sign and the data size (number of bits). As an example, an integer generally used in a calculation operation may be expressed as eight-bit data (two's complement) including a sign, as shown in FIG3(B), where the most significant bit represents a sign, the following seven bits represent an integer, and its size may be represented by Equation 2:

[0062] (Equation 2)

[0063]

[0064] When the data is an unsigned number, it is a positive number, and its size can be expressed by Equation 3:

[0065] (Equation 3)

[0066]

[0067] Figure 4 A calculation circuit according to a comparative example is shown. Specifically, the calculation circuit 400 according to the comparative example can perform multiplication-accumulation calculations on floating-point data.

[0068] refer to Figure 4 , the calculation circuit 400 according to the comparative example may receive the first input data (IN1[31:0]) and the second input data (IN2[31:0]), and may perform a multiplication-accumulation calculation on the input data. In one example, when the first input data IN1[31:0] and the second input data IN2[31:0] are 32-bit floating-point data, the first input data IN1[31:0] and the second input data IN2[31:0] may include 1-bit sign data S_IN1

[31] and S_IN2

[31] , 8-bit exponent data E_IN1[30:23] and E_IN2[30:23], and 23-bit mantissa data M_IN1[22:0] and M_IN2[22:0], respectively.

[0069] The calculation circuit 400 according to the comparative example may include a multiplication section 401 and an accumulation section 402. The multiplication section 401 may include a sign calculation section 410, an exponent calculation section 420, and a mantissa calculation section 430, and the accumulation section 402 may include an adder 450 and an accumulator 460.

[0070] Referring to the multiplication section 401, the sign calculation section 410 may include an exclusive OR (hereinafter referred to as an XOR) gate. The sign calculation section 410 may receive first sign data S_IN1

[31] of the first input data IN1 and second sign data S_IN2

[31] of the second input data IN2. When both the first sign data S_IN1

[31] and the second sign data S_IN2

[31] have a value "0" representing a positive number or a value "1" representing a negative number, the XOR gate of the sign calculation section 410 may output "0" representing a positive number. On the other hand, when one of the first sign data S_IN1

[31] and the second sign data S_IN2

[31] has a value "0" representing a positive number and the other has a value "1" representing a negative number, the XOR gate of the sign calculation section 410 may output "1" representing a negative number. The sign calculation section 410 may output 1-bit sign data OUT

[31] of the output data (OUT) as data generated as a result of the XOR calculation.

[0071] The exponential calculation section 420 may include a first exponential adder 421 and a second exponential adder 423. The first exponential adder 421 may receive first exponential data E_IN1[30:23] of the first input data IN1 and second exponential data E_IN2[30:23] of the second input data IN2. The first exponential adder 421 may perform a first addition calculation on the first exponential data E_IN1[30:23] and the second exponential data E_IN2[30:23]. The first exponential adder 421 may output first addition result data generated as a result of the first addition calculation. The first exponential data E_IN1[30:23] and the second exponential data E_IN2[30:23] are both added with an exponential deviation value (e.g., "127"). Since the exponential deviation value is doubled by the first addition calculation in the first exponential adder 421, it is necessary to subtract the exponential deviation value from the first addition result data. Accordingly, the second exponential adder 423 may receive the first addition result data output from the first exponential adder 421, and may perform a second addition calculation in which an exponent bias value (Bias), for example, "127", is subtracted from the first addition result data. The second exponential adder 423 may output data generated as a result of the second addition calculation as 8-bit exponent data (E_OUT[7:0]).

[0072] The mantissa calculation unit 430 may include a multiplier 431. The multiplier 431 may receive first mantissa data M_IN1[22:0] of the first input data IN1 and second mantissa data M_IN2[22:0] of the second input data IN2. The multiplier 431 may perform a multiplication calculation on the first mantissa data M_IN1[22:0] and the second mantissa data M_IN2[22:0]. The multiplier 431 may output data generated as a result of the multiplication calculation as 48-bit mantissa data M_OUT[47:0].

[0073] The normalizer 440 may normalize the exponential data E_OUT[7:0] output from the exponent calculation section 420 and the mantissa data M_OUT[47:0] output from the mantissa calculation section 430. Normalization may refer to shifting the calculated data so that the most significant bit becomes 1. The normalized data OUT[30:0] may be output to the accumulation section 402 as a multiplier result value (OUT) together with the sign data OUT

[31] output from the sign processing circuit 410.

[0074] Referring to the accumulation section 402 , the adder 450 performs an accumulation calculation by adding the data received from the multiplication section 401 and the data received from the accumulator 460 .

[0075] As described above, the calculation circuit 400 according to the comparative example can perform dot product calculation on a matrix by performing multiplication-accumulation calculation on input data in the form of a matrix. However, the calculation circuit 400 according to the comparative example can receive only floating-point data as input data.

[0076] Meanwhile, when the multiplier 400 receives floating-point data as input data, the floating-point data is composed of a predetermined precision, and when the size of the mantissa data of the floating-point data (i.e., the number of bits) is smaller than the size of the predetermined mantissa, the remaining bits are filled with "0". The problem here is that, although the size of the input data becomes smaller, the performance of the multiplier (e.g., computing device) is not improved because the multiplier still performs calculations on all values ​​of the mantissa.

[0077] Figure 5 is a flowchart for explaining a method of operating a neural processing unit according to at least one embodiment. Figure 5 The neural network processing unit 120 (see Figure 1 ) to execute.

[0078] refer to Figure 5 , the controller in the neural processing unit 120 receives input data from the memory device 140 (S510). The input data may include input feature values, weight values, function coefficients, etc. required for neural network calculation. In addition, when the neural network device 120 processes a quantized neural network, the input data may include quantization parameters. For example, the quantization parameters may include a proportional value (or an inverse proportional value), a bias value, etc.

[0079] In at least one embodiment, the controller in the neural processing unit 120 outputs data type information indicating the data type of the input data (S520). In at least one embodiment, the input data may be the same and / or different data types. For example, the input data may be floating point data; and / or the input data may be integer data. The input data may be floating point data and integer data, but is not limited thereto. Figures 6 to 8 Describes data type information.

[0080] In at least one embodiment, the controller in the neural network device 120 outputs a clock signal based on the number of bits of the input data (S530). In at least one embodiment, the neural network device 120 may receive the input data and count the bits of the input data. In at least one embodiment, the controller in the neural network device 120 may output a clock signal equal to the number of bits of the input data. In at least one embodiment, the controller in the neural network device 120 may output a clock signal equal to the number of bits of one of the input data. In at least one embodiment, the neural network device 120 may not receive new input data until a clock cycle output based on the bit information of the input data has passed.

[0081] In at least one embodiment, the neural network device 120 performs a multiplication-accumulation calculation on the input data (S540). The neural network device 120 may include an adder and a shifter to perform the multiplication calculation. The neural network device 120 may perform an exponent calculation based on the data type information. The neural network device 120 may perform a mantissa calculation based on a clock signal.

[0082] In at least one embodiment, the neural network device 120 stores the final data generated in operation S540 in the memory 140 ( S550 ).

[0083] Figure 6 A schematic block diagram of a computing device according to at least one embodiment is shown. Figure 6 The computing device 600 can perform dot product calculations on the input data. Figure 6 The computing device 600 in FIG. 6 may be a MAC computing device for performing multiplication-accumulation calculations on input data.

[0084] In at least one embodiment, computing device 600 may include a controller 610 and computing circuitry 620 .

[0085] In at least one embodiment, the computing device 600 may receive input data IN1 and IN2 of the same or different types. The controller 610 may output data type signals DS1 and DS2 indicating the data types of the input data IN1 and IN2. For example, when the first input data IN1 is integer data, the first data type signal DS1 may output a "low" signal, and when the second input data IN2 is floating point data, the second data type signal DS2 may output a "high" signal, but the present disclosure is not limited thereto. In at least one embodiment, the computing circuit 620 may perform exponential calculations based on the data type signals DS1 and DS2 of the input data IN1 and IN2. This will be referred to later. Figure 8 The exponential calculation of the calculation circuit 620 is described.

[0086] In at least one embodiment, the controller 610 can output the clock signal CLK based on information about the number of data bits of the input data IN1 and IN2. For example, when the first input data IN1 is integer data and includes a sign bit, bits other than the most significant bit can be obtained as information about the number of bits of the first input data IN1. When the second input data IN2 is floating point data, the number of bits in the mantissa can be obtained as information about the number of bits of the second input data IN2.

[0087] In at least one embodiment, the controller 610 may output a clock signal CLK based on information about the number of data bits of the input data IN1 and IN2. The controller 610 may output a clock signal whose cycle is repeated as many times as the number of bits of the input data IN1 and IN2. In at least one embodiment, the controller 610 may output a clock signal based on the number of bits of any data among the multiple input data IN1 and IN2. In at least one embodiment, the controller 610 may compare the number of bits of the multiple input data IN1 and IN2 to output a clock signal based on the number of bits of the input data with fewer bits. In at least one embodiment, the calculation circuit 620 may perform mantissa calculation based on the clock signal based on the information about the number of data bits of the input data IN1 and IN2. Reference will be made later Fig. 9 The mantissa calculation of the calculation circuit 620 is described.

[0088] In at least one embodiment, the controller 610 may not receive new input data until all clock signals CLK are output.

[0089] Figure 7 A schematic block diagram of a computing circuit according to at least one embodiment is shown.

[0090] refer to Figure 7 , the calculation circuit 700 may include a multiplication section 701 and an accumulation section 702. In at least one embodiment, the multiplication section 701 may include a sign calculation section 710, an exponent calculation section (E_CAL) 720, and a mantissa calculation section (M_CAL) 730. In at least one embodiment, the accumulation section 702, the adder 750, and the accumulator 760 may be the same and / or substantially similar to the accumulation section 402, the adder 450, and the accumulator 460 described above, respectively.

[0091] In at least one embodiment, the sign calculation unit 710 may receive first sign data S_IN1[A] of the first input data IN1 and second sign data S_IN2[B] of the second input data IN2. When both the first sign data S_IN1[A] and the second sign data S_IN2[B] have a value "0" representing a positive number or a value "1" representing a negative number, the XOR gate of the sign calculation unit 710 may output "0" representing a positive number. On the other hand, when one of the first sign data S_IN1[A] and the second sign data S_IN2[B] has a value "0" representing a positive number and the other has a value "1" representing a negative number, the XOR gate of the sign calculation unit 710 may output "1" representing a negative number. The sign calculation unit 710 may output data generated as a result of the XOR calculation as 1-bit sign data (S_OUT).

[0092] The exponential processing circuit 720 may receive first exponential data E_IN1[A-1:a+1] of the first input data IN1 and second exponential data E_IN2[B-1:b+1] of the second input data IN2. In at least one embodiment, the first input data IN1 and the second input data IN2 may be floating point data or integer data, and / or one of the first input data IN1 and the second input data IN2 may be integer data, while the other may be floating point data. Meanwhile, when all the input data are integer data, the exponential data of the corresponding input data does not exist. The exponential processing circuit 720 may also receive the exponential data from the controller ( Figure 6 610) receives data type signals DS1 and DS2. Figure 8 The components and specific methods of operation of the exponential processing circuit 720 are described.

[0093] Figure 8 An index calculation section according to at least one embodiment is shown.

[0094] The exponential calculation section 800 according to at least one embodiment may further include an exponential processing circuit 840 including a first exponential adder 841 and a second exponential adder 843 , and multiplexers MUX_E1 , MUX_E2 , and MUX_E3 connected to input nodes of the respective adders.

[0095] In at least one embodiment, the first multiplexer (MUX_E1) 810 includes a first input node receiving first exponential data E_IN1 [A-1: a+1] of the first input data IN1 and a second input node receiving "0". The first multiplexer (MUX_E1) 810 may receive a first data type signal DS1 as a selection signal. The first multiplexer (MUX_E1) 810 may include a selection node receiving the first data type signal DS1. For example, when the first input data IN1 is integer data, the first exponential data does not exist. Accordingly, the first multiplexer 810 may output "0" based on the first data type signal DS1.

[0096] In at least one embodiment, the second multiplexer (MUX_E2) 820 includes a first input node receiving second exponential data E_IN2[B-1:b+1] of the second input data IN2 and a second input node receiving "0". The second multiplexer (MUX_E2) 820 may receive the second data type signal DS2 as a selection signal. The second multiplexer (MUX_E2) 820 may include a selection node receiving the second data type signal DS2. For example, when the second input data IN2 is floating point data, the second multiplexer 820 may output the second exponential data (E_IN2[B-1:b+1]) based on the second data type signal DS2.

[0097] According to at least one example, the first input data IN1 and the second input data IN2 may be floating point data. When the first input data IN1 and the second input data IN2 are floating point data, the first multiplexer 810 may output the first index data E_IN1[A-1:a+1] based on the first data type signal DS1, and the second multiplexer 820 may output the second index data E_IN2[B-1:b+1] based on the second data type signal DS2. The first exponential adder 841 may receive the first exponential data E_IN1[A-1:a+1] and the second exponential data E_IN2[B-1:b+1] and perform a first addition calculation. The first exponential data E_IN1[A-1:a+1] and the second exponential data E_IN2[B-1:b+1] are both added with an exponential deviation value (e.g., "127"). Since the exponential deviation value is doubled by the first addition calculation of the first exponential adder 841, it is necessary to subtract the exponential deviation value from the first addition result data.

[0098] Meanwhile, the third multiplexer (MUX_E3) 830 may include a first input node receiving a negative exponent bias value (-Bias) and a second input node receiving "0". The third multiplexer 830 may receive a signal generated by AND operator 831 performing AND calculation on the first data type signal DS1 and the second data type signal DS2 as a selection signal SEL. The third multiplexer 830 may include a selection node receiving a signal generated by AND operator 831 performing AND calculation on the first data type signal DS1 and the second data type signal DS2. For example, when both the first input data and the second input data are floating point data, that is, when the selection signal SEL is "1", the third multiplexer 830 may output a negative exponent bias value. When one of the first input data and the second input data is integer data, that is, when the selection signal SEL is "0", the third multiplexer 830 may output "0". In an example, since both the first input data IN1 and the second input data IN2 are floating point data, the third multiplexer 830 may output a negative exponent bias value, for example, "-127", based on the selection signal SEL. Accordingly, the second exponential adder 843 may receive the first addition result data output from the first exponential adder 841, and may perform a second addition calculation in which an exponent bias value (Bias), for example, "127", is subtracted from the first addition result data. The second exponential adder 843 may output data generated as a result of the second addition calculation as exponential data E_OUT of the multiplication data OUT.

[0099] According to another example, the first input data IN1 may be floating point data, and the second input data IN2 may be integer data. When the first input data IN1 is floating point data and the second input data IN2 is integer data, the first multiplexer 810 of the exponential calculation section 800 may output the first index data E_IN1[A-1:a+1] based on the first data type signal DS1, and the second multiplexer 820 may output "0" based on the second data type signal DS2. The first exponential adder 841 may receive the first exponential data E_IN1[A-1:a+1] and "0", and perform a first addition calculation. The first exponential data E_IN1[A-1:a+1] is in a state of adding an exponential deviation value (e.g., "127"), and the exponential deviation value does not need to be subtracted from the first addition result data. Accordingly, the third multiplexer 830 may output "0" based on the selection signal SEL. Accordingly, the second exponential adder 843 may receive the first addition result data output from the first exponential adder 841, and may perform a second addition calculation of subtracting "0" from the first addition result data. The second exponential adder 843 may output data generated as a result of the second addition calculation as exponential data E_OUT of the multiplication data OUT.

[0100] According to another example, the first input data IN1 and the second input data IN2 may be integer data. When the first input data IN1 and the second input data IN2 are integer data, the first multiplexer 810 of the exponential calculation section 800 may output "0" based on the first data type signal DS1, and the second multiplexer 820 may output "0" based on the second data type signal DS2. The first exponential adder 841 may receive the output data of the first multiplexer 810 and the output data of the second multiplexer 820, and may perform a first addition calculation. The third multiplexer 830 may output "0" based on the selection signal SEL. Accordingly, the second exponential adder 843 may receive the first addition result data output from the first exponential adder 841, and may perform a second addition calculation of subtracting "0" from the first addition result data. The second exponential adder 843 may output data generated as a result of the second addition calculation, that is, "0", as the exponential data E_OUT of the multiplication data OUT.

[0101] Return to reference Figure 7 , the mantissa calculation unit 730 may receive first mantissa data M_IN1[a:0] of the first input data IN1 and second mantissa data M_IN2[b:0] of the second input data IN2. When the input data is floating point data, the mantissa data refers to the mantissa, and when the input data is integer data, the mantissa data may correspond to an integer. The mantissa calculation unit 730 may also receive first mantissa data M_IN1[a:0] of the first input data IN1 and second mantissa data M_IN2[b:0] of the second input data IN2. Figure 6 610) receives the clock signal CLK. Fig. 9 The components and specific operation method of the mantissa calculation unit 730 are described.

[0102] Fig. 9 A mantissa calculation unit according to at least one embodiment is shown. Here, for better understanding and ease of description, it is referred to as a mantissa calculation unit, and when the input data is integer data, the data input to the mantissa calculation unit may be an integer part.

[0103] The mantissa calculation unit 900 according to at least one embodiment may include a first multiplexer (MUX_E1) 910, a second multiplexer (MUX_E2) 920, a counter 911, a shifter 921, an adder 930, and an accumulator (ACC) 940. The mantissa calculation unit 900 according to at least one embodiment may perform a multiplication calculation on input data. The mantissa calculation unit 900 according to at least one embodiment may perform a multiplication calculation using the shifter 921 and the adder 930.

[0104] The multiplication calculation of the mantissa calculation unit 900 according to at least one embodiment may be performed by addition calculation and shift calculation. Fig.10 The calculation method of the fraction calculation unit 900 will be described.

[0105] Fig.10 Is used to explain the Fig. 9 A diagram showing the calculation method of the decimal calculation part.

[0106] refer to Fig.10 ,exist middle, can be called the multiplicand, can be called a multiplier. First, when the least significant bit value of the multiplier is "1" (1001), the multiplicand can be output as is as the multiplicand value ( )(1005), and the multiplicand can be shifted left by 1 bit ( )(1013).

[0107] Next, since the next bit value of the multiplier is "1" (1011), the multiplicand can be output as is ( ) (1015) and can be compared with the previous output value ( ) performs addition calculation. As a result of addition, you can output (1017), and the multiplicand can be shifted left by 1 bit ( )(1023).

[0108] Next, since the next bit value of the multiplier is "0" (1021), the output value can become "0" (1025) and can be compared with the previous output value ( ) performs addition calculation (1017). As a result of the addition, the output (1027), and the multiplicand can be shifted left by 1 bit.

[0109] When the calculation is performed in the above manner, the final output value is (1037), which is similar to using a multiplier to perform The results of the calculations are the same. That is, binary multiplication can be performed using shifters and adders.

[0110] Return to reference Fig. 9 According to at least one embodiment, the calculation of the mantissa calculation unit 900 can be based on Fig.10 In at least one embodiment, the first mantissa data M_IN1[a:0] of the first input data IN1 may be a multiplier, and the second mantissa data M_IN2[b:0] of the second input data IN2 may be a multiplicand.

[0111] In at least one embodiment, the first multiplexer 910 may include a plurality of input nodes that receive the first mantissa data M_IN1[a:0] as input data. The first multiplexer 910 may receive the respective bit values ​​(M_IN1[a], ..., M_IN1[1], M_IN1[0]) of the first mantissa data M_IN1[a:0] as input data. The first multiplexer 910 may receive the output signal of the counter 911 as the selection signal SEL1. The first multiplexer 910 may include a selection node that receives the output signal of the counter 911. The counter 911 may count the number of times the clock signal CLK output from the controller 610 is activated. That is, the counter 911 may count the cycle of the clock signal CLK. The first multiplexer 910 may output the input data (M_IN1[a], ..., M_IN1[1], M_IN1[0]) in sequence, one bit at a time, starting from the least significant bit, based on the selection signal SEL1 output from the counter 911. The number of cycles of the clock signal CLK may be the same as the number of bits of the first mantissa data M_IN1 [a:0]. The value output from the first multiplexer 910 may be input as the selection signal SEL2 of the second multiplexer 920 .

[0112] In at least one embodiment, the second multiplexer 920 may include a first input node for receiving the second mantissa data M_IN2[b:0] and a second input node for receiving “0”. The second multiplexer 920 may receive the output value of the first multiplexer 910 (i.e., the respective bit values ​​(M_IN1[a], ..., M_IN1[1], M_IN1[0]) of the first mantissa data M_IN1[a:0]) as the selection signal SEL2. The second multiplexer 920 may include a selection node for receiving the output value of the first multiplexer 910 (i.e., the respective bit values ​​(M_IN1[a], ..., M_IN1[1], M_IN1[0]) of the first mantissa data M_IN1[a:0]. In at least one embodiment, when the selection signal SEL2 is “0”, the second multiplexer 920 may output “0” among the input data of the second multiplexer 920. In at least one embodiment, when the selection signal SEL2 is “1”, the second multiplexer 920 may output the second mantissa data M_IN2 [b:0] of the second input data IN2 among the input data of the second multiplexer 920 .

[0113] In at least one embodiment, the adder 930 may add the output data of the second multiplexer 920 and the output data from the accumulator 940 , and send the added data to the accumulator 940 .

[0114] In at least one embodiment, the shifter 921 may shift the second mantissa data M_IN2[b:0] to the left by 1 bit. In at least one embodiment, the shifter 921 may send the shifted data to the second input node of the second multiplexer 920 .

[0115] In at least one embodiment, the number of cycles of the clock signal CLK is the same as the number of bits of the first mantissa data M_IN1[a:0], so the above operation can be repeated as many times as the number of bits of the first mantissa data M_IN1[a:0].

[0116] The mantissa calculation unit 900 according to at least one embodiment does not include a multiplier, and can be configured to perform shift calculation and addition calculation on input data to output the multiplication value of the input data. The mantissa calculation unit 900 according to at least one embodiment can receive data with different digits as input data, and perform shift calculation and addition calculation on the input data. The mantissa calculation unit 900 according to at least one embodiment has no restriction on the digit of the input data, and performs shift calculation and addition calculation as many as the digit of the input data, so that the performance of the computing device is improved as the digit is reduced.

[0117] refer to Figure 7In at least one example, when at least one of the first input data IN1 and the second input data IN2 is floating point data, the normalizer 740 may normalize the exponential data E_OUT output from the exponent calculation section 720 and the mantissa data M_OUT output from the mantissa calculation section 730. The output data OUT output from the normalizer may be output to the accumulation section 702 together with the sign data S_OUT output from the sign calculation section 710. In another example, when the first input data IN1 and the second input data IN2 are integer data, that is, when the exponential data E_OUT output from the exponent calculation section 720 is "0", the normalizer 740 may output the mantissa data M_OUT output from the mantissa calculation section 730 as the output data OUT. The output data OUT output from the normalizer 740 may be output to the accumulation section 702 together with the sign data S_OUT output from the sign calculation section 710.

[0118] exist Figure 7 In the accumulation section 702 , the adder 750 may perform an accumulation calculation by adding the data received from the multiplier 701 and the data received from the accumulator 760 .

[0119] Fig.11 A timing diagram illustrating calculations performed in a mantissa calculation section according to at least one embodiment is shown.

[0120] refer to Fig.11 , the first mantissa data M_IN1[5:0] can be 6 bits , and the second mantissa data M_IN2 can be 4 bits The first mantissa data M_IN1 is a multiplier, the second mantissa data M_IN2 is a multiplicand, and the clock signal CLK is output for 6 cycles (6 CYCLE) which is the same as the number of bits of the first mantissa data M_IN1 as the multiplier. Fig. 9 describe Fig.11 Timing diagram of .

[0121] Referring to the first cycle (1 CYCLE), the counter 911 may output a first counter value ( ), and the first multiplexer 910 may output the first mantissa data as a multiplier based on the output signal of the counter 911 ( ) is the least significant bit "1". The second multiplexer 920 may receive "0" as input data and receive the second mantissa data ( The output data (“1”) of the first multiplexer 910 is input as the selection signal SEL2 of the second multiplexer 920, and the second multiplexer 920 may output the second mantissa data ( Since the accumulator 940 does not output any value in the first cycle, the adder (ADDER) can output the output data of the second multiplexer 920 as it is ( ), and sends it to the accumulator (ACC).

[0122] Referring to the second cycle (2CYCLE), the counter 911 may output a second counter value ( ), and the first multiplexer 910 may output the first mantissa data based on the output signal of the counter 911 ( ) of the second bit "0". The second multiplexer 920 can receive "0" as input data, and receive the second mantissa data ( ) is shifted left by 1 bit. The output data (“0”) of the first multiplexer 910 is input as the selection signal SEL2 of the second multiplexer 920, and the second multiplexer 920 can output “0” based on the selection signal SEL2. In the second cycle, the accumulator 940 receives the output data ( ), so the adder 930 can add the output data (“0”) of the second multiplexer 920 to the output data ( ) and the data obtained by adding ) is sent to accumulator 940.

[0123] Referring to the third cycle (3CYCLE), the counter 911 may output a third counter value ( ), and the first multiplexer 910 may output the first mantissa data based on the output signal of the counter 911 ( ) of the third bit "1". The second multiplexer 920 can receive "0" as input data, and receive the second mantissa data ( ) shifted left by 2 bits The output data (“1”) of the first multiplexer 910 is input as the selection signal SEL2 of the second multiplexer 920, and the second multiplexer 920 can output the input data ( In the second cycle, the accumulator 940 receives the output data of the adder 930 in the first cycle ( ), so the adder 930 can convert the output data of the second multiplexer 920 ( ) and the output data of the accumulator 940 ( ) and the data obtained by adding ) is sent to accumulator 940.

[0124] When six cycles are executed using the above method, the accumulator 940 can store the data received from the adder. As output data is output to the normalizer ( Figure 7 of 740).

[0125] Fig.12 A timing chart showing calculations performed in a fraction calculation section according to another embodiment is shown.

[0126] In at least one embodiment, the controller 610 in the computing device 600 can output the clock signal CLK based on information about the number of data bits of the input data IN1 and IN2. In this case, the number of data bits of the input data IN1 and IN2 may be different. Accordingly, by comparing the number of bits of the plurality of input data IN1 and IN2, the clock signal can be output based on the number of bits of the input data with fewer bits. Among the plurality of input data IN1 and IN2, the controller 610 can determine the data with fewer bits as the multiplier, and determine the data with more bits as the multiplicand. Therefore, the performance of the multiplier (e.g., computing device) is improved because the multiplier is configured to skip the calculation of those values ​​outside the loop of the mantissa.

[0127] Fig.12 Shown in Fig.11 In at least one embodiment of the invention, the multiplier and the multiplicand are changed. Specifically, Fig.11 The multiplier in becomes Fig.12 as the multiplicand of the second mantissa data M_IN2, and Fig.11 The multiplicand in becomes Fig.12 The clock signal CLK is a signal for the first mantissa data M_IN1 ( ) is output in 4 cycles with the same number of bits. Fig. 9 describe Fig.12 Timing diagram of .

[0128] Referring to the first cycle (1CYCLE), the counter 911 may output a first counter value ( ), and the first multiplexer 910 may output the first mantissa data as a multiplier based on the output signal of the counter 911 ( ) is the least significant bit "1". The second multiplexer 920 can receive "0" as input data and the second mantissa data ( The output data (“1”) of the first multiplexer 910 is input as the selection signal SEL2 of the second multiplexer 920, and the second multiplexer 920 may output the second mantissa data ( Since the accumulator 940 does not output any value in the first cycle, the adder 900 can output the output data of the second multiplexer 920 as it is ( ), and sends it to accumulator 940.

[0129] Referring to the second cycle (2CYCLE), the counter 911 may output a second counter value ( ), and the first multiplexer 910 may output the first mantissa data based on the output signal of the counter 911 ( ) of the second bit "0". The second multiplexer 920 can receive "0" as input data, and receive the second mantissa data ( ) shifted left by 1 bit to obtain the input data ( ). The output data (“0”) of the first multiplexer 910 is input as the selection signal SEL2 of the second multiplexer 920, and the second multiplexer 920 can output “0” based on the selection signal SEL2. In the second cycle, the accumulator 940 outputs the output data ( ), so the adder 930 can add the output data (“0”) of the second multiplexer 920 to the output data ( ) and the data obtained by adding ) is sent to accumulator 940.

[0130] When four cycles are executed using the above method, the accumulator 940 can store the data received from the adder. As output data is output to the normalizer ( Figure 7 of 740).

[0131] In this way, when the number of bits of the mantissa data input to the mantissa calculation unit is different, the data with fewer bits among the input data can be determined as the multiplier (e.g., the input data of the first multiplexer 910). This has the advantage of reducing the delay required for calculating the input data, because the cycle of the clock signal CLK is reduced, so that the number of repetitions of the calculation performed in the mantissa calculation unit is reduced. Therefore, the performance of the multiplier (e.g., the computing device) can be improved.

[0132] Fig.13 is an example diagram for explaining multi-head attention according to at least one embodiment.

[0133] The multi-head attention 1300 is a mechanism used in a transformer model and an automatic speech recognition (ASR) model among neural network models, and it has a scaled dot product attention 1320 structure of an overlapping form (h). In at least one embodiment, the multi-head attention 1300 can perform self-attention on the embedding of input data. Embedding can refer to the result or a series of processes of converting natural language used by humans into a vector form that can be understood by machines.

[0134] When processing sequence data such as natural language, long-term dependencies can be a problem. Self-attention intends to solve long-term dependencies, and when using self-attention, the relationship between words in a sentence can be measured. Specifically, the relationship value with other words can be calculated based on each word, and this value is called an attention score. The attention scores between words with large relationships can be high. A table of attention scores is called an attention map. Multi-head attention 1300 can check the attention to various feature values ​​by making multiple attention maps.

[0135] In the multi-head attention 1300, the input of the scaled dot product attention layer 1320 includes a query (Q), a key (K), and a value (V). For example, when looking up the meaning of a specific word through an English dictionary, the specific word may correspond to the query, the word registered in the dictionary may correspond to the key, and the meaning of the key may correspond to the value.

[0136] The multi-head attention 1300 can reduce the dimensionality of the value (V), key (K) and / or query (Q) through a first linear computation layer 1310, self-attention can be performed through h scaled dot product attention layers 1320, and linear computation can be performed through a cascade 1330 and a second linear computation layer 1340 on the attention result.

[0137] In at least one embodiment, the linear calculations performed in the first linear calculation layer 1310 and the second linear calculation layer 1340 may refer to matrix dot product calculations. For example, in order to reduce the dimension of the value (V), key (K), and query (Q) derived from the embedded information of the input data, the first linear calculation layer 1310 may perform a dot product calculation of multiplying the embedded vector of the input data by a specific weight, and the second linear calculation layer 1340 may perform a dot product calculation of multiplying the cascaded output matrix by a specific weight.

[0138] The first linear computing layer 1310 and / or the second linear computing layer 1340 may use a computing device according to at least one of the above-described embodiments. For example, the first linear computing layer 1310 and the second linear computing layer 1340 may receive an input matrix and a weight matrix as input data. In at least one embodiment, the elements of the input matrix and the weight matrix may be real numbers, such as floating point data expressed as floating point numbers, or integer data. In at least one embodiment, the elements of the input matrix and the weight matrix may have different precisions (i.e., bits). The computing device according to at least one embodiment ( Figure 1 121) in can perform dot product calculations on the input matrix and the weight matrix.

[0139] Fig.14 It is used to explain Fig.13 Example graph of scaled dot-product attention.

[0140] The matrix multiplication layer 1311 of the scaled dot product attention 1320 may perform a dot product calculation of the key (K) and the query (Q) and output an attention score indicating similarity. The scaling layer 1313 may perform scaling to adjust the size of the attention score output from the first matrix multiplication layer 1311. The masking layer 1315 may prevent attention to incorrect connections by masking. The softmax calculation layer 1317 may calculate weights based on the attention scores, and the second matrix multiplication layer 1319 may perform a dot product calculation on the weights and values ​​(V) which are outputs of the softmax calculation layer 1317.

[0141] The first matrix multiplication layer 1311 and the second matrix multiplication layer 1319 may use a computing device according to at least one embodiment. For example, the first matrix multiplication layer 1311 may receive a key (K) vector and a query (Q) vector as input data, and the second matrix multiplication layer 1319 may receive a value (V) vector and a weight matrix as input data. In at least one embodiment, the elements of the input data vectors of the first matrix multiplication layer 1311 and the second matrix multiplication layer 1319 may be real numbers, such as floating-point data expressed as floating-point numbers, or integer data. In at least one embodiment, the elements of the input data vectors of the first matrix multiplication layer 1311 and the second matrix multiplication layer 1319 may have different precisions (i.e., bits). The computing device ( Figure 1 121) in can perform dot product calculation on the input vector and the weight matrix.

[0142] While at least one embodiment of the disclosure has been described in connection with what is presently considered to be practical embodiments, it is to be understood that the disclosure is not limited to the disclosed embodiments, but on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A computing device, comprising: processing circuitry, configured as receiving first input data of a first data type and second input data of a second data type different from the first data type, outputting a first signal representing the first data type, a second signal representing the second data type, and a clock signal based on the number of bits included in the first input data and the second input data, and A multiplication calculation is performed based on the first signal, the second signal, and the clock signal, and output data is generated such that a result of the multiplication calculation is based on the first input data and the second input data.

2. The computing device of claim 1, wherein The first input data includes at least one of symbolic data or integer data, and The second input data includes at least one of sign data, exponent data, or mantissa data.

3. The computing device of claim 2, wherein: The processing circuit includes a first circuit configured to output sign data of the output data, a second circuit configured to output exponential data of the output data based on the first signal and the second signal, and A third circuit is configured to output mantissa data of the output data based on the clock signal.

4. The computing device of claim 3, wherein: The first circuit includes an exclusive OR (XOR) gate configured as receiving the sign data of the first input data and the sign data of the second input data, and The sign data of the output data is output.

5. The computing device of claim 3, wherein: The second circuit is configured to output the exponential data of the second input data.

6. The computing device of claim 5, wherein: The second circuit includes a first multiplexer configured to output a first value based on the first signal, a second multiplexer configured to output the exponential data of the second input data based on the second signal, a first adder configured to add the output data of the first multiplexer and the output data of the second multiplexer, a third multiplexer configured to receive a negative (-) exponential deviation value and the first value, and output the first value based on a third signal generated by performing an AND calculation on the first signal and the second signal, and The second adder is configured to add the output data of the first adder and the output data of the third multiplexer.

7. The computing device of claim 3, wherein: The third circuit is configured to receive first data corresponding to integer data of the first input data and second data corresponding to mantissa data of the second input data, and perform the multiplication calculation on the first data and the second data.

8. The computing device of claim 7, wherein: The third circuit includes a counter configured to generate an output signal based on the clock signal, a shifter configured to shift the second data by one (1) bit, a first multiplexer configured to sequentially output the first data bit by bit starting from the least significant bit based on the output signal of the counter, a second multiplexer configured to output one of a first value, the second data, or the output data of the shifter based on the output data of the first multiplexer, and An adder is configured to add the output data of the second multiplexer based on the clock signal.

9. The computing device of claim 7, wherein: The number of bits of the first data is different from the number of bits of the second data.

10. The computing device of claim 9, wherein: The number of bits of the first data is smaller than the number of bits of the second data.

11. The computing device of claim 1 , wherein: The processing circuit is configured not to receive new input data until the clock signal ends.

12. A computing device configured to perform a multiplication calculation on first data of a first digit and second data of a second digit different from the first digit, the computing device comprising: a first multiplexer comprising a first selection node and a plurality of input nodes, the first selection node being configured to receive an output signal of a counter, the counter being configured to count the number of cycles of a clock signal based on the first bit number, and the plurality of input nodes being configured to receive respective bit values ​​of the first data, a second multiplexer including a second selection node, a first input node, and a second input node, the second selection node being configured to receive output data of the first multiplexer, the first input node being configured to receive a first value, and the second input node being configured to receive one of the second data or data obtained by shifting the second data by one (1) bit from a shifter, and The first adder is configured to add the output data of the second multiplexer based on the clock signal.

13. The computing device of claim 12, wherein The first multiplexer is configured to sequentially output 1 bit one by one starting from the least significant bit value among the bit values ​​of the first data based on the output signal of the counter, and The second multiplexer is configured to output the first value when the output data of the first multiplexer is "0", and to output one of the second data or the output data of the shifter when the output data of the first multiplexer is "1".

14. The computing device of claim 12, wherein: The first digit is smaller than the second digit.

15. The computing device of claim 12, further comprising: The controller is configured to receive first input data including the first data and second input data including the second data, and output a first signal indicating a data type of the first input data and a second signal indicating a data type of the second input data.

16. The computing device of claim 15, wherein: The first input data and the second input data are at least one of the following: data of a first data type including sign data and integer data, or a second data type data including sign data, exponent data, and mantissa data, and The first data corresponds to integer data or mantissa data of the first input data, and the second data corresponds to integer data or mantissa data of the second input data.

17. The computing device of claim 16, further comprising: a third multiplexer configured to output one of the exponential data or the first value of the first input data based on the first signal, a fourth multiplexer configured to output one of the exponential data or the first value of the second input data based on the second signal, a fifth multiplexer configured to output one of a negative (-) exponential deviation value or a first value based on a third signal, wherein the third signal is obtained by performing an AND calculation on the first signal and the second signal, a second adder configured to add the output data of the third multiplexer and the output data of the fourth multiplexer, and A third adder is configured to add the output data of the second adder and the output data of the fifth multiplexer.

18. The computing device of claim 17, further comprising The normalization circuit is configured to normalize the output data of the first adder and the output data of the third adder when at least one of the first input data or the second input data is data of the second data type.

19. The computing device of claim 16, further comprising An exclusive OR (XOR) gate is configured to receive the sign data of the first input data and the sign data of the second input data.

20. A neural processing system comprising: non-transitory memory; as well as a processing circuit configured to receive an input matrix and a weight matrix corresponding to the input data from the memory, perform a matrix dot product calculation on the input matrix and the weight matrix, and output a result of performing the matrix dot product calculation as an output matrix, wherein the input matrix comprises elements of a first data type, and the weight matrix comprises elements of a second data type different from the first data type, and The processing circuit includes a computing device configured to output one of exponential data of elements of the input matrix or exponential data of elements of the weight matrix as exponential data of elements of the output matrix based on the first data type and the second data type.

Images