Data processing system, method of operation thereof, and computing system using the system

Abstract

This disclosure relates to a data processing system, which may include: a matrix splitting circuit configured to split a matrix into a positive matrix and a negative matrix, and store the positive matrix and the negative matrix in a first subarray and a second subarray in a computing memory, respectively; a vector conversion circuit configured to generate an offset vector by adding an element of a vector to an offset, the offset being used to convert the negative element with the largest absolute value among the elements of the vector into a zero element or a positive element, and applying the offset vector to the row lines of the first subarray and the second subarray; and an offset correction circuit configured to generate an offset correction value by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix, and subtracting the offset correction value from the calculated value output from the first subarray and the second subarray.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2022-0008499, filed on January 20, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] The various embodiments disclosed herein generally relate to a data processing technique, and more specifically, to a data processing system, a method of operating the same, and a computing system using the data processing system. Background Technology

[0004] As the importance of artificial intelligence (AI) applications and big data analytics increases, they are receiving more and more attention, leading to a growing demand for computing systems capable of efficiently processing big data.

[0005] Artificial neural networks are one method for realizing AI. The computations performed by AI applications mainly consist of vector-matrix multiplication, and various methods are being researched to compute large amounts of data quickly and accurately. Summary of the Invention

[0006] In embodiments of this disclosure, a data processing system may include: a computing memory including one or more subarrays, each subarray including a plurality of memory cells connected between a plurality of row lines and a plurality of column lines; a matrix splitting circuit configured to: when a matrix received from a host device includes negative elements, split the matrix into a positive matrix consisting of the positive elements of the matrix and a negative matrix consisting of the absolute values ​​of the negative elements of the matrix, and store the positive matrix and the negative matrix in a first subarray and a second subarray within the computing memory, respectively; a vector conversion circuit configured to: when a vector received from a host device includes negative elements, generate an offset vector by adding an offset to the elements of the vector, the offset being used to convert the negative element with the largest absolute value among the elements of the vector into a zero element or a positive element, and apply the offset vector to the row lines of the first subarray and the second subarray; and an offset correction circuit configured to: generate an offset correction value by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix, and subtract the offset correction value from the calculated value output from the first subarray and the second subarray.

[0007] In embodiments of this disclosure, an operation method of a data processing system may include: providing a computing memory comprising one or more subarrays, each subarray comprising a plurality of memory cells connected between a plurality of row lines and a plurality of column lines; when a matrix received from a host device includes negative elements, splitting the matrix into a positive matrix composed of the positive elements of the matrix and a negative matrix composed of the absolute values ​​of the negative elements of the matrix by a negative number calculation control circuit; storing the positive matrix and the negative matrix in a first subarray and a second subarray within the computing memory, respectively; when a vector received from the host device includes negative elements, generating an offset vector by adding the elements in the vector to an offset by the negative number calculation control circuit, the offset being used to convert the negative element with the largest absolute value among the elements in the vector into a zero element or a positive element; applying the offset vector to the row lines of the first subarray and the second subarray by the negative number calculation control circuit; generating an offset correction value by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix; and subtracting the offset correction value from the calculated value output from the first subarray and the second subarray by the negative number calculation control circuit.

[0008] In embodiments of this disclosure, a computing system may include: a host device; a data processing system configured to process computations of an application according to a request from the host device, and including a computing memory comprising one or more subarrays, each subarray including a plurality of memory cells connected between a plurality of row lines and a plurality of column lines; and a negative number calculation control circuit configured to: split a matrix into a positive matrix and a negative matrix when a matrix received from the host device includes a negative element; generate an offset vector by adding the vector to an offset when a vector received from the host device includes a negative element; and correct a calculated value based on an offset correction value generated based on the offset, the calculated value being output from the computing memory as the result of multiplying each of the positive and negative matrices by the offset vector.

[0009] In embodiments of this disclosure, an operational method of a computing device with built-in memory may include: generating an offset vector comprising non-negative elements by adding an adjustment vector to a provided vector; converting a provided matrix into a positive matrix and a negative matrix to program the positive and negative matrices into first and second memristor memory cell arrays, respectively; generating a positive computation vector and a negative computation vector by applying offset signals to corresponding rows of the respective first and second arrays and, column by column, adding the results of applying the offset signals, where the offset signals represent individual elements of the offset vectors; generating an intermediate vector by subtracting the negative computation vector from the positive computation vector; and generating an intermediate vector by applying adjustment signals to the rows of the respective first and second arrays and, column by column. The unit adds the results of the applied adjustment signal to generate a positive correction vector and a negative correction vector, the adjustment signal representing a single value of each element in the adjustment vector; an offset correction vector is generated by subtracting the negative correction vector from the positive correction vector; and a result vector is generated by subtracting the offset correction vector from the intermediate vector, wherein: the provided vector is of [1xN] dimension, where N is an integer greater than 0, each of the provided positive and negative matrices and the first and second arrays are of [NxM] dimension, where M is an integer greater than 0, the positive matrix includes the positive elements of the provided matrix and one or more zero elements, and the negative matrix includes the absolute value elements of the negative elements of the provided matrix and one or more zero elements. Attached Figure Description

[0010] Figure 1 This is a configuration diagram illustrating a computing system according to an embodiment of the present disclosure.

[0011] Figure 2 This is a configuration diagram illustrating a neural network processor according to an embodiment of the present disclosure.

[0012] Figure 3 This is a diagram illustrating the configuration of a computing memory according to an embodiment of the present disclosure.

[0013] Figure 4 This is a configuration diagram showing a negative number calculation control circuit according to an embodiment of the present disclosure.

[0014] Figure 5 This is a flowchart describing an operation method of a data processing system according to embodiments of the present disclosure.

[0015] Figures 6A to 6E This is a conceptual diagram used to describe a VMM (vector-matrix multiplication) including negative numbers according to embodiments of the present disclosure.

[0016] Figure 7 This is a conceptual diagram used to describe an offset correction value generation method according to embodiments of the present disclosure.

[0017] Figure 8This is a configuration diagram showing a negative number calculation control circuit according to an embodiment of the present disclosure.

[0018] Figure 9 This is a diagram illustrating the configuration of processing elements according to an embodiment of the present disclosure.

[0019] Figures 10A to 10E This is a conceptual diagram used to describe an offset vector providing method according to embodiments of the present disclosure.

[0020] Figure 11 This is a flowchart describing an operation method of a data processing system according to embodiments of the present disclosure. Detailed Implementation

[0021] In the following, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings.

[0022] Figure 1 This is a configuration diagram illustrating a computing system according to an embodiment of the present disclosure.

[0023] Reference Figure 1 The computing system 10 according to an embodiment may include a host device 100 and a data processing system 200. The data processing system 200 may include a neural network processor 300 configured to process computations of an application in response to requests from the host device 100.

[0024] The host device 100 may include at least a main processor 110, RAM 120, memory 130 and input / output (I / O) devices 140, and further include other general-purpose components (not shown).

[0025] In one embodiment, the components of the host device 100 may be integrated into a single semiconductor chip and implemented as a system-on-a-chip (SoC). However, the embodiment is not limited thereto, and the components of the host device 100 may be implemented as multiple semiconductor chips.

[0026] The main processor 110 can control all operations of the computing system 10. For example, a central processing unit (CPU) can be used as the main processor 110. The main processor 110 may include one or more cores. The main processor 110 can process or execute programs, data, or instructions stored in RAM 120 and memory 130. For example, the main processor 110 can control the functions of the computing system 10 by executing programs stored in memory 130.

[0027] RAM 120 can temporarily store programs, data, or instructions. Depending on the boot code or under the control of the main processor 110, programs and / or data stored in memory 130 can be temporarily loaded into RAM 120. RAM 120 can be implemented as a memory such as dynamic RAM (DRAM) or static RAM (SRAM).

[0028] Memory 130 can be used as storage space for storing data (e.g., operating system (OS), various programs, and various data). Memory 130 may include one or more of volatile memory and non-volatile memory. Non-volatile memory may be selected from read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, phase-change RAM (PRAM), magnetic RAM (MRAM), resistive RAM (RRAM), ferroelectric RAM (FRAM), etc. Volatile memory may be selected from DRAM, SRAM, synchronous DRAM (SDRAM), etc. In embodiments, memory 130 may be implemented as a storage device such as a hard disk drive (HDD), solid-state drive (SSD), compact flash memory (CF), secure digital storage (SD), micro-secure digital storage (micro-SD), mini secure digital storage (mini-SD), extreme digital storage (xD), or memory stick.

[0029] I / O device 140 can receive user input or input data from the outside and output the processing results of computing system 10. I / O device 140 can be implemented as a touch screen panel, keyboard, or various types of sensors. In embodiments, I / O device 140 can collect information around computing system 10. For example, I / O device 140 may include imaging device and image sensor to sense or receive image signals from outside computing system 10, convert the sensed or received image signals into image data, and store the image data in memory 130 or provide the image data to data processing system 200.

[0030] The data processing system 200 can process application calculations in response to requests from the host device 100. Specifically, the data processing system 200 can extract useful information by analyzing input data based on artificial neural networks, and determine the situation based on the extracted information or control components of electronic devices equipped with the data processing system 200. For example, the data processing system 200 can be applied to drones, advanced driver assistance systems (ADAS), smart TVs, smartphones, medical devices, mobile devices, image display devices, measuring devices, Internet of Things (IoT) devices, etc. Furthermore, the data processing system 200 can be installed on any type of computing system 10.

[0031] In an embodiment, the host device 100 can offload neural network computation to the data processing system 200 and provide the data processing system 200 with initial parameters for neural network computation, such as input vectors and weight matrices.

[0032] In one embodiment, the data processing system 200 may be an application processor installed in a mobile device.

[0033] The data processing system 200 may include at least a neural network processor 300.

[0034] The neural network processor 300 can generate neural network models by training or learning from input data, generate information signals by inferring input data from the neural network model, or retrain the neural network model. Neural networks can include, but are not limited to, various types of neural network models such as: Convolutional Neural Networks (CNNs), Region Convolutional Neural Networks (R-CNNs), Region Proposal Networks (RPNs), Recurrent Neural Networks (RNNs), Stacked Deep Neural Networks (S-DNNs), State-Space Dynamic Neural Networks (S-SDNNs), Deconvolutional Networks, Deep Belief Networks (DBNs), Restricted Boltzmann Machines (RBMs), Fully Convolutional Networks, Long Short-Term Memory (LSTM) Networks, and Classification Networks.

[0035] Figure 2 This is a configuration diagram illustrating a neural network processor according to an embodiment of the present disclosure.

[0036] The neural network processor 300 can be a processor or accelerator specifically designed for neural network computations, and as... Figure 2 As shown, the device includes a computing device 310 with built-in memory, a controller 320, and RAM 330. In one embodiment, the neural network processor 300 can be implemented as a System-on-a-Chip (SoC) integrated into a single semiconductor chip. However, the embodiment is not limited thereto, and the neural network processor 300 can be implemented as multiple semiconductor chips.

[0037] The controller 320 can control all operations of the neural network processor 300. The controller 320 can set and manage parameters related to neural network computation, enabling the computing device 310 with built-in memory to perform neural network computations correctly. The controller 320 can be implemented in hardware, software (firmware), or a combination of hardware and software.

[0038] The controller 320 can be implemented as one or more processors, such as a central processing unit (CPU), a microprocessor, etc., and executes instructions that constitute various functions stored in RAM 330.

[0039] When the host device 100 offloads neural network computation to the neural network processor 300 by transferring operands, including vectors and matrices, to the neural network processor 300, the controller 320 can transfer the operands and the addresses to which the operands are to be stored to the computing device 310 with built-in memory.

[0040] RAM 330 can be implemented as DRAM, SRAM, etc., and temporarily stores various programs and data for the operation of controller 320, as well as data generated by controller 320.

[0041] The computing device 310 with built-in memory can be configured to perform neural network calculations under the control of the controller 320. The computing device 310 with built-in memory may include a computing memory 311, a global buffer 313, an accumulation circuit (ACCU) 315, an activation circuit (ACTIV) 317, a pooling circuit (POOL) 319, and a negative number calculation control circuit 500.

[0042] The computation memory 311 may include multiple processing elements PE. Each processing element PE may receive vectors and matrices as operands from the global buffer 313 and perform vector-matrix multiplication (VMM). In an embodiment, the vector may be an input feature map for neural network computation, and the matrix may be a weight matrix.

[0043] Each processing element (PE) may include multiple subarrays. Each subarray may include multiple memory cells connected between multiple row lines and multiple column lines. For neural network computation, a weight matrix serving as a first operand can be stored in the memory cells of the subarray, and a vector corresponding to the input feature map and serving as a second operand can be applied to the row lines of the subarray. Built-in memory computation, such as a Virtual Memory Model (VMM), can then be performed. The VMM can be a convolution computation, such as element-wise multiplication and addition. In an embodiment, the first operand may be an NxM matrix, and the second operand may be a 1xN vector. Here, N and M are both positive integers.

[0044] In an embodiment, the subarray may be a cross array of memory devices including memristor elements. The subarray can be programmed such that the memristor memory cells arranged at the intersections of the cross array have conductances corresponding to the values ​​of the individual elements of the matrix, and the elements of the vector can be converted into analog input voltages, which can then be applied to the row lines. Therefore, the input voltage applied to each row line of the cross array can be increased by the conductance of the memristor memory cells, and accumulated and output as the current value of each column line.

[0045] The global buffer 313 can store operands therein and then provide the stored operands to the computation memory 311. Furthermore, the global buffer 313 can receive computation results from the computation memory 311 and store the received computation results in the global buffer. The global buffer 313 can be implemented as DRAM or SRAM.

[0046] The accumulator circuit 315 can be configured to obtain a weighted sum by accumulating the processing results of each processing element PE.

[0047] The activation circuit 317 can be configured to increase nonlinearity by applying the weighted sum of the accumulator circuit 315 to an activation function such as ReLU.

[0048] The pooling circuit 319 can sample the output value of the activation circuit 317 and reduce and optimize the dimensionality.

[0049] The process of calculating memory 311, accumulation circuit 315, activation circuit 317 and pooling circuit 319 can represent the process of training or retraining a neural network model or inferring input data.

[0050] When an operand transmitted from the host device 100 or an operand generated during the intermediate process of neural network computation includes a negative element, the negative number calculation control circuit 500 can convert the negative element into a non-negative element, i.e., a zero element or a positive element, and store the operand including the converted non-negative element in the global buffer 313.

[0051] In an embodiment, when negative elements are included in a matrix used as a first operand, the negative number calculation control circuit 500 can split the elements constituting the matrix into a positive matrix composed of positive elements and a negative matrix composed of the absolute values ​​of negative elements, and store the positive and negative matrices in a global buffer 313. The positive and negative matrices stored in the global buffer 313 can be stored in a first subarray and a second subarray, respectively.

[0052] In an embodiment, when negative elements are included in a vector used as a second operand, the negative number calculation control circuit 500 can generate an offset vector by adding an offset to each element of the vector, and can apply the offset vector to the row lines of the first subarray and the second subarray storing the positive matrix and the negative matrix, respectively. The offset is used to convert the negative element with the largest absolute value among the elements of the vector into a non-negative element.

[0053] The computation memory 311 can perform a virtual machine model (VMM) on operands including transformed non-negative elements and store the VMM result in a global buffer 313.

[0054] When computation memory 311 performs a Virtual Model (VMM) on a matrix and a vector containing negative elements, it can store the positive matrix in a first subarray of the processing element PE and apply an offset vector to the row lines of the first subarray to output a positive calculated value. Similarly, it can store the negative matrix in a second subarray and apply an offset vector to the row lines of the second subarray to output a negative calculated value. Since a VMM has been performed on the absolute values ​​of the negative elements included in the matrix, computation memory 311 combines the calculation results by subtracting the negative calculated value from the positive calculated value.

[0055] When performing a Virtual Model (VMM) on the offset value obtained by applying an offset to a vector, the negative number calculation control circuit 500 can derive the final calculation result by correcting the VMM result based on the offset. For example, the negative number calculation control circuit 500 can calculate a first correction value as the result of the VMM between the offset and the positive matrix and a second correction value as the result of the VMM between the offset and the negative matrix, and obtain the offset correction value by subtracting the second correction value from the first correction value. Then, the negative number calculation control circuit 500 can obtain the final calculation result by subtracting the offset correction value from the combined calculation result. In an embodiment, each of the first subarray and the second subarray can be an [NxM] array, and the above parameters can have the following dimensions.

[0056] parameter Dimension positive matrix [NxM] matrix negative matrix [NxM] matrix Offset [1xN] vector offset vector [1xN] vector Positive calculated value [1xM] vector negative calculated value [1xM] vector Calculation results [1xM] vector First correction value [1xM] vector Second correction value [1xM] vector Offset correction value [1xM] vector

[0057] Computational memory using memristor memory cells configured as a cross array stores matrices as conductances in the memory cells and applies vectors as voltages to the row lines, performing a virtual matrix model (VMM) based on the principle of adding the current corresponding to the product of voltage and conductance for each column line. That is, it is impossible to perform a VMM on negative elements because conductance and voltage are used to perform the simulation calculations.

[0058] However, according to the embodiment, the matrix containing negative elements is divided into a positive matrix and a negative matrix containing the absolute values ​​of the negative elements. In order to perform VMM, the negative elements are converted into non-negative elements according to the offset, and the VMM results are combined and corrected. Therefore, even when the matrix or operands contain negative elements, the built-in memory computation can be performed in a simulated manner.

[0059] Figure 3 This is a diagram illustrating the configuration of a computing memory according to an embodiment of the present disclosure.

[0060] Reference Figure 3 According to the embodiment, the computing memory 311 can be divided into multiple tiles.

[0061] Each block may include a block input buffer 410, multiple processing elements PE, and a block output buffer 420.

[0062] Each processing element PE may include a PE input buffer 430, multiple subarrays SA, and a cumulative / PE output buffer 440.

[0063] The subarray SA, also known as a synaptic array, includes multiple word lines WL1, WL2, ..., WLN, multiple bit lines BL1, BL2, ..., BLM, and multiple memory cells MC. Word lines WL1, WL2, ..., WLN can also be called row lines, and bit lines BL1, BL2, ..., BLM can also be called column lines. In embodiments, each memory cell MC may each include a resistive memory element RE, or preferably a memristor element. However, embodiments are not limited to this. The conductance can be changed by a write voltage applied through the word lines WL1, WL2, ..., WLN and the bit lines BL1, BL2, ..., BLM, thus changing the data value stored in the memory cell MC, and the resistive memory cell can store data by such a change in resistance.

[0064] In the embodiments, each resistive memory cell may be implemented as a phase-change random access memory (PRAM) cell, a resistive random access memory (RRAM) cell, a magnetic random access memory (MRAM) cell, or a ferroelectric random access memory (FRAM) cell.

[0065] Examples of resistive elements constituting a resistive memory cell may include, but are not limited to, phase change materials, perovskite compounds, transition metal oxides, magnetic materials, ferromagnetic materials, or antiferromagnetic materials whose crystalline state changes according to the amount of current.

[0066] When the unit cell of the subarray SA is configured as a memristor element, the processing element PE can store the data corresponding to each element of the weight matrix in the memristor, apply the voltage corresponding to each element of the input feature map to the word lines WL1, WL2, ..., WLN, and perform VMM by utilizing Kirchhoff's current law and Ohm's law.

[0067] Each of the bit lines BL1, BL2, ..., BLM can also be called an output channel and is connected to a combiner / analog-to-digital converter (COMB / ADC). The COMB / ADC can sense the VMM result applied to the bit lines BL1, BL2, ..., BLM and output the sensed result as a digital value.

[0068] Specifically, when a matrix containing negative elements is split into positive and negative matrices such that the positive and negative matrices are processed in different subarrays SA, the positive and negative computed values ​​can be converted into digital values ​​using a COMB / ADC, and then combined using subtraction by a digital subtractor. Specifically, the negative computed value can be converted to the complement of 2, and the complement can be added to the negative computed value and combined with the positive computed value.

[0069] In an embodiment, the COMB / ADC can perform subtraction on positive and negative calculated values ​​using an analog subtractor, and then convert the result value into a digital value.

[0070] Figure 4 This is a configuration diagram showing a negative number calculation control circuit according to an embodiment of the present disclosure.

[0071] Reference Figure 4 The negative number calculation control circuit 500 may include a matrix splitting circuit 510, a vector conversion circuit 520, and an offset correction circuit 530.

[0072] When the first operand includes a negative element, the matrix splitting circuit 510 can split the elements constituting the matrix into a positive matrix composed of positive elements and a negative matrix composed of the absolute values ​​of negative elements, and store the positive and negative matrices in the global buffer 313, wherein the first operand is a matrix generated by intermediate calculation results from the host device 100 or the computing memory 311.

[0073] When negative elements are included in a vector that is the second operand, the vector conversion circuit 520 can determine an offset for converting the negative element with the largest absolute value among the elements of the vector into a non-negative element, and add the offset to each element of the vector to generate an offset vector.

[0074] When the vector conversion circuit 520 determines the offset, the offset correction circuit 530 can calculate a first correction value as the VMM result between the positive matrix and the offset, and a second correction value as the VMM result between the negative matrix and the offset, and obtain the offset correction value by subtracting the second correction value from the first correction value. When the positive calculated value obtained from the positive matrix and the offset vector, and the negative calculated value obtained from the negative matrix and the offset vector, are combined and output through the computation memory 311, the offset correction circuit 530 can calculate the final calculation result by subtracting the offset correction value from the combined calculation result.

[0075] Figure 5 This is a flowchart describing an operation method of a data processing system according to embodiments of the present disclosure.

[0076] In operation S101, when the first operand and the second operand are provided to the neural network processor 300 of the data processing system 200, in operation S103, the negative number calculation control circuit 500 can check whether the negative element is included in the first operand and / or the second operand.

[0077] When the matrix used as the first operand includes negative elements (in operation S103, it is a matrix), in operation S105, the negative number calculation control circuit 500 can split the elements constituting the matrix into a positive matrix composed of positive elements and a negative matrix composed of the absolute values ​​of negative elements, and store the positive and negative matrices in the global buffer 313. The positive and negative matrices stored in the global buffer 313 can be stored in the first subarray and the second subarray, respectively.

[0078] When the vector used as the second operand includes negative elements (in operation S103, it is a vector), in operation S107, the negative number calculation control circuit 500 can determine an offset for converting the negative element with the largest absolute value among the elements of the vector into a non-negative element (i.e., a zero element or a positive element), and add the offset to each element of the vector to generate an offset vector. The offset vector can be applied to the row lines of the first subarray and the second subarray, which store the positive matrix and the negative matrix respectively.

[0079] When the matrix values ​​are stored in the first subarray and the second subarray and the offset vector is applied to the row lines, in operation S109, the built-in memory VMM is executed when the current corresponding to the product between the matrix (conductance) and the vector (voltage) is added for each column line.

[0080] As a result of the VMM, positive computed values ​​can be output from the first subarray, and negative computed values ​​can be output from the second subarray. Since the VMM has already been performed on the absolute values ​​of the negative elements included in the matrix, the computed results are combined in operation S111 by subtracting the negative computed values ​​from the positive computed values.

[0081] In one embodiment, to combine the calculation results, the positive and negative calculated values ​​can be converted into numerical values ​​respectively, and a digital subtractor can be used to perform subtraction on the numerical values. In another embodiment, to combine the calculation results, an analog subtractor can be used to subtract the negative calculated value from the positive calculated value, and the subtraction result can be converted into a numerical value.

[0082] When performing a Virtual Model (VMM) on the offset vector obtained by applying the offset to the vector, in operation S113, the negative number calculation control circuit 500 can derive the final calculation result based on the offset-corrected VMM result. For example, the negative number calculation control circuit 500 can calculate a first correction value as the VMM result between the offset and the positive matrix and a second correction value as the VMM result between the offset and the negative matrix, and obtain the offset correction value by subtracting the second correction value from the first correction value. Then, the negative number calculation control circuit 500 can obtain the final calculation result by subtracting the offset correction value from the calculation result combined in operation S111.

[0083] In operation S115, the final calculation result can be output to the global buffer 313.

[0084] When neither the first operand nor the second operand contains a negative element (no in operation S103), in operation S117, the operand can be provided to the computation memory 311 to execute the VMM, and in operation S115, the VMM result can be output to the global buffer 313.

[0085] Figures 6A to 6E This is a conceptual diagram used to describe a VMM including negative elements according to embodiments of the present disclosure.

[0086] Figures 6A to 6E The operands, intermediate calculated values, and final calculation results are shown as decimal numbers. However, the actual calculations in the data processing system 200 are performed based on binary or hexadecimal numbers. Figures 6A to 6E Even though they are shown as column vectors, each of the vectors VEC and OFS_VEC is a 1xN vector (i.e., a row vector).

[0087] Figure 6A The procedure for performing a VMM on the first operand MAT and the second operand VEC is shown.

[0088] Since the matrix MAT, which is the first operand, contains negative elements, therefore... Figure 6B As shown, the negative number calculation control circuit 500 can split the matrix MAT into a positive matrix PMAT composed of positive elements and a negative matrix NMAT composed of the absolute values ​​of negative elements, and then store the positive matrix PMAT and the negative matrix NMAT in the global buffer 313. The positive matrix PMAT and the negative matrix NMAT stored in the global buffer 313 can be stored in the first subarray and the second subarray, respectively.

[0089] When the vector VEC, used as the second operand, includes negative elements, the negative number calculation control circuit 500 determines an offset for converting the negative element (-3) with the largest absolute value among the elements of the vector into a non-negative element (i.e., a zero element or a positive element). For example... Figure 6B As shown, the negative number calculation control circuit 500 can determine the offset OFS as 4 and generate the offset vector OFS_VEC by adding the offset to each element of the vector. The offset vector can be applied to the row lines of the first and second subarrays that store the positive and negative matrices, respectively.

[0090] When the matrix values ​​are stored in the first and second subarrays and the offset vectors are applied to the row lines, the built-in memory VMM is executed when the current corresponding to the multiplication between the matrix (conductance) and the vector (voltage) is added for each column line, as shown. Figure 6C As shown.

[0091] Figure 6D The combined calculation result UVMM is shown by subtracting the negative calculation value NVMM, which is the result of VMM, from the positive calculation value PVMM output from the first subarray.

[0092] Since the offset vector OFS_VEC, obtained by adding the offset OFS to the vector VEC, has already been used to perform the VMM, the negative calculation control circuit 500 drives the final calculation result OUT by subtracting the offset correction value COF from the combined calculation result UVMM. Figure 6E As shown.

[0093] Figure 7 This is a conceptual diagram used to describe an offset correction value generation method according to embodiments of the present disclosure.

[0094] To generate the offset correction value COF, the negative number calculation control circuit 500 calculates a first correction value PCOFS as the VMM result between the offset OFS and the positive matrix PMAT, and a second correction value NCOFS as the VMM result between the offset OFS and the negative matrix NMAT. By subtracting the second correction value NCOFS from the first correction value PCOFS, the negative number calculation control circuit 500 obtains the offset correction value COFS. Figure 6E As shown, the negative number calculation control circuit 500 obtains the final calculation result OUT by subtracting the offset correction value COF from the combined calculation result UVMM.

[0095] Figure 8 This is a configuration diagram showing a negative number calculation control circuit according to an embodiment of the present disclosure.

[0096] Reference Figure 8 The negative number calculation control circuit 500-1 may include a matrix splitting circuit 510, a vector conversion circuit 520, an offset vector splitting circuit 540, and an offset correction circuit 530.

[0097] When the first operand of a matrix generated as an intermediate computation result of the host device 100 or the computation memory 311 includes negative elements, the matrix splitting circuit 510 can split the elements constituting the matrix into a positive matrix composed of positive elements and a negative matrix composed of the absolute values ​​of negative elements, and store the positive and negative matrices in the global buffer 313.

[0098] When negative elements are included in a vector that is the second operand, the vector conversion circuit 520 can determine an offset for converting the negative element with the largest absolute value among the elements of the vector into a non-negative element, and add the offset to each element of the vector to generate an offset vector.

[0099] The offset vector splitting circuit 540 can generate a sequential vector VEC_SEQ comprising one or more partial offset vectors by splitting the elements constituting the offset vector bit by bit according to their bit values. For example, when each element of the offset vector is a binary number consisting of X bits (where X is a positive integer), the offset vector splitting circuit 540 can split the elements constituting the second offset vector... M Each element of the offset vector is assigned a bit (M is 0 or a positive integer between 1 and X), and a partial offset vector is generated. The sequence vector VEC_SEQ can be configured as multiple partial offset vectors.

[0100] Partial offset vectors can be sequentially provided to the first and second subarrays by the vector conversion circuit 520, and pass through the VMM together with the positive and negative matrices.

[0101] When the offset is determined by the vector conversion circuit 520, the offset correction circuit 530 can calculate a first correction value as the VMM result between the positive matrix and the offset, and a second correction value as the VMM result between the negative matrix and the offset, and obtain the offset correction value by subtracting the second correction value from the first correction value. When the positive calculated value obtained from the positive matrix and the sequentially provided partial offset vectors, and the negative calculated value obtained from the negative matrix and the sequentially provided partial offset vectors are combined and output by the computation memory 311, the offset correction circuit 530 can calculate the final calculation result by subtracting the offset correction value from the combined calculation result.

[0102] Figure 9 This is a diagram illustrating the configuration of processing elements according to an embodiment of the present disclosure.

[0103] Reference Figure 9 According to the embodiment, the processing element PE may include a bit-to-analog converter (DAC) 610, a first subarray 620, a second subarray 630, an ADC 640, a bit sorting circuit 650, a subtractor 660, and an output buffer 670.

[0104] The 1-bit DAC 610 is configured to convert a portion of the offset vector generated by the offset vector splitting circuit 540 into analog values ​​and apply the analog values ​​to the row lines of the first subarray 620 and the second subarray 630.

[0105] The first subarray 620 can store positive matrices, and the second subarray 630 can store negative matrices. Partially positive calculated values ​​between the positive matrices and the sequentially applied partial offset vectors are sequentially output from the first subarray 620. Partially negative calculated values ​​between the negative matrices and the sequentially applied partial offset vectors are sequentially output from the second subarray 630.

[0106] The ADC 640 can convert some of the positive computed values ​​sequentially output from the first subarray 620 into digital values, and convert some of the negative computed values ​​sequentially output from the second subarray 630 into digital values.

[0107] The bit value sorting circuit 650 can sort the partially positive and partially negative calculated values ​​output sequentially from the ADC 640 according to the bit values ​​of a portion of the offset vector, and then add the calculated results of the bit value sorting.

[0108] For example, when each element of the offset vector consists of X bits (where X is a positive integer), the bit-sorting circuit 650 can sort the offset vector by the second bit. M The partially positive and partially negative computed values ​​of the corresponding partial offset vector (where M is 0 or a positive integer between 1 and X) are converted to 2. M Bit value. Then, the bit value sorting circuit 650 can add all the positive calculated values ​​of the sorted bit value and output a positive calculated value, and add all the negative calculated values ​​of the sorted bit value and output a negative calculated value.

[0109] Subtractor 660 can subtract a negative calculated value from a positive calculated value and store the subtraction result in output buffer 670. In an embodiment, subtractor 660 can convert a negative calculated value to a two's complement, convert a negative calculated value to a negative number, and add the negative number to the positive calculated value.

[0110] Figures 10A to 10E This is a conceptual diagram used to describe an offset vector providing method according to embodiments of the present disclosure. In Figures 10 to 10E, although shown as column vectors, each of the vectors OFS_VEC and SOFS_VECx can be a 1×N vector (row vector).

[0111] like Figure 6B The elements (7, 1, 5) of the generated offset vector OFS_VEC shown can be represented as follows: Figure 10AThe 4-bit binary number shown is used, and the offset vector splitting circuit 540 configures the sequence vector VEC_SEQ by splitting the offset vector OFS_VEC bit by bit according to the bit value. That is, the sequence vector VEC_SEQ can be configured as multiple partial offset vectors SOFS_VEC1, SOFS_VEC2, SOFS_VEC3, and SOFS_VEC4.

[0112] The partial offset vectors SOFS_VEC1, SOFS_VEC2, SOFS_VEC3, and SOFS_VEC4 that constitute the sequence vector VEC_SEQ are sequentially provided to the first subarray 620 and the second subarray 630, which respectively store the positive matrix PMAT and the negative matrix NMAT.

[0113] The bit value sorting circuit 650 can sort the partially positive calculated values ​​between the partially offset vectors SOFS_VEC1, SOFS_VEC2, SOFS_VEC3, and SOFS_VEC4 and the positive matrix, and the partially negative calculated values ​​between the partially offset vectors SOFS_VEC1, SOFS_VEC2, SOFS_VEC3, and SOFS_VEC4 and the negative matrix, based on the bit values ​​of the partially offset vectors SOFS_VEC1, SOFS_VEC2, SOFS_VEC3, and SOFS_VEC4.

[0114] like Figure 10B As shown, the first portion of the offset vector SOFS_VEC1 is applied to the row lines of the first and second subarrays 620 and 630 to execute the VMM at the first time point t1. When the first portion of the offset vector SOFS_VEC1 corresponds to the second offset vector OFS_VEC... 3 When the bit is selected, the bit value sorting circuit 650 sorts the partially positive and partially negative calculated values ​​of the first part of the offset vector SOFS_VEC1 into values ​​corresponding to the second bit. 3 The value of the bit.

[0115] like Figure 10C As shown, the second part of the offset vector SOFS_VEC2 is applied to the row lines of the first subarray 620 and the second subarray 630 to execute the VMM at the second time point t2. When the second part of the offset vector SOFS_VEC2 corresponds to the second offset vector OFS_VEC... 2 At bit 1, the bit value sorting circuit 650 sorts the partially positive and partially negative calculated values ​​of the second part of the offset vector SOFS_VEC2 into those corresponding to the second bit. 2 The value of the bit.

[0116] like Figure 10DAs shown, the third offset vector SOFS_VEC3 is applied to the row lines of the first subarray 620 and the second subarray 630 to perform VMM at the third time point t3. When the third offset vector SOFS_VEC3 corresponds to the second offset vector OFS_VEC... 1 At bit 1, the bit value sorting circuit 650 sorts the partially positive and partially negative calculated values ​​of the third part of the offset vector SOFS_VEC3 into those corresponding to the second bit. 1 The value of the bit.

[0117] like Figure 10E As shown, the fourth offset vector SOFS_VEC4 is applied to the row lines of the first subarray 620 and the second subarray 630 to perform VMM at the fourth time point t4. Since the fourth offset vector SOFS_VEC4 corresponds to the second offset vector OFS_VEC... 0 At bit 1, the bit value sorting circuit 650 sorts the partially positive and partially negative calculated values ​​of the fourth part of the offset vector SOFS_VEC4 into those corresponding to the second bit. 0 The value of the bit.

[0118] After the bit values ​​are sorted, the bit value sorting circuit 650 can add up all the positive calculated values ​​of the positive matrix and output the positive calculated value, and add up all the negative calculated values ​​of the negative matrix and output the negative calculated value.

[0119] As mentioned above, after combining the positive and negative calculated values, the final calculation result can be calculated by subtracting the offset correction value from the combined calculation result.

[0120] Figure 11 This is a flowchart describing an operation method of a data processing system according to embodiments of the present disclosure.

[0121] When the first operand and the second operand are provided to the neural network processor 300 of the data processing system 200 in operation S201, the negative number calculation control circuit 500 can check whether the negative element is included in the first operand and / or the second operand in operation S203.

[0122] When the matrix used as the first operand includes negative elements (in operation S203, it is a matrix), in operation S205, the negative number calculation control circuit 500 can split the elements constituting the matrix into a positive matrix composed of positive elements and a negative matrix composed of the absolute values ​​of negative elements, and store the positive and negative matrices in the global buffer 313. The positive and negative matrices stored in the global buffer 313 can be stored in the first subarray and the second subarray, respectively.

[0123] When the vector used as the second operand includes negative elements (in operation S203, it is (vector)), in operation S207, the negative number calculation control circuit 500 can determine the offset used to convert the negative element with the largest absolute value among the elements of the vector into a non-negative element (i.e., a zero element or a positive element), and add the offset to each element of the vector to generate an offset vector.

[0124] In operation S209, the negative number calculation control circuit 500 can generate a sequential vector consisting of one or more partial offset vectors by splitting each element constituting the offset vector bit by bit according to the bit value.

[0125] For example, in operation S211, a portion of the offset vector can be sequentially provided from the most significant bit (MSB) to the first subarray and the second subarray, and passed through the VMM together with the positive matrix and the negative matrix, respectively.

[0126] When the sequentially provided partial offset vectors, together with the positive and negative matrices, pass through the VMM and are converted into digital values, in operation S213, the bit value sorting circuit 650 sorts the sequentially output partial positive and partial negative calculated values ​​according to the bit values ​​of the partial offset vectors.

[0127] In operation S215, the negative number calculation control circuit 500 checks whether there is a partial offset vector that has not yet been input, or whether the partial offset vector processed at a previous time point is the last partial offset vector. When the check result indicates that the last partial offset vector has not been processed (no in operation S215), in operation S211, the negative number calculation control circuit 500 applies the partial offset vector of the next bit value to the first subarray 620 and the second subarray 630, thereby performing built-in memory calculation. When calculating the calculated value of the last partial offset vector (yes in operation S215), in operation S217, the negative number calculation control circuit 500 obtains a positive calculated value by adding all the partially positive calculated values ​​of the sorted bit values, obtains a negative calculated value by adding all the partially negative calculated values ​​of the sorted bit values, and then combines the positive and negative calculated values. Operations S209 to S214 can be applied... Figure 5 The operations on offset vectors described in [the document].

[0128] In one embodiment, to combine calculated values, the negative number calculation control circuit 500 can convert the positive and negative calculated values ​​into digital values, convert the negative calculated value into the complement of 2, and then add the complement to the positive calculated value. In another embodiment, to combine the calculation results, a simulated subtractor can be used to subtract the negative calculated value from the positive calculated value, and then the subtraction result can be converted into a digital value.

[0129] When performing a Virtual Model (VMM) on the offset vector obtained by applying the offset to the vector, in operation S219, the negative number calculation control circuit 500 can derive the final calculation result based on the offset-corrected VMM result. For example, the negative number calculation control circuit 500 can calculate a first correction value as the VMM result between the offset and the positive matrix and a second correction value as the VMM result between the offset and the negative matrix, and obtain the offset correction value by subtracting the second correction value from the first correction value. Then, the negative number calculation control circuit 500 can obtain the final calculation result by subtracting the offset correction value from the calculation result combined in operation S217.

[0130] In operation S221, the final calculation result can be output to the global buffer 313.

[0131] When neither the first operand nor the second operand contains a negative element (no in operation S203), in operation S223, the operand can be provided to the computation memory 311 to execute the VMM, and in operation S221, the VMM result can be output to the global buffer 313.

[0132] Even when the vectors or matrices used as operands include negative elements in the cross array that performs analog calculations through resistance and voltage, negative number calculations can be performed in a simple way without increasing system complexity, thus enabling efficient neural network calculations.

[0133] Thus, those skilled in the art will understand that the present invention can be implemented in other specific forms without altering its technical spirit or essential characteristics. Therefore, it should be understood that the above embodiments are not restrictive but illustrative in all respects. The scope of this disclosure is defined by the claims described below, rather than by a detailed description, and all variations or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as included within the scope of this disclosure. Furthermore, embodiments may be combined to form additional embodiments.

Claims

1. A data processing system, comprising: Computational memory, comprising one or more subarrays, each subarray comprising multiple memory cells connected between multiple row lines and multiple column lines; Matrix splitting circuit: When the matrix received from the host device includes negative elements, the matrix is ​​split into a positive matrix composed of the positive elements of the matrix and a negative matrix composed of the absolute values ​​of the negative elements of the matrix. The positive matrix and the negative matrix are stored in the first subarray and the second subarray in the computing memory, respectively; Vector conversion circuit: When the vector received from the host device includes negative elements, an offset vector is generated by adding the elements of the vector to an offset, the offset being used to convert the negative elements with the largest absolute value among the elements of the vector into zero elements or positive elements. The offset vector is applied to the row lines of the first subarray and the second subarray; as well as Offset correction circuit: An offset correction value is generated by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix. The offset correction value is subtracted from the calculated value output from the first subarray and the second subarray.

2. The data processing system according to claim 1, wherein the computation memory generates the computation value by subtracting the negative computation value from the positive computation value, the positive computation value is output from the first subarray as the result of multiplying the positive matrix and the offset vector, and the negative computation value is output from the second subarray as the result of multiplying the negative matrix and the offset vector.

3. The data processing system according to claim 2, wherein the computing memory further comprises: An analog-to-digital converter converts the positive calculated value and the negative calculated value into digital positive calculated value and digital negative calculated value, respectively. as well as A digital subtractor subtracts the digital negative calculated value from the digital positive calculated value to generate the calculated value.

4. The data processing system according to claim 2, wherein the computing memory further comprises: A simulated subtractor subtracts the negative calculated value from the positive calculated value; as well as An analog-to-digital converter converts the output of the analog subtractor to generate the calculated value.

5. The data processing system according to claim 1, Each element constituting the offset vector is a binary number consisting of multiple bits. The data processing system further includes an offset vector splitting circuit, wherein the offset vector splitting circuit: An ordered vector comprising one or more partial offset vectors is generated by splitting the offset vector bit by bit according to its bit value, and The vector conversion circuit sequentially applies the partial offset vectors to the row lines of the first subarray and the second subarray.

6. The data processing system according to claim 5, wherein the computing memory further includes a bit value sorting circuit, the bit value sorting circuit: The partial positive calculated values, which are the result of multiplying the partial offset vector with the positive matrix, are sorted according to the bit values ​​of the partial offset vector. The partial negative calculated values, which are the result of multiplying the partial offset vector with the negative matrix, are sorted according to the bit values ​​of the partial offset vector. The positive calculated value is obtained by adding all the sorted partial positive calculated values, and The negative calculated value is obtained by adding up all the sorted partial negative calculated values.

7. A method for operating a data processing system, comprising: A computing memory is provided, the computing memory comprising one or more subarrays, each subarray comprising a plurality of memory cells connected between a plurality of row lines and a plurality of column lines; When the matrix received from the host device includes negative elements, the negative number calculation control circuit splits the matrix into a positive matrix composed of the positive elements of the matrix and a negative matrix composed of the absolute values ​​of the negative elements of the matrix. The positive matrix and the negative matrix are stored in the first subarray and the second subarray in the computing memory, respectively; When the vector received from the host device includes negative elements, the negative number calculation control circuit adds the elements in the vector to the offset to generate an offset vector. The offset is used to convert the negative elements with the largest absolute value in the vector into zero elements or positive elements. The negative number calculation control circuit applies the offset vector to the row lines of the first subarray and the second subarray; The negative number calculation control circuit generates an offset correction value by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix; and The negative number calculation control circuit subtracts the offset correction value from the calculated values ​​output from the first subarray and the second subarray.

8. The method of operation according to claim 7, further comprising generating the calculated value by subtracting the negative calculated value from the positive calculated value, wherein the positive calculated value is output from the first subarray as the result of multiplying the positive matrix and the offset vector, and the negative calculated value is output from the second subarray as the result of multiplying the negative matrix and the offset vector.

9. The operation method according to claim 8, wherein the generation of the calculated value includes: The computing memory converts the positive calculated value and the negative calculated value into digital positive calculated value and digital negative calculated value, respectively; as well as The computational memory generates the calculated value by subtracting the digitized negative calculated value from the digitized positive calculated value.

10. The method of operation according to claim 8, wherein the generation of the calculated value comprises: The positive calculated value minus the negative calculated value; and The calculated value is generated by converting the output of the analog subtractor into a digital value.

11. The operating method according to claim 7, Each element constituting the offset vector is a binary number consisting of multiple bits. The application of the offset vector includes: An ordered vector comprising one or more partial offset vectors is generated by splitting the offset vector bit by bit according to its bit value; and The partial offset vectors are sequentially applied to the row lines of the first subarray and the second subarray.

12. The operating method according to claim 11, further comprising: The partial positive calculated values, which are the result of multiplying the partial offset vector with the positive matrix, are sorted according to the bit values ​​of the partial offset vector. The partial negative calculated values, which are the result of multiplying the partial offset vector with the negative matrix, are sorted according to the bit values ​​of the partial offset vector. The positive calculated value is obtained by adding all the sorted partial positive calculated values, and The negative calculated value is obtained by adding up all the sorted partial negative calculated values.

13. A computing system, comprising: Main unit; A data processing system that processes application calculations according to a request from the host device, and includes a computing memory comprising one or more subarrays, each subarray comprising a plurality of memory cells connected between a plurality of row lines and a plurality of column lines; as well as Negative number calculation control circuit: When the matrix received from the host device includes negative elements, the matrix is ​​split into a positive matrix and a negative matrix; When the vector received from the host device includes a negative element, an offset vector is generated by adding the vector to the offset; and Based on the offset correction value generated from the offset, a correction calculation value is performed, and the calculation value is output from the computation memory as the result of multiplying each of the positive matrix and the negative matrix with the offset vector.

14. The computing system of claim 13, wherein the subarray comprises: The first subarray stores the positive matrix and receives the offset vector through its row lines; as well as The second subarray stores the negative matrix and receives the offset vector through its row lines.

15. The computing system of claim 13, wherein the negative number calculation control circuit further generates the offset correction value by subtracting the result of multiplying the offset by the negative matrix from the result of multiplying the offset by the positive matrix.

16. The computing system of claim 13, wherein the computing memory generates the computed value by subtracting a negative computed value from a positive computed value, the positive computed value being output from a first subarray as the result of multiplying the positive matrix and the offset vector, and the negative computed value being output from a second subarray as the result of multiplying the negative matrix and the offset vector.

17. The computing system of claim 13, wherein the negative number calculation control circuit further subtracts the offset correction value from the calculated value.

18. The computing system according to claim 13, Each element constituting the offset vector is a binary number consisting of multiple bits. The negative number calculation control circuit further includes: An ordered vector comprising one or more partial offset vectors is generated by splitting the offset vector bit by bit according to its bit value, and The partial offset vectors are sequentially applied to the row lines of the subarrays that store the positive and negative matrices, respectively.

19. The computing system of claim 18, wherein the computing memory further includes: The partial positive calculated values, which are the result of multiplying the partial offset vector with the positive matrix, are sorted according to the bit values ​​of the partial offset vector. The partially negative calculated values, which are the result of multiplying the partially offset vector with the negative matrix, are sorted according to the bit values ​​of the partially offset vector, and Subtract the negative values ​​from the sorted positive values.

Images