Neural network computing device and method for operating the same

Abstract

A neural network computing device and a method of operating the same are disclosed. The method includes: receiving data for neural network computing; verifying whether a competition occurs between a first data traversal path corresponding to a first computing device and a second data traversal path corresponding to a second computing device; determining first operand data and second operand data from the data using the verification result and the priority between the first data traversal path and the second data traversal path; sending the first operand data to the first computing device and sending the second operand data to the second computing device; and performing neural network computing based on the first operand data by the first computing device and performing neural network computing based on the second operand data by the second computing device.

Description

[0001] This application claims the benefit of Korean Patent Application No. 10-2021-0016943, filed on Feb. 5, 2021, and Korean Patent Application No. 10-2021-0036060, filed on Mar. 19, 2021, in the Korean Intellectual Property Office, the entire disclosures of all of the Korean patent applications being incorporated herein by reference for all purposes. Technical Field

[0002] The following description relates to a neural network operation device and a method of operating the same. Background Art

[0003] A neural network or an artificial neural network (ANN) can generate a mapping between an input pattern and an output pattern and can have the ability to generate a relatively correct output for an input pattern not yet used for training. A neural processor is designed to accelerate the operation of a neural network. Accelerating neural network operations may involve reducing the time to obtain an output by minimizing the number of multiplication operations that are the core of neural network operations.

[0004] For efficient neural network operations, various techniques (such as pruning and quantization) have been used.

[0005] Pruning is a compression method that involves removing nodes, weights, and connections that are elements of a neural network. Pruning is intended to maintain the accuracy of a neural network while improving its efficiency.

[0006] Pruning and quantization utilize sparsity to efficiently perform neural network operations. However, when sparsity is utilized, a load imbalance may occur between operation devices. That is, since the elements to be excluded from the arranged operation set are concentrated on a specific device, an uneven load is applied to all devices, which may lead to a reduction in performance. Summary of the Invention

[0007] This Summary of the Invention is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter.

[0008] In one general aspect, a method for operating a neural network computing device is provided. The method includes: receiving data for neural network computing; verifying whether a competition occurs between a first data traversal path corresponding to a first computing device and a second data traversal path corresponding to a second computing device; determining first operand data and second operand data from the data using the verification result and the priority between the first data traversal path and the second data traversal path; sending the first operand data to the first computing device and sending the second operand data to the second computing device; and performing neural network computing based on the first operand data by the first computing device and performing neural network computing based on the second operand data by the second computing device.

[0009] In one general aspect, a neural network computing device is provided. The neural network computing device includes: a receiver configured to receive data for neural network computing; and a processor configured to: verify whether a competition occurs between a first data traversal path corresponding to a first computing device and a second data traversal path corresponding to a second computing device, determine first operand data and second operand data from the data using the verification result and the priority between the first data traversal path and the second data traversal path, send the first operand data to the first computing device and send the second operand data to the second computing device, and perform neural network computing based on the first operand data by the first computing device and perform neural network computing based on the second operand data by the second computing device.

[0010] In one general aspect, a method for operating neural network computing is provided. The method includes: receiving data for neural network computing; verifying whether a competition occurs between a first data traversal path corresponding to a first computing device and a second data traversal path corresponding to a second computing device; determining first operand data and second operand data from the data using the verification result and the priority between the first data traversal path and the second data traversal path; and performing neural network computing based on the first operand data and the second operand data.

[0011] The method may include: determining whether to skip the operations on the data on the first data traversal path and the second data traversal path among the data.

[0012] The step of determining whether to skip the operations may include: determining to skip the operations on the data in response to the data being "0"; or determining to skip the operations on the data in response to the data being a value within a range.

[0013] The verification step may include: verifying that a competition occurs between the first data traversal path and the second data traversal path in response to the first computing device and the second computing device accessing the same data at a time point.

[0014] The steps of determining the first operand data and the second operand data may include: setting priorities for the first data traversal path and the second data traversal path; and in response to the occurrence of a contention, determining the first operand data and the second operand data based on the priorities.

[0015] The step of setting may include: setting a first priority such that nodes corresponding to data on the first data traversal path have different priorities; and setting a second priority such that nodes corresponding to data on the second data traversal path have different priorities.

[0016] The steps of determining the first operand data and the second operand data may include: comparing a first priority corresponding to the first data traversal path and a second priority corresponding to the second data traversal path to determine the traversal path with the higher priority; and determining the data at the position where the contention occurs as the operand data of the arithmetic device corresponding to the traversal path with the higher priority.

[0017] The step of determining the data at the position where the contention occurs may include: in response to the first priority being higher than the second priority, determining the data at the position where the contention occurs as the first operand data; and determining the subsequent data on the second data traversal path as the second operand data.

[0018] The method may include: in response to the first operand data and the second operand data being determined, excluding the addresses of the first operand data and the second operand data from the first data traversal path and the second data traversal path.

[0019] The first data traversal path and the second data traversal path may have a predetermined traversal range, and the neural network arithmetic method may include: in response to completing traversal within the predetermined traversal range, updating the first data traversal path and the second data traversal path.

[0020] In another general aspect, there is provided a neural network arithmetic device, the neural network arithmetic device including: a receiver configured to receive data for neural network arithmetic; and a processor configured to: verify whether a contention occurs between a first data traversal path corresponding to a first arithmetic device and a second data traversal path corresponding to a second arithmetic device, determine the first operand data and the second operand data from the data using the verification result and the priorities between the first data traversal path and the second data traversal path, and perform neural network arithmetic based on the first operand data and the second operand data.

[0021] The processor may be configured to: determine whether to skip arithmetic operations on the data on the first data traversal path and the second data traversal path among the data.

[0022] The processor can be configured to: in response to the data being "0", determine to skip the operation for the data; or in response to the data being a value within a range, determine to skip the operation for the data.

[0023] The processor can be configured to: in response to the first arithmetic unit and the second arithmetic unit accessing the same data at a time point, verify that a competition occurs between the first data traversal path and the second data traversal path.

[0024] The processor can be configured to: set the priorities of the first data traversal path and the second data traversal path; and in response to the occurrence of the competition, determine the first operand data and the second operand data based on the priorities.

[0025] The processor can be configured to: set a first priority such that the nodes corresponding to the data on the first data traversal path have different priorities; and set a second priority such that the nodes corresponding to the data on the second data traversal path have different priorities.

[0026] The processor can be configured to: compare the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path to determine the traversal path with the higher priority; and determine the data at the position where the competition occurs as the operand data of the arithmetic unit corresponding to the traversal path with the higher priority.

[0027] The processor can be configured to: in response to the first priority being higher than the second priority, determine the data at the position where the competition occurs as the first operand data; and determine the subsequent data on the second data traversal path as the second operand data.

[0028] The processor can be configured to: in response to the first operand data and the second operand data being determined, exclude the address of the first operand data and the address of the second operand data from the first data traversal path and the second data traversal path.

[0029] The first data traversal path and the second data traversal path can have a predetermined traversal range, and the processor can be configured to: in response to completing the traversal within the predetermined traversal range, update the first data traversal path and the second data traversal path.

[0030] Other features and aspects will be clear from the following detailed description, the drawings, and the claims. Description of the Drawings

[0031] Figure 1 An example of a neural network arithmetic device is shown.

[0032] Figure 2 Shown Figure 1 Examples of embodiments of the neural network arithmetic device of

[0033] Figure 3 Shows Figure 1 An example of the data traversal process of a neural network computing device.

[0034] Figure 4 Shows an example of skipping data.

[0035] Figure 5 Shows an example of traversing data in an arithmetic unit.

[0036] Figure 6 Shows an example of a data traversal path.

[0037] Figures 7A to 7C Shows an example of the data traversal process over time.

[0038] Figure 8 Shows an example of a data traversal path.

[0039] Figure 9 Shows an example of performing neural network operations while executing data traversal.

[0040] Figure 10 Shows Figure 1 An example of an embodiment of a neural network computing device.

[0041] Figure 11 Shows Figure 1 An example of the operation of a neural network computing device.

[0042] Throughout the drawings and the detailed description, unless otherwise described or provided, the same reference numerals will be understood to refer to the same elements, features, and structures. The drawings may not be to scale, and for clarity, illustration, and convenience, the relative sizes, proportions, and depictions of the elements in the drawings may be exaggerated. Detailed Description

[0043] The following detailed description is provided to assist the reader in obtaining a comprehensive understanding of the methods, devices, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, devices, and / or systems described herein will be apparent after understanding the disclosure of this application. For example, the order of operations described herein is merely exemplary and is not limited to those set forth herein, but may be changed as will be apparent after understanding the disclosure of this application, except for operations that must occur in a specific order. Additionally, descriptions of known features may be omitted for greater clarity and conciseness.

[0044] The features described herein can be implemented in different forms and should not be construed as limited to the examples described herein. On the contrary, the examples described herein have been provided only to illustrate some of the many possible ways of implementing the methods, devices, and / or systems described herein that will be apparent after understanding the disclosure of the present application.

[0045] Although terms such as first, second, A, B, (a), (b) may be used to explain various components, the components are not limited to these terms. These terms should only be used to distinguish one component from another. For example, within the scope of the rights according to the concept of the present disclosure, a "first" component may be referred to as a "second" component, or similarly, a "second" component may be referred to as a "first" component.

[0046] Throughout the specification, when an element (such as a layer, region, or substrate) is described as "on", "connected to", or "coupled to" another element, the element may be directly "on", "connected to", or "coupled to" the other element, or there may be one or more other elements therebetween. In contrast, when an element is described as "directly on", "directly connected to", or "directly coupled to" another element, there may be no other elements therebetween. Similarly, expressions such as "between" and "immediately between" and "adjacent to" and "immediately adjacent to" can also be interpreted as described above.

[0047] The terms used herein are for the purpose of describing particular examples only and are not intended to limit the disclosure. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. As used herein, the term "and / or" includes any one and any combination of any two or more of the associated listed items. As used herein, the terms "comprises", "comprising", and "having" specify the presence of the recited features, quantities, operations, elements, components, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, elements, components, and / or combinations thereof.

[0048] When describing example embodiments with reference to the drawings, the same reference numerals denote the same constituent elements, and repeated descriptions related to the same constituent elements will be omitted. In the description of example embodiments, when a detailed description of a known related structure or function is considered to result in an ambiguous interpretation of the present disclosure, such a detailed description will be omitted.

[0049] The use of the term "may" herein with respect to an example or embodiment (e.g., with respect to what an example or embodiment may include or implement) means that there is at least one example or embodiment that includes or implements such a feature, and all examples are not limited thereto.

[0050] Hereinafter, example embodiments will be described in detail with reference to the accompanying drawings. When describing the example embodiments with reference to the accompanying drawings, the same reference numerals denote the same components, and repeated descriptions related to the same components will be omitted.

[0051] Figure 1 An example of a neural network computing device is shown.

[0052] Referring to Figure 1 , the neural network computing device 10 can generate the result of neural network operations by processing data. The neural network computing device 10 can perform neural network operations by traversing data based on the sparsity of the data, thereby accelerating neural network operations. Sparsity can be the ratio of the number of unimportant elements in neural network operations to the number of elements used for neural network operations. For example, sparsity can be the ratio of the number of elements having a zero value to the number of all elements.

[0053] The neural network computing device 10 can skip the data that does not require operations and the processing competition that occurs during data traversal processing based on the priority, thereby effectively reducing the cost of computing.

[0054] The neural network computing device 10 can train a neural network. The neural network computing device 10 can perform inference based on the trained neural network.

[0055] The neural network computing device 10 can use an accelerator to perform neural network operations. The neural network computing device 10 can be implemented inside or outside the accelerator.

[0056] The accelerator can include, for example, a microprocessor, a single processor, a stand-alone processor, a parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, multiple instruction multiple data (MIMD) multiprocessing, a microcomputer, a processor core, a multi-core processor, a multi-processor, a central processing unit (CPU), a controller and an arithmetic logic unit (ALU), a digital signal processor (DSP), a graphics processing unit (GPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC) or an application processor (AP), a neural processor (NPU) or a programmable logic unit (PLU). In another example, the accelerator can be implemented as a software computing environment (such as a virtual machine).

[0057] A neural network (or artificial neural network) can include statistical training algorithms that simulate biological neurons in machine learning and cognitive science. A neural network can represent a general model with the ability to solve problems, where artificial neurons (nodes) that form a network through synaptic combinations change the connection strength of the synapses through training.

[0058] The neurons of a neural network may include a combination of weights and biases. A neural network may include one or more layers, each layer including one or more neurons or nodes. A neural network may infer a desired result from an input by changing the weights of the neurons via learning.

[0059] The neural network may include a deep neural network (DNN). The neural network may include any one or any combination of a convolutional neural network (CNN), a recurrent neural network (RNN), a perceptron, a multi-layer perceptron, a feed-forward (FF), a radial basis network (RBF), a deep feed-forward (DFF), a long short-term memory (LSTM), a gated recurrent unit (GRU), an autoencoder (AE), a variational autoencoder (VAE), a denoising autoencoder (DAE), a sparse autoencoder (SAE), a Markov chain (MC), a Hopfield network (HN), a Boltzmann machine (BM), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a deep convolutional network (DCN), a deconvolution network (DN), a deep convolutional inverse graphics network (DCIGN), a generative adversarial network (GAN), a liquid state machine (LSM), an extreme learning machine (ELM), an echo state network (ESN), a deep residual network (DRN), a differentiable neural computer (DNC), a neural Turing machine (NTM), a capsule network (CN), a Kohonen network (KN), and an attention network (AN). In one example, at least a portion of the multiple layers in the neural network may correspond to a CNN, and another portion of the multiple layers may correspond to an FCN. In this case, the CNN may be referred to as a convolutional layer, and the FCN may be referred to as a fully connected layer.

[0060] The neural network operation device 10 may be implemented by a printed circuit board (PCB) (such as a main board, an integrated circuit (IC), or a system on a chip (SoC)). For example, the neural network operation device 10 may be implemented by an application processor.

[0061] In addition, the neural network operation device 10 may be implemented in a personal computer (PC), a data server, or a portable device.

[0062] The portable device may be implemented as a laptop computer, a mobile phone, a smart phone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital still camera, a digital video camera, a portable multimedia player (PMP), a personal navigation device or a portable navigation device (PND), a handheld game console, an e-book, a digital television (DTV), an artificial intelligence (AI) speaker, a household appliance (such as a television, a smart TV, a refrigerator, a smart home device), a vehicle (such as a smart vehicle), an Internet of Things (IoT) device, or a smart device. The smart device may be implemented as a smart watch, a smart bracelet, smart glasses, or a smart ring.

[0063] The neural network computing device 10 includes a receiver 100 and a processor 200. The neural network computing device 10 may further include a memory 300.

[0064] The receiver 100 may include a receiving interface. The receiver 100 may receive data for performing neural network operations. The receiver 100 may receive data from the memory 300.

[0065] The processor 200 may process the data stored in the memory 300. The processor 200 may execute computer-readable code (e.g., software) stored in the memory 300 and instructions triggered by the processor 200.

[0066] The "processor 200" may be a data processing device implemented by hardware including a circuit having a physical structure for performing desired operations. For example, the desired operations may include code or instructions included in a program.

[0067] For example, the hardware-implemented data processing device may include, for example, a microprocessor, a single processor, a stand-alone processor, a parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, multiple instruction multiple data (MIMD) multiprocessing, a microcomputer, a processor core, a multi-core processor, a multi-processor, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a controller, and an arithmetic logic unit (ALU), a digital signal processor (DSP), a graphics processing unit (GPU), or an application processor (AP), a neural processor (NPU), or a programmable logic unit (PLU).

[0068] The processor 200 may determine whether to skip operations on the data on the first data traversal path and the second data traversal path among the data. The processor 200 may determine to skip the operations on the data in response to the data being "0", or determine to skip the operations on the data in response to the data being a value within a range. In one example, the range may be predetermined. Data skipping will be described in detail with reference to Figure 4 Describe data skipping in detail.

[0069] The processor may verify whether a competition occurs between the first data traversal path corresponding to the first computing device and the second data traversal path corresponding to the second computing device. The processor 200 may verify that a competition occurs between the first data traversal path and the second data traversal path in response to the first computing device and the second computing device accessing the same data at a certain time point.

[0070] In one example, the processor 200 may use the result of verification and the priorities between the first data traversal path and the second data traversal path to determine the first operand data and the second operand data among the data.

[0071] The processor 200 may set the priorities of the first data traversal path and the second data traversal path. The processor 200 may set a first priority such that nodes corresponding to data on the first data traversal path have different first priorities. The processor 200 may set a second priority such that nodes corresponding to data on the second data traversal path have different second priorities.

[0072] In one example, when a contention occurs, the processor 200 may determine the first operand data and the second operand data based on the priorities. The processor 200 may determine the traversal path with the higher priority by comparing the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path.

[0073] The processor 200 may determine the data at the position where the contention occurs as the operand data of the arithmetic device corresponding to the traversal path with the higher priority. When the first priority is higher than the second priority, the processor 200 may determine the data at the position where the contention occurs as the first operand data. The processor 200 may determine the subsequent data on the second data traversal path as the second operand data.

[0074] When the first priority is lower than the second priority, the processor 200 may determine the data at the position where the contention occurs as the second operand data. The processor 200 may determine the subsequent data on the first data traversal path as the first operand data.

[0075] In one example, when the first operand data and the second operand data are determined, the processor 200 may exclude the addresses of the first operand data and the second operand data from the first data traversal path and the second data traversal path.

[0076] The processor 200 may perform neural network operations based on the first operand data and the second operand data.

[0077] In one example, the first data traversal path and the second data traversal path may have traversal ranges. In one example, the traversal ranges may be predetermined. The processor 200 may update the first data traversal path and the second data traversal path of the data in response to the completion of the traversal in the traversal ranges.

[0078] The memory 300 stores instructions (or programs) executable by the processor 200. For example, the instructions may include instructions for performing the operations of the processor and / or the operations of each element of the processor.

[0079] The memory 300 is implemented as a volatile memory device or a non-volatile memory device.

[0080] The volatile memory device can be implemented as a dynamic random access memory (DRAM), a static random access memory (SRAM), a thyristor RAM (T-RAM), a zero-capacitor RAM (Z-RAM), or a two-transistor RAM (TTRAM).

[0081] The non-volatile memory device can be implemented as an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic RAM (MRAM), a spin-transfer torque (STT)-MRAM, a conductive-bridge RAM (CBRAM), a ferroelectric RAM (FeRAM), a phase-change RAM (PRAM), a resistive RAM (RRAM), a nanotube RRAM, a polymer RAM (PoRAM), a nano-floating-gate memory (NFGM), a holographic memory, a molecular electronic memory device, or an insulator resistance change memory.

[0082] Figure 2 illustrates Figure 1 an example of a neural network operation device.

[0083] Referring to Figure 2 , a device for operating a neural network (e.g., Figure 1 the neural network operation device 10) can load data from a memory storing the data (e.g., Figure 1 the memory 300) and allocate the data to an arithmetic unit, where neural network operations will be performed on the data using a hardware accelerator or a processor (such as, taking Figure 1 the processor 200 as an example). A device for operating a neural network can include an arithmetic unit (e.g., a processing unit) and a memory system, and the memory system includes a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), and a neural processing unit (NPU). The arithmetic unit can include a multiplier, an adder, or a multiply-accumulate (MAC).

[0084] The processor 200 can use the sparsity of the data used in neural network operations to accelerate the neural network operations. The processor 200 can provide such a scheme: traversing an arbitrary set of data during the driving time and obtaining the data to be used by each arithmetic unit.

[0085] In one example, the processor 200 can skip data based on a skip condition and perform neural network operations without loading redundant data among the arithmetic units according to the conditions of the arithmetic units. In one example, the skipped data can be predetermined.

[0086] The processor 200 can skip data by using the sparsity of data and select multiple operand data within a specified range to skip operations, thereby improving the performance of neural network operations and reducing the computational cost.

[0087] The obtained operand data to be input to the arithmetic device can represent a candidate among candidates for obtaining the right of the arithmetic device to exclusively use data that can be shared among neural network operations.

[0088] Skipping data can mean supplying only data that meets conditions (such as a predetermined range (or threshold)) to the arithmetic device and excluding data that does not meet the conditions according to the operation. For example, in the case of pruning a network, the processor 200 can accelerate neural network operations for input data without compression or structuring.

[0089] The neural network operation device 10 may include an external input / output (IO) 210, a data memory 230, a data traversal manager 250, and a data processor 270 (e.g., an arithmetic device).

[0090] The external IO 210 may include a data input / output interface. The data memory 230 may be included in the memory 300. The data traversal manager 250 may be included in the processor 200. The data processor 270 may be located separately outside the neural network operation device.

[0091] The data traversal manager 250 can update and manage the address of the memory 300 where data is stored, and the memory 300 is limited in size. The data traversal manager 250 can use a data traversal path to traverse data and output the obtained operand data to the data processor 270.

[0092] The data memory 230 can also store indexes for determining data skip conditions and whether to use data (e.g., whether to use data in neural network operations). In one example, the data memory 230 may include a write manager and a memory manager.

[0093] When data transfer is completed in a stage or cycle in the time stream, the data traversal manager 250 can perform an address update on the area where data usage is completed. For example, the data traversal manager 250 can exclude the address of the operand data of the data processor 270 from the data traversal path in response to the determination of the operand data.

[0094] The data traversal manager 250 can traverse data along the data traversal path and determine operand data, so as to send the operand data, the order of obtaining the data, and the metadata about the location of the data to the data processor 270 together.

[0095] Figure 3 Show Figure 1Example of data traversal processing of a neural network computing device.

[0096] Referring to Figure 3 , the memory (e.g., Figure 1 memory 300) may include memory cells 310. The processor (e.g., Figure 1 processor 200) may include computing means (e.g., processing unit 330). In one example, the computing means may be implemented separately outside the processor 200.

[0097] The processor 200 may perform load balancing, thereby reducing the time for processing neural network operations and the energy consumed by the hardware.

[0098] Load balancing may represent the allocation of data D to be used by the processing unit 330 (e.g., computing means) n so that it can be processed by one of the processing units (P n-3 , P n-2 , P n-1 , P n , P n+1 , P n+2 , P n+3 , ……). In one example, the data D n of the data may be used exclusively by only one of the processing units 330.

[0099] The processor 200 can efficiently solve the competition problem that occurs in the processing of obtaining the same data by the processing unit 330 during the load balancing process, thereby performing neural network operations without reducing performance according to the competition elimination in the hardware.

[0100] The memory cell 310 may be allocated the data D to be processed by the processing unit P n . The processing unit P n may be one of the processing units formed by a set of m processing units P = {P n , P n+a , P n+b , P n+c , P n+d ,...}. Different processing units may access a part or all of the set formed by m data D = {D n+a , D n+b , D n+c , D n+d ,...}, search for data that meets the conditions, and load the discovered data.

[0101] If the set formed by L data (which is a subset of D to be accessed by the processing unit P n in the set) is D n ' = {D n , Dn+a , D n+b , D n+c , D n+d ,...}, then it will be processed by another processing unit P n+i The data set to be accessed can be D n+i ' = {D n+i , D n+i+a , D n+i+b , D n+i+c , D n+i+d ,...}. Here, the access order of a, b, c, and d can be the same for all processing units.

[0102] For the data set D in a stage or cycle n ' For each data in it, the processing unit P n can traverse the data m times within the range of D n '[k:k + t - 1] = {D n [k:k + t - 1], D n+a [k:k + t - 1], D n+b [k:k + t - 1], D n+c [k:k + t - 1], D n+d [k:k + t - 1],...} (for example, through a traversal engine). Here, m may not exceed L × t, and each D n '[i] can be traversed only once in a single stage or cycle.

[0103] According to the above conditions, the processing unit P n can traverse the data set D n ' m times. The processing unit P n can access the data in the way of [D n+d [q] → D n+e [r] → D n+f [s],...], and another processing unit P n+1 can access the data in the way of [D n+d+i [q] → D n+e+i [r] → D n+f+i [s],...].

[0104] In the above example, P n can perform operations on the data D n ' allocated to P n 's D n '[k:k + t - 1] = {D n [k:k + t - 1], D n+a [k:k + t - 1], D n+b [k:k + t - 1], D n+c [k:k + t - 1], Dn+d [k:k + t - 1], ...} perform at least one traversal.

[0105] In the traversal process described above, the processing unit that obtains data considering the skip condition can transfer the data as its own input and set the data to the in - use state to exclusively occupy the data, so that the data cannot be used by other processing units. The processing unit that obtains the data for its input can be changed to the end - of - traversal state.

[0106] When the data does not meet the operation conditions or the data is not used by other processing units because multiple processing units 330 access the same data D p [j], the processing units 330 can compete to use the data.

[0107] The processor 200 can set a unique priority for the order of accessing data to resolve the competition. For example, the processor 200 can set the priority of P n such as D n+d [q]=1, D n+e [r]=2, D n+f [s]=3, ……. In the same way, the processor 200 can set the priority of P n+1 such as D n+d+i [q]=1, D n+e+i [r]=2, D n+f+i [s]=3, …….

[0108] In the above example, the priority can be assigned to the data access order of each processing unit in the same way, and different priorities can be assigned to all access orders. The processor 200 can set the priority regardless of the data access order. That is, the processor 200 can set the highest priority to the data that will be accessed first and the lowest priority to the data that will be accessed last, or set the priority in the reverse order.

[0109] When a competition occurs, the processor 200 can compare the priorities set for each data traversal path and allow the processing unit with a higher priority to obtain the data. In one example, the processing unit that obtains the data can terminate the traversal, and the processing unit that fails to obtain the data can continue to traverse the data along the traversal path specified above.

[0110] Finally, the processing unit that fails to obtain the data can perform a null operation or generate an invalid result. For example, if the processing unit is a MAC, the processing unit can generate "0".

[0111] In D of each stage or cycle nIn [k:k+t-1], if the data corresponds to a skip condition or the data includes D n [y] (e.g., D n [y] can be data corresponding to a skip condition), then the data can be excluded from D n [k:k+t-1], and k and t can be updated, and then each processing unit can iterate through.

[0112] Figure 4 An example of skipping data is shown.

[0113] Referring to Figure 4 , a processor (e.g., Figure 1 processor 200) can determine whether to skip operations on data on the first data traversal path and the second data traversal path among the data.

[0114] Processor 200 can determine to skip operations on the data in response to the data being "0", or can determine to skip operations on the data in response to the data being a value within a range.

[0115] Figure 4 The example of shows the case of skipping operations when the data is "0". However, in some examples, operations can be skipped for data other than "0". For two inputs A n and B n of an arithmetic device (e.g., a multiplier), A and B can represent the two inputs of the arithmetic device, and n can represent the operation order.

[0116] Stages 0, 1, and 2 can represent the time points of data traversal. Processor 200 can traverse the data in the data set from right to left.

[0117] In stage 0, processor 200 can use A0 which is the data at the first position in data set 410 and B0 which is the data at the first position in data set 420 to perform neural network operations (e.g., multiplication).

[0118] In stage 1, processor 200 can use A1 in data set 430 and B1 in data set 440 to perform neural network operations.

[0119] In stage 2, since the data at the position corresponding to A2 is "0", processor 200 can determine to skip the data at the position corresponding to A2 in data set 450. In this example, processor 200 can also skip the data B2 in data set 460 so as to correspond to the skip performed on data set 450. In other words, when skipping data, the jump offset of data traversal corresponding to the skipped data can be the same for the data used in the same arithmetic device.

[0120] InFigure 4 In the example of , when skipping is not performed, a total of four multiplications need to be executed. However, the processor 200 can perform the operation in three stages (or cycles) by skipping some data, thereby reducing the operation time and energy by a quarter.

[0121] Figure 5 An example of traversing data in an arithmetic device is shown. Figure 6 An example of a data traversal path is shown.

[0122] Referring to Figure 5 and Figure 6 , a processor (e.g., the processor 200 of Figure 1 ) can traverse the data stored in a memory (e.g., the memory 300 of Figure 1 ) based on the data traversal path. While traversing along the data traversal path, the processor 200 can determine the operand data to be output to an arithmetic device (e.g., a multiplier), and send the determined operand data to the arithmetic device.

[0123] The data traversal path may include nodes corresponding to the positions where data is stored in the memory 300, and edges connecting the nodes. In the example of Figure 5 , the dashed and solid lines in the data sets 510 and 530 may indicate examples of the data traversal path. The data traversal path may have a data traversal direction.

[0124] The processor 200 can generate a data traversal path based on the data traversal range. The traversal range may indicate the number of data to be traversed. For example, in the example of Figure 5 , the processor 200 may have a data traversal path of "7", Figure 6 and the data traversal range in the traversal path of may be "6".

[0125] To perform an operation in a first arithmetic device (e.g., a multiplier), the processor 200 can traverse the data in the data set 510 in the order of A 0,0 , A 0,1 , A 1,0 , A 1,1 , A 2,0 , A 2,1 , and A 3,0 , and traverse the data in the data set 530 in the order of B 0,0 , B 0,1 , B 1,0 , B 1,1 , B 2,0 , B 2,1 , and B 3,0 .

[0126] In the same way, in order to perform operations in a second arithmetic device (e.g., a multiplier), the processor 200 may traverse the data in the data set 510 in the order of A 0,1 、A 0,2 、A 1,1 、A 1,2 、A 2,1 、A 2,2 and A 3,1 and traverse the data in the data set 530 in the order of B 0,1 、B 0,2 、B 1,1 、B 1,2 、B 2,1 、B 2,2 、B 3,1 .

[0127] The processor 200 can allow all arithmetic devices participating in data traversal to execute traversal simultaneously during the driving time without increasing the traversal time of the arithmetic devices for traversing data while traversing the data, thereby solving the problem of restricting the number of arithmetic devices or restricting the performance of the arithmetic devices.

[0128] The processor 200 can assign different priorities to the traversal paths, thereby solving the competition for data calls that occurs when multiple arithmetic devices perform operations.

[0129] The processor 200 can generate a data traversal path according to the order in which each arithmetic device traverses data for a specified area of the memory 300. The processor 200 can set the position where the data is stored as a node and use edges to connect the nodes, thereby setting a data traversal path from the start node to the last node. In this example, the data traversal path can have a direction.

[0130] The processor 200 can generate a data traversal path such that the edges and nodes on the data traversal path do not overlap. The processor 200 can set priorities for the nodes on the data traversal path.

[0131] The processor 200 can set priorities for the first data traversal path and the second data traversal path. The processor 200 can set a first priority such that the nodes corresponding to the data on the first data traversal path have different first priorities. The processor 200 can set a second priority such that the nodes corresponding to the data on the second data traversal path have different second priorities.

[0132] For example, in Figure 6In the example, the processor 200 may set "1" as the number corresponding to the priority of the node on the upper right side, set "2" as the number corresponding to the priority of the node on the lower right side, and set "3" as the number corresponding to the priority of the node on the upper middle side. In the same way, the processor 200 can set priorities for six nodes.

[0133] The processor 200 may determine the node corresponding to a relatively small number as the node with a relatively high priority. In another example, the processor 200 may determine the node corresponding to a relatively large number as the node with a relatively high priority.

[0134] When a competition occurs, the processor 200 may determine the first operand data and the second operand data based on the priority. The processor 200 may determine the traversal path with a higher priority by comparing the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path.

[0135] The processor 200 may determine the data at the position where the competition occurs as the operand data of the arithmetic unit corresponding to the traversal path with a higher priority. When the first priority is higher than the second priority, the processor 200 may determine the data at the position where the competition occurs as the first operand data. The processor 200 may determine the subsequent data on the second data traversal path as the second operand data.

[0136] When the first priority is lower than the second priority, the processor 200 may determine the data at the position where the competition occurs as the second operand data. The processor 200 may determine the subsequent data on the first data traversal path as the first operand data.

[0137] As described above, in Figure 6 the example, the data traversal range (or the traversal of valid data) is limited to at most six times, and the processor 200 may set priorities for the memory locations on each data traversal path according to the traversal length.

[0138] The processor 200 may traverse the memory up to six times until the available data for performing neural network operations is found. In one example, if the operation candidates (e.g., operand data) are not determined within a limited number of times, the data traversal may be terminated, and a predetermined value may be sent to the arithmetic unit.

[0139] In one example, the processor 200 may start from the starting point where the stored data is updated or the address in the memory 300, and traverse a predetermined and finite-length data traversal path according to the directionality. In one example, the directionality may be predetermined.

[0140] Nodes on the data traversal path may have different priorities within the data traversal path. When performing traversal along the data traversal paths corresponding to computing devices to send to different computing devices, the processor 200 may compare the priorities of each data traversal path at the competing nodes.

[0141] The processor 200 may send the data at the competing nodes as operand data to the computing device corresponding to the data traversal path with a higher priority. The data traversal path that fails to obtain the data may have survivability, and the processor 200 may use the data traversal path that fails to obtain the data to continue traversal within a preset traversal range.

[0142] Figure 5 and Figure 6 Examples of show data traversal paths set in a toothed shape. However, the shape of the data traversal path may vary according to the example.

[0143] Figures 7A to 7C shows an example of data traversal processing over time.

[0144] Referring to Figures 7A to 7C , the processor 200 may determine the operand data to be used for neural network operations along the data traversal paths (e.g., the first data traversal path and the second data traversal path) corresponding to computing devices (e.g., the first computing device and the second computing device).

[0145] In Figures 7A to 7C 's example, when the processor 200 performs data traversal along the first data traversal path corresponding to the first computing device (e.g., MUL 0) and the second data traversal path corresponding to the second computing device (e.g., MUL 1), it may determine the first operand data to be used by the first computing device and the second operand data to be used by the second computing device.

[0146] Figure 7A shows the traversal operation in stage 0 (or cycle 0). The processor 200 may skip the data that meets the skip conditions as described above.

[0147] The processor 200 may verify that the result data obtained by traversing the first data on the first data traversal path is "0", and skip the data that is "0". The processor 200 attempts to use the subsequent data A on the first data traversal path 0,1 as the first operand data. However, since A 0,1 is the first data on the second data traversal path, a competition may occur.

[0148] That is, the first computing device skips the data that is "0", and attempts to obtain the subsequent data A on the first data traversal path0,1 However, since the second arithmetic unit also attempts to obtain the same data, competition may occur.

[0149] When competition occurs, the processor 200 can determine the first operand data and the second operand data based on priorities. The processor 200 can determine the traversal path with a higher priority by comparing the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path.

[0150] Priorities and the numbers corresponding to the priorities can be assigned in the same way as described with reference to Figure 6 . Therefore, in Figure 7A , the number corresponding to the priority of data A on the first data traversal path 0,1 can be "2", and the number corresponding to the priority of data A on the second data traversal path 0,1 is "1".

[0151] If the node corresponding to the smaller value has a higher priority, since the first data traversal path has a lower priority than the second data traversal path, the processor 200 can determine A 0,1 as the second operand data. Therefore, the processor 200 can send A 0,1 to the second arithmetic unit and terminate the traversal.

[0152] Since the first arithmetic unit fails to obtain data A 0,1 , the processor 200 can continue the traversal along the first data traversal path. Since the subsequent data on the first data traversal path is data A 1,0 , the processor 200 can determine A 1,0 as the first operand data, send A 1,0 to the first arithmetic unit, and terminate the traversal at stage 0.

[0153] In response to the determination of the operand data, the processor 200 can exclude the address of the data determined as the operand data from the data traversal path. In the example of Figure 7A , the processor 200 can indicate in data A 0,1 and A 1,0 that have been determined as the operand data and sent to the arithmetic unit that the data has been obtained, or change the data to "0", so as to exclude the data from the data traversal in subsequent stages.

[0154] Figure 7B shows the traversal operation in stage 1 (or cycle 1). In stage 1, the subsequent data A 1,1 on the first data traversal path and the subsequent data A 0,2Neither of them is "0". Therefore, the processor 200 may not perform a skip.

[0155] The processor 200 may perform data traversal in a state where the already traversed data 0, A 0,1 and A 1,0 are excluded from the first data traversal path and the already traversed data A 0,1 is excluded from the second data traversal path.

[0156] The processor 200 may determine the subsequent data A on the first data traversal path 1,1 as the first operand data and send the data A 1,1 to the first arithmetic unit, and determine the subsequent data A on the second data traversal path 0,2 as the second operand data and send the data A 0,2 to the second arithmetic unit. In addition, the data A 1,1 and A 0,2 sent to the arithmetic unit for arithmetic operations may be excluded from the data traversal path.

[0157] Figure 7C Shows the traversal operation in stage 2 (or cycle 2). In stage 2, the subsequent data on the first data traversal path is "0". Therefore, the processor 200 may skip the data that is "0".

[0158] The subsequent data on the second data traversal path is A 1,1 , A 1,1 is the data excluded from the previous stage. Therefore, the processor 200 may skip A 1,1 . Since the subsequent data of A on the second data traversal path 1,1 is "0", the processor 200 may skip this data.

[0159] In this example, the subsequent data on the first data traversal path is A 2,1 , and the subsequent data on the second data traversal path is also A 2,1 . Therefore, a competition may occur. The processor 200 may determine the arithmetic unit based on the priority to send A 2,1 . The numbers corresponding to the priority may be set as shown in the example of Figure 6 .

[0160] The number corresponding to the priority of A on the first data traversal path 2,1 is "6", and the number corresponding to the priority of A on the second data traversal path 2,1 is "5". Therefore, the processor 200 may determine that the second data traversal path has a higher priority and determine A 2,1 as the second operand data. ​

[0161] In this example, since the predetermined traversal range of the first data traversal path ends, the processor 200 may output NA indicating that no data is available to the first arithmetic unit.

[0162] The processor 200 may update the first data traversal path and the second data traversal path of the data in response to the traversal in the predetermined traversal range being completed. In Figure 7C this case, the processor 200 may update the first data traversal path or the second data traversal path to traverse the data on a new memory area.

[0163] Figure 8 An example of a data traversal path is shown.

[0164] Referring to Figure 8 , the processor 200 may perform data traversal on data that is not arranged in a square form. In addition, if three or more arithmetic units are provided, the processor 200 may also perform data traversal in parallel.

[0165] As Figure 8 shown in the example of Figures 7A to 7C , even when three arithmetic units expect to share a single data set 830, data traversal can be performed in the manner described with reference to

[0166] When a competition between data traversal paths occurs or data with a value of "0" is traversed, the processing scheme may be the same as the processing scheme shown in Figures 7A to 7C . Through the data traversal described above, the processor 200 can reduce the power used for neural network operations and improve the operation efficiency.

[0167] Figure 9 An example of performing neural network operations while performing data traversal is shown.

[0168] Referring to Figure 9 , an example of performing neural network operations using a neural network operation device (e.g., Figure 1 the neural network operation device 10 of

[0169] A processor (e.g., Figure 1 the processor 200 of

[0170] While traversing the data to be input to the corresponding processing units of the arithmetic unit (e.g., MAC array 910) along the data traversal path, the processor 200 can determine the operand data based on the priority.

[0171] For example, a receiver (e.g., Figure 1 receiver 100) can receive the data 950 and output the data 950 to the processor 200.

[0172] The processor 200 can traverse the received data 970 (e.g., the received data 970 corresponds to the data 950). For the area corresponding to a part of the data 970, the processor 200 can skip a predetermined amount of data and perform data traversal based on the priority of the data traversal path in the manner described above.

[0173] The processor 200 can send the operand data determined by the traversal to the MAC array 910 and generate the output activation value 990 based on the result of the operation output from the MAC array 910. In one example, the MAC array 910 can include a multiplexer (MUX).

[0174] By using the priority-based data traversal, the processor 200 can perform neural network operations while achieving load balancing in the operations. Thus, when implementing the actual hardware, the processor 200 can suppress the increase in complexity according to the number of arithmetic units, thereby achieving relatively high operation performance compared to the traditional scheme.

[0175] Figure 10 Shown Figure 1 is an example of an implementation of a neural network operation device.

[0176] In Figure 10 is shown an example of an implementation of a sparsity MAC arithmetic unit applying a neural network operation device (e.g., Figure 1 neural network operation device 10).

[0177] 0]The sparsity MAC arithmetic unit can include a sparsity unit 1030 and a MAC array 1050. The processor (e.g., Figure 1 processor 200) can be implemented inside or outside the sparsity unit 1030.

[0178] The MAC array 1050 can be included in the arithmetic unit described above. The MAC array 1050 can perform operations and generate the result of the operation.

[0179] The sparsity unit 1030 can perform the priority-based traversal 1010. The sparsity unit 1030 can traverse the data along the data traversal path in the same manner as described above and send the data to the MAC array 1050. In Figure 10Among them, c2, b2, a2, a1, c1, b1, d1, d0, c0, b0, and a0 can represent non-zero data. W 0,0 to W 0,15 can represent weights corresponding to 16 input channels.

[0180] Figure 11 shows Figure 1 an example of the operation of a neural network computing device. Figure 11 The operations in can be performed in the order and manner shown, but the order of some operations can be changed or some operations can be omitted without departing from the spirit and scope of the described illustrative examples. Figure 11 Many of the operations shown in can be performed in parallel or simultaneously. Figure 11 One or more blocks of and combinations of blocks can be implemented by a computer based on dedicated hardware (such as a processor) that performs a specified function or a combination of dedicated hardware and computer instructions. Except for the following Figure 11 description of Figures 1 to 10 the description of also applies to Figure 11 and is incorporated herein by reference. Therefore, the above description may not be repeated here.

[0181] Referring to Figure 11 , in operation 1110, the receiver 100 can receive data for performing neural network operations.

[0182] The processor 200 can determine whether to skip the operations on the data on the first data traversal path and the second data traversal path among the data. The processor 200 can determine to skip the operations on the data in response to the data being "0", or determine to skip the operations on the data in response to the data being a value within a range.

[0183] In operation 1130, the processor can verify whether a competition occurs between the first data traversal path corresponding to the first computing device and the second data traversal path corresponding to the second computing device. When the first computing device and the second computing device access the same data at a certain point in time, the processor 200 can verify that a competition occurs between the first data traversal path and the second data traversal path.

[0184] In operation 1150, the processor 200 can determine the first operand data and the second operand data from the data using the verified result and the priority between the first data traversal path and the second data traversal path.

[0185] The processor 200 can set the priorities of the first data traversal path and the second data traversal path. The processor 200 can set a first priority such that nodes corresponding to data on the first data traversal path have different first priorities. The processor 200 can set a second priority such that nodes corresponding to data on the second data traversal path have different second priorities.

[0186] When a contention occurs, the processor 200 can determine first operand data and second operand data based on the priorities. The processor 200 can determine the traversal path with a higher priority by comparing the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path.

[0187] The processor 200 can determine the data at the position where the contention occurs as the operand data of the arithmetic unit corresponding to the traversal path with a higher priority. When the first priority is higher than the second priority, the processor 200 can determine the data at the position where the contention occurs as the first operand data. The processor 200 can determine the subsequent data on the second data traversal path as the second operand data.

[0188] When the first priority is lower than the second priority, the processor 200 can determine the data at the position where the contention occurs as the second operand data. The processor 200 can determine the subsequent data on the first data traversal path as the first operand data.

[0189] The processor 200 can exclude the addresses of the first operand data and the second operand data from the first data traversal path and the second data traversal path in response to the determination of the first operand data and the second operand data.

[0190] In operation 1170, the processor 200 can perform neural network operations based on the first operand data and the second operand data.

[0191] The first data traversal path and the second data traversal path can have a predetermined traversal range. The processor 200 can update the first data traversal path and the second data traversal path of the data in response to the completion of the traversal within the predetermined traversal range.

[0192] The neural network computing device 10, data traversal manager 250, data processor 270, write manager, memory manager, and other devices, apparatuses, units, modules, and components described herein are implemented by hardware components. Examples of hardware components that can be used to perform the operations described in this application suitably include: controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtracters, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components that perform the operations described in this application are implemented by computing hardware (e.g., by one or more processors or computers). A processor or computer can be implemented by one or more processing elements (such as logic gate arrays, controllers and arithmetic logic units, digital signal processors, microcomputers, programmable logic controllers, field programmable gate arrays, programmable logic arrays, microprocessors, or any other device or combination of devices configured to respond and execute instructions in a defined manner to achieve a desired result). In one example, a processor or computer includes or is connected to one or more memories that store instructions or software executed by the processor or computer. The hardware components implemented by the processor or computer can execute instructions or software for performing the operations described in this application (such as an operating system (OS) and one or more software applications running on the OS). The hardware components can also access, manipulate, process, create, and store data in response to the execution of the instructions or software. For simplicity, the singular terms "processor" or "computer" may be used in the description of the examples described in this application, but in other examples, multiple processors or computers may be used, or a processor or computer may include multiple processing elements or multiple types of processing elements or both. For example, a single hardware component or two or more hardware components can be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components can be implemented by one or more processors, or a processor and a controller, and one or more other hardware components can be implemented by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, can implement a single hardware component or two or more hardware components.Hardware components can have any one or more of different processing configurations. Examples of different processing configurations include: a single processor, a stand-alone processor, a parallel processor, single instruction single data (SISD) multiprocessing, single instruction multiple data (SIMD) multiprocessing, multiple instruction single data (MISD) multiprocessing, and multiple instruction multiple data (MIMD) multiprocessing, a controller and an arithmetic logic unit (ALU), a DSP, a microcomputer, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic unit (PLU), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), or any other device capable of responding and executing instructions in a defined manner.

[0193] Figures 1 to 11 The method of performing the operations described in this application shown in to Figures 1 to 11 is performed by computing hardware (e.g., by one or more processors or computers), which is implemented as described above to execute instructions or software to perform the operations performed by the method described in this application. For example, a single operation or two or more operations can be performed by a single processor, or two or more processors, or a processor and a controller. One or more operations can be performed by one or more processors, or a processor and a controller, and one or more other operations can be performed by one or more other processors, or additional processors and additional controllers. One or more processors, or a processor and a controller, can perform a single operation or two or more operations.

[0194] Instructions or software for controlling computing hardware (e.g., a processor or computer) to implement the hardware components and perform the methods described above are written as a computer program, code segment, instruction, or any combination thereof to individually or jointly instruct or configure the processor or computer to operate as a machine or a special-purpose computer to perform the operations performed by the hardware components and methods described above. In one example, the instructions or software include machine code directly executed by the processor or computer (such as machine code generated by a compiler). In one example, the instructions or software include at least one of a small program, a dynamic link library (DLL), middleware, firmware, a device driver, an application that stores a method of operating a neural network operation. In another example, the instructions or software include high-level code executed by the processor or computer using an interpreter. The instructions or software can be written in any programming language based on the block diagrams and flowcharts shown in the figures and the corresponding descriptions in the specification, and the block diagrams and flowcharts shown in the figures and the corresponding descriptions in the specification disclose algorithms for performing the operations performed by the hardware components and methods described above.

[0195] Instructions or software for controlling a processor or computer to implement the hardware components and execute the methods as described above, as well as any associated data, data files, and data structures, are recorded, stored, or fixed in one or more non-transitory computer-readable storage media, or are recorded, stored, or fixed on one or more non-transitory computer-readable storage media. Examples of non-transitory computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), magnetic RAM (MRAM), spin transfer torque (STT)-MRAM, static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), dual transistor RAM (TTRAM), conductive bridge RAM (CBRAM), ferroelectric RAM (FeRAM), phase change RAM (PRAM), resistive RAM (RRAM), nanotube RRAM, polymer RAM (PoRAM), nano floating gate memory (NFGM), holographic memory, molecular electronic memory devices, resistive change memory, dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk memory, hard disk drive (HDD), solid state drive (SSD), card memory (such as, multimedia card or micro card (e.g., secure digital (SD) or extreme digital (XD))), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, and any other device, any other device being configured to store instructions or software and any associated data, data files, and data structures in a non-transitory manner and provide the instructions or software and any associated data, data files, and data structures to a processor or computer such that the processor and computer can execute the instructions. In one example, the instructions or software and any associated data, data files, and data structures are distributed across a networked computer system such that the instructions and software and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner by one or more processors or computers.

[0196] While the present disclosure includes specific examples, it will be apparent after understanding the disclosure of this application that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein will be considered merely illustrative and not for purposes of limitation. The description of a feature or aspect in each example will be considered applicable to similar features or aspects in other examples. Appropriate results may be achieved if the techniques described are performed in a different order, and / or if the components in the described system, architecture, device, or circuit are combined in a different manner, and / or replaced or supplemented by other components or their equivalents.

[0197] Accordingly, the scope of the disclosure is not limited by the specific embodiments, but by the claims and their equivalents, and all variations within the scope of the claims and their equivalents should be construed as being included in the disclosure.

Claims

1. A method for operating a neural network computing device, the method comprising: Receiving data for neural network computing; Verifying whether competition occurs between a first data traversal path corresponding to a first computing device and a second data traversal path corresponding to a second computing device; Determining first operand data and second operand data from the data using the result of verifying whether competition occurs between the first data traversal path and the second data traversal path and the priority between the first data traversal path and the second data traversal path; Sending the first operand data to the first computing device and sending the second operand data to the second computing device; And Performing neural network computing by the first computing device based on the first operand data and performing neural network computing by the second computing device based on the second operand data, wherein the first data traversal path and the second data traversal path are paths for traversing data stored in a memory.

2. The method according to claim 1, further comprising: Determining whether to skip the operation on the first data on the first data traversal path and the second data traversal path among the data.

3. The method according to claim 2, wherein The step of determining whether to skip the operation on the first data includes: In response to the first data being "0", determining to skip the operation on the first data; or In response to the first data being a value within a predetermined range, determining to skip the operation on the first data.

4. The method according to claim 1, wherein The verifying step includes: in response to the first computing device and the second computing device accessing the same data at a time point, verifying that competition occurs between the first data traversal path and the second data traversal path.

5. The method according to claim 1, wherein, The step of determining the first operand data and the second operand data includes: Setting the priorities of the first data traversal path and the second data traversal path; and In response to the occurrence of competition, determining the first operand data and the second operand data based on the priorities.

6. The method according to claim 5, wherein The setting step includes: Setting a first priority such that nodes corresponding to data on the first data traversal path have different first priorities; and Setting a second priority such that nodes corresponding to data on the second data traversal path have different second priorities.

7. The method according to claim 5, wherein The step of determining the first operand data and the second operand data includes: Comparing the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path to determine the higher-priority traversal path; and Determining the data at the position where competition occurs as the operand data of the computing device corresponding to the higher-priority traversal path.

8. The method according to claim 7, wherein The step of determining the data at the position where competition occurs as the operand data of the computing device corresponding to the higher-priority traversal path includes: In response to the first priority being higher than the second priority, determining the data at the position where competition occurs as the first operand data; and Determining the subsequent data on the second data traversal path as the second operand data.

9. The method according to any one of claims 1 to 8, further comprising: In response to the first operand data and the second operand data being determined, exclude the addresses of the first operand data and the second operand data from the first data traversal path and the second data traversal path.

10. The method according to any one of claims 1 to 8, wherein, The first data traversal path and the second data traversal path have a predetermined traversal range, and the method further includes: in response to completing traversal within the predetermined traversal range, updating the first data traversal path and the second data traversal path.

11. A neural network operation device, comprising: a receiver configured to receive data for neural network operations; and a processor configured to: verify whether a competition occurs between a first data traversal path corresponding to a first operation device and a second data traversal path corresponding to a second operation device, determine the first operand data and the second operand data from the data using the result of verifying whether a competition occurs between the first data traversal path and the second data traversal path and the priority between the first data traversal path and the second data traversal path, send the first operand data to the first operation device and send the second operand data to the second operation device, and perform a neural network operation by the first operation device based on the first operand data and perform a neural network operation by the second operation device based on the second operand data, wherein the first data traversal path and the second data traversal path are paths for traversing data stored in a memory.

12. The neural network computing device according to claim 11, wherein, The processor is further configured to: determine whether to skip the operation on the first data on the first data traversal path and the second data traversal path among the data.

13. The neural network computing device according to claim 12, wherein, The processor is further configured to: in response to the first data being "0", determine to skip the operation on the first data; or in response to the first data being a value within a predetermined range, determine to skip the operation on the first data.

14. The neural network computing device according to claim 11, wherein, The processor is further configured to: in response to the first operation device and the second operation device accessing the same data at a certain time point, verify that a competition occurs between the first data traversal path and the second data traversal path.

15. The neural network computing device according to claim 11, wherein, The processor is further configured to: set the priorities of the first data traversal path and the second data traversal path; and in response to the occurrence of the competition, determine the first operand data and the second operand data based on the priorities.

16. The neural network computing device according to claim 15, wherein, The processor is further configured to: set a first priority such that nodes corresponding to data on the first data traversal path have different first priorities; and set a second priority such that nodes corresponding to data on the second data traversal path have different second priorities.

17. The neural network computing device according to claim 15, wherein, The processor is further configured to: compare the first priority corresponding to the first data traversal path and the second priority corresponding to the second data traversal path to determine the traversal path with a higher priority; and determine the data at the position where the competition occurs as the operand data of the operation device corresponding to the traversal path with a higher priority.

18. The neural network computing device according to claim 17, wherein, The processor is further configured to: in response to the first priority being higher than the second priority, determine the data at the position where the competition occurs as the first operand data; and determine the subsequent data on the second data traversal path as the second operand data.

19. The neural network computing device according to any one of claims 11 to 18, wherein The processor is further configured to: in response to the first operand data and the second operand data being determined, exclude the addresses of the first operand data and the second operand data from the first data traversal path and the second data traversal path.

20. The neural network computing device according to any one of claims 11 to 18, wherein, The first data traversal path and the second data traversal path have a predetermined traversal range, and The processor is further configured to: in response to completing traversal within the predetermined traversal range, update the first data traversal path and the second data traversal path.

Images