Data processing method, processing array, and processing device

Abstract

A data processing method, a processing array, and a processing device for neural network computation. The data processing method is used for performing multiply-accumulate operations on at least one first vector and a second vector, and comprises: in each operation cycle, sequentially acquiring first data elements from the first vector; and, in response to a first data element acquired from the first vector being non-zero data, performing, in parallel, multiplication operations between the acquired first data element and a plurality of second data elements in the second vector, or, in response to a first data element acquired from the first vector being zero, skipping a multiplication operation for the acquired first data element. The data processing method can fully utilize the sparsity of input feature maps, and achieves computational acceleration while maintaining design simplicity.

Description

Data processing method, processing array and processing device

[0001] This application claims priority to Chinese Patent Application No. 202410089553.2 filed on January 22, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] Embodiments of the present disclosure relate to a data processing method, a processing array, and a processing device for neural network computing. Background Art

[0003] Neural network algorithms often generate many zeros. While these zeros do not affect the results of multiplication and addition operations, they still consume computational time and resources. Therefore, skipping zeros in neural network calculations can reduce the amount of computation without compromising accuracy. Exploiting zeros in data to speed up computation and reduce dynamic power consumption has been a major focus.

[0004] Summary of the Invention

[0005] At least some embodiments of the present disclosure provide a data processing method for neural network computing, for performing multiplication and addition operations on at least one first vector and a second vector, the method comprising: in each operation cycle, obtaining first data elements from the first vector one by one, and in response to the first data element obtained from the first vector being non-zero data, performing multiplication operations on the obtained first data element in parallel with multiple second data elements in the second vector, or in response to the first data element obtained from the first vector being 0, skipping the multiplication operation on the obtained first data element.

[0006] For example, in a data processing method provided by at least some embodiments of the present disclosure, in response to a first data element obtained from the first vector being non-zero data, multiplying the obtained first data element with multiple second data elements in the second vector in parallel, including: in response to a first data element obtained from the first vector being non-zero data, multiplying the obtained first data element with all second data elements in the second vector in parallel.

[0007] For example, in a data processing method provided by at least some embodiments of the present disclosure, in response to the first data element obtained from the first vector being non-zero data, the obtained first data element is multiplied in parallel with multiple second data elements in the second vector, including: in response to the data obtained from the first vector being non-zero data, the non-zero data is sent to a processing module, so that the processing module performs multiplication and addition operations on the first vector and the second vector, wherein the processing module includes multiple multiplication processing sub-units, and each multiplication processing sub-unit correspondingly receives a second data element for multiplication operation.

[0008] For example, in a data processing method provided by at least some embodiments of the present disclosure, the first vector is one of the parallel vectors in the input feature map calculated by the neural network; the second vector is one of the parallel vectors in the weight map calculated by the neural network.

[0009] For example, in a data processing method provided by at least some embodiments of the present disclosure, the vector in the input feature map calculated by the neural network includes a first data element.

[0010] For example, a data processing method provided by at least some embodiments of the present disclosure further includes: caching each first vector as a whole at a time, and obtaining first data elements from the first vector one by one, including: obtaining the first data elements from the cached first vector one by one.

[0011] For example, in a data processing method provided by at least some embodiments of the present disclosure, at least one first vector includes multiple first vectors, and each of the multiple first vectors is multiplied and added in parallel with the second vector.

[0012] For example, at least some embodiments of the present disclosure provide a data processing array for neural network computing, comprising at least one input module and at least one processing module, wherein the input module is configured to, in each operation cycle, retrieve first data elements from the first vector one by one to input the first vector, and the processing module is configured to, in response to the first data element retrieved from the first vector being non-zero data, perform multiplication operations on the retrieved first data element in parallel with multiple second data elements in the second vector, or, in response to the first data element retrieved from the first vector being 0, skip the multiplication operation on the retrieved first data element to perform multiplication-addition operations on at least one first vector and a second vector.

[0013] For example, in the data processing array provided by at least some embodiments of the present disclosure, the input module includes a buffer module, and the buffer module is configured to cache the entire first vector at one time.

[0014] For example, in the data processing array provided in at least some embodiments of the present disclosure, the processing module includes a plurality of multiplication processing subunits, each multiplication processing subunit being configured to perform multiplication and addition operations on a corresponding second data element and the acquired first data element.

[0015] For example, in the data processing array provided by at least some embodiments of the present disclosure, the multiplication processing subunit includes a multiplier.

[0016] In the data processing array provided in at least some embodiments of the present disclosure, the at least one processing module includes a plurality of processing modules, the at least one input module includes a plurality of input modules corresponding to the plurality of processing modules respectively, each of the plurality of input modules is configured to input a plurality of first vectors respectively, and the plurality of processing modules are configured to perform multiplication and addition operations on each of the plurality of first vectors in parallel with the second vector respectively.

[0017] At least some embodiments of the present disclosure provide a data processing device for neural network computing, including a processing array provided by an embodiment of the present disclosure.

[0018] At least some embodiments of the present disclosure provide a data processing device for neural network computing, comprising: a processor and a memory, wherein the memory stores one or more computer program modules; wherein the one or more computer program modules are configured to, when executed by the processor, execute the data processing method provided by an embodiment of the present disclosure.

[0019] At least some embodiments of the present disclosure further provide a non-transitory storage medium that non-transitorily stores computer-readable instructions, wherein when the computer-readable instructions are executed by a computer, the data processing method provided by the embodiments of the present disclosure is executed.

[0020] At least some embodiments of the present disclosure further provide a computer program product, including a computer program / instructions, wherein when the computer program / instructions are executed by a processor, the data processing method provided by the embodiments of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, rather than limiting the present disclosure.

[0022] FIG1A is a schematic diagram showing a convolutional layer performing multi-channel convolution operations;

[0023] FIG1B shows a schematic diagram of the architecture of a neural network processor;

[0024] FIG1C is a schematic block diagram of a processing unit performing data processing;

[0025] FIG2 is a flow chart of a data processing method for neural network calculation provided by some embodiments of the present disclosure;

[0026] FIG3 is a schematic diagram of a processing unit provided in some embodiments of the present disclosure;

[0027] FIG4A is a schematic diagram of a data processing process within an nth operation cycle provided by some embodiments of the present disclosure;

[0028] FIG4B is a schematic diagram of a data processing process in an (n+1)th operation cycle according to some embodiments of the present disclosure;

[0029] FIG4C is a schematic diagram of a data processing process in an (n+2)th operation cycle according to some embodiments of the present disclosure;

[0030] FIG5 is a schematic diagram of a data processing array provided by some embodiments of the present disclosure;

[0031] FIG6 is a schematic structural diagram of an electronic device provided by some embodiments of the present disclosure; and

[0032] FIG7 is a schematic diagram of a storage medium provided in some embodiments of the present disclosure. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0034] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0035] The present disclosure is described below using several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numeral in each drawing.

[0036] Neural networks are mathematical computational models inspired by the structure of neurons in the brain and the principles of neural conduction. The method of implementing intelligent computing based on this model is called brain-inspired computing. For example, neural networks include various network structures, such as back propagation (BP) neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs). Convolutional neural networks can also be further divided into fully convolutional networks, deep convolutional networks, and U-nets.

[0037] For example, a common convolutional neural network usually includes an input end, an output end, and multiple processing layers. For example, the input end is used to receive data to be processed, such as an image to be processed, and the output end is used to output processing results, such as a processed image. These processing layers may include convolution layers, pooling layers, batch normalization layers (Batch Normalization, abbreviated as BN), fully connected layers, etc. Depending on the structure of the convolutional neural network, the processing layers may include different contents and combinations. After the input data is input into the convolutional neural network, it passes through several processing layers to obtain the corresponding output. For example, the input data can pass through several processing layers to complete operations such as convolution, upsampling, downsampling, standardization, full connection, and flattening.

[0038] The convolutional layer is the core layer of a convolutional neural network. It applies several filters to the input data (input image or input feature map), which is then used to extract various types of features. The result of applying a filter to the input data is called a feature map, and the number of feature maps is equal to the number of filters. The feature map output by a convolutional layer can be fed into the next convolutional layer for further processing to produce a new feature map.

[0039] Figure 1A shows a schematic diagram of a convolutional layer performing a multi-channel convolution operation. As shown in Figure 1A, M groups of R×S convolution kernels with C channels are used to perform convolution operations on N groups of H×W input images (or input feature maps) with C channels, respectively. N groups of E×F output feature maps with M channels are obtained, so the output feature maps include multiple dimensions F / E / M in total. Convolution operations have the characteristics of high parallelism and high data reuse. High parallelism is reflected in the fact that multiple convolution kernels can operate on multiple input feature maps simultaneously.

[0040] Because the computational complexity of neural networks, especially for convolutional layers with large-scale input feature maps, is very high, it is usually necessary to decompose the computational operations of a convolutional layer in the neural network. For example, the convolution operations of different parts of the same convolutional layer can be performed independently of each other. These decomposed tasks are assigned to multiple processing units to perform calculations in parallel. The calculation results of these processing units are then merged to obtain the calculation results of the entire convolutional layer. The calculation results of this network layer can then be used as the input of the next convolutional layer.

[0041] A neural-network processing unit (NPU) is a type of microprocessor or computing system dedicated to hardware acceleration of artificial intelligence (especially artificial neural networks, machine vision, machine learning, etc.), sometimes also called an artificial intelligence accelerator (AI Accelerator).

[0042] Figure 1B shows a schematic diagram of the architecture of a neural network processor. As shown in Figure 1B, the neural network processor 100 includes a processing element (PE) array 110, a global cache 120, and a memory 130. The processing element array 110 includes multiple rows and columns (e.g., 12 rows × 12 columns) of processing elements, which are coupled to each other via on-chip interconnects and share a global cache 120. The on-chip interconnect is, for example, a network on chip (NoC). Each processing element has a computing function and, for example, may also have its own local cache, such as a cache or register array including a multiply-accumulator (MAC) and a vector (or matrix) for caching inputs. Each PE can access other PEs around it, the PE's own local cache, and the global cache. The global cache 120 is further coupled to the memory 130 via, for example, a bus.

[0043] During operation, for example, the convolution kernel (Flt) and input feature map (Ifm) data required for calculations of a network layer (e.g., a convolutional layer) are read from memory 130 into global cache 120. The convolution kernel (Flt) and input image (Img) are then input from global cache 120 into processing unit array 110 for calculation. Computational tasks for different image pixels are assigned to different processing units (i.e., mapping is performed). The partial cumulative sum (Psum1) generated during the calculation process is temporarily stored in the global cache. If a subsequent calculation requires further accumulation of the previously generated partial cumulative sum (Psum1), the required partial cumulative sum (Psum2) can be read from global cache 120 and returned to processing unit array 110 for calculation. The output feature map (Ofm) obtained after completing the calculations of a convolutional layer can be output from global cache 120 to memory 130 for storage, for example, to be used for calculations of the next network layer (e.g., a convolutional layer).

[0044] Figure 1C shows a schematic diagram of a processing element (PE) in a processing layer of a neural network. This processing element is used for data processing. Input data, such as an input feature map (IFMAP), is fed into the processing element shown in Figure 1C , where it performs operations on data in the processing module, such as weights, and outputs the results.

[0045] In neural network algorithms, zeros in data can be used to reduce dynamic power consumption. The inventors of this disclosure noted that, for example, an enable signal can be set for the multiply-add unit as a flag to start / permit execution. Before performing the multiply-add operation, the data is checked for non-zero values. If zero is detected, the corresponding enable bit in the pipeline is pulled low, preventing the multiply-add unit and pipeline from flipping, but allowing the pipeline to operate normally.

[0046] Using zeros in data to accelerate calculations is relatively complex, and there are very few AI accelerators that support this function.

[0047] There are three ways to accelerate operations using 0 in data:

[0048] ① Use 0 in weight to achieve acceleration, for example, the calculation speed can be doubled;

[0049] ②Use 0 in the input feature map to achieve acceleration;

[0050] ③ Use the input feature map and 0 in the weight to achieve acceleration at the same time.

[0051] Due to the sparse and unpredictable nature of the input feature map, using both the input feature map and zeros in the weights to achieve acceleration will dramatically increase the complexity of the AI ​​accelerator. Therefore, this type of AI accelerator product is still very rare on the market.

[0052] The inventors of the present disclosure attempted to develop a method for accelerating the neural network processor (NPU) by simultaneously utilizing the input feature map and zeros in the weights. However, they also discovered that this method significantly increased the complexity of the relationship between input feature map retrieval, weight retrieval, and accumulation.

[0053] An embodiment of the present disclosure provides a data processing method for neural network calculations and a corresponding data processing device, which are used to perform multiplication and addition operations on at least one first vector and a second vector. The data processing method includes: in each operation cycle, obtaining first data elements from the first vector one by one, and in response to the first data element obtained from the first vector being non-zero data, multiplying the obtained first data element with multiple second data elements in the second vector in parallel, or skipping the multiplication operation on the obtained first data element in response to the first data element obtained from the first vector being 0.

[0054] The data processing method provided by the above embodiments of the present disclosure can fully utilize the sparsity of the input feature map, strike a balance between computational complexity and computational speed, and achieve computational acceleration while maintaining a simple design.

[0055] As shown in FIG2 , the data processing method provided by some embodiments of the present disclosure includes steps S100 to S200 .

[0056] Step S100 : in each operation cycle, obtaining first data elements from the first vector one by one.

[0057] Step S200: In response to a first data element obtained from the first vector being non-zero data, multiplying the obtained first data element with a plurality of second data elements in the second vector in parallel; or in response to a first data element obtained from the first vector being zero, skipping the multiplication operation on the obtained first data element.

[0058] It should be noted that the "first vector" and "second vector" here are used to refer to the vector currently serving as the description object, which can be any vector applicable to the embodiments of the present disclosure; similarly, the first data element and the second data element are respectively used to refer to the data elements in the first vector and the second vector currently serving as the description object.

[0059] The above steps S100 to S200 are described below in conjunction with Figures 3 and 4A to 4C. Figure 3 shows a schematic structural diagram of a processing unit provided in some embodiments of the present disclosure. The processing unit 10 shown in Figure 3 includes an input module 20 and a processing module 30. It should be noted that for ease of explanation, Figure 3 has been simplified and only shows one input module and one processing module. Based on the structure shown in Figure 3, multiple input modules and processing modules can be obtained.

[0060] In the embodiment shown in FIG3 , the input module can retrieve a first data element from a first vector during each operation cycle to input the first vector to the processing module. The first vector can be one of the vectors listed in parallel in an input feature map calculated by the neural network, and the vectors in the input feature map can include the first data element. In response to the first data element retrieved from the first vector being non-zero data, the processing module performs a multiplication operation on the retrieved first data element in parallel with multiple second data elements in the second vector, or in response to the first data element retrieved from the first vector being zero, skips the multiplication operation on the retrieved first data element, and performs a multiplication-addition operation on at least one first vector and a second vector. In this embodiment, the second vector can be one of the vectors listed in parallel in a weight map calculated by the neural network.

[0061] For example, the processing module shown in Figures 3 and 4A to 4C includes eight independent multipliers for performing multiplication and addition operations on the first vector and the second vector. It should be understood that in the embodiments of the present disclosure, the number of multipliers can also be 16, 24, etc., and eight multipliers are merely used as an example for illustration. Moreover, the number of elements in a row or column of the first vector and the second vector used in the calculation is not limited to the number shown in the example. These should not be construed as limitations of the present disclosure.

[0062] As shown in FIG4A , in the nth operation cycle (n is a positive integer), taking the first vector of the input feature map including 7 data elements as an example, for example, the white squares in the figure represent data 0, and the gray squares represent non-zero data. The embodiments of the present disclosure do not limit the type of data, for example, the data type can be Int8, bf16, fp16, fp32, etc.

[0063] In FIG4A , the first data element in the first vector is, for example, the first non-zero data element closest to the multiplier and the second vector in the figure. Step S200 may include step S210:

[0064] Step S210 : In response to the first data element obtained from the first vector being non-zero data, performing a multiplication operation on the obtained first data element and all second data elements in the second vector in parallel.

[0065] According to step S210 , a parallel multiplication operation is performed on the non-zero first data element and all second data elements in the second vector.

[0066] In the embodiment shown in FIG4A , the second vector includes, for example, 8-bit weight data, and the processing module correspondingly includes, for example, 8 multiplication processing subunits. Each multiplication processing subunit includes a multiplier and, if necessary, may further include an adder, thereby obtaining a multiplier-accumulator (MAC). Step S210 may further include step S211:

[0067] In step S211, in response to the data obtained from the first vector being non-zero data, the non-zero data is sent to a processing module so that the processing module performs a multiplication and addition operation on the first vector and the second vector, wherein the processing module includes a plurality of multiplication processing sub-units, each of which receives a corresponding second data element for multiplication operation.

[0068] According to step S211 , the non-zero first data element is sent to the processing module, and each of the eight multiplication processing sub-units of the processing module correspondingly receives a second data element to perform a multiplication operation.

[0069] Next, in the (n+1)th operation cycle shown in FIG4B , since the second bit of data following the first non-zero data is 0, according to step S200 , the multiplication operation on this 0 data element is skipped, and the multiplication-addition operation on the third non-zero data element and the second vector is performed.

[0070] As shown in FIG4B , since the fourth and fifth bits following the third non-zero data bit are both 0, the multiplication operation of these two 0 data elements is also skipped according to step S200 .

[0071] In the (n+2)th operation cycle shown in FIG. 4C , a multiplication-addition operation is performed on the sixth non-zero data element and the second vector.

[0072] To simplify the skip-0 operation, the data processing method provided in the above embodiments of the present disclosure only sends one data element in each operation cycle. This is because the inventors of the present disclosure have discovered that if too much data is acquired and sent in a single operation cycle, the design becomes more complex, and may even make the skip-0 operation impossible in some scenarios, which also reduces computational efficiency.

[0073] The data processing method provided by the above embodiment may further include step S300: caching each first vector as a whole at a time, and obtaining first data elements from the first vectors one by one, including: obtaining first data elements from the cached first vectors one by one.

[0074] Some embodiments of the present disclosure also provide a data processing array for neural network calculations. The data processing array 200 shown in FIG5 includes three processing units 10. For example, data processed by each processing unit can be output for subsequent processing such as addition operations, wherein each processing unit includes an input module 20 and a processing module 30.

[0075] It should be noted that the three processing units in Figure 5 are exactly the same and are only used as an example. The present disclosure does not limit this. The number of processing units, the number of input modules and the number of processing modules are also only used as an example and should not be regarded as a limitation on the present disclosure.

[0076] At least one embodiment of the present disclosure provides a data processing array comprising at least one input module and at least one processing module, wherein the input module is configured to, within each operation cycle, retrieve first data elements from a first vector one by one to input the first vector, and each processing module is configured to, in response to the first data element retrieved from the first vector being non-zero data, perform a multiplication operation on the retrieved first data element in parallel with a plurality of second data elements in a second vector, or, in response to the first data element retrieved from the first vector being zero, skip the multiplication operation on the retrieved first data element to perform a multiplication-addition operation on at least one first vector with a second vector. For the data processing method performed by the data processing array, reference may be made to the data processing method of a single processing unit shown in FIG. 4A to FIG. 4C .

[0077] The processing array provided in the above embodiment may further include a buffer module (not shown in FIG5 ), and the buffer module is configured to cache the entire first vector at a time.

[0078] In the above processing array, the processing module may include multiple multiplication processing subunits, each of which is configured to perform multiplication and addition operations on a corresponding second data element and the obtained first data element. For this, please refer to Figure 3 or Figures 4A to 4C.

[0079] In the data processing array provided by the above embodiment, only one bit of data is sent to the data processing module of the processing unit during each operation cycle, simplifying input feature map address calculation and buffer module access. Furthermore, the addressing of the second vector, such as the weight, depends solely on this single input feature map, further simplifying buffer module access and second vector addressing, thereby unrestricting the width of the second vector. Furthermore, because the order of the second vectors remains unchanged, the accumulation relationship of the multiplication-addition unit is fixed, maintaining a simple and clear design for the multiplication-addition unit.

[0080] In the above-mentioned processing array, at least one processing module includes multiple processing modules, at least one input module includes multiple input modules corresponding to the multiple processing modules respectively, each of the multiple input modules is configured to input multiple first vectors respectively, and the multiple processing modules are configured to perform multiplication and addition operations on each of the multiple first vectors in parallel with the second vector.

[0081] Some embodiments of the present disclosure provide a data processing device for neural network computing, including the processing array provided by an embodiment of the present disclosure.

[0082] Some embodiments of the present disclosure provide a data processing device for neural network computing, comprising a processor and a memory, wherein the memory stores one or more computer program modules; the one or more computer program modules are configured to, when executed by the processor, execute the data processing method provided by the embodiments of the present disclosure.

[0083] Some embodiments of the present disclosure also provide an electronic device comprising the above-mentioned data processing device for neural network calculation.

[0084] Figure 6 illustrates an electronic device provided in some embodiments of the present disclosure. For example, as shown in Figure 6 , the electronic device 300 is used to implement the data processing method provided in any embodiment of the present disclosure. For example, the electronic device 300 may be a terminal device such as a personal computer, a laptop, a tablet computer, a mobile phone, or a workstation, a server, a cloud service, etc. It should be noted that the electronic device 300 shown in Figure 6 is merely an example and does not impose any limitations on the functionality and scope of use of the embodiments of the present disclosure.

[0085] As shown in FIG6 , the electronic device 300 may include a processing device (e.g., one or more central processing units, one or more graphics processing units, etc., or the aforementioned data processing device) 310, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 320 or a program loaded from a storage device 380 into a random access memory (RAM) 330. Various executable programs and data required for the operation of the electronic device 300 are also stored in the RAM 330. The processing device 310, the ROM 320, and the RAM 330 are connected to each other via a bus 340. An input / output (I / O) interface 350 is also connected to the bus 340.

[0086] Typically, the following devices may be connected to the I / O interface 350: an input device 360 ​​including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, or a gyroscope; an output device 370 including, for example, a liquid crystal display (LCD), a speaker, or a vibrator; a storage device 380 including, for example, a magnetic tape, a hard disk, or the like; and a communication device 390. The communication device 390 may allow the electronic device 300 to communicate with other electronic devices wirelessly or by wire to exchange data. Although FIG6 shows the electronic device 300 including various devices, it should be understood that it is not required to implement or have all of the devices shown, and the electronic device 300 may alternatively implement or have more or fewer devices.

[0087] For example, according to an embodiment of the present disclosure, the above-mentioned data processing method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the above-mentioned data processing method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 390, or installed from the storage device 380, or installed from the ROM 320. When the computer program is executed by the processing device 310, the functions defined in the data processing method provided in the embodiment of the present disclosure can be executed.

[0088] Some embodiments of the present disclosure also provide a storage medium for storing non-transitory computer program executable code (e.g., computer executable instructions). When the non-transitory computer program executable code is executed by a computer (e.g., including one or more processors), it can implement the data processing method of any embodiment of the present disclosure; or, when the non-transitory computer program executable code is executed by a computer, it can implement the data processing method provided by the embodiments of the present disclosure.

[0089] Figure 7 is a schematic diagram of a storage medium provided by some embodiments of the present disclosure. As shown in Figure 7, storage medium 400 non-transitorily stores computer program executable code 401. For example, when computer program executable code 401 is executed by a computer (e.g., including one or more processors), it can perform the data processing method provided by embodiments of the present disclosure.

[0090] For example, the storage medium 400 can be used in the above-mentioned data processing device. For another example, the storage medium 400 can be the memory 320 in the electronic device 300 shown in FIG6 . For example, the relevant description of the storage medium 400 can refer to the corresponding description of the memory 320 in the electronic device 300 shown in FIG6 , and will not be repeated here.

[0091] Although the present disclosure has been described in detail above using general descriptions and specific embodiments, it will be apparent to those skilled in the art that modifications or improvements may be made based on the embodiments of the present disclosure. Therefore, such modifications or improvements, as long as they do not depart from the spirit of the present disclosure, are within the scope of protection claimed by the present disclosure.

[0092] In addition to the above exemplary description, the following points need to be explained for this disclosure:

[0093] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure. Other structures may refer to conventional designs.

[0094] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale.

[0095] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to form new embodiments.

[0096] The above description is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. A data processing method for neural network computing, used to perform a multiply-accumulate operation on at least one first vector and a second vector. The method includes: In each operation cycle, obtain first data elements from the first vector one by one, and In response to the first data element obtained from the first vector being a non-zero data, perform a multiplication operation on the obtained first data element and multiple second data elements in the second vector in parallel, or in response to the first data element obtained from the first vector being 0, skip the multiplication operation for the obtained first data element.

2. The data processing method according to claim 1, wherein, In response to the first data element obtained from the first vector being a non-zero data, performing a multiplication operation on the obtained first data element and multiple second data elements in the second vector in parallel includes: In response to the first data element obtained from the first vector being a non-zero data, perform a multiplication operation on the obtained first data element and all second data elements in the second vector in parallel.

3. The data processing method according to claim 2, wherein, In response to the first data element obtained from the first vector being a non-zero data, performing a multiplication operation on the obtained first data element and multiple second data elements in the second vector in parallel includes: In response to the data obtained from the first vector being non-zero data, send the non-zero data to a processing module to perform the multiply-accumulate operation on the first vector and the second vector by the processing module. Wherein, the processing module includes multiple multiplication processing sub-units, and each multiplication processing sub-unit correspondingly receives a second data element to perform a multiplication operation.

4. The data processing method according to any one of claims 1 to 3, wherein, The first vector is one of the parallel vectors in the input feature map of the neural network computing; The second vector is one of the parallel vectors in the weight map of the neural network computing.

5. The data processing method according to any one of claims 1-4, wherein, The vectors in the input feature map of the neural network computing include first data elements.

6. The data processing method according to any one of claims 1-5 further includes: Cache each first vector as a whole once. Obtaining first data elements from the first vector one by one includes: Obtain the first data elements from the cached first vector one by one.

7. The data processing method according to any one of claims 1-6, wherein, At least one first vector includes multiple first vectors. Perform the multiply-accumulate operation on each of the multiple first vectors and the second vector in parallel.

8. A data processing array for neural network computing, including at least one input module and at least one processing module. Wherein, The input module is configured to obtain first data elements from the first vector one by one in each operation cycle to input the first vector, and The processing module is configured to, in response to the first data element obtained from the first vector being a non-zero data, perform a multiplication operation on the obtained first data element and multiple second data elements in the second vector in parallel, or in response to the first data element obtained from the first vector being 0, skip the multiplication operation for the obtained first data element, so as to perform a multiply-accumulate operation on at least one first vector and a second vector.

9. The processing array according to claim 8, wherein, The input module includes a buffer module, and the buffer module is configured to cache the first vector as a whole once.

10. The processing array according to claim 7 or 8, wherein, The processing module includes a plurality of multiplication processing sub-units, and each multiplication processing sub-unit is configured to perform a multiply-accumulate operation on a corresponding second data element and the acquired first data element.

11. The processing array according to claim 10, wherein The multiplication processing sub-unit includes a multiplier.

12. The processing array according to any one of claims 7-10, wherein, The at least one processing module includes a plurality of processing modules. The at least one input module includes a plurality of input modules respectively corresponding to the plurality of processing modules. Each of the plurality of input modules is configured to respectively input a plurality of first vectors. The plurality of processing modules are configured to respectively perform a multiply-accumulate operation on each of the plurality of first vectors and the second vector in parallel.

13. A data processing device for neural network computing, comprising the processing array according to any one of claims 7-12.

14. A data processing device for neural network computing, comprising: a processor, a memory having one or more computer program modules stored thereon; wherein, when the one or more computer program modules are configured to be executed by the processor, they are used to execute the data processing method according to any one of claims 1-7.

15. A non-transitory storage medium that non-transitorily stores computer-readable instructions, wherein, When the computer-readable instructions are executed by a computer, the data processing method according to any one of claims 1-7 is executed.

16. A computer program product comprising a computer program / instructions, wherein, When the computer program / instructions are executed by the processor, the data processing method according to any one of claims 1-7 is implemented.

Images