Method of transporting data, direct memory access device and computer system

Abstract

Disclosed are a data moving method, a direct memory access device and a computer system. The data moving method is used for a neural network processor including at least one processing unit array, and the method includes: receiving a first instruction, wherein the first instruction indicates address information of target data to be moved, and the address information of the target data is obtained based on a mapping relationship between the target data and at least one processing unit in the processing unit array; generating a data moving request according to the address information of the target data; and moving the target data for the neural network processor according to the data moving request.

Description

Technical Field

[0001] This disclosure relates to the field of chips, and more specifically to a method for transporting data, a direct memory access device, and a computer system. Background Technology

[0002] In computer systems, data exchange between I / O memory and between memory modules is frequently required. Direct Memory Access (DMA) technology is a high-speed data transfer method that allows direct data transfer between external devices and memory, and between memory modules. The DMA process is primarily implemented in hardware. In this case, data exchange between external devices and memory is not controlled by the Central Processing Unit (CPU), but rather utilizes the system bus. The DMA hardware controller directly controls the bus to complete the data exchange between external devices and memory. Summary of the Invention

[0003] According to a first aspect of this disclosure, a method for moving data is proposed for a neural network processor, the neural network processor including at least one processing unit array, the method comprising: receiving a first instruction, wherein the first instruction indicates address information of target data to be moved, the address information of the target data being obtained based on a mapping relationship between the target data and at least one processing unit in the processing unit array; generating a data moving request based on the address information of the target data; and moving the target data for the neural network processor according to the data moving request.

[0004] In some embodiments, before generating the data transfer request based on the address information of the target data, the method further includes: decoding the first instruction to obtain the address information of the target data.

[0005] In some embodiments, the first instruction further includes a channel identifier indicating a target channel among a plurality of channels in the neural network processor, and the method further includes: decoding the first instruction to obtain the channel identifier; and writing the first instruction into a target storage area according to the channel identifier, wherein the target storage area corresponds to the target channel.

[0006] In some embodiments, the method further includes: reading the first instruction from the target storage area to generate the data transfer request based on a handshake signal sent by the neural network processor.

[0007] In some embodiments, the first instruction is received by the target direct memory access device and the data transfer request is responded to. The method further includes: receiving an initial instruction, decoding the initial instruction to obtain the first instruction and a DMA identifier of the target direct memory access device for executing the first instruction; and providing the first instruction to the target direct memory access device according to the DMA identifier.

[0008] In some embodiments, the initial instruction further indicates an instruction type, and the method further includes: decoding the initial instruction to obtain the instruction type in the process of obtaining the first instruction; and writing the first instruction into the target storage area in response to the instruction type being a preset instruction type.

[0009] In some embodiments, the method further includes: controlling the decrease or increase of a valid number in response to writing or reading the first instruction, wherein the valid number indicates the space size of the target storage area for storing instructions.

[0010] In some embodiments, reading the first instruction from the target storage area to generate the data transfer request based on a handshake signal sent by the neural network processor includes: polling the plurality of channels to select the target channel; and reading the first instruction from the target storage area to generate the data transfer request based on a handshake signal sent by the target channel.

[0011] In some embodiments, before reading the first instruction from the target storage area, the method further includes: determining whether the target storage area is empty; and reading the first instruction from the target storage area in response to the target storage area being non-empty.

[0012] In some embodiments, before writing the first instruction to the target storage area, the method further includes: determining whether the target storage area is full; and writing the first instruction to the target storage area in response to the target storage area being non-full.

[0013] According to a second aspect of this disclosure, a direct memory access device is provided for a neural network processor, wherein the neural network processor includes at least one processing unit array, the device comprising: a receiving unit configured to receive a first instruction, wherein the first instruction indicates address information of target data to be transferred, the address information of the target data being obtained based on a mapping relationship between the target data and at least one processing unit in the processing unit array; and an interface control unit configured to generate a data transfer request based on the address information of the target data, and to transfer the target data to the neural network processor according to the data transfer request.

[0014] In some embodiments, the interface control unit further includes a first decoding unit configured to decode the first instruction to obtain address information of the target data.

[0015] In some embodiments, the first instruction further includes a channel identifier, the channel identifier indicating a target channel among a plurality of channels in the neural network processor, and the apparatus further includes: a second decoding unit configured to decode the first instruction to obtain the channel identifier; an instruction virtual channel storage unit; and an instruction write control unit configured to write the first instruction into a target storage area of ​​the instruction virtual channel storage unit according to the channel identifier, wherein the target storage area corresponds to the target channel.

[0016] In some embodiments, the system further includes an instruction read control unit configured to read the first instruction from the target storage area based on a handshake signal sent by the neural network processor to generate the data transfer request.

[0017] In some embodiments, a target direct memory access device receives the first instruction and responds to the data transfer request. The target direct memory access device is coupled to an interface decoder, which includes: a third decoding unit configured to receive an initial instruction, decode the initial instruction to obtain the first instruction and a DMA identifier of the target direct memory access device for executing the first instruction; and provide the first instruction to the target direct memory access device according to the DMA identifier.

[0018] In some embodiments, the initial instruction further indicates an instruction type, and the interface decoder further includes: a fourth decoding unit configured to decode the initial instruction to obtain the instruction type in the process of obtaining the first instruction, and to write the first instruction into the target storage area in response to the instruction type being a preset instruction type.

[0019] In some embodiments, the system further includes: a valid number recording unit configured to control the decrease or increase of a valid number in response to writing or reading the first instruction, wherein the valid number indicates the space size of the target storage area for storing instructions.

[0020] In some embodiments, the instruction read control unit further includes: a polling scheduling unit configured to poll and schedule the plurality of channels to select the target channel; and an instruction read control module configured to read the first instruction from the target storage area based on a handshake signal sent by the target channel to generate the data transfer request.

[0021] In some embodiments, the instruction read control unit is further configured to: determine whether the target storage area is empty before reading the first instruction from the target storage area; and read the first instruction from the target storage area in response to the target storage area being non-empty.

[0022] In some embodiments, the instruction write control unit is further configured to: determine whether the target storage area is full before writing the first instruction to the target storage area; and store the first instruction to the target storage area in response to the target storage area being non-full.

[0023] According to a third aspect of this disclosure, a computer system is proposed, comprising: a direct memory access device according to any of the second aspects of this disclosure; and a neural network processor, wherein the neural network processor is electrically coupled to the direct memory access device.

[0024] In some embodiments, the device further includes: an interface decoder configured to receive an initial instruction, decode the initial instruction to obtain the first instruction, and provide the decoded first instruction to the direct memory access device.

[0025] In some embodiments, it further includes an auxiliary processor configured to provide the initial instructions to the interface decoder.

[0026] In some embodiments, the auxiliary processor is coupled to the interface decoder via a first data transmission protocol, and the neural network processor is coupled to the direct memory access device via a second data transmission protocol, wherein the first data transmission protocol and the second data transmission protocol are different.

[0027] The data transfer method, direct memory access device, and computer system provided in this disclosure can simplify the hardware structure of DMA and improve the flexibility of neural network data processing systems. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The following drawings are not intentionally drawn to scale to actual size; their focus is on illustrating the main points of the present invention.

[0029] Figure 1 A schematic example of convolution operation using a 2×2 filter is shown;

[0030] Figure 2 A schematic diagram of the architecture of a neural network processor is shown;

[0031] Figure 3 This illustrates a schematic example of a single PE performing a one-dimensional convolution operation;

[0032] Figures 4 to 7 A schematic example is shown where at least some of the PEs in the PE array perform a two-dimensional convolution operation;

[0033] Figure 8 and Figure 9 Illustrative examples of some methods for cutting filters are shown;

[0034] Figure 10 A schematic diagram of the structure of R-type instructions in the RISC-V instruction set is shown;

[0035] Figure 11 This provides a conceptual illustration of the offset information in this disclosure;

[0036] Figure 12 This provides a conceptual illustration of the length information in this disclosure;

[0037] Figure 13 A conceptual illustration of the channel information for the DMA in this disclosure is provided;

[0038] Figure 14 This conceptually illustrates a description of the cutting status information of the data to be transferred in this disclosure;

[0039] Figure 15 An example of a first type of instruction according to this disclosure is shown;

[0040] Figure 16 An example of a second type of instruction according to this disclosure is shown;

[0041] Figure 17 An example of a third type of instruction according to this disclosure is shown;

[0042] Figure 18 This illustrates a schematic example of multiple segments of continuous data;

[0043] Figure 19 An example of an instruction is shown, which includes encoding indicating the start address and characteristic information of multiple consecutive data segments;

[0044] Figure 20 Another example of instructions for moving multiple consecutive segments of data according to this disclosure is shown;

[0045] Figure 21 An example architecture diagram of a system that applies DMA in this disclosure is shown;

[0046] Figure 22 A schematic diagram of an example DMA architecture according to this disclosure is shown;

[0047] Figure 23 A schematic diagram of another example of a DMA architecture according to this disclosure is shown. Detailed Implementation

[0048] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are also within the scope of protection of the present invention.

[0049] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0050] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0051] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0052] Neural networks are mathematical computational models inspired by the structure of neurons in the brain and the principles of neural transmission. The method of achieving intelligent computing based on such models is called brain-inspired computing. For example, neural networks include various network structures, such as backpropagation (BP) neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs). For instance, convolutional neural networks can be further subdivided into fully convolutional networks, deep convolutional networks, and U-Nets.

[0053] The convolution operation of a neural network is to perform a matrix inner product between the input feature map and the filter / convolution kernel. Figure 1 A schematic example of convolution operation using a 2×2 filter is shown. Figure 1 As shown, the filter F is a 2×2 matrix with input features. Figure X The input features are a 3×3 matrix. Figure X After performing a convolution operation with filter F, the output feature map O is obtained. The size of output feature map O is a 2×2 matrix. The result of the convolution operation on output feature map O is as follows:

[0054] O 11 =F 11 X 11 +F 12 X 12 +F 21 X 21 +F 22 X 22

[0055] O 12 =F 11 X 12 +F 12 X 13 +F 21 X 22 +F 22 X 23

[0056] O 21 =F 11 X 21 +F 12 X 22 +F 21 X 31 +F 22 X 32

[0057] O 22 =F 11 X 22 +F 12 X 23 +F21 X 32 +F 22 X 33

[0058] Because neural networks are computationally intensive, especially convolutional layers with large input feature maps, it is often necessary to decompose the computational operations of a convolutional layer within a neural network. For example, convolutional operations for different parts of the same convolutional layer can be performed independently. These decomposed tasks are then computed in parallel by multiple processing units. The results of these processing units are then combined to obtain the computational result of the entire convolutional layer, which can then be used as the input to the next convolutional layer.

[0059] A neural network processing unit (NPU) is a type of microprocessor or computing system specifically designed for hardware acceleration of artificial intelligence (especially artificial neural networks, machine vision, machine learning, etc.), and is sometimes referred to as an AI accelerator.

[0060] Figure 2 A schematic diagram of a neural network processor architecture is shown, such as the Eyeriss architecture neural network processor. Figure 2 As shown, the neural network processor includes a processing unit (PE) array 110, a global cache 120, and memory 130. The processing unit array 110 includes multiple rows and columns (e.g., 12 rows × 12 columns) of processing units, which are coupled to each other via on-chip interconnects and share the global cache 120, such as a network-on-chip (NoC). Each processing unit has computational capabilities and may also have its own local cache, such as including a multiply-accumulate (MAC) unit and a cache or register array for caching input vectors (or matrices). Each PE can access other surrounding PEs, its own local cache, and the global cache. The global cache 120 is further coupled to memory 130 via, for example, a bus.

[0061] During operation, data such as the convolutional kernel (Flt) and input feature map (Ifm) required for computation in a network layer (e.g., a convolutional layer) are read from memory 130 into global cache 120. Then, from global cache 120, the convolutional kernel (Flt) and input image (Img) are input into processing unit array 110 for computation. Computation tasks for different image pixels are assigned to different processing units (i.e., mapping is performed). The partial sum (Psum1) generated during the computation process is temporarily stored in global cache. If subsequent computations require further accumulation operations on the previously generated partial sum (Psum1), the required partial sum (Psum2) can be read from global cache 120 into processing unit array 110 for computation. The output feature map (Ofm) obtained after completing the computation of a convolutional layer can be output from global cache 120 and stored in memory 130, for example, for use in the computation of the next network layer (e.g., a convolutional layer).

[0062] For example, data generated by the processing unit array 110, especially data involving sparse matrices, can be compressed and stored. One compression method for sparse matrices is RLC encoding, which encodes consecutive zeros as the number of zeros, thus saving storage space. During the process of data being stored from the processing unit array 110 into the memory 130, an encoder can be used to compress and encode the data; correspondingly, during the process of data being read from the memory 130 into the processing unit array 110, a decoder can be used to decompress the data.

[0063] Figure 3 This illustrates a schematic example of a single PE performing a one-dimensional convolution operation. Figure 3 As shown, the rows of the filter and the rows of the input feature map are one-dimensional vectors. During the convolution operation, the filter weights remain unchanged. The window of the input feature map is slid, and the multiplication and addition operations are performed sequentially through the multiplier / adder (MAC) to obtain the multiplication and addition results. The output feature map can be obtained by sorting the obtained multiplication and addition results.

[0064] Figures 4 to 7 A schematic example is shown where at least a portion of the PEs in an array perform a two-dimensional convolution operation. For example... Figure 4 As shown, performing a convolution operation on a 5×5 input feature map and a 3×3 filter yields a 3×3 output feature map. Figure 4 In the example of convolution operation shown, the input feature map has 5 rows, the filter has 3 rows, and the output feature map has 3 rows. Therefore, a 3×3 PE array can be used to perform the operation. Figure 4 The convolution operation is shown in the figure.

[0065] like Figure 5 As shown, each PE in the 3×3 PE array can perform the following: Figure 3 The diagram illustrates a one-dimensional convolution operation. PE1, PE2, and PE3 represent the first row of a 3×3 PE array. The filter weights in PE1, PE2, and PE3 are the first, second, and third rows of the filters, respectively. The eigenvalues ​​of the input feature maps in PE1, PE2, and PE3 are the first, second, and third rows of the input feature maps, respectively. The multiply-accumulate result in PE3 is accumulated into the multiply-accumulate result in PE2, and the multiply-accumulate result in PE2 is accumulated into the multiply-accumulate result in PE1, finally yielding the first row of the output feature map. PE 4, PE 5, and PE 6 form the second row of a 3×3 PE array. The filter weights in PE 4, PE 5, and PE 6 are the first, second, and third rows of the filters, respectively. The eigenvalues ​​of the input feature maps in PE 4, PE 5, and PE 6 are the second, third, and fourth rows of the input feature maps, respectively. The multiply-accumulate result in PE 6 is added to the multiply-accumulate result in PE 5, and the multiply-accumulate result in PE 5 is added to the multiply-accumulate result in PE 4, finally resulting in the second row of the output feature map. PE 7, PE 8, and PE 9 form the third row of a 3×3 PE array. The filter weights in PE 7, PE 8, and PE 9 are the first, second, and third rows of the filters, respectively. The eigenvalues ​​of the input feature maps in PE 7, PE 8, and PE 9 are the third, fourth, and fifth rows of the input feature maps, respectively. The multiply-accumulate result in PE 9 is added to the multiply-accumulate result in PE 8, and the multiply-accumulate result in PE 8 is added to the multiply-accumulate result in PE 7, finally resulting in the third row of the output feature map.

[0066] Figure 6 and Figure 7 Each illustrated schematically Figure 5 The reuse method of filter weights and input feature map feature values ​​in a 3×3 PE array during convolution operations. For example... Figure 6 As shown, the first, second, and third rows of filter weights are repeated horizontally in the first, second, and third rows of a 3×3 PE array, respectively. Figure 7 As shown, the second row of input feature map feature values ​​is repeated between PE 4 and PE 2 at its diagonal position; the third row of input feature map feature values ​​is repeated between PE 7 and PE 5 and PE 3 at its diagonal position; and the fourth row of input feature map feature values ​​is repeated between PE 8 and PE 6 at its diagonal position.

[0067] As described above, the PE array in the Eyeris architecture neural network processor achieves row-stationary (RS) data flow by horizontally reusing rows of filter weights between PEs and reusing rows of input feature map feature values ​​diagonally between PEs. Furthermore, as described above, in row-stationary data flow, there is a mapping relationship between the rows of input feature map feature values ​​and the rows of filter weights and at least one PE in the PE array, for example, as... Figure 6 and Figure 7 As shown, the first row of the input feature map features and the first row of the filter weights are mapped to PE1, and the second row of the input feature map features and the second row of the filter weights are mapped to PE4, etc. For ease of description, this paper can refer to the above mapping relationship as the mapping relationship of the input feature map, the mapping relationship of the filter, or the mapping relationship of the weights, etc.

[0068] To achieve the row-fixed data stream described in this disclosure, one approach proposes segmenting the input feature map and the filter. Figure 8 and Figure 9 Illustrative examples of some methods for cutting filters are shown. Figure 8 Figure 801 illustrates a filter that includes four dimensions: height, length, number of channels, and number of filters. The height of a single filter is R, the length is S, the number of channels is C, and the number of filters is M. Figure 8 Figure 802 illustrates a method for splitting a filter. As shown at 802, the channel C of the first filter can be split into at least one channel group, the number of channels in the channel group being G, where G < C. Figure 8 Figure 803 illustrates another method for splitting filters. As shown at 803, the filters can be split one by one based on a single weight in the height and length dimensions of the first filter, while the number of channels in each split filter remains C. Figure 9 Figure 901 shows a two-dimensional filter that includes two dimensions: height R and length S. The filter has a size of 5×5. Figure 9 902 shows a method for cutting the filter. As shown at 902, a 5×5 filter can be cut based on two 3×3 filters. Figure 9 903 shows another method for cutting the filter. As shown at 903, a 5×5 filter can be cut based on a 1×5 filter or a 5×1 filter.

[0069] Some algorithms can be used to segment the data of the input feature map or filter, and the segmented data of the input feature map or filter has the mapping relationship of the input feature map or filter described in this paper.

[0070] Direct Memory Access (DMA) is an interface technology that allows data exchange directly with system memory without going through the Central Processing Unit (CPU). It is commonly used for data transfer in computer architectures. In this disclosure, the hardware structure implementing the DMA process can be referred to as a direct memory access device or a DMA device; for ease of description, DMA device and DMA can be used interchangeably. The data of the filter or input feature map is typically stored contiguously in memory, for example... Figure 8 The storage method for the first filter in the 801 is as follows: First, store them sequentially in the channel direction (e.g., C0 to C...). n Then store them sequentially along the length direction (e.g., S0 to S). n This forms a "Z"-shaped continuous storage as shown in Figure e; finally, it is stored sequentially in the height direction (e.g., from R0 to R). n ).

[0071] In some neural network processing units (NPUs) with fixed-flow data (such as the Eyeriss architecture described above), DMA (Data Transfer Module) needs to move data based on the mapping relationships of input feature maps or filters. Some algorithms can slice the input feature maps or filter data in various dimensions, resulting in discontinuous data with complex mapping relationships. If DMA is used to parse the mapping relationships of input feature maps or filters and move the data based on the parsed results, a relatively complex DMA hardware architecture is required to perform this parsing function.

[0072] To address the aforementioned technical issues, this disclosure proposes an instruction scheme based on coprocessor-controlled DMA. The coprocessor executes algorithms to segment the data of the input feature map or filter and outputs instructions. The segmented data of the input feature map or filter has a mapping relationship with the input feature map or filter. The DMA decodes the instructions output by the coprocessor and performs data transfer based on the decoding results, thereby simplifying the hardware structure of the DMA and improving the flexibility of neural network processor (e.g., RS-type NPU) systems.

[0073] The following is combined Figures 10 to 17 The structure and combination of instructions in this disclosure are described.

[0074] The RISC-V instruction set is an open-source instruction set architecture based on the Reduced Instruction Set Computing (RISC) principle. Figure 10 This diagram illustrates the structure of R-type instructions (also known as register-type instructions) in the RISC-V instruction set. Figure 10 As shown, the R-type instruction has a 32-bit field, including a 7-bit opcode, a 5-bit rd, a 3-bit func3, a 5-bit rs1, a 5-bit rs2, and a 7-bit func7. Here, opcode indicates the opcode, rd indicates the destination register number, func3 is an expandable opcode / function code, rs1 indicates the first source register number, rs2 indicates the second source register number, and func7 is an expandable opcode / function code.

[0075] In the RISC-V instruction set, the opcode is used to indicate the type of instruction. The opcode of R-type instructions is a fixed value [0110011]. The func3 in R-type instructions is used to indicate different functions of the R-type instructions, including address, logic, and operation. Among them, the func3 of R-type instructions that indicate addresses is a fixed value

[000] .

[0076] In this disclosure, opcode and func3 can indicate that the instruction type is R-type instruction and the instruction is a data moving instruction indicating an address. For ease of description, the instruction indicating an address and the data moving instruction can be used interchangeably. For example, the instruction indicating an address and the data moving instruction indicated by func3 can be used interchangeably.

[0077] In this disclosure, the type of a data transfer instruction can be encoded using some bits (e.g., three bits) of func7 in the R-type instruction. In one example, the following five types of data transfer instructions can be encoded using three bits of func7 in the R-type instruction:

[0078] ①: Request for uncompressed data of the input feature map;

[0079] ②: Request for uncompressed data of the filter;

[0080] ③: Request for compressed data for the filter;

[0081] ④: Read the valid number of requests for data from the input feature map;

[0082] ⑤: Read the valid number of data requests for the filter.

[0083] For example, the encoding of the five types of data transfer instructions above is as follows:

[0084]

[0085]

[0086] In this disclosure, the process of parsing a data transfer instruction by a DMA and executing the instruction based on the parsed information is called executing the data transfer instruction by the DMA, and the data transfer instruction is called an object data transfer instruction. For example, ①, ②, and ③ here are object data transfer instructions. As will be described below in this disclosure, executing object data transfer instructions by the DMA can realize the transfer of data (e.g., input feature map or filter data) or the transfer of instructions (e.g., the second type of instructions described below in this disclosure). That is, data transfer instructions can realize the transfer of different objects, including both data transfer and instruction transfer.

[0087] In this disclosure, the 5-bit rs1, rs2, and rd in the data transfer instruction respectively indicate the addresses of the first source register, the second source register, and the destination register. In this disclosure, at least some bits of the first source register, the second source register, and the destination register can be used to encode information associated with data transfer performed by DMA according to the mapping relationship of the input feature map or filter.

[0088] For ease of description, this article will refer to the data transfer performed by DMA based on the mapping relationship of the input feature map or filter as data transfer.

[0089] In one example, a 32-bit first source register can be used to encode the address information of the data to be moved during data transfer. In this disclosure, the address information of the data to be moved may include base address information and offset address information. Figure 11 This provides a conceptual illustration of the offset information used in this disclosure. For example... Figure 11As shown, the four dimensions of the filter are length R, width S, channels C, and number M. The base address information can be the address information of the first weight in the filter (as shown in the "black box" in the upper left corner of the first channel of the first filter in Figure 11) in memory. This memory can be Dynamic Random Access Memory (DRAM) or Static Random Access Memory (SRAM). If the weight data to be moved is the data of the first channel of the Mth filter, then the offset information can be the first weight in the filter to be moved (e.g., ...). Figure 11 The address information in memory (shown in the upper left corner of the first channel of the Mth filter) and the offset information can indicate the offset information of the first weight to be moved in the filter relative to the weight at the base address.

[0090] In another example, the base address information of the data to be moved can be preset address information (e.g., configured by software), while the offset address information of the data to be moved can be obtained by the coprocessor executing some algorithms. In this way, the first source register can use at least a portion of the bits to encode the offset information without encoding the base address information, thereby simplifying the encoding method and improving the decoding efficiency of DMA data moving instructions. For example, the encoding method of the offset information of the five types of data moving instructions described in this disclosure using the first source register is as follows:

[0091]

[0092] In one example, at least a portion of the bits in the second source register can be used to encode information related to data transfer other than the address information of the data to be transferred as described above. For example, a portion of the bits (e.g., 13 bits) in the 32-bit second source register can be used to encode the length information of the data to be transferred.

[0093] Figure 12 This provides a conceptual illustration of the length information used in this disclosure. For example... Figure 12 As shown, if the weight data to be transferred is the data of the first channel of the Mth filter, then the length information is the length of the data of the first channel of the Mth filter in memory. This length information can be reflected as the length of the data of the first channel of the Mth filter, that is, the first weight and the last weight of the weight data to be transferred (e.g., ...). Figure 12 The length information is shown in the lower right corner of the first channel of the Mth filter (indicated by the black box).

[0094] For example, the encoding method for the length information of the five types of data transfer instructions described in this disclosure using a portion of the bits in the second source register is as follows:

[0095]

[0096] In another example, at least a portion of the bits in the second source register can be used to encode the identification information (ID) of the DMA executing the data transfer instructions. In a row-fixed data stream NPU system, multiple DMAs can be used to perform data transfer tasks; for example, the first data transfer task can be performed by the first DMA, the second data transfer task can be performed by the second DMA, and so on.

[0097] For example, the DMA IDs of the five types of data transfer instructions described in this disclosure are encoded using a portion of the bits in the second source register as follows:

[0098]

[0099] In yet another example, at least a portion of the bits in the second source register can be used to encode channel information in the DMA executing data transfer instructions. The following will combine... Figure 13 The channel information of the DMA in this disclosure is explained.

[0100] Eyeriss V2 is a neural network processor with a fixed-row data stream, which proposes, for example... Figure 13 The architecture shown. (As illustrated) Figure 13 As shown, Eyeriss V2 includes multiple Global Buffers (GLBs) and multiple Entities Executable (PEs). Each GLB stores data or instructions moved by the DMA through the execution of data transfer instructions. Each PE is configured with a GLB, and the PEs and GLBs are connected together via a 2D mesh network. When the DMA executes data transfer instructions, the data or instructions being transferred can be stored in different GLBs or a cluster of GLBs. For example, the data or instructions being transferred can be stored in... Figure 13 The GLB cluster at point A can also be stored in Figure 13The data or instructions being moved can be stored in the GLB cluster at location B; or, the data or instructions being moved can be stored in the first GLB of the GLB cluster at location A, or in the second GLB of the GLB cluster at location A, and so on. For ease of description, the GLB or GLB cluster that can store the data or instructions being moved will be referred to as GLB below. In this example, when the DMA executes data moving instructions, the data or instructions being moved can be stored in different GLBs, and the data channels in the corresponding DMA can also be different. For example, the data or instructions being moved can be stored in 8 different GLBs, and the DMA can include 8 data channels corresponding to these 8 GLBs. The data channels here can be the DMA channels described above in this disclosure.

[0101] For example, the encoding method for channel information of the five types of data transfer instructions described in this disclosure using a portion of the bits in the second source register is as follows:

[0102]

[0103] In yet another example, at least a portion of the bits in the second source register can be used to encode the cutting status information of the data to be transported during data transfer.

[0104] The following is combined Figure 14 This explanation details the segmentation status information of the data to be transferred in this disclosure. For example... Figure 14 As shown, in the channel dimension of the input feature map, "C0" represents the range of the cut performed for data transfer of a single PE, "C1" represents the range of the cut performed for data transfer of a single PE cluster, "C2" represents the range of the cut performed for data transfer of a single GLB, and "C3" represents the range of the cut performed for this data transfer task. In this disclosure, the data to be transferred for a single data transfer task can be the original data, for example... Figure 14 The input feature map data before slicing shown in the figure can be the original data. The slicing of the input feature map in other dimensions is similar to the slicing in the channel dimension shown in Figure 14, and the slicing of the output feature map and filter in their respective dimensions is similar to the slicing of the input feature map in their respective dimensions, which will not be repeated here. The slicing status information in this disclosure can be information about the last slice of the data to be transported in at least one dimension. For example, the encoding of the last slice information (denoted as _last) in multiple dimensions using 4 bits in the second source register is as follows:

[0105]

[0106] In another example, at least a portion of the bits in the second source register can be used to encode operation information of the data to be moved during data transfer. In this disclosure, the operation information of the data to be moved can be information about performing operations on the data before performing data transfer to adapt the manipulated data to the operation of the neural network. In one example, the operation information of the data to be moved can be information about a padding operation. Exemplarily, the padding operation information is encoded using 4 bits in the second source register as follows:

[0107]

[0108] In yet another example, at least a portion of the bits in the second source register can be used to jointly encode the cutting status information and operation information of the data to be transported during data transfer.

[0109] In this example, a subset of bits from the second source register can be used as identification codes for the cutting status information and operation information. For instance, 8 bits from the second source register can be used to jointly encode the cutting status information and operation information, with the higher 4 bits serving as identification codes and the lower 4 bits serving as the encoding for either the cutting status information or the operation information. For example, the encoding of the cutting status information and operation information of the data to be transferred during data transfer using 8 bits from the second source register is as follows:

[0110]

[0111]

[0112] In yet another example, at least a portion of the bits in the second register can be used to encode transport status information during data transport.

[0113] In this example, the transport status information can be for a portion of the data to be transported, or it can be for all the data to be transported. For example, the transport status information can include information that the data transport to the GLB is complete, or it can include information that the data transport task is complete. In this example, the transport status information can be encoded using 4 bits of the second register; for example, information that the data transport to the GLB is complete and / or information that the data transport task is complete can be encoded as 0001.

[0114] In yet another example, at least a portion of the bits in the second source register can be used to jointly encode multiple transport status information.

[0115] In this example, a subset of bits from the second source register can be used as identification codes for each transport status message. For instance, two bits from the second source register can be used as identification codes for the first transport status message (e.g., information indicating that data transport to the GLB has been completed) and the second transport status message (e.g., information indicating that the data transport task has been completed). Preferably, the identification information for the first and second transport status messages can be encoded using a field in the ACMD FLAG field of the R-type instruction. For example, the encoding of the first and second transport status messages using 6 bits from the second source register is as follows:

[0116]

[0117] In one example, at least a portion of the bits in the destination register can be used to encode the number of valid requests for data on the input feature map or the number of valid requests for data on the filter. In this example, the number of valid requests for data can be the number of requests for data. For example, if the number of valid requests is 5, it indicates that there are 5 requests for data.

[0118] For example, the encoding of the valid numbers described in this disclosure using at least a portion of the bits in the destination register is as follows:

[0119]

[0120] The above content combined Figures 10 to 14 The encoding method for the structure and combination of instructions in this disclosure has been explained. The following will continue to discuss this in conjunction with... Figures 15 to 17 The functions of the instructions in this disclosure are explained.

[0121] The instructions in this disclosure include data transfer instructions for performing transfer on data to be transferred and data transfer instructions for performing transfer on control information. For ease of description, this disclosure refers to data transfer instructions for performing transfer on data to be transferred as first type instructions and data transfer instructions for performing transfer on control information as second type instructions.

[0122] Figure 15 An example of a first type of instruction according to this disclosure is shown. For example... Figure 15As shown, the instruction is an R-type instruction (indicated by an opcode of a fixed value [0110011]), the instruction is a data transfer instruction (indicated by func3 of a fixed value

[000] ), and the type of the data transfer instruction corresponding to this instruction is ① (indicated by the encoding of func7 in this disclosure). At least some bits in the first source register of this instruction indicate the offset information of the data to be transferred, and at least some bits in the second source register of this instruction indicate the length information of the data to be transferred. In an optional example, the first type instruction may also include at least one of the encoding of the DMA identification information described above in this disclosure and the encoding of the channel information in the DMA. Exemplarily, in Figure 15 In the first type of instruction shown, at least some bits in its second source register can indicate the identification information of the DMA and / or the channel information in the DMA.

[0123] Figure 16 An example of a second type of instruction according to this disclosure is shown. For example... Figure 16 As shown, the instruction is an R-type instruction (indicated by the fixed value [0110011] of opcode), the instruction is a data transfer instruction (indicated by the fixed value

[000] of func3), and the type of the data transfer instruction corresponding to this instruction is ① (indicated by the encoding of func7 in this disclosure). The first source register of this instruction no longer indicates the offset information of the data to be transferred, and the second source register in this instruction no longer indicates the length information of the data to be transferred. At least some bits in the second source register of this instruction indicate the operation information of the data to be transferred as described above in this disclosure. In an optional example, the second type instruction may also include the encoding of the cutting status information of the data to be transferred as described above in this disclosure and at least one of the encodings of one or more transfer status information. For example, in Figure 16 In the second type of instruction shown, at least some bits in its second source register can indicate the cutting status information and / or one or more transport status information of the data to be transported.

[0124] In an optional example, at least a portion of the bits in the second source register of the first type instruction and the second type instruction in this disclosure may be used to encode the priority of the first type instruction and the second type instruction. In this example, the instruction priority indicates the order in which the instructions are sent, with higher priority instructions sent in a higher order and lower priority instructions sent in a lower order. In this example, the second type instruction including the encoded operation information of the data to be transferred as described above in this disclosure, and the second type instruction including the encoded cutting status information of the data to be transferred as described above in this disclosure, are at a first priority, the first type instruction of this disclosure are at a second priority, and the second type instruction including the encoded one or more transfer status information described above in this disclosure are at a third priority.

[0125] In an optional example, the first type instruction may include encoding indicating transport status information. For example, the first type instruction may include encoding of the first transport status information described above in this disclosure. To distinguish between first type instructions that include encoding indicating transport status information and first type instructions that do not include encoding indicating transport status information, this distinguishing information may be encoded. Exemplarily, this distinguishing information may be encoded using 4 bits of a second source register; for example, the first type instruction that includes encoding indicating transport status information may be encoded as 0010, and the first type instruction that does not include encoding indicating transport status information may be encoded as 0001. Optionally, the first type instruction that includes encoding indicating transport status information has a lower priority than the first type instruction that does not include encoding indicating transport status information.

[0126] The instructions disclosed herein also include data transfer instructions for reading the significant numbers described above in this disclosure. For ease of description, the data transfer instructions for reading significant numbers may be referred to as third-type instructions.

[0127] Figure 17 An example of a third type of instruction according to this disclosure is shown. For example... Figure 17 As shown, this instruction is an R-type instruction (indicated by an opcode of a fixed value [0110011]), this instruction is a data transfer instruction (indicated by func3 of a fixed value

[000] ), and the type of the data transfer instruction corresponding to this instruction is ④ (indicated by the encoding of func7 in this disclosure). At least some bits in the destination register of this instruction indicate the valid number described above in this disclosure, and at least some bits in the second source register of this instruction indicate the DMA ID described above in this disclosure. This instruction can be used to read the number of data requests stored in the corresponding DMA. In another example, Figure 17At least some bits in the second source register of the third type of instruction shown also indicate the channel ID described above in this disclosure.

[0128] In the data transfer process of this disclosure, the data to be transferred can be a continuous data segment. The starting address of this continuous data segment can be determined by the offset information described above, and the size of this continuous data segment can be determined by the length information described above. Through the first type of instructions and the second type of instructions described above, the transfer of a continuous data segment and the transfer of control information for that continuous data segment can be realized.

[0129] However, this disclosure is not limited to this; in the data transfer process of this disclosure, the data to be transferred can also be multiple consecutive data segments. The following is in conjunction with... Figures 18 to 20 An example of instructions for moving multiple consecutive segments of data in this disclosure is illustrated.

[0130] Figure 18 This illustrates a schematic example of multiple segments of continuous data. For example... Figure 18 As shown, the starting address of the first segment of continuous data is the address of its first data element. The first segment contains four data elements; excluding the first, the remaining data has a length of 3, therefore the length of the first segment is 3. The interval between the address of the first data element and the address of the last data element in the second segment is called the step size, and the length of the second segment is the same as that of the first segment (3). The interval between the address of the first data element and the address of the last data element in the third segment is also the step size, and the length of the third segment is the same as that of the first segment (3). This continues, resulting in a total of N segments of continuous data. Figure 18 It can be seen that the address of each data in multiple consecutive data segments can be determined by the starting address, length, step size, and number of segments.

[0131] In the data transfer process disclosed herein, in order to transfer multiple consecutive data segments, it is necessary to encode the starting addresses and characteristic information of the multiple consecutive data segments. Figure 19 An example of an instruction is shown, which includes encoding indicating the starting address and characteristic information of multiple consecutive data segments.

[0132] like Figure 19 As shown, the instruction includes an address field and a control field. The address field can use at least one bit to encode the starting address of multiple consecutive data segments, and the control field can use multiple bits to encode the feature information of multiple consecutive data segments, wherein the feature information includes... Figure 18 The length information, step size information, and number of segments are shown in the figure.

[0133] In this disclosure, the instruction that includes the encoding of the starting address and characteristic information of multiple consecutive data segments can be an R-type instruction. Figure 20 Another example of instructions for moving multiple segments of continuous data according to this disclosure is shown.

[0134] like Figure 20 As shown, this instruction is an R-type instruction (indicated by the fixed value [0110011] of opcode), this instruction is a data transfer instruction (indicated by the fixed value

[000] of func3), and the type of the data transfer instruction corresponding to this instruction is ① (indicated by the encoding of func7 in this disclosure). At least some bits in the first source register of this instruction can indicate the offset information of multiple consecutive data segments. For example, the offset information is encoded using 27 bits in the first source register. At least some bits in the second source register of this instruction can indicate the offset information of multiple consecutive data segments in this disclosure. Figure 18 The length information is described. For example, the length information is encoded using 8 bits in a second source register. At least some bits in the second source register of this instruction can indicate the combination of this disclosure. Figure 18 The segment number information is described. For example, the segment number information is encoded using 5 bits in the second source register. In this example, partial bits from the first source register and partial bits from the second source register of this instruction can be used to combine this disclosure. Figure 18 The described step size information is encoded. For example, 5 bits from the first source register are used as the high-order bits of the step size information, and 8 bits from the second source register are used as the low-order bits. By combining some bits from the first source register and some bits from the second source register to encode the step size information, the source register resources can be fully utilized.

[0135] In an optional example, the above combination Figure 19 and Figure 20 The described instructions may also include an encoding indicating at least one of the DMA ID and channel ID described above in this disclosure.

[0136] To form a complete instruction set, the instructions in this disclosure can be encoded using 2 bits of the ACDM FLAG field in the R-type instruction. For example, the following four types of instructions: data transfer instructions for performing data transfer, data transfer instructions for performing control information transfer, data transfer instructions for performing transfer of multiple consecutive data segments, and data transfer instructions containing encoded information indicating a second transfer status (e.g., information that the data transfer task has been completed), have the following corresponding ACDM FLAG field encodings:

[0137]

[0138] The instructions in this disclosure have been described above. This disclosure proposes an instruction scheme based on coprocessor-controlled DMA. The coprocessor executes some algorithms to cut the data of the input feature map or filter and output instructions. The cut input feature map or filter data has a mapping relationship with the input feature map or filter. The DMA decodes the instructions output by the coprocessor and performs data transfer according to the decoding result, thereby simplifying the hardware structure of DMA and improving the flexibility of the RS NPU system.

[0139] As described above in this disclosure, in a complex NPU system with a fixed row data flow, DMA needs to move data according to the data mapping relationship. Due to the discontinuity of data and the complexity of the mapping relationship in this NPU system, if a corresponding DMA hardware structure is designed according to the specific data mapping relationship to perform data moving, the DMA hardware structure will be extremely complex. Furthermore, a DMA hardware structure designed according to a certain data mapping relationship will fix the data mapping relationship, making the NPU system inflexible.

[0140] To address the aforementioned technical issues, this disclosure proposes a coprocessor-based DMA scheme. The coprocessor executes algorithms to segment the data of the input feature map or filter and outputs instructions. The segmented data of the input feature map or filter has a mapping relationship with the input feature map or filter. The DMA decodes the instructions output by the coprocessor and performs data transfer based on the decoding results, thereby simplifying the hardware structure of the DMA and improving the flexibility of the RS NPU system.

[0141] Figure 21 An example architecture diagram of a system applying the DMA of this disclosure is shown. Figure 21 As shown, the system includes memory, DMA, and a neural network processor. The memory is used to store neural network data, such as convolutional kernels or input feature maps. The memory can be off-chip DRAM or on-chip SRAM. The neural network processor can be integrated with the features described in this disclosure. Figure 2 or Figure 13 The described neural network processor includes an on-chip network and multiple PE arrays interconnected via the on-chip network. DMA, connected via a bus to the on-chip network and PE arrays within the neural network processor, is responsible for data transfer between memory and the PE arrays in the neural network processor.

[0142] Figure 22 A schematic diagram of an example DMA architecture according to this disclosure is shown. Figure 22As shown, the DMA includes an instruction read control unit, an instruction write control unit, an instruction virtual channel storage unit, and an interface control unit. The instruction write control unit writes a received first instruction to the instruction virtual channel storage unit. This first instruction indicates the address information of the target data to be transferred. The address information of the target data is obtained based on the mapping relationship between the target data and at least one PE in the PE array of the neural network processor. For example, the target data has a mapping relationship with the input feature map or filter described in this disclosure. In this example, the address information of the target data is the address information of the target data in memory. In another example, the first instruction can be a combination of this disclosure and the appendix... Figure 15 The first type of instruction is described. An instruction virtual channel storage unit, which can be SRAM, is used to store the first instruction. An instruction read control unit is used to read the first instruction from the instruction virtual channel storage unit into the interface control unit based on the channel's status information. For example, if the channel is idle, the first instruction can be retrieved and transmitted to the interface control unit; if the channel is busy, retrieving the first instruction is prohibited. The interface control unit generates a data transfer request based on the target data's address information and transmits the data transfer request to the on-chip network. The on-chip network then transfers the target data from memory to the interface control unit based on the data transfer request. The interface control unit then transmits the target data to the corresponding channel's subsequent module (e.g., GLB).

[0143] Figure 23 A schematic diagram of another example of a DMA architecture according to this disclosure is shown. Figure 23 As shown, the SCIE (Extended Serial Interface) decoder resides in the coprocessor's clock domain, and the coprocessor outputs instructions through the SCIE bus interface. It should be understood that the instructions output by the coprocessor can be those described above in this disclosure. Exemplarily, the instructions output by the coprocessor can be those described in conjunction with the appendix. Figure 15 The first type of instruction described herein includes a portion of bits in the second source register that also indicates the DMA ID and the channel ID in the DMA. Exemplarily, the instructions output by the coprocessor may also be those described in conjunction with the appendix of this disclosure. Figure 17 The third type of instruction described herein includes a portion of bits in the second source register that also indicate the channel ID in the DMA. For ease of description, this disclosure will refer to the first type of instruction as instruction 1 and the third type of instruction as instruction 3.

[0144] The SCIE decoder includes a first instruction decoder (e.g., corresponding to the fourth decoding unit in this disclosure), to which instruction 1 or instruction 3 output by the coprocessor is transmitted. The first instruction decoder performs a first-level decoding on instruction 1 or instruction 3 to determine whether the instruction output by the coprocessor is of a preset type and whether it is a read instruction or a write instruction. For example, the first instruction decoder can determine whether the instruction output by the coprocessor is an R-type data transfer instruction. In this example, the first instruction decoder decodes the opcode field and func3 field in instruction 1 or instruction 3 to determine that instruction 1 or instruction 3 is an R-type data transfer instruction. The first instruction decoder can also determine whether the instruction is a read instruction or a write instruction. If the decoding result of the first instruction decoder indicates that the instruction output by the coprocessor is a data transfer instruction of type ①, ②, or ③ described above in this disclosure, then the instruction is a write instruction; if the decoding result of the first instruction decoder indicates that the instruction output by the coprocessor is a data transfer instruction of type ④ or ⑤ described above in this disclosure, then the instruction is a read instruction. In this example, the first instruction decoder decodes the func7 field in instruction 1 or instruction 3 to obtain that instruction 1 is a write instruction and instruction 3 is a read instruction.

[0145] The SCIE decoder may further include a First Input First Output (FIFO) queue and a valid number decoder. If the decoding result of the first instruction decoder indicates that the instruction output by the coprocessor is of a preset type and is a write instruction, the decoded instruction is written into the instruction FIFO. If the decoding result of the first instruction decoder indicates that the instruction output by the coprocessor is of a preset type and is a read instruction, the value of the valid number counter in the DMA is read according to the decoded read instruction. For example, after the first instruction decoder decodes the func7 field in instruction 1, it can write the decoded instruction 1 into the instruction FIFO. After the first instruction decoder decodes the func7 field in instruction 3, the SCIE decoder can read the value of the valid number counter in the DMA according to the decoded instruction 3, and decode the read valid number through the valid number decoder to obtain the number of write instructions that can be written in one or more channels of the DMA (i.e., the number of data requests mentioned above in this disclosure).

[0146] In the coprocessor clock domain, each write instruction written to the instruction FIFO decrements the valid number counter in the DMA by 1. For example, when the decoded instruction 1 is written to the instruction FIFO, the valid number counter is decremented by 1. If the SCIE decoder determines that the valid number of instruction 3 is 0 (i.e., the DMA has no space to continue storing instructions), then the decoded instruction 1 needs to wait to be written to the instruction FIFO until the decoded valid number is not 0 (i.e., the DMA has space to continue storing instructions) before the decoded instruction 1 is written to the instruction FIFO.

[0147] The SCIE decoder also includes a second instruction decoder (e.g., corresponding to the third decoding unit in this disclosure), and the instruction FIFO can transmit the decoded instruction 1 to the second instruction decoder. The second instruction decoder can perform a second layer of decoding on the instruction to determine which DMA the instruction is being transferred to. In this example, the second instruction decoder decodes the field indicating the DMA ID in instruction 1 to obtain a signal indicating which DMA instruction 1 will be written to. Figure 23 The DMA in the code is determined based on the decoded DMA ID. After the second instruction decoder decodes instruction 1, it can transfer the decoded instruction 1 to the third instruction decoder.

[0148] Figure 23 The DMA resides in the NPU clock domain, and the third instruction decoder (e.g., corresponding to the second decoding unit in this disclosure) is located within the DMA. Since the DMA and coprocessor are in different clock domains, the instruction FIFO in the SCIE decoder can use an asynchronous FIFO for instruction synchronization. The third instruction decoder can perform a third-level decoding of the instruction to determine which channel in the DMA the instruction will be written to. In this example, the third instruction decoder decodes the channel ID field in instruction 1 to determine which channel instruction 1 will be written to.

[0149] DMA also includes an instruction write control module and an instruction virtual channel storage unit. The instruction virtual channel storage unit includes a storage area corresponding to each channel. For example... Figure 23As shown, the DMA includes eight channels: channel 1, channel 2, channel 3, channel 4, channel 5, channel 6, channel 7, and channel 8. Each channel corresponds to the GLB described above in this disclosure. The DMA is used to move data to and from the GLB on the corresponding channels. The instruction virtual channel storage unit includes eight storage areas, each used to store write instructions on the corresponding channel. For example, if the DMA is used to move data from channel 1 to GLB#1, then storage area #1 stores the write instructions for channel 1; if the DMA is used to move data from channel 2 to GLB#2, then storage area #2 stores the write instructions for channel 2, and so on. The instruction write control module is used to write the decoded instructions into the storage area of ​​the corresponding channel in the instruction virtual channel storage unit according to the decoding result of the third instruction decoder. For example, if the decoding result of the third instruction decoder is channel 1, then the instruction write control module can write the decoded instruction 1 into storage area #1 of the instruction virtual channel storage unit corresponding to channel 1.

[0150] In an optional example, the instruction write control module can control the write address. The start and end addresses of each memory region in the instruction virtual channel memory unit can be configured via a configuration unit in the DMA. In this example, the configuration unit is located in the Advanced Peripheral Bus (APB) clock domain and interacts via the APB interface. If the instruction write control module successfully writes an instruction to a write address in a memory region, it can increment the write address by 1 to allow the next instruction to be written from the next write address in that memory region. When the write address reaches the end address of the memory region, the instruction write control module can toggle the write address to allow the next instruction to be written from the start address of that memory region.

[0151] In an optional example, the instruction write control module can also determine whether the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is full. If the storage area of ​​the corresponding channel is full (i.e., there is no address space to store another write instruction), a signal indicating that the corresponding channel is full is output. Optionally, if the storage area of ​​the corresponding channel is full and there is still a write request, an error signal is output. If the storage area of ​​the corresponding channel is not full (i.e., there is address space to store another write instruction), a write enable signal (denoted as wr_en) is output. The instruction virtual channel storage unit can write the decoded instruction to the storage area of ​​the corresponding channel according to the write enable signal. For example, if the decoding result of the third instruction decoder is channel 1 and the instruction write control module outputs the write enable signal, the instruction write control module can write the decoded instruction 1 to the storage area #1 of the instruction virtual channel storage unit corresponding to channel 1. At this point, the DMA has completed the writing of instruction 1, and the data stream of instruction 1 written to the instruction virtual channel storage unit is as follows: Figure 23The α line is shown in the diagram.

[0152] The following will continue to combine Figure 23 The reading process of instruction 1 will be explained.

[0153] DMA also includes a polling scheduling module and an instruction read control module. The polling scheduling module determines which channel in the instruction virtual channel unit to read the write instruction from. It generates a channel tag signal (ch_tag) and transmits it to the instruction read control module. The instruction read control module reads the write instruction for the corresponding channel in the instruction virtual channel memory unit based on the channel tag. For example, if instruction 1 is written to memory area #1 corresponding to channel 1 in the instruction virtual channel memory unit, and the signal generated by the polling scheduling module is the channel 1 tag signal, the instruction read control module can read instruction 1 from memory area #1.

[0154] The instruction read control module can control the read address. The start and end addresses of each memory region in the instruction virtual channel memory unit can be configured through the configuration unit in the DMA. If the instruction read control module successfully reads an instruction from a read address in a memory region, it can increment the read address by 1 so that the next instruction can be read from the next read address in that memory region. When the read address reaches the end address of the memory region, the instruction read control module can flip the read address so that the next instruction can be read from the start address of that memory region.

[0155] In an optional example, the instruction read control module can also determine whether the storage area of ​​the corresponding channel in the instruction virtual channel storage unit is empty based on the channel tag signal. If there is no readable instruction in the storage area of ​​the corresponding channel, an instruction empty signal is returned. Optionally, if the storage area of ​​the corresponding channel is empty and there is still a read request, an error signal is output. If there is a readable instruction in the storage area of ​​the corresponding channel, a read enable signal (denoted as rd_en) is returned. The polling scheduling module can select the channel to read the instruction based on the read enable signal.

[0156] Each time the instruction read control module successfully reads one instruction from the instruction virtual channel storage unit, it can control the valid number counter to increment by 1. For example, each time the instruction read control module successfully reads an instruction, it can generate a valid number increment signal (denoted as credit_add), and then synchronize this credit_add signal to the valid number counter through a synchronizer to increment the value of the valid number counter by 1. Figure 23 In the example, by dynamically adjusting the value of the valid number counter, the available space size of the storage area in the instruction virtual channel storage unit can be reflected in real time, which reduces the error rate of instruction writing and improves the performance of the NPU system.

[0157] DMA also includes an AXI interface control module, and the polling scheduling module receives feedback from the AXI interface control module's subsequent modules (such as in...). Figure 23 The example uses GLB and AXI handshake signals to control the channel status and switching timing. For example, if data transfer is in progress on channel 1, indicating that channel 1 is not idle, no handshake signal (e.g., a valid AXI request) will be generated on channel 1. If data transfer has been completed on channel 1 or channel 1 is idle, a handshake signal may be generated. Similarly, if the AXI interface control module is currently processing data (described in detail below), no handshake signal (e.g., a valid AXI request) will be generated. If the AXI interface control module is idle, a handshake signal may be generated.

[0158] The instruction read control module can read instructions from the instruction virtual channel storage unit to the AXI interface control module. After receiving the instruction read from the instruction virtual channel storage unit, the AXI interface control module performs fourth-level decoding on the instruction (e.g., corresponding to the first decoding unit in this disclosure), which is used to connect the instruction to the subsequent modules of the AXI interface control module (e.g., in...). Figure 23 In this example, the data content required by the GLB is extracted and transformed based on the extracted data content, simultaneously generating an AXI request corresponding to the extracted data content. In this example, after receiving instruction 1, the AXI interface control module decodes the fields indicating offset and length information in instruction 1 to obtain the address information of the data to be transferred. Then, the AXI interface control module performs operations such as burst length control and 4K address cross-checking based on the address information of the data to be transferred, while simultaneously generating the corresponding AXI request. At this point, the DMA has completed reading instruction 1 and converting it into an AXI request that the NPU system can recognize.

[0159] DMA can transmit AXI requests to the on-chip network (BTC). The BTC can then read the data to be moved from SRAM or DRAM based on the AXI request and perform the data transfer. The data stream for moving data from SRAM or DRAM to the corresponding channel's GLB is as follows: Figure 23 The γ dashed line is shown in the diagram.

[0160] The DMA also includes an interrupt control module located in the APB clock domain. After all data and / or instruction transfers are complete, the AXI interface control module can generate a transfer completion signal (denoted as trans_done) to indicate task completion. The interrupt control module generates and outputs an interrupt signal based on the received transfer completion signal. The DMA also includes a performance monitor located in the APB clock domain for testing DMA performance.

[0161] In this disclosure, Figure 23 The DMA shown in the figure demonstrates a process for writing and reading the second type of instructions in this disclosure, which is similar to the process for writing and reading instructions 1. The difference is that after the second type of instructions are read from the instruction virtual channel storage unit, they are directly transmitted to the subsequent module of the DMA (e.g., GLB) without converting the second type of instructions into AXI requests, thereby realizing the transmission of the information (e.g., control information or status information described above in this disclosure) carried by the second type of instructions.

[0162] This application uses specific terms to describe embodiments of the application. Terms such as "first / second embodiment," "an embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of the application. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics in one or more embodiments of the application can be appropriately combined.

[0163] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several patentable types or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Accordingly, aspects of this application can be implemented entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. All of the above hardware or software may be referred to as a “data block,” “module,” “engine,” “unit,” “component,” or “system.” Furthermore, aspects of this application may manifest as a computer product located on one or more computer-readable media, the product including computer-readable program code.

[0164] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in a common dictionary shall be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or highly formalized meaning, unless expressly defined herein.

[0165] The foregoing description is illustrative of the invention and should not be construed as limiting it. Although several exemplary embodiments of the invention have been described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and advantages of the invention. Therefore, all such modifications are intended to be included within the scope of the invention as defined in the claims. It should be understood that the foregoing description is illustrative of the invention and should not be construed as limiting it to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The invention is defined by the claims and their equivalents.

Claims

1. A method for transporting data, used in a neural network processor, the neural network processor comprising at least one array of processing units, The method includes: Receive a first instruction, wherein the first instruction indicates the address information of the target data to be transferred, the address information of the target data is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array, the mapping relationship is the mapping relationship in the row-fixed data stream implemented by the processing unit array, the target data has the mapping relationship, the target data is the data after the coprocessor has cut the data of the input feature map or filter, and the first instruction is the instruction output by the coprocessor; Generate a data transfer request based on the address information of the target data; and According to the data transfer request, the target data is transferred between the memory and the processing unit array in the neural network processor; The first instruction further includes a channel identifier, which indicates a target channel among multiple channels in the neural network processor. The method for transferring data also includes: Decode the first instruction to obtain the channel identifier; and The first instruction is written to the target storage area according to the channel identifier, wherein the target storage area corresponds to the target channel.

2. The method according to claim 1, further comprising, before generating the data transfer request based on the address information of the target data: The first instruction is decoded to obtain the address information of the target data.

3. The method according to claim 1, further comprising: Based on the handshake signal sent by the neural network processor, the first instruction is read from the target storage area to generate the data transfer request.

4. The method according to any one of claims 1-3, wherein, The method further includes receiving the first instruction and responding to the data transfer request by the target direct memory access device, and the method further includes: Receive an initial instruction, decode the initial instruction to obtain a first instruction and a DMA identifier of the target direct memory access device for executing the first instruction; and The first instruction is provided to the target direct memory access device according to the DMA identifier.

5. The method of claim 4, wherein the initial instruction further indicates an instruction type, and the method further comprises: The initial instruction is decoded to obtain the instruction type in the process of obtaining the first instruction; as well as In response to the instruction type being a preset instruction type, the first instruction is written to the target storage area.

6. The method according to claim 5, further comprising: In response to writing or reading the first instruction, the number of valid data points is controlled to decrease or increase, wherein the number of valid data points indicates the size of the target storage area used to store the instruction.

7. The method according to claim 3, The step of reading the first instruction from the target storage area to generate the data transfer request based on the handshake signal sent by the neural network processor includes: The multiple channels are polled and scheduled to select the target channel; as well as Based on the handshake signal sent by the target channel, the first instruction is read from the target storage area to generate the data transfer request.

8. The method of claim 3, wherein before reading the first instruction from the target storage area, the method further comprises: Determine whether the target storage area is empty; as well as In response to the target storage area being non-empty, the first instruction is read from the target storage area.

9. The method according to claim 1, further comprising, before writing the first instruction to the target storage area: Determine whether the target storage area is full; as well as In response to the target storage area being non-full, the first instruction is written to the target storage area.

10. A direct memory access device for a neural network processor, wherein, The neural network processor includes at least one array of processing units, and the device includes: A receiving unit is configured to receive a first instruction, wherein the first instruction indicates the address information of target data to be transferred, the address information of the target data is obtained based on the mapping relationship between the target data and at least one processing unit in the processing unit array, the mapping relationship is the mapping relationship in the row-fixed data stream implemented by the processing unit array, the target data has the mapping relationship, the target data is the data after the coprocessor has segmented the data of the input feature map or filter, and the first instruction is the instruction output by the coprocessor. An interface control unit is configured to generate a data transfer request based on the address information of the target data, and to transfer the target data between the memory and the processing unit array in the neural network processor according to the data transfer request. The first instruction further includes a channel identifier, which indicates a target channel among multiple channels in the neural network processor. The device further includes: The second decoding unit is configured to decode the first instruction to obtain the channel identifier; Instruction virtual channel storage unit; and The instruction write control unit is configured to write the first instruction into the target storage area of ​​the instruction virtual channel storage unit according to the channel identifier, wherein the target storage area corresponds to the target channel.

11. The apparatus of claim 10, wherein the interface control unit further comprises: The first decoding unit is configured to decode the first instruction to obtain the address information of the target data.

12. The apparatus of claim 11, further comprising: An instruction read control unit is configured to read the first instruction from the target storage area based on a handshake signal sent by the neural network processor to generate the data transfer request.

13. The apparatus according to any one of claims 10-12, wherein the first instruction is received and the data transfer request is responded to by a target direct memory access device, the target direct memory access device being coupled to an interface decoder. The interface decoder includes: The third decoding unit is configured to receive an initial instruction, decode the initial instruction to obtain the first instruction and the DMA identifier of the target direct memory access device for executing the first instruction. as well as The first instruction is provided to the target direct memory access device according to the DMA identifier.

14. The apparatus according to claim 13, wherein, The initial instruction also indicates the instruction type, and the interface decoder further includes: The fourth decoding unit is configured to decode the initial instruction to obtain the instruction type in the process of obtaining the first instruction, and to write the first instruction into the target storage area in response to the instruction type being a preset instruction type.

15. The apparatus of claim 14, further comprising: A valid number recording unit is configured to control the decrease or increase of a valid number in response to writing or reading the first instruction, wherein the valid number indicates the space size of the target storage area for storing instructions.

16. The apparatus according to claim 12, wherein, The instruction read control unit also includes: A polling scheduling unit configured to perform polling scheduling on the plurality of channels to select the target channel; and The instruction read control module is configured to read the first instruction from the target storage area based on the handshake signal sent by the target channel to generate the data transfer request.

17. The apparatus of claim 12, wherein the instruction read control unit is further configured to: Before reading the first instruction from the target storage area, determine whether the target storage area is empty; and In response to the target storage area being non-empty, the first instruction is read from the target storage area.

18. The apparatus of claim 10, wherein the instruction write control unit is further configured to: Before writing the first instruction to the target storage area, determine whether the target storage area is full; and In response to the target storage area being non-full, the first instruction is stored in the target storage area.

19. A computer system comprising: The direct memory access apparatus according to any one of claims 10 to 18; as well as The neural network processor is electrically coupled to the direct memory access device.

20. The computer system according to claim 19 further includes: An interface decoder is configured to receive an initial instruction, decode the initial instruction to obtain a first instruction, and provide the decoded first instruction to the direct memory access device.

21. The computer system according to claim 19, further comprising: An auxiliary processor, configured to provide initial instructions to the interface decoder.

22. The computer system according to claim 21, wherein, The auxiliary processor is coupled to the interface decoder via a first data transmission protocol, and the neural network processor is coupled to the direct memory access device via a second data transmission protocol, wherein the first data transmission protocol and the second data transmission protocol are different.

Images