Data rearrangement apparatus, method, medium, device, and neural network processor

Abstract

Disclosed are a data rearrangement apparatus, method, medium, device and neural network processor. An input rearrangement circuit converts an input data block arranged according to a first data arrangement mode into a first intermediate data block arranged according to a second data arrangement mode. A data write-in circuit writes valid data elements in the first intermediate data block into a plurality of storage banks. After a data read-out circuit reads the valid data elements from a plurality of storage units in the same storage row in the plurality of storage banks, an output rearrangement circuit performs a second data rearrangement on the read data to obtain an output data block arranged according to a third data arrangement mode. In the write-in stage, only valid data is written into the plurality of storage banks, and the original data corresponding to the hollow position can be shielded, so that the purpose of eliminating the hollow is achieved, thereby reducing the complexity of the data rearrangement task, improving the data rearrangement efficiency and reducing the hardware overhead.

Description

Technical Field

[0001] This disclosure relates to the field of integrated circuit technology, and in particular to a data rearrangement apparatus, method, medium, device, and neural network processor. Background Technology

[0002] Neural network inference computation in integrated circuits typically involves a large amount of data processing. During the data processing, the data to be processed can be divided into blocks according to a specific data layout. For example, the data elements in the data blocks can be rearranged to change the layout of the data blocks. When there are gaps in the data blocks, since there are no specific rules for handling gaps, it is difficult to handle gaps, resulting in low data rearrangement processing efficiency, which in turn affects the efficiency of neural network inference computation. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a data rearrangement apparatus, method, storage medium, electronic device, and neural network processor to solve the problem of low data processing efficiency caused by gaps in the data rearrangement process.

[0004] The first aspect of this disclosure provides a data rearrangement apparatus, including a storage array, an input rearrangement circuit, a data writing circuit, a data reading circuit, and an output rearrangement circuit. The storage array includes multiple storage banks, and each storage bank includes multiple storage cells. The input rearrangement circuit performs a first data rearrangement on multiple data elements in an input data block arranged according to a first data arrangement, to obtain a first intermediate data block arranged according to a second data arrangement. The data writing circuit writes multiple valid data elements from the first intermediate data block into the multiple storage banks, the valid data elements including data elements corresponding to non-hole positions in the input data block. The data reading circuit reads multiple valid data elements from multiple storage cells located in the same storage row in the multiple storage banks. The output rearrangement circuit performs a second data rearrangement on the read multiple valid data elements, to obtain an output data block arranged according to a third data arrangement.

[0005] A second aspect of this disclosure provides a data rearrangement method, comprising: performing a first data rearrangement on multiple data elements in an input data block arranged according to a first data arrangement, to obtain a first intermediate data block arranged according to a second data arrangement; writing multiple valid data elements in the first intermediate data block into multiple storage banks; reading multiple valid data elements from multiple storage cells located in the same storage row in the multiple storage banks; and performing a second data rearrangement on the read multiple valid data elements to obtain an output data block arranged according to a third data arrangement.

[0006] A third aspect of this disclosure provides a neural network processor, including a data rearrangement device and a memory; wherein the data rearrangement device includes a data rearrangement device for implementing the first aspect of this disclosure.

[0007] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program for implementing the data rearrangement method of the second aspect of this disclosure.

[0008] A fifth aspect of this disclosure provides an electronic device comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the data rearrangement method of the second aspect of this disclosure.

[0009] The data rearrangement apparatus provided in this disclosure converts an input data block arranged according to a first data arrangement method into a first intermediate data block arranged according to a second data arrangement method through an input rearrangement circuit. A data writing circuit writes valid data elements from the first intermediate data block into multiple memory banks. A data reading circuit reads valid data elements from multiple memory cells located in the same memory row from the multiple memory banks. An output rearrangement circuit then performs a second data rearrangement on the read data to obtain an output data block arranged according to a third data arrangement method. The data rearrangement apparatus provided in this disclosure can efficiently rearrange data blocks directly through hardware circuitry. Furthermore, during the data writing stage, the data writing circuit only writes valid data into multiple memory banks, effectively masking the original data corresponding to gap locations and thus eliminating gaps. Therefore, using the data rearrangement apparatus of this disclosure eliminates the need for migration and filling operations on data elements at gap locations during data processing, reducing the complexity of the data rearrangement task, improving data rearrangement efficiency, and lowering hardware overhead. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of a data rearrangement device provided in this disclosure.

[0011] Figure 2 This is a schematic diagram of the arrangement of data blocks to be spliced, provided in this publication.

[0012] Figure 3 This is a schematic diagram of the circuit structure of a data rearrangement device provided in this disclosure.

[0013] Figure 4 This is a flowchart illustrating a data rearrangement method provided in this disclosure.

[0014] Figure 5 This is a flowchart illustrating a first data rearrangement method provided in this disclosure.

[0015] Figure 6 This is a flowchart illustrating a target data element selection method provided in this disclosure.

[0016] Figure 7 This is a flowchart illustrating another first data rearrangement method provided in this disclosure.

[0017] Figure 8 This is a flowchart illustrating a method for determining a first spatial location mapping relationship provided in this disclosure.

[0018] Figure 9 This is a flowchart illustrating a method for writing target data elements provided in this disclosure.

[0019] Figure 10 This is a flowchart illustrating another method for writing target data elements provided in this disclosure.

[0020] Figure 11 This is a flowchart illustrating another method for writing target data elements provided in this disclosure.

[0021] Figure 12 This is a flowchart illustrating a method for writing data elements into storage, as provided in this disclosure.

[0022] Figure 13 This is a flowchart illustrating a method for reading data elements provided in this disclosure.

[0023] Figure 14 This is a flowchart illustrating another method for reading data elements provided in this disclosure.

[0024] Figure 15 This is a flowchart illustrating a second data rearrangement method provided in this disclosure.

[0025] Figure 16 This is a flowchart illustrating a method for outputting target valid data elements provided in this disclosure.

[0026] Figure 17 This is a flowchart illustrating a second data rearrangement method provided in this disclosure.

[0027] Figure 18 This is a flowchart illustrating a method for determining a second spatial location mapping relationship provided in this disclosure.

[0028] Figure 19 This is a flowchart illustrating a method for determining an access address provided in this disclosure.

[0029] Figure 20 This is a schematic diagram of the structure of an electronic device provided in this disclosure.

[0030] Figure 21This is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present disclosure. Detailed Implementation

[0031] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0032] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0033] Application Overview With the rapid development of neural network technology, neural network models have been widely used in fields such as image recognition and natural language processing. The computation of neural network models usually involves a large amount of data operations, such as tensor computation, matrix multiplication, and attention mechanism operations. In order to make full use of the storage resources on integrated circuits and improve the computing throughput, the data to be processed is usually divided into blocks according to a specific data layout. This divides the data to be processed into multiple data blocks, and writes multiple data blocks into the storage circuit in sequence. The data layout is used to indicate the arrangement order of each dimension within the data block and the block size (or block size) of each dimension.

[0034] When performing neural network inference operations, the storage circuit typically outputs the data to be processed to the computing circuit according to a fixed read / write bandwidth. Taking the storage circuit including Static Random-Access Memory (SRAM) as an example, if the SRAM read / write bandwidth is 64B and the on-chip bus bandwidth is also 64B, in each data rearrangement cycle, the SRAM can provide a 64B input data block to the data rearrangement device through the on-chip bus. If the size of the data to be processed is not aligned with the data arrangement, there will be empty positions in the input data block obtained by the data rearrangement device. The empty positions are invalid data and will occupy the positions of data elements in the input data block. When rearranging the input data block, the invalid data in the empty positions and the valid data are generally treated as the same type of data and moved, which increases the complexity of position mapping, cross-row movement and write control, affects the data rearrangement efficiency, and thus affects the efficiency of neural network inference calculation.

[0035] Typically, to prevent holes from impacting data rearrangement efficiency, some data elements from the subsequent data to be processed can be written into holes in the previous data, and the arrangement of other data elements in the previous data can be adjusted to eliminate the holes. Since holes may be distributed across different data element positions in the data to be processed, valid data from the subsequent data cannot be written continuously according to the original data arrangement. Instead, it is necessary to write into holes, skip invalid positions, and in some cases, migrate data across storage rows or data blocks. Therefore, eliminating holes in the data to be processed requires complex operations such as position mapping, hole detection, and write address control, resulting in low data rearrangement efficiency.

[0036] To address the issue of low data rearrangement efficiency caused by gaps, this disclosure provides a data rearrangement apparatus. An input rearrangement circuit converts an input data block arranged according to a first data arrangement method into a first intermediate data block arranged according to a second data arrangement method. A data writing circuit writes only valid data elements from the first intermediate data block to multiple storage banks, preventing invalid data at gap locations in the first intermediate data block from being written as valid data to the storage array. A data reading circuit reads valid data elements from multiple storage cells located in the same storage row from multiple storage banks. An output rearrangement circuit rearranges the read valid data elements to obtain an output data block. The data rearrangement apparatus provided by this disclosure can efficiently rearrange data blocks directly through hardware circuitry. Furthermore, by writing only valid data elements to the storage array during the data writing stage, it can shield invalid data corresponding to gap locations in the input data block, thereby eliminating gaps in the input data block, reducing the complexity of the data rearrangement task, and improving data rearrangement efficiency.

[0037] Exemplary device Figure 1This is a schematic diagram of a data rearrangement device provided in this disclosure. The data rearrangement device 100 of this disclosure can be a hardware circuit in a neural network processor, a brain processing unit (BPU), a graphics processing unit (GPU), or other chips with tensor processing capabilities. It can also be an integrated circuit in a neural network processor, brain processing unit (BPU), graphics processing unit (GPU), or other integrated circuits with neural network processing capabilities. To fully explain this disclosure, the data rearrangement device 100 is exemplified as a hardware circuit disposed in a neural network processor. The data rearrangement device 100 of this disclosure can be coupled to one or more devices in the neural network processor, such as memory, on-chip cache, and direct memory access controller (DMA). Figure 1 As shown, the data rearrangement device 100 includes a storage array 110, an input rearrangement circuit 120, a data writing circuit 130, a data reading circuit 140, and an output rearrangement circuit 150.

[0038] like Figure 1 As shown, exemplarily, the input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 can operate in a pipelined manner. In one data rearrangement cycle, the input rearrangement circuit 120 can input a new input data block; the data writing circuit 130 can write the first intermediate data block obtained in the previous data rearrangement cycle into the storage array 110; the data reading circuit 140 can read the target data element from the already written storage cell and generate an intermediate data block; the output rearrangement circuit 150 can perform a second data rearrangement on the intermediate data block read in the previous data rearrangement cycle.

[0039] It is important to note that the actual operating cycles of the input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 are not exactly the same as the data rearrangement cycle. For example, in the first data rearrangement cycle, the input rearrangement circuit 120 reads the first input data block, but the data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 do not operate during this data rearrangement cycle. Therefore, the first data rearrangement cycle is the first actual operating cycle of the input rearrangement circuit 120 (i.e., the first data input cycle corresponding to the input rearrangement circuit 120). Similarly, the second data rearrangement cycle is the second actual operating cycle of the input rearrangement circuit (i.e., the second data input cycle corresponding to the input rearrangement circuit 120), the first actual operating cycle of the data writing circuit 130 (i.e., the first data writing cycle corresponding to the data writing circuit 130), and the first actual operating cycle of the data reading circuit 140 (i.e., the first data reading cycle corresponding to the data reading circuit 140). The third data rearrangement cycle is the third actual working cycle of the input rearrangement circuit (i.e., the third data input cycle corresponding to the input rearrangement circuit 120), the second actual working cycle of the data writing circuit 130 (i.e., the second data writing cycle corresponding to the data writing circuit 130), the second actual working cycle of the data reading circuit 140 (i.e., the second data reading cycle corresponding to the data reading circuit 140), and the first actual working cycle of the output rearrangement circuit 150 (i.e., the first data rearrangement cycle corresponding to the output rearrangement circuit 150).

[0040] The storage array 110 includes multiple storage banks, each containing multiple storage cells. The number of storage cells in a storage bank is referred to as the depth of the storage bank. The multiple storage banks can operate in parallel, and each storage bank is used to store data elements written by the data writing circuit 130 during the data rearrangement cycle or data input cycle of the data rearrangement device 100.

[0041] For example, the number of data elements in the input data block can be associated with the number of storage units. Taking the storage array 110 as an example that includes 64 storage units, the input data block can be configured as 64B, and each data element is 1B. The positions of the 64 data elements in the input data block correspond to the 64 storage units respectively, and the 64 data elements can be written into the 64 storage units respectively.

[0042] The depth of the memory bank can be designed based on hardware area and throughput requirements. In some embodiments, each memory bank includes 64 memory cells to improve pipeline throughput; in other embodiments, the depth of each memory bank can be reduced to 2 memory cells or other numbers of memory cells to reduce hardware area overhead. Memory cells may include registers, latches, SRAM cells, or other circuitry capable of storing data elements.

[0043] In practical applications, the depth of the memory bank can be configured according to the needs of the neural network processor. If the neural network processor has high throughput requirements, a deeper memory bank can be configured so that the memory array can store more intermediate data rows. If the neural network processor is a low-power chip that is sensitive to hardware area, a shallower memory bank can be configured to reduce the number of registers and wiring resources.

[0044] For example, the number of memory banks in storage array 110 may not be 64. The input data block size, data element bit width, or bus width corresponds to the number of memory banks. For example, if storage array 110 may include 32 memory banks, the input data block may include 32 data elements.

[0045] The input rearrangement circuit 120 is configured to perform a first data rearrangement on multiple data elements in the input data block arranged according to a first data arrangement method to obtain a first intermediate data block arranged according to a second data arrangement method.

[0046] For example, the input rearrangement circuit 120 is coupled to a storage circuit and a controller to receive input data blocks stored in the storage circuit under the control of the controller. The output of the input rearrangement circuit 120 is coupled to the input of the data writing circuit 130 to output a first intermediate data block to the data writing circuit 130.

[0047] The data rearrangement device 100 can perform related operations on various types of function operators, including but not limited to the concat operator, flip operator, or tile operator. This embodiment does not limit the specific type of function operator. The input data block can be a data block in the data to be concatenated, and the data to be concatenated is the input data corresponding to the function operator. In order to clearly illustrate this disclosure, the following embodiment takes the concat data concatenation operator as an example for illustrative explanation.

[0048] Figure 2 This is a schematic diagram illustrating the arrangement of data blocks to be spliced, provided in an embodiment of this disclosure. For example... Figure 2As shown, the data to be spliced ​​(i.e., the data to be processed) includes a first data to be spliced ​​21 and a second data to be spliced ​​22. The tensor size of both the first data to be spliced ​​21 and the second data to be spliced ​​22 is 4H3W15C, and the first data arrangement is 2H4W8C. Therefore, the first data to be spliced ​​21, arranged in 2H4W8C, may include a first input data block 211, a second input data block 212, a third input data block 213, and a fourth input data block 214. The second data to be spliced ​​22, arranged in 2H4W8C, may include a fifth input data block 221, a sixth input data block 222, a seventh input data block 223, and an eighth input data block 224. Since the tensor size of both the first data to be spliced ​​21 and the second data to be spliced ​​22 is 4H3W15C, this tensor size is not aligned with the first data arrangement. Therefore, when the first data to be spliced ​​21 and the second data to be spliced ​​22 are arranged according to 2H4W8C, each corresponding input data block will include multiple valid data elements and holes. For example, since the actual size of the data to be spliced ​​in the W dimension is 3, while the size of the W dimension in the first data arrangement is 4, each input data block of the data to be spliced ​​will generate corresponding holes in the W dimension. Figure 2 For example, the first input data block 211 includes H0W0C0-7, H0W1C0-7, H0W2C0-7, blank 8B, H1W0C0-7, H1W1C0-7, H1W2C0-7, and blank 8B. Similarly, the second input data block 212 includes H0W0C8-14 and blank 1B, H0W1C8-14, blank 1B, H0W2C8-14, blank 1B, and blank 8B.

[0049] If it is necessary to remove the holes in the C dimension of the input data block, the input rearrangement circuit 120 can rearrange the element positions of the data elements in each input data block so that the data elements in the C dimension of the rearranged data block are located on the outermost side, thereby obtaining the first intermediate data block arranged according to the second data arrangement method.

[0050] For example, the second data arrangement can be determined based on the first data arrangement of the input data block and the processing requirements of the function operator. Furthermore, the second data arrangement can be determined according to the required concatenation dimension of the function operator. For example, the required concatenation dimension of the concat operator corresponds to the outermost dimension of the second data arrangement. That is, by arranging the required concatenation dimension of the function operator on the outermost dimension, it is convenient to write the concatenated data.

[0051] Continuing with the example of the first data arrangement of the input data block being 2H4W8C and the function operator being the concat operator, if the concat operator needs to concatenate along dimension C, then dimension C can be arranged in the outermost dimension, hence the second data arrangement is 8C2H4W. If the concat operator needs to concatenate along dimension W, then dimension W can be arranged in the outermost dimension, hence the second data arrangement is 4W2H8C. Thus, the input rearrangement circuit 120, by converting the first data arrangement to the second data arrangement, places the data elements corresponding to the required concatenation dimensions of the function operator in the outermost position within the first intermediate data block for easy continuation of writing.

[0052] For example, the input rearrangement circuit 120 can rearrange the valid data elements H0W0C8-14, H0W1C8-14, H0W2C8-14, etc. in the second input data block 212, so that the data elements in the C dimension are arranged in the outermost dimension, thereby improving the hole processing efficiency. For example, taking the first data arrangement as 2H4W8C as an example, in order to arrange the data in the C dimension in the outermost dimension, the input rearrangement circuit 120 can convert the first data arrangement 2H4W8C into the second data arrangement 8C2H4W, or convert it into the second data arrangement 8C4W2H. In this embodiment of the disclosure, the type of the second data arrangement is not limited. Optionally, the difference between the second data arrangement and the first data arrangement is whether the data elements in the C channel are arranged in the outermost dimension.

[0053] The data writing circuit 130 is configured to write multiple valid data elements from the first intermediate data block into multiple memory banks. The valid data elements include data elements corresponding to non-hole positions in the input data block.

[0054] The output of the data writing circuit 130 is coupled to the storage array 110 to write valid data elements to the target storage cell in the corresponding storage bank. The data writing circuit 130 can use a write mask signal to block the write path corresponding to the hole position, thereby writing only valid data elements to the storage array 110 and blocking the data elements corresponding to the hole position during the data write operation. Therefore, the data writing circuit 130 will not write the data elements corresponding to the hole position to the storage array 110 during the data write operation.

[0055] For example, the data writing circuit 130 can write the valid data elements H0W0C0-7, H0W1C0-7, H0W2C0-7, H1W0C0-7, H1W1C0-7, and H1W2C0-7 from the first input data block 211 into the storage array 110, but not the invalid data elements corresponding to the 8B hole positions in the first input data block 211. In a scenario where the function operator indicates that the data to be spliced ​​is to be spliced ​​in the C dimension, the valid data elements in the second data to be spliced ​​22 can be written into the storage array 110 following the valid data elements in the first data to be spliced ​​21, and the data elements corresponding to the holes in the first data to be spliced ​​21 are not written into the storage array 110, thereby eliminating the influence of the holes.

[0056] Data read circuit 140 is configured to read multiple valid data elements from multiple memory cells located in the same memory row in multiple memory banks. Data read circuit 140 is coupled to memory array 110, and can read multiple valid data elements from multiple memory cells located in the same memory row in multiple memory banks after data write circuit 130 writes valid data elements to multiple memory banks.

[0057] For example, the same storage row refers to a group of storage cells with the same row address in multiple storage banks. For instance, the data read circuit 140 can read 64B of data from the 0th row of Bank0 to Bank63 based on the 0th row address, or it can read 64B of data from the 1st row of Bank0 to Bank63 based on the 1st row address. Since each storage bank stores 1B of data element in one storage cell, the data read circuit 140 can read 64B of data from the same storage row of 64 storage banks.

[0058] For example, a storage row of the storage array 110 can store multiple valid data elements from the first data to be concatenated 21 and the second data to be concatenated 22. For instance, the first storage row in the storage array 110 may include the valid data element corresponding to H3W2C8-14 in the first data to be concatenated 21, and the valid data elements in the second data to be concatenated 22. When the data reading circuit 140 reads data elements from the same storage row, it reads the set of valid data elements concatenated based on that storage row after removing the effects of voids.

[0059] The output rearrangement circuit 150 is configured to perform a second data rearrangement on multiple valid data elements read, resulting in an output data block arranged according to a third data arrangement. The output rearrangement circuit 150 is coupled to the output terminal of the data reading circuit 140, and can output the output data block to the storage circuit.

[0060] The first, second, and third data arrangement methods are used to represent different data arrangement methods for data blocks. The first data arrangement method can represent the data arrangement method corresponding to the input data block when it is read from the storage circuit. The second data arrangement method can represent the data arrangement method corresponding to the first intermediate data block, which facilitates the writing of valid data elements into the storage array. The third data arrangement method can represent the target data arrangement method that the output data block needs to satisfy. The second and third data arrangement methods can be the same or different; the first and third data arrangement methods can also be the same or different, and this disclosure does not limit this. In a specific example, the first data arrangement method is 2H4W8C, the second data arrangement method is 8C2H4W, and the third data arrangement method is 2H4W8C.

[0061] For example, the input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 can operate in a pipelined manner. In one data rearrangement cycle, the input rearrangement circuit 120 can process a new input data block; the data writing circuit 130 can write the first intermediate data block obtained in the previous data rearrangement cycle into the storage array 110; the data reading circuit 140 can read the second intermediate data block from the target storage row that has been written; and the output rearrangement circuit 150 can perform a second data rearrangement on the second intermediate data block read in the previous data rearrangement cycle.

[0062] For example, in a data rearrangement cycle, the data read circuit 140 can read multiple data elements from multiple storage rows to obtain a second intermediate data block. For instance, the data read circuit 140 can read multiple valid data elements from a target storage cell that has already been written according to a predetermined data read rule, and determine a second intermediate data block according to the data arrangement corresponding to the data read rule, wherein there are no gaps in the second intermediate data block.

[0063] In some optional embodiments, the data rearrangement device 100 further includes a controller. The controller receives configuration information from the compiler or upper-level scheduling logic, and generates a select signal and a write mask signal based on the configuration information to control the input rearrangement circuit 120, data write circuit 130, data read circuit 140, and output rearrangement circuit 150 to perform their respective functions. The controller can be connected to… Figure 1The input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 are coupled to each other. The controller changes the data path for each data rearrangement cycle by using selection signals and mask signals, thereby enabling the operation of different function operators (such as concat, flip, or tile operators) without changing the physical hardware connection of the data rearrangement device 100.

[0064] Figure 3 This is a structural diagram of another data rearrangement apparatus provided in an embodiment of this disclosure. Figure 3 As shown, the data rearrangement device includes a storage array 110, an input rearrangement circuit 120, a data writing circuit 130, a data reading circuit 140, and an output rearrangement circuit 150. The storage array 110 includes multiple storage cells, each storage cell having multiple storage units, and each storage unit storing data elements.

[0065] In some embodiments, the input rearrangement circuit 120 includes a plurality of first multiplexers, each configured to correspond to a plurality of memory banks. Each first multiplexer is configured to select a first target data element from a plurality of data elements in an input data block and output the first target data element; wherein the element position of the first target data element in a first intermediate data block corresponds to the arrangement position of the first multiplexer in the input rearrangement circuit.

[0066] For example, the number of first multiplexers included in the input rearrangement circuit 120 is the same as the number of memory banks included in the memory array 110, and the data elements output by each first multiplexer can be written into the corresponding memory bank.

[0067] like Figure 3 As shown, taking a storage array comprising 64 memory banks as an example, the input rearrangement circuit 120 (i.e., MUX0) may include 64 first multiplexers, such as first multiplexer 331, first multiplexer 332, and first multiplexer 333, etc. The set of data elements output by these 64 first multiplexers can constitute the first intermediate data block 31.

[0068] For example, if the data width of the input data block is 64B, the MUX0 may include 64 first multiplexers, each of which is a 64-to-1 selector. The first multiplexer is used to select 1B of data elements from the 64B input data block for output, and the output 1B data element will be subsequently written into the memory corresponding to the first multiplexer.

[0069] The first multiplexer selects a first target data element from multiple data elements in the input data block and outputs the first target data element. The position of the first target data element in the first intermediate data block 31 corresponds to the arrangement position of the first multiplexer in the input rearrangement circuit 120.

[0070] For example, the first multiplexer 331 can correspond to the first element position in the first intermediate data block 31, and the first multiplexer 332 can correspond to the second element position in the first intermediate data block 31. Different first multiplexers can select data elements corresponding to different element positions from the input data block, thereby completing the first data rearrangement.

[0071] In some embodiments, the first multiplexer is configured to determine the first element position of the first target data element in the input data block based on a first selection signal, and then output the first target data element based on the first element position.

[0072] The first element position can be one of the element positions in the sequence of positions 0 to 63 in the input data block. The first selection signal can be a select signal, which instructs the first multiplexer to select and output the data element from a certain element position in the input data block. For the 64 first multiplexers in the input rearrangement circuit 120, the 64 first multiplexers will receive 64 first selection signals from the controller within the same data rearrangement cycle. These 64 first selection signals are used to instruct the corresponding first selection signal to select a first element position in the input data block and output the target data element corresponding to that first element position. It should be noted that the first element positions selected by the 64 first selection signals are all different, thereby realizing the conversion of the input data block of the first data arrangement method into the first intermediate data block of the second data arrangement method.

[0073] In one example, assuming the input data block is arranged in 2H4W8C, and the input rearrangement circuit 120 needs to convert the input data block into a first intermediate data block 31 arranged in the second data arrangement 8C2H4W, each first multiplexer can have its own position number, and the position number of each first multiplexer is consistent with the memory bank address of the memory bank corresponding to that first multiplexer. For example, if the position number of the first multiplexer is 21, the address (bank id) of the memory bank corresponding to that first multiplexer is 21. If the position number is 6 bits, the selection value can be determined by rearranging the bit order of these 6 bits. For example, when converting the arrangement of the input data block from the first data arrangement 2H4W8C to the second data arrangement 8C2H4W, after conversion, bit 0 is equivalent to the original bit 3, after conversion, bit 1 is equivalent to the original bit 4, after conversion, bit 2 is equivalent to the original bit 5, after conversion, bit 3 is equivalent to the original bit 0, after conversion, bit 4 is equivalent to the original bit 1, and after conversion, bit 5 is equivalent to the original bit 2. For the first multiplexer with position number 15, if its bank id is binary 001111, after rearranging the bit order as described above, it can be obtained as binary 111001, which corresponds to the 57th element position in the input data block.

[0074] For example, the input rearrangement circuit 120 can support different data element bit widths. If a data element is 1B, a 64B input data block includes 64 element positions. If a data element is 2B, a 64B input data block includes 32 element positions. The input rearrangement circuit 120 can select according to the 32 element positions, or split a data element into multiple byte paths for synchronous selection. The specific implementation of the input rearrangement circuit 120 can be determined based on one or more of the on-chip bus bit width, data type, and subsequent memory organization method.

[0075] In some embodiments, the data rearrangement apparatus 100 further includes a controller. The controller is configured to: determine a first data arrangement method, the size of the data to be spliced, and a second data arrangement method; determine a first spatial position mapping relationship between the first data arrangement method and the second data arrangement method, and the shift number corresponding to the plurality of data elements, based on the first data arrangement method, the size of the data to be spliced, and the second data arrangement method; and control the input rearrangement circuit 120 to perform a first data rearrangement on the plurality of data elements in the input data block based on the first spatial position mapping relationship and the shift number.

[0076] The controller can obtain configuration parameters such as the first data layout, the size of the data to be concatenated, and the second data layout by parsing the compiler instructions corresponding to the function operators.

[0077] The first spatial location mapping relationship represents the correspondence between the element positions in the input data block and the element positions in the first intermediate data block 31 (or, the memory cells in the storage array). This first spatial location mapping relationship indicates the correspondence between data element positions in the first data arrangement and the second data arrangement, i.e., it indicates the transformation relationship of element positions between different data arrangement methods. For example, the conversion from the first data arrangement 2H4W8C to the second data arrangement 8C2H4W can be represented by a mapping between the element position sequence 0 to 63 in the input data block and the element position sequence 0 to 63 in the first intermediate data block 31.

[0078] The first spatial location mapping relationship can be generated according to the dimensional order. Taking the first data arrangement corresponding to the input data block as 2H4W8C as an example, the position of the element in the input data block can be determined by the H-dimensional index, W-dimensional index, and C-dimensional index. If the second data arrangement is 8C2H4W, the controller can place the C-dimensional index of the element position in the input data block in a higher priority position or a position closer to the continuous writing direction, and then map the H-dimensional index and W-dimensional index to the subsequent positions of the C-dimensional index.

[0079] For hardware implementation, the controller can store the first spatial position mapping relationship by explicitly saving the three-dimensional coordinates of H, W, and C. For example, the controller can explicitly save the position of element H0W1C2 in the input data block as corresponding to storage bank 63, and the position of element H0W1C3 in the input data block as corresponding to storage bank 61.

[0080] The controller can determine the element position in the input data block corresponding to the memory address by performing bit splitting and bit rearrangement on the memory address according to predetermined rules. The controller can also perform bit splitting and bit rearrangement on the element position in the input data block according to predetermined rules to obtain the corresponding memory address. Bit splitting involves extracting multiple bit fields corresponding to different dimension indices from the binary element position index value.

[0081] For example, based on the size 4H3W15C of the data to be concatenated, the data includes three dimensions: H, W, and C. Based on the first data arrangement 2H4W8C, it can be determined that the input data block has 2 element positions in the H dimension, 4 element positions in the W dimension, and 8 element positions in the C dimension. Therefore, the H dimension of the input data block needs to be represented by 1 binary bit, the W dimension by 2 binary bits, and the C dimension by 3 binary bits. Taking element position 15 in the input data block as an example, its binary index value can be determined to be 001111. After performing bit splitting on the binary index value, it can be determined that the bit field corresponding to the H dimension is "0", the bit field corresponding to the W dimension is "01", and the bit field corresponding to the C dimension is "111".

[0082] Bit order reordering involves rearranging the order of multiple bit fields obtained from the splitting process according to predetermined rules to determine the memory bank corresponding to the element position. For example, the predetermined rule is to convert the first data layout 2H4W8C to the second data layout 8C2H4W, thereby swapping the bit fields in the H dimension with the bit fields in the C dimension. The resulting binary index value is 111010, so the address of the memory bank corresponding to the element position numbered 15 in the input data block is bank57, which corresponds to 111010. Thus, the controller can determine the first spatial location mapping relationship between the memory bank and the element position in the input data block based on the predetermined rules.

[0083] For example, the selection signal of the first multiplexer in the input rearrangement circuit 120 can be generated based on the element position in the input data block and the address (bank id) of the memory bank. Taking the input data block arranged according to the first data arrangement 2H4W8C and the first intermediate data block 31 arranged according to the second data arrangement 8C2H4W as an example, the controller can parse the 6-bit number of the memory bank address bank id and rearrange the bit order to form a new 6-bit selection value. For example, based on the first data arrangement 2H4W8C and the second data arrangement 8C2H4W, the first spatial position mapping relationship can be determined as follows: after conversion, bit 0 corresponds to the original bit 3, after conversion, bit 1 corresponds to the original bit 4, after conversion, bit 2 corresponds to the original bit 5, after conversion, bit 3 corresponds to the original bit 0, after conversion, bit 4 corresponds to the original bit 1, and after conversion, bit 5 corresponds to the original bit 2. Therefore, through this 6-bit mapping relationship, the arrangement of the input data block can be converted from the first data arrangement 2H4W8C to the second data arrangement 8C2H4W.

[0084] For example, for the first multiplexer numbered 15, its bank address (bank id) can be represented as binary 001111. Following the bit rearrangement rules described above, the controller can obtain binary 111001, corresponding to decimal 57. Therefore, the controller can generate a first selection signal to instruct the first multiplexer numbered 15 to select data from the 57th element position of the input data block. Other first multiplexers can also select data elements from different positions in the input data block in the same manner.

[0085] In some embodiments, besides storing a complete selection table for each multiplexer, the controller can also generate a selection signal in real time based on the bank ID and the mode field. The mode field indicates the current conversion type, such as conversion from a first data layout 2H4W8C to a second data layout 8C2H4W, from a second data layout 8C2H4W to a third data layout 2H4W8C, or other hardware-supported data layout conversions. For commonly used data layout conversions, the controller can use fixed bit splitting and bit rearrangement logic. For less frequently used or more complex data layout conversions, the controller can use small-capacity lookup table logic.

[0086] The controller can determine the shift number based on the first data arrangement method, the second data arrangement method, and the size of the data to be spliced. For a given set of data to be spliced, the controller can determine the location and number of holes in each input data block corresponding to the data to be spliced, based on the first data arrangement method and the size of the data. For example... Figure 2 As shown, if the size of the data to be spliced ​​is 4H3W15C and the first data arrangement is 2H4W8C, it can be determined that there are two 8-byte holes in the first input data block corresponding to the data to be spliced, and six 1-byte holes and two 8-byte holes in the second input data block corresponding to the data to be spliced. Therefore, the controller can determine the number of valid data elements in each input data block based on the first data arrangement and the size of the data to be spliced, and thus determine the number of storage units occupied by each input data block in the storage array after it is written to the storage array.

[0087] For an input data block, the controller can determine the number of valid data elements included in the previous input data block based on the position of the input data block in the data to be concatenated (e.g., the index of the input data block in a certain dimension), thereby determining the shift number corresponding to the input data block.

[0088] The controller can determine the first selection signal based on the first spatial location mapping relationship and the shift number. The controller can first determine the element position corresponding to each memory bank in the input data block based on the bit splitting and bit rearrangement results. Then, based on the shift number, it shifts the element position corresponding to each memory bank forward or backward, that is, performs a left or right shift operation on the data sequence output by multiple first multiplexers. The position of the data element output by the first multiplexer in the data sequence is determined by the number of the first multiplexer.

[0089] For example, if the first multiplexer 1 corresponds to the memory bank 3 and the shift bit is 56, the element position corresponding to each memory bank can be shifted forward or backward by 56 element positions. The data element output by the first multiplexer 1 corresponds to the memory bank 59 after the shift. In some embodiments, the number of data elements in the input data block may be greater than the number of memory banks in the storage array. Therefore, the element position obtained after shifting can be moduloed (i.e., the element position obtained after shifting % the number of memory banks) to determine the correspondence between the element position of each data element and the memory bank. In this way, when the second data to be concatenated 22 needs to be written to the tail position of the first data to be concatenated 21, based on the first selection signal determined by the first spatial position mapping relationship and the shift bit, the data elements in the first intermediate data block 31 can be output to the corresponding memory bank, so as to write the data elements in the first intermediate data block 31 to the corresponding memory bank.

[0090] The aforementioned shift number represents the step size by which data elements in the function operator need to be cyclically moved before being written to storage array 110. The shift number is related to the cumulative size of the data to be concatenated in the concatenation dimension, the group size in the data arrangement, and the size of the data to be concatenated to which the current input data block belongs. For multiple data elements in the same data block included in the data to be concatenated, the shift number is the same. The controller can determine the starting offset (i.e., the shift number) of the data elements of the next input data block when writing to the storage array based on the number of data elements already written to the storage array in the current input data block. For example, if the current input data block includes 52 valid data elements, and these 52 valid data elements are written to the third storage row of the storage array, then the shift number of the data elements in the next input data block when writing to the third storage row of the storage array is 52, so that the data elements in the next input data block are written to the third storage row of the storage array immediately following the current input data block.

[0091] For example, when writing data elements from the first input data block of the first data to be concatenated 21 into the storage array 110, the shift bit can be 0. Due to the presence of gaps in the data, if the data elements in the first input data block do not fill a storage row of the storage array 110, when writing data elements from the second input data block of the first data to be concatenated 21 into the storage array 110, it is necessary to continue writing data elements from the end of the first input data block.

[0092] In some embodiments, the data elements in the input data block to be written can be cyclically shifted right based on the number of blank storage cells written in the storage row during the previous data rearrangement cycle. For example, if the first storage row of the storage array has 64 storage cells and the first input data block includes 56 valid data elements, then after the first input data block is written to the storage array, the first storage row will still have 8 blank storage cells remaining. Therefore, the data elements in the second input data block can be shifted by 8 bits to write the first 8 data elements output from the second input data block into the first storage row, and the subsequent 56 data elements output into the second storage row.

[0093] The controller can generate a first selection signal based on the shift bit and the first spatial position mapping relationship. For example, the controller can first determine the first spatial position mapping relationship according to the first data arrangement and the second data arrangement, and then cyclically move the mapped output position according to the shift bit. Alternatively, the controller can first calculate the shift bit for the element positions in the input data block, and then determine the position allocated to the first intermediate data block 31 according to the second data arrangement. Both implementation methods can determine the positional correspondence when the input data block is converted to the first intermediate data block 31.

[0094] The input rearrangement circuit performs a first data rearrangement on the input data block arranged according to the first data arrangement method to obtain a first intermediate data block arranged according to the second data arrangement method. As a result, multiple data elements in the input data block are first adjusted into an intermediate data arrangement that is conducive to writing to the memory array. The subsequent data writing circuit does not need to handle scattered gaps according to the first data arrangement method, thereby reducing the control complexity of the writing stage.

[0095] In some embodiments, the controller determines a first correspondence between the positions of multiple data elements in the input data block and the addresses of multiple memory banks based on a first data arrangement, the size of the data to be concatenated, and a second data arrangement. This first correspondence may include the relationship between the element positions in the input data block, the memory bank numbers, and the row addresses within the memory banks. Based on this first correspondence, the controller determines a first spatial location mapping relationship and a shift value.

[0096] For example, the controller can first determine the element position index value of each data element in the input data block according to the first data arrangement method, and then determine the corresponding element position of the data element in the first intermediate data block according to the second data arrangement method.

[0097] For example, when the first data layout is 2H4W8C and the second data layout is 8C2H4W, the element position index value in the input data block can be split into multiple bit fields according to the dimensions of H, W, and C. The controller then recombines these multiple bit fields according to the dimensions of C, H, and W to obtain the corresponding element position. The corresponding element position is used to represent the element position of the data element after the conversion from the first data layout to the second data layout.

[0098] The controller can determine the shift value based on the cumulative number of data elements in the concatenation dimension of the preceding input data blocks of the current input data block. The preceding input data blocks of the current input data block include all input data blocks in the data to be concatenated that have been written to the storage array. For example, if the current input data block is the third input data block in the first data to be concatenated that has been written to the storage array, the preceding data blocks include the first and second input data blocks in the first data to be concatenated that have been written to the storage array. If the current input data block is the first data block in the second data to be concatenated and the first data to be concatenated has been completely written to the storage array, the preceding data blocks include all input data blocks in the first data to be concatenated.

[0099] Taking C-dimensional stitching as an example, if the size of the first data to be stitched in the C-dimensional dimension is 15 (i.e., the first input data block has 15 data elements in the C-dimensional dimension) and the C-dimensional dimension is grouped according to 8C in the second data arrangement, then the position of the second input data block in the C-dimensional group when it starts stitching is 7, and the corresponding shift is 7 × 8 storage units. The controller adds this shift to the corresponding element position to obtain the shifted mapping index value, and determines the corresponding storage unit number by taking the modulo of the number of storage units based on the shifted mapping index value. When the shifted mapping index value exceeds the number of storage units corresponding to a storage row, the controller also determines the row address offset in the storage row based on the excess part, and determines the shift based on the row address offset. Thus, the first correspondence can represent the relationship between the element position in the input data block, the storage unit number, and the row address in the storage unit. Based on the first correspondence, the controller can determine the position transformation relationship corresponding to the position of the corresponding element as the first spatial position mapping relationship, and determine the offset caused by the number of cumulative data elements in the splicing dimension of the input data block as the shift number, thereby generating control signals for controlling the input rearrangement circuit and the data writing circuit.

[0100] The controller can first determine the position of the valid data element in the data to be concatenated, and then determine the storage bank number and row address of the valid data element in the storage array 110 based on the position. When the storage bank number corresponds to the element position in the first intermediate data block 31, the controller can reverse the process to determine the input position that the first multiplexer should select.

[0101] In some embodiments, the controller can receive configuration information from the compiler via the configuration bus. The configuration information may include the number of data to be concatenated, the size of the data to be concatenated, the data arrangement method of the data to be concatenated, a first data arrangement method, a second data arrangement method, the number of memory banks, and the concatenation dimension. The controller parses the configuration information and, in conjunction with the memory bank address, position mapping relationship, and concatenation offset, calculates to generate the selection signal, row address, and mask signal for the control input rearrangement circuit 120. The memory bank address may include the memory bank index, the location code of the memory bank in the memory array, or the memory bank number. The controller can also generate the selection signal, row address, and mask signal corresponding to the data writing circuit 130, the data reading circuit 140, and the output rearrangement circuit 150 based on the configuration information.

[0102] In some embodiments, the controller can determine the mapping relationship between the shift bits and the first spatial location in multiple ways. In one implementation, the controller can pre-generate the selection signal, mask signal, and row address required for each data rearrangement cycle based on configuration information during the task startup phase, and store the selection signal, mask signal, and row address in a local control storage circuit. During task execution, the controller reads the corresponding control items according to the data rearrangement cycle. In another implementation, the controller calculates the control signals (such as selection signals, row addresses, and mask signals) in real time in each data rearrangement cycle based on the position of the current input data block in the data to be rearranged, its size in the data to be processed, the position of the data element in the data to be concatenated, the storage address, and the size of the storage array.

[0103] In some embodiments, the controller can be located inside a BPU, GPU, or neural network processor and coupled to the input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150. Since the input rearrangement circuit 120, data writing circuit 130, data reading circuit 140, and output rearrangement circuit 150 need to perform corresponding functions based on multiple control signals sent by the controller in each data rearrangement cycle, the controller can be arranged close to these circuits to reduce long-distance wiring and transmission delays, and lower control signal distribution overhead.

[0104] The data writing circuit 130 includes a plurality of data distributors. Each data distributor is configured to correspond to a plurality of first multiplexers in the input rearrangement circuit 120 and is coupled to a plurality of memory cells in the memory bank. Each data distributor is configured to write a second target data element into a target memory cell in the memory bank corresponding to its corresponding data distributor in response to the output of the first multiplexer of the data distributor being a valid data element.

[0105] For example, the number of data distributors included in the data writing circuit 130 is the same as the number of first multiplexers included in the input rearrangement circuit 120, and also the same as the number of memory banks included in the memory array 110. Each data distributor corresponds to a different memory bank, and each data distributor is used to write data elements into its corresponding memory bank.

[0106] like Figure 3 As shown, taking a memory bank comprising 64 memory cells as an example, the data writing circuit 130 (such as DEMUX) includes 64 data distributors, such as data distributors 341, 342, and 343, etc. Each data distributor can correspond to one memory bank. For example, data distributor 341 can correspond to memory bank 111, data distributor 342 can correspond to memory bank 112, data distributor 343 can correspond to memory bank 113, and so on. Each data distributor receives the second target data element output from the first multiplexer and writes the second target data element into the target memory cell of the corresponding memory bank.

[0107] The second target data element is a valid data element output by any of the first multiplexers. In some embodiments, since the first target data element may be a valid data element or an invalid data element located at a hole position, the first target data element and the second target data element may be the same data element if the first target data element is a data element corresponding to a non-hole position in the input data block.

[0108] Data distributors can employ a one-input, multiple-output gating structure (e.g., 1 to 64). A data distributor takes one valid data element as input and outputs multiple row positions in the corresponding memory bank. Instead of selecting other memory banks, the data distributor chooses one row address from the multiple row addresses in its corresponding memory bank for writing. Thus, each data distributor can write one data element to its corresponding memory bank within a single data rearrangement cycle, and different memory banks can be written to in parallel.

[0109] In some embodiments, the data distributor can be implemented using a decoder and write enable hardware. The second selection signal is first decoded through row address to determine multiple row write enable signals, and then ANDed with the write mask signal to drive the write terminal of the corresponding memory cell. This reduces the number of data lines and makes it more compatible with memory arrays. If the memory bank uses SRAM, the second selection signal can serve as the SRAM row address and the write mask signal for write enable.

[0110] In some embodiments, the data distributor is configured to write the second target data element into the target storage cell in the corresponding storage bank in response to the second target data element being a valid data element output by the corresponding first multiplexer.

[0111] The target storage unit is determined by the row address of the corresponding storage bank. The storage bank address can be determined by hardware connection relationships, and the row address is determined by the second selection signal. For example, the data distributor 341 can correspond to storage bank 111, and the second selection signal indicates that the row address is 12, then the target storage unit is the storage unit with a depth of 12 in storage bank 111.

[0112] In some embodiments, the data distributor is configured to determine the row address of the target storage cell in the corresponding memory bank based on a second selection signal.

[0113] If multiple data distributors have the same row address within a data rearrangement cycle, multiple valid data elements can be written to the same storage row. If the number of free storage units in the current storage row is less than the number of valid data elements in the current input data block, then some valid data in the current input data block can be stored in the current storage row, and the other part of the valid data can be stored in the next storage row, thus allowing valid data elements in the current input data block to be stored in multiple storage rows. Therefore, the row addresses corresponding to different data distributors can be different. For example, some data distributors may select the first storage row, while others may select the second storage row. For a single data distributor, within a data rearrangement cycle, there is only one corresponding row address, and data is written only to one storage unit located at that row address in the storage bank corresponding to that data distributor.

[0114] The data writing circuit 130 can also control the write path based on a write mask signal. The write mask signal enables the corresponding write path when the second target data element is a valid data element. When the second target data element corresponds to a hole, the write mask signal blocks the corresponding write path. Thus, C-dimensional holes are not written to the storage array 110, and the subsequent data reading circuit 140 can read the rearranged valid data elements when reading row by row.

[0115] Masking of void locations can be implemented at multiple levels. In one implementation, the controller can generate a write mask bit for each data distributor. If the write mask bit corresponding to a data distributor is in a first state, the data distributor writes a data element to the corresponding bank. If the write mask bit corresponding to a data distributor is in a second state, the data distributor does not trigger write enable and does not write a data element to the bank. In another implementation, the data write circuit 130 can receive a byte-represented write mask vector, where each bit in the write mask vector corresponds to an element position in a first intermediate data block 41. In yet another implementation, the controller can convert void locations into invalid row addresses or write inhibit signals, preventing the data distributor from driving the write port during the corresponding data rearrangement cycle.

[0116] In some embodiments, the controller is configured to determine the location of the void in the input data block according to the first data arrangement and the size of the data to be spliced; and to control the data writing circuit 130 to write the multiple valid data elements in the first intermediate data block 31 into multiple storage units based on the element positions of multiple valid data elements in the data to be spliced, the first spatial position mapping relationship and the void positions.

[0117] For example, consider a data block to be stitched with dimensions 4H3W15C and a first data layout of 2H4W8C. Since the actual size of the data to be stitched in dimension W is 3, while the size of dimension W in the layout is 4, each input data block will generate a corresponding hole in dimension W. Similarly, since the actual size of the data to be stitched in dimension C is 15, while the size of dimension C in the layout is 8, a 1B hole will be generated in the second C group when arranged in 2H4W8C. The controller can determine whether a valid data element exists at a certain position in an input data block based on the effective range of the three dimensions H, W, and C, thus determining whether that position is a hole. After the input rearrangement circuit 120 completes the position conversion, the controller can synchronously convert the hole information to the corresponding position in the first intermediate data block 41, and then form a write mask signal.

[0118] In some embodiments, the controller is configured to determine the row addresses of multiple storage units based on the element positions of multiple valid data elements in the data to be concatenated and a first spatial position mapping relationship; and to control the data writing circuit 130 to perform a write operation based on the first spatial position mapping relationship, the hole positions and the row addresses of the multiple storage units.

[0119] When determining the row address of multiple memory banks, the row address is related to the shift bit. If a valid data element of the current input data block to be concatenated can continue to be written to the current memory row, the controller can control the corresponding data distributor to write the corresponding data element to the current memory row. If a valid data element crosses the current row after being shifted, the controller can control some data distributors to write the corresponding data element to the next memory row.

[0120] The location of holes can be determined by the relationship between the size of the data to be stitched and the first data layout. The first data layout represents the size of each dimension within an input data block, and the size of the data to be stitched represents its size in each dimension. When the actual size of the data to be stitched in a certain dimension is smaller than the size of the first data layout in that dimension, there will be element positions in the input data block where no valid data element corresponds, i.e., holes appear. For example, for multi-dimensional data to be stitched, holes can appear at the end of dimension W, the end of dimension C, or at the block boundaries of dimension H.

[0121] exist Figure 2 In the example, the first data arrangement is 2H4W8C, the size of the data to be concatenated is 4H3W15C, and the data element is 1B. The input data block (64B) can be arranged according to 2 H, 4 W, and 8 C. Since the actual size of the W dimension is 3, there is no valid data element at the W=3 position in each H layer, corresponding to an 8B hole. Since the actual size of the C dimension is 15, a 1B hole will be left at each H and W position in the second C group. The above hole occupies the position in the original input data block, but should not continue to be written as a data element in the concatenated valid output data element.

[0122] The controller can determine the H, W, and C ranges covered by the input data block number, and then determine whether the position of each element falls within the valid range of the size of the data to be concatenated. If the H index corresponding to the element position is less than the size of the H dimension, the W index is less than the size of the W dimension, and the C index is less than the size of the C dimension, then the position is a valid data element; if the index of any dimension exceeds the size of the corresponding dimension, then a hole position will appear at that position. The determination of this hole position can be implemented through a comparator or configured by pre-generating a mask by the compiler.

[0123] After the input rearrangement circuit 120 performs the first data rearrangement, the hole positions will change with the data element positions. The controller can convert the hole mask on the input side into a write mask in the first intermediate data block 31 based on the first spatial position mapping relationship. Therefore, when the data writing circuit 130 receives the first intermediate data block 31, it can directly decide whether to write based on the write mask at the corresponding position.

[0124] In some embodiments, the write mask can correspond one-to-one with a data distributor. If the storage array 110 includes 64 memory banks and the data writing circuit 130 includes 64 data distributors, a 64-bit write mask can be generated in one data rearrangement cycle. The 0th write mask corresponds to the 0th data distributor, the 1st write mask corresponds to the 1st data distributor, and so on. When the write mask is in a first state, the data distributor writes the input data to the target storage row; when the write mask is in a second state, the data distributor does not write data elements to the storage array.

[0125] The data writing circuit writes valid data elements from the first intermediate data block into multiple memory banks. Therefore, empty positions are not written as valid data elements into the memory array, and the data writing phase eliminates the impact of empty positions on subsequent data element reading and output rearrangement.

[0126] The data read circuit 140 includes a plurality of second multiplexers. Each of the plurality of second multiplexers is configured corresponding to a plurality of memory banks and is coupled to a plurality of memory cells in the corresponding memory bank. Figure 3 Second multiplexers 351, 352, 353, and others not individually labeled are shown. Each second multiplexer can correspond to a memory bank. For example, second multiplexer 351 can correspond to memory bank 111, second multiplexer 352 can correspond to memory bank 112, and second multiplexer 353 can correspond to memory bank 113.

[0127] The second multiplexer reads valid data elements from the memory cells located in the target memory row of the corresponding memory bank and outputs them. The target memory row can be a memory row in the memory array 110 that has been written to and meets the output conditions. The data read circuit 140 can read multiple valid data elements from the same target memory row in multiple memory banks to form a second intermediate data block 32.

[0128] In some embodiments, the second multiplexer determines the row address of the target storage row that has been written in the storage array 110 based on the third selection signal, and selects the storage cell located in the target storage row in the corresponding storage bank based on the row address of the target storage row, so as to read valid data elements from the storage cell located in the target storage row in the storage bank corresponding to the second multiplexer. For example, when the target storage row is row 0, multiple second multiplexers can select the storage cell in row 0 in their respective corresponding storage banks. When the target storage row is row 1, multiple second multiplexers can select the storage cell in row 1 in their respective corresponding storage banks.

[0129] In some embodiments, in response to the completion of writing to a target storage row in the storage array 110, the controller determines the row address of the target storage row and controls the data reading circuit 140 to read multiple valid data elements based on the row address of the target storage row.

[0130] The completion of writing a target storage line can be determined by the line's write count, the line's row validity flag, or pipeline timing. The line write count includes the number of data elements written to the line and can be counted using a counting circuit. The line validity flag indicates whether the line has been completely written. If a line has been completely written, its corresponding row validity flag is valid; if a line has not been completely written, its corresponding row validity flag is invalid, and this flag can be stored in a status register.

[0131] Pipeline timing refers to the timing of the data rearrangement circuit 100 in processing input data blocks. The data rearrangement circuit 100 processes the data to be concatenated or the input data blocks in a pipelined manner. For example, at t=1, the data rearrangement circuit 100 processes the second input data block, and the target storage line corresponding to the pipeline timing is the second storage line; at t=4, the data rearrangement circuit 100 processes the fourth input data block, and the target storage line corresponding to the pipeline timing is the fourth storage line.

[0132] The controller can determine whether a write operation to a specific storage row is complete by counting the number of valid rows. For a storage array 110 consisting of 64 banks, the controller can maintain a 64-bit write status bit for each storage row. This write status bit can be stored, for example, but not limited to, in the controller's corresponding first status register. After a data distributor writes a valid data element to the target storage row of the corresponding bank, the controller sets the write status bit for that bank. If a position is masked by a hole mask, the controller can determine whether a valid data element needs to be written to that position in the subsequent output data block layout. If it is determined that a valid data element needs to be written, the write status bit remains in an incomplete state; if it is determined that a valid data element does not need to be written, the write status bit can be marked as complete according to the output layout requirements.

[0133] In some embodiments, the controller can determine whether to write data elements to the target storage row based on the write status bit and write decision signal of the target storage row. The write decision signal is determined based on the output layout requirements and indicates whether a certain storage cell in the storage row needs to be written with data elements. If a target storage row requires 64 valid positions, the controller can count the write status bits of the corresponding positions (i.e., the multiple storage cells corresponding to each bank) in the storage row. When the positions corresponding to multiple banks have been written, the controller can set the row validity flag of the row to 1; if there are empty positions corresponding to some banks, the controller can determine whether these empty positions need to be written with valid data elements based on the write decision signal corresponding to the output layout requirements; if no valid data elements need to be written, the row validity flag of the storage row is set to 1; otherwise, the row validity flag of the storage row is set to 0.

[0134] In some embodiments, the controller can determine the write status bit of the target storage row by combining the row validity count and the concatenation task boundary. The concatenation task boundary is the dividing position between different data to be concatenated, determined by the concat function operator. The concatenation task boundary includes the end position of the first data to be concatenated, the start position of the second data to be concatenated, and whether a certain storage cell in the storage array should be written with data elements from the second data to be concatenated. For example, when concatenating along the C dimension, if the C dimension of the first data to be concatenated is 15, then the C0 of the second data to be concatenated should be continued to the concatenated position C15. The C15 position is the concatenation boundary jointly determined by the end position of the first data to be concatenated and the start position of the second data to be concatenated. The row validity count is the number of valid data elements written in a certain storage row.

[0135] For example, after some data blocks of the first data to be concatenated 21 are written to the storage array, if there are still some unwritten free storage cells in a certain storage row, it is determined that the free storage cells in that storage row still need to be written with valid data elements of the second data to be concatenated 22. In this case, the controller will not immediately confirm the write status bit as completed, but will wait for the relevant valid data elements of the second data to be concatenated 22 to be written to the free storage cells in the above storage row before marking the write status bit of that storage row as completed. Thus, the second intermediate data block 32 read by the data reading circuit 140 already includes the data elements that should enter the same output data block after concatenation.

[0136] The storage row in the storage array indicated by the third selection signal of the data read circuit 140 corresponds to the data rearrangement cycle. For example, in the second data rearrangement cycle, the third selection signal can indicate the second storage row. After the 0th storage row, the 1st storage row, and the 2nd storage row are written sequentially, the controller can control the second multiplexer MUX1 in the data read circuit 140 to read the data elements in these storage rows according to the storage row order. If the storage array 110 adopts a circular buffer queue, the data read circuit can read multiple valid data elements from multiple storage cells located in the same target storage row in multiple storage banks based on the row address of the target storage row. Thus, valid data elements that are scattered across multiple storage banks can be read uniformly according to storage rows, so that the subsequent output rearrangement circuit processes the data elements based on the storage rows, preventing gaps from recurring.

[0137] In some embodiments, the usage of each storage row in the storage array during each data rearrangement cycle can be determined based on the size of the data to be concatenated, the location and number of holes in the input data block, the first data arrangement, the second data arrangement, the third data arrangement, and the storage volume size, thereby determining the storage rows that have been written. From the storage rows that have been written, the target storage row to be read is determined, and a third selection signal is generated based on the target storage row.

[0138] Taking the data to be concatenated as 4H3W15C, the first data arrangement as 2H4W8C, the second data arrangement as 8C2H4W, and the third data arrangement as 2H4W8C, and the storage array as 64 memory banks, each containing 64 memory cells, as an example, we can determine that the first input data block contains 16 bytes of holes, the second input data block contains 22 bytes of holes, and so on. First, we determine the number of valid data elements in each input data block. In the first data rearrangement cycle (cycle0), the 0th storage row is not yet written; in the second data rearrangement cycle (cycle1), the 0th storage row is written, but the 1st storage row is not yet written; in the third data rearrangement cycle (cycle2), the 0th and 1st storage rows are written, but the 2nd storage row is not yet written. In the third data rearrangement cycle (cycle2), a third selection signal can be generated based on either the 0th or 1st storage row.

[0139] In some embodiments, the output rearrangement circuit 150 includes a plurality of third multiplexers. The plurality of third multiplexers are configured to correspond to the element positions of a plurality of data elements in the output data block. Each third multiplexer is configured to select a target valid data element from the plurality of valid data elements read from the data read circuit 140 and output the target valid data element to the target position corresponding to that third multiplexer.

[0140] Figure 3Several third multiplexers, such as third multiplexer 361, third multiplexer 362, and third multiplexer 363, are shown. Each third multiplexer can output one target valid data element from a data block.

[0141] In some embodiments, the controller can manage the read and write states of the storage array 110 to avoid undefined read and write contention within the same storage cell during the same data rearrangement cycle. The controller may include a second status register storing the write state, readable state, and release state for each storage row. A write state indicates that the storage row is receiving data elements being written; a readable state indicates that the storage row is ready to be read by the data read circuit 140; and a release state indicates that the data elements in the storage row have been read, and the storage row can be used again for writing data elements.

[0142] The controller can also maintain write row pointers and read row pointers. The write row pointer indicates the currently writable row address of the data write circuit 130. The read row pointer indicates the currently readable row address of the data read circuit 140. When there is no cross-row writing, the storage row indicated by the write row pointer can correspond to a data rearrangement cycle; for example, in the second data rearrangement cycle, the write row pointer can indicate the second storage row. The storage row indicated by the write row pointer can also correspond to the position of the input data block within the data to be processed; for example, if the current input data block is the second input data block, the write row pointer can indicate the second storage row.

[0143] When a cross-row write occurs, the controller can determine different row addresses for different data distributors based on the shift bit and the memory bank address corresponding to the data distributor. The row pointer can then be read in the order in which the storage rows are written. For example, after the 0th storage row in the storage array is written, the 1st storage row, the 2nd storage row, and so on are written in sequence, then the row pointer can be read according to the order in which the storage rows are written.

[0144] During a data rearrangement cycle, if the data write circuit 130 needs to write to a certain storage row, and the data read circuit 140 is also ready to read the same storage row, the controller can determine the processing order based on the write status of the storage row. The write status of the storage row indicates whether the write operation is complete or incomplete. If the storage row has not yet been written, the controller can delay reading. If the storage row has been written and the corresponding read / write port allows simultaneous read / write access, the controller can allow the data read circuit 140 to read the storage row while prohibiting overwriting of storage cells within the storage row. If the storage row corresponds to a single-port storage structure, the controller can time-sharing access to the storage row by pausing write or pausing read.

[0145] In some embodiments, the third multiplexer is configured to determine the second element position of the target valid data element in the second intermediate data block 32 based on the fourth selection signal and the second intermediate data block 32 read by the data read circuit 140; and output the target valid data element based on the second element position.

[0146] The output rearrangement circuit may include multiple third multiplexers. Each of the multiple third multiplexers may correspond to multiple output positions within the output data block. Each third multiplexer may receive the second intermediate data block 32 read by the data read circuit 140 and, based on a corresponding fourth selection signal, select a second element position from multiple element positions in the second intermediate data block 32. This second element position indicates the position of the target valid data element within the second intermediate data block 32.

[0147] The second intermediate data block 32 may include multiple valid data elements read from the same target storage line from multiple storage banks. Since the multiple valid data elements in the second intermediate data block 32 can be arranged according to the second data arrangement, while the output data block needs to satisfy the third data arrangement, the third multiplexer does not directly output the data elements according to the original arrangement order in the second intermediate data block 32, but determines the target valid data element that should be output according to the fourth selection signal.

[0148] The fourth selection signal can be used to indicate which element position in the second intermediate data block 32 the third multiplexer selects. For a third multiplexer, its corresponding output position can be fixed, while the position of the data element required for that output position in the second intermediate data block 32 can be specified by the fourth selection signal. After determining the position of the second element according to the fourth selection signal, the third multiplexer selects the target valid data element corresponding to that second element position and outputs the target valid data element to the corresponding output position.

[0149] For example, the second intermediate data block 32 may include 64 element positions, and the third multiplexer can select one element position from these 64 element positions. If a third multiplexer corresponds to the i-th output position in the output data block, and the fourth selection signal indicates that the third multiplexer selects the j-th element position in the second intermediate data block 32, then the third multiplexer outputs the target valid data element at the j-th element position in the second intermediate data block 32 to the i-th output position in the output data block. Thus, after multiple third multiplexers perform the above selection operation respectively, multiple valid data elements in the second intermediate data block 32 can be rearranged into an output data block that conforms to the third data arrangement method.

[0150] The fourth selection signal can be related to the third data arrangement. The controller can determine the fourth selection signal corresponding to each third multiplexer based on the current arrangement of the second intermediate data block 32 and the third data arrangement required by the output data block. Each third multiplexer selects the target valid data element from different element positions of the second intermediate data block 32 based on its own fourth selection signal, so that multiple output positions in the output data block can obtain the corresponding data element.

[0151] In this way, the third multiplexer can select the position of valid data elements on the output side without changing the data elements in the second intermediate data block 32. This structure allows the data arrangement of the second intermediate data block 32, which is read row by row by the data reading circuit 140, to be reorganized in the output rearrangement circuit, thereby obtaining an output data block that meets the target arrangement requirements.

[0152] In some embodiments, the controller is configured to determine a second spatial position mapping relationship between the second data arrangement and the third data arrangement based on the second data arrangement and the third data arrangement; and to control the output rearrangement circuit 150 to perform a second data rearrangement on the read multiple valid data elements based on the second spatial position mapping relationship.

[0153] In some embodiments, the second spatial location mapping relationship is used to represent the correspondence between the element positions in the second intermediate data block 32 and the element positions in the output data block. The controller can determine the second spatial location mapping relationship based on the second data layout method and the third data layout method. When the third data layout method is the same as the first data layout method, the second spatial location mapping relationship and the first spatial location mapping relationship are inverse mappings of each other; when the third data layout method is different from the first data layout method used in the input data block, the second spatial location mapping relationship can be determined based on the third data layout method (i.e., the output layout).

[0154] For example, taking a first data arrangement of 2H4W8C, a second data arrangement of 8C2H4W, and a third data arrangement of 2H4W8C as an example, the second spatial position mapping relationship can represent rearranging the data elements in the second intermediate data block 32 arranged according to the second data arrangement 8C2H4W into an output data block arranged according to the third data arrangement 2H4W8C. Since the third data arrangement is the same as the first data arrangement used by the input data block, the second spatial position mapping relationship can be the inverse mapping of the first spatial position mapping relationship. That is, the input rearrangement circuit maps the data elements from the element positions corresponding to the first data arrangement 2H4W8C to the element positions corresponding to the second data arrangement 8C2H4W through the first spatial position mapping relationship, and the output rearrangement circuit maps the data elements from the element positions corresponding to the second data arrangement 8C2H4W back to the element positions corresponding to the third data arrangement 2H4W8C through the second spatial position mapping relationship.

[0155] For example, the third data arrangement may differ from the first data arrangement. In this case, the second spatial location mapping relationship is not an inverse mapping relationship with the first spatial location mapping relationship, but is calculated and determined by the controller based on the second and third data arrangement methods.

[0156] For example, the second intermediate data block 32 read by the data reading circuit 140 is arranged according to the second data arrangement 8C2H4W. The output data block needs to meet the third data arrangement required by the subsequent operator or storage module, and this third data arrangement is different from the first data arrangement. The controller can then determine the source element position corresponding to each output position based on this third data arrangement and generate a fourth selection signal accordingly. The third multiplexer selects the corresponding target valid data element from the second intermediate data block 32 based on the fourth selection signal and outputs it. Therefore, the output rearrangement circuit provided in this disclosure can both restore the intermediate arrangement to the input-side arrangement and support the formation of output data blocks different from the input-side arrangement according to the third data arrangement.

[0157] In some embodiments, the controller is configured to determine a second correspondence between the element positions of multiple data elements in a second intermediate data block 32 and the element positions of multiple data elements in an output data block, based on a second data arrangement and a third data arrangement, and to determine a second spatial position mapping relationship based on the second correspondence.

[0158] The second correspondence can be represented as element position pairs. For example, the i-th element position in the second intermediate data block 32 corresponds to the j-th element position in the output data block. The controller then converts multiple element position pairs into fourth selection signals corresponding to each third multiplexer. In one example, a target position in the output data block corresponds to H0W1C2. The controller determines the output number of the target position according to the second data arrangement, and then determines the element position of H0W1C2 in the second intermediate data block 32 according to the first data arrangement. This element position is the second element position that the corresponding third multiplexer needs to select.

[0159] The second spatial location mapping relationship is used to represent the correspondence between the element positions in the second intermediate data block 32 and the element positions in the output data block. For example, if the second intermediate data block 32 is arranged according to the second data arrangement method 8C2H4W, and the output data block needs to be arranged according to the third data arrangement method 2H4W8C, the controller can convert the H, W, and C indices corresponding to each target position in the output data block into position indices in the second intermediate data block 32, and generate a fourth selection signal to control each third multiplexer accordingly. Thus, the output rearrangement circuit 150 can rearrange the target data elements that have been read according to the target storage row into the data arrangement required by the subsequent storage circuit or subsequent operator.

[0160] The controller generates or configures a fourth selection signal based on the second spatial location mapping relationship, controls multiple third multiplexers in the output rearrangement circuit 150, and enables multiple third multiplexers to select corresponding data from multiple target data elements read out and output them to the corresponding positions in the output data block, thereby realizing data rearrangement.

[0161] The data read circuit 140 can read multiple target data elements from multiple memory cells located in the same target memory row in the memory array 110 and output a second intermediate data block 32. The multiple target data elements in the second intermediate data block 32 can be arranged according to the arrangement order of multiple memory banks, for example, in the order of Bank0 to Bank63. This arrangement order can be different from the third data arrangement required for the output data block.

[0162] The controller can determine a second spatial position mapping relationship based on the positions of multiple target data elements in the intermediate data block and the positions of multiple data elements in the output data block. This second spatial position mapping relationship represents the correspondence between the positions of elements in the intermediate data block and the positions of elements in the output data block. The controller can generate a fourth selection signal based on the second spatial position mapping relationship and provide this fourth selection signal to the corresponding third multiplexer.

[0163] For example, for a target element position in the output data block, the controller can determine the position of the target data element required for that target element position in the intermediate data block based on the second spatial position mapping relationship. The third multiplexer corresponding to that target element position selects the target data element corresponding to that position from multiple target data elements in the intermediate data block based on the fourth selection signal, and outputs it to the target element position in the output data block. Other third multiplexers can also perform selection operations according to their respective corresponding fourth selection signals.

[0164] If the intermediate data blocks read by the data reading circuit 140 are arranged in the order of the memory bank, while the output data blocks need to be arranged according to the third data arrangement method, the controller can map the element positions under the memory bank order to the element positions under the third data arrangement method. Therefore, the output rearrangement circuit 150 can rearrange multiple target data elements read by memory row into an output data block that conforms to the third data arrangement method.

[0165] For example, the input rearrangement circuit 120 can convert an input data block arranged according to the first data layout 2H4W8C into a data block arranged according to the second data layout 8C2H4W, and the output rearrangement circuit 150 can convert the second intermediate data block 32 from the second data layout 8C2H4W to the third data layout 2H4W8C. If the subsequent operator requires other layouts, the output rearrangement circuit 150 can also output other forms of data layout according to the third data layout.

[0166] The output rearrangement circuit 150 can employ a multiplexer array similar to that of the input rearrangement circuit 120. Each third multiplexer receives data elements corresponding to multiple element positions in the second intermediate data block 32 and selects one data element corresponding to an element position for output according to a fourth selection signal. If the second data arrangement is 8C2H4W and the third data arrangement is 2H4W8C, the controller can generate the corresponding second element position based on the output position number. For an input data block of size 64B, the output rearrangement circuit 150 can include 64 64-input multiplexers.

[0167] The output rearrangement circuit performs a second data rearrangement on the multiple valid data elements read, resulting in an output data block arranged according to the third data arrangement method. Since the valid data elements do not include invalid data elements corresponding to the holes in the input data block, the second data rearrangement can be based on the valid data elements whose hole effects have been removed, so that the output data block meets the format requirements of the subsequent storage circuit or subsequent operator.

[0168] For example, when the arrangement order of the intermediate data blocks read by the data reading circuit 140 is inconsistent with the required arrangement order of the output data blocks, the controller can generate a fourth selection signal according to the second correspondence, so that the output rearrangement circuit 150 outputs the target data element to the target position in the output data block.

[0169] The second spatial location mapping relationship is used to represent the correspondence between the element positions in the second intermediate data block 32 and the element positions in the output data block. For example, if the second intermediate data block 32 is arranged according to the second data arrangement method 8C2H4W, and the output data block needs to be arranged according to the third data arrangement method 2H4W8C, the controller can convert the H, W, and C indices corresponding to each target position in the output data block into source position indices in the second intermediate data block 32, and generate a fourth selection signal to control each third multiplexer accordingly. Thus, the output rearrangement circuit 150 can rearrange the target data elements read from the target storage row of the storage array into the data arrangement required by the subsequent storage circuit or subsequent operator.

[0170] Therefore, the output rearrangement circuit can adjust the position of the target data elements that have been read without re-accessing the data to be rearranged, so that the output data block meets the requirements of the subsequent storage or subsequent calculation for the data arrangement.

[0171] In some embodiments, the output rearrangement circuit 100 further includes a controller, which includes an input address generation unit and an output address generation unit. The input address generation unit is configured to generate a first access address for reading an input data block from memory based on the size of the data to be rearranged, a first data arrangement method, a first dimension step size, a first access start address, and a first access range. The output address generation unit is configured to generate a second access address for writing an output data block to memory based on a third data arrangement method, a second dimension step size, a second access start address, and a second access range.

[0172] The first access start address represents the base address of the data to be read in memory. The first dimension step size represents the address offset corresponding to moving one index unit in different dimensions according to the first data arrangement. The first access range represents the valid range of the data to be read. The second access start address represents the base address of the data to be written in memory. The second dimension step size represents the address offset corresponding to moving one index unit in different dimensions according to the third data arrangement. The second access range represents the valid range of the output data to be written.

[0173] Taking the first data arrangement as 2H4W8C as an example, adjacent data elements in dimension C can be arranged continuously, adjacent positions in dimension W can be spaced 8 C-dimensional positions apart, and adjacent positions in dimension H can be spaced 4×8 positions apart. The controller can convert the H, W, and C-dimensional indices into element positions in the input data block based on the above dimensional step sizes, and determine whether the element position is within the valid access range by combining the size of the data to be concatenated. After obtaining the output data block, the output rearrangement circuit 150 can write the output data block to the storage circuit periodically via the write-back interface. Specifically, the writing method of the output data block can be determined based on the output data block number, the third data arrangement, and the size of the concatenated data.

[0174] The input address generation unit can generate a first access address for reading the input data block based on the first access start address, the first dimension step size, and the H, W, and C ranges covered by the current input data block. For example, in the case of a first data arrangement of 2H4W8C, an input data block covers the data element positions corresponding to 2 H, 4 W, and 8 C. The input address generation unit can determine the H, W, and C index ranges covered by the current data block number, and then calculate the read address of the input data block in memory based on the first dimension step size.

[0175] The output address generation unit can generate a second access address based on the output data block number, the third data arrangement method, and the second access start address, enabling the output data blocks obtained by the output rearrangement circuit 150 to be written back to the memory in a predetermined block order. When the output data blocks correspond to consecutive positions in the concatenation result, the output address generation unit can update the second access address based on the number of data blocks already output. In the case of boundary data blocks, the output address generation unit can determine whether the current output data block falls within the valid write-back range by combining the second access range, and generate the corresponding write-back address based on the second access address, the second dimension step size, and the second access range.

[0176] In some embodiments, the number of inputs to the first, second, and third multiplexers can be adjusted according to the number of memory banks or the memory bank depth. If the number of memory banks is 64, both the first and third multiplexers can be 64-input structures. If the memory bank depth is 64, the second multiplexer can select one of the 64 memory cells in the corresponding memory bank. The actual circuitry of the first, second, and third multiplexers can be implemented using tree multiplexers, hierarchical multiplexers, or cross switches.

[0177] The following is combined Figure 2 and Figure 3 The process of data rearrangement and data splicing is explained. Figure 3In the circuit, the input rearrangement circuit 120 includes multiple first multiplexers, the data writing circuit 130 includes multiple data distributors, the storage array 110 includes multiple memory banks, the data reading circuit 140 includes multiple second multiplexers, and the output rearrangement circuit 150 includes multiple third multiplexers.

[0178] After the input rearrangement circuit 120 receives the input data block, multiple first multiplexers perform gating operations respectively. First multiplexer 331 can select a first element position from 0 to 63 element positions in the input data block based on its first selection signal, and output the first target data element corresponding to that first element position. First multiplexers 332, 333, and other first multiplexers can also perform gating according to their respective first selection signals. The set of data elements output by the multiple first multiplexers constitutes the first intermediate data block 31.

[0179] After the first intermediate data block 31 enters the data writing circuit 130, multiple data distributors perform write operations respectively. Data distributor 341 can receive the second target data element output from its corresponding first multiplexer and determine a row address in the memory bank 111 based on the second selection signal. When the write mask signal indicates that the second target data element is a valid data element, data distributor 341 writes the second target data element into the memory cell corresponding to that row address in the memory bank 111. When the write mask signal indicates that the position is an empty position, data distributor 341 does not write to that position.

[0180] Data distributor 342 can similarly write the corresponding second target data element to the target storage row address in memory bank 112. Data distributor 343 can write the corresponding second target data element to the target storage row address in memory bank 113. In the same data rearrangement cycle, data distributors 341, 342, and 343 can select different row addresses. For example, data distributor 341 selects row 1, data distributor 342 selects row 1, and data distributor 343 selects row 2. In this way, data writing circuit 130 can support cross-row writing caused by concat offsets.

[0181] After the target storage row in the storage array 110 is written, the data read circuit 140 performs a row-by-row read operation. Taking the second multiplexer 351 as an example, the second multiplexer 351 corresponds to the storage bank 111. The second multiplexer 351 can determine the row address of the target storage row based on the third selection signal and read valid data elements from the storage cells located in the target storage row in the storage bank 111. The second multiplexer 352 can read valid data elements from the same target storage row in the storage bank 112, and the second multiplexer 353 can read valid data elements from the same target storage row in the storage bank 113. And so on, the set of data elements output by multiple second multiplexers forms the second intermediate data block 32.

[0182] After the second intermediate data block 32 enters the output rearrangement circuit 150, multiple third multiplexers perform output position selection respectively. Third multiplexer 361 can select a second element position from multiple element positions in the second intermediate data block 32 based on a fourth selection signal, and output the target valid data element corresponding to that second element position to the target position in the output data block. Third multiplexers 362 and 363 can also output their respective selected target valid data elements to their corresponding target positions. This process continues, and the collection of data elements output by multiple third multiplexers forms an output data block arranged according to the third data arrangement method.

[0183] See Figure 3 The circuit diagram of the data rearrangement device shown illustrates that the input data block arranged according to the first data arrangement is first converted by the input rearrangement circuit 120 into a first intermediate data block 31 arranged according to the second data arrangement, which is convenient for writing to the storage array 110. Then, the data writing circuit 130 writes the valid data elements into multiple storage banks. The data reading circuit 140 reads the second intermediate data block 32 from the target storage line, and the output rearrangement circuit 150 converts the second intermediate data block 32 arranged according to the second data arrangement into an output data block arranged according to the third data arrangement. This multiple data rearrangement method maintains the processing capabilities and boundaries of each hardware circuit, facilitating hardware pipeline implementation.

[0184] The data rearrangement device 100 can read input data blocks from the storage circuit during a data rearrangement cycle. The input rearrangement circuit 120 converts the input data blocks arranged according to the first data arrangement 2H4W8C into intermediate data blocks arranged according to the second data arrangement 8C2H4W. The data writing circuit 130 writes the valid data elements after the first mapping into the storage array 110. After a storage row is written, the data reading circuit 140 reads multiple valid data elements from that storage row. The output rearrangement circuit 150 then remaps the read valid data elements according to the output requirements. For example, the output rearrangement circuit 150 can remap the second data arrangement 8C2H4W to the third data arrangement 2H4W8C. Finally, the data rearrangement device 100 can write the output data blocks into the storage circuit cycle by cycle.

[0185] In the 0th data rearrangement cycle (cycle 0), the data rearrangement device 100 can read the first input data block from the storage circuit. This input data block is arranged according to the first data arrangement 2H4W8C, and the input rearrangement circuit 120 converts it into a first intermediate data block 31 arranged according to the second data arrangement 8C2H4W. In the 1st data rearrangement cycle (cycle 1), multiple data distributors in the data writing circuit 130 can select different storage cells located in the 0th storage row of multiple storage banks to write valid data elements to the storage cells located in the 0th storage row of the multiple storage banks. For empty locations in the input data block, the controller generates a corresponding write mask to block the write path corresponding to the empty location.

[0186] In the first data rearrangement cycle (cycle 1), the data rearrangement device 100 can read the second input data block from the storage circuit. The input rearrangement circuit 120 continues to convert the input data block arranged according to the first data arrangement 2H4W8C into a first intermediate data block 31 arranged according to the second data arrangement 8C2H4W. If the valid data elements in the second input data block correspond to the storage cells in the first 56 storage banks, then in the second data rearrangement cycle (cycle 2), the controller can control a portion of the data distributors in the data writing circuit 130 to select the first storage row to write the 56 valid data elements into the storage cells in the first storage row of the corresponding storage bank. For other storage cells in the first storage row, if their corresponding hole positions are not determined to be valid data elements to be written in the current cycle, the controller can close the corresponding write path through a write mask.

[0187] In the second data rearrangement cycle (cycle 2), the data rearrangement device 100 can read the third input data block from the storage circuit. Since there are free storage cells in the first storage row that have not yet been written with valid data elements, the valid data elements corresponding to the input data block can be shifted to write some data elements from the third input data block into the free storage cells in the first storage row. For example, if there are 8 free storage cells in the first storage row, the valid data elements in the input data block can be shifted. Therefore, in the third data rearrangement cycle (cycle 3), the controller can control the data distributor to write the data elements output by the first multiplexer corresponding to storage banks 56 to 63 into the first storage row, and the data elements output by the first multiplexer corresponding to storage banks 0 to 55 into the second storage row. A portion of the valid data elements in the third input data block can be written to the end of the first storage row, and another portion can be written to the beginning of the second storage row. Therefore, by writing data elements to different row addresses through a data distributor during the data element writing phase to complete cross-row writing, the additional data movement operations after writing are reduced.

[0188] In the third data rearrangement cycle (cycle 3), the data rearrangement device 100 can read the fourth input data block from the storage circuit. After the input rearrangement circuit 120 completes the first data rearrangement, the controller generates row addresses and write masks based on the current splicing offset, hole positions, and the valid range in the input data block. For example, some valid data elements can be written to the second storage row of storage banks 56 to 63, and other valid data elements can be written to the third storage row of storage banks 0 to 47. If storage banks 48 to 55 correspond to the hole positions in the current input data block, the controller can close the corresponding write path using the write mask.

[0189] In subsequent data rearrangement cycles, the data rearrangement device 100 can continue to process the fifth to eighth input data blocks in the manner described above. For each input data block, the input rearrangement circuit 120 first converts it from the first data arrangement mode 2H4W8C to the second data arrangement mode 8C2H4W; then the data writing circuit 130 writes the valid data elements into the target storage row in the corresponding memory bank according to the shift bit, hole position, and row address.

[0190] After multiple storage lines, such as the first, second, and third storage lines, have been written, the multiple second multiplexers in the data read circuit 140 can read data elements from these storage lines. If part of a storage line is written by the previous input data block and another part is written by the next input data block, the controller can wait until the storage line is completely written before initiating a read operation. Therefore, the data read circuit 140 always outputs a complete second intermediate data block 32 on a storage line basis.

[0191] In some embodiments, the first input data block, the last input data block, and the input data block at the transition point of the data to be concatenated can have different hole distributions. The controller can determine the shift number and write mask corresponding to the input data block based on the position of the current input data block in the corresponding data to be concatenated. For the first input data block of a certain data to be concatenated, the controller can determine the shift number based on the starting position of the data to be concatenated in the concatenation result. For example, when the second data to be concatenated is written following the first data to be concatenated, the shift number of the first input data block of the second data to be concatenated is determined by the size of the first data to be concatenated in the concatenation dimension. For subsequent input data blocks in the same data to be concatenated, the controller can use the concatenation offset corresponding to the data to be concatenated. For the last input data block of the data to be concatenated, the controller can determine the remaining valid data elements and hole positions based on the size of the data to be concatenated, and generate a write mask accordingly.

[0192] Taking a data block of size 4H3W15C and a first data arrangement of 2H4W8C as an example, one input data block covers the positions of 2 H, 4 W, and 8 C. Since the actual size of the W dimension is 3, an 8B hole will be formed at the tail position of the W dimension. Since the actual size of the C dimension is 15, a 1B hole will be formed at each valid H / W position in the second C group. When processing the last C group of a certain data block to be spliced, the controller can identify the position corresponding to C15 as a hole and prevent the writing of data elements to that position by using a write mask signal.

[0193] When the second data to be concatenated 22 begins to be written, the controller can determine the starting offset of the second data to be concatenated 22 based on the C-dimensional length already occupied by the first data to be concatenated 21. If the C-dimensional length of the first data to be concatenated 21 is 15, then C0 of the second data to be concatenated 22 can be continued to the concatenated position C15. Since the C group size in the layout is 8, this position may fall at the blank 1B position in the original input data block, or it may fall into the beginning of the next line due to position mapping and cyclic right shift. The controller writes the valid data element to the target position through the shift bit and storage row address selection signal.

[0194] During the writing process of input data blocks corresponding to the boundaries of the first and second data to be concatenated (e.g., the last input data block of the first data to be concatenated and the first input data block of the second data to be concatenated) into a certain storage row, some storage cells in that storage row may receive data from the current input data block, while other storage cells may receive data from the next input data block. The data writing circuit 130 can independently select the storage row address through each data distributor to write data elements to target storage cells in different storage rows. For example, the first storage row from Bank0 to Bank55 in the storage array can be written with valid data elements from the previous input data block, the first storage row from Bank56 to Bank63 can be written with valid data elements from the current input data block, and the second storage row from Bank0 to Bank55 can be written with valid data elements from the current input data block. Thus, the input data blocks at the boundaries of the data to be concatenated are split and written into multiple storage rows of multiple storage banks during the writing phase.

[0195] Exemplary methods Figure 4 This is a flowchart illustrating a data rearrangement method provided in this disclosure. For example... Figure 4 The data rearrangement method shown can be derived from... Figure 1 The data rearrangement device 100 shown is executed. For example... Figure 4 As shown, data rearrangement methods include: Step 410: Based on the input data block arranged according to the first data arrangement method, perform a first data rearrangement on multiple data elements in the input data block to obtain a first intermediate data block arranged according to the second data arrangement method.

[0196] Step 420: Write multiple valid data elements from the first intermediate data block into multiple storage units; valid data elements include data elements corresponding to non-hole positions in the input data block.

[0197] Step 430: Read multiple valid data elements from multiple storage cells located in the same storage row in multiple storage banks.

[0198] Step 440: Perform a second data rearrangement on the read multiple valid data elements to obtain an output data block arranged according to the third data arrangement method.

[0199] Figure 5 This is a flowchart illustrating a first data rearrangement method provided in this disclosure. For example... Figure 5 As shown, step 410 above, based on the input data block arranged according to the first data arrangement method, involves performing a first data rearrangement on multiple data elements in the input data block, including: Step 510: Select a first target data element from multiple data elements in the input data block and output the first target data element; wherein the element position of the first target data element in the first intermediate data block corresponds to the arrangement position of the first multiplexer.

[0200] Figure 6 This is a flowchart illustrating a target data element selection method provided in this disclosure. For example... Figure 6 As shown, step 510 above, which involves selecting a first target data element from multiple data elements of the input data block and outputting the first target data element, includes: Step 610: Based on the first selection signal, determine the position of the first target data element in the input data block.

[0201] Step 620: Output the first target data element based on the position of the first element.

[0202] Figure 7 This is a flowchart illustrating another first data rearrangement method provided in this disclosure. For example... Figure 7 As shown, step 410 above, which involves performing a first data rearrangement on multiple data elements in the input data block, includes: Step 710: Determine the first data layout method, the size of the data to be spliced, and the second data layout method; wherein, the data to be spliced ​​includes input data blocks; Step 720: Based on the first data arrangement method, the size of the data to be spliced, and the second data arrangement method, determine the first spatial position mapping relationship between the first data arrangement method and the second data arrangement method, and the shift number corresponding to multiple data elements; Step 730: Based on the first spatial location mapping relationship and the shift number, perform a first data rearrangement on multiple data elements in the input data block.

[0203] Figure 8 This is a flowchart illustrating a method for determining a first spatial location mapping relationship provided in this disclosure. For example... Figure 8 As shown, step 720 above, determining the first spatial position mapping relationship between the first data arrangement method and the second data arrangement method, and the shift number corresponding to multiple data elements, includes: Step 810: Based on the first data arrangement method, the size of the data to be spliced, and the second data arrangement method, determine the first correspondence between the element positions of multiple data elements in the input data block and the addresses of multiple storage entities.

[0204] Step 820: Based on the first correspondence, determine the first spatial location mapping relationship and the number of shifts.

[0205] Figure 9This is a flowchart illustrating a method for writing target data elements provided in this disclosure. For example... Figure 9 As shown, step 420 above, writing multiple valid data elements from the first intermediate data block into multiple storage banks, includes: Step 910: In response to the second target data element output by the first multiplexer corresponding to the data distributor being a valid data element, the second target data element is written into the target storage unit in the storage bank corresponding to the data distributor.

[0206] Figure 10 This is a flowchart illustrating another method for writing target data elements provided in this disclosure. For example... Figure 10 As shown, the step 910 above, writing the second target data element into the target storage unit in the storage bank corresponding to the data distributor, includes: Step 1010: Based on the second selection signal, determine the row address of the target storage unit in the storage bank corresponding to the data distributor.

[0207] Step 1020: Based on the write mask signal and the row address of the target storage unit in the storage bank corresponding to the data distributor, the second target data element is written to the target storage unit; the write mask signal is used to enable control of the write path of the second target data element when the second target data element is a valid data element.

[0208] Figure 11 This is a flowchart illustrating another method for writing target data elements provided in this disclosure. For example... Figure 11 As shown, step 420 above, writing multiple valid data elements from the first intermediate data block into multiple storage banks, includes: Step 1110: Determine the location of the holes in the input data block according to the first data arrangement and the size of the data to be spliced; wherein, the data to be spliced ​​includes the input data block; Step 1120: Based on the element positions of multiple valid data elements in the data to be spliced, the first spatial position mapping relationship, and the hole positions, write multiple valid data elements in the first intermediate data block into multiple storage units.

[0209] Figure 12 This is a flowchart illustrating a method for writing data elements into storage, as provided in this disclosure. Figure 12 As shown, step 1120 above, which involves writing multiple valid data elements from the first intermediate data block into multiple storage units based on the element positions of multiple valid data elements in the data to be concatenated, the first spatial position mapping relationship, and the hole positions, includes: Step 1210: Determine the row addresses of multiple storage volumes based on the mapping relationship between the element positions of multiple valid data elements in the data to be concatenated and the first spatial position.

[0210] Step 1220: Based on the first spatial location mapping relationship, the hole location and the row address of multiple storage bodies, write multiple valid data elements in the first intermediate data block into multiple storage bodies.

[0211] Figure 13 This is a flowchart illustrating a method for reading data elements provided in this disclosure. For example... Figure 13 As shown, step 430 above, reading multiple valid data elements from multiple storage cells located in the same storage row in multiple storage banks, includes: Step 1310: Based on the third selection signal, determine the row address of the target storage row that has been written in the storage array.

[0212] Step 1320: Read multiple valid data elements based on the row address of the target storage row.

[0213] Step 1330: Read the valid data element from the storage cell located in the target storage row in the storage bank corresponding to the second multiplexer.

[0214] Figure 14 This is a flowchart illustrating another method for reading data elements provided in this disclosure. For example... Figure 14 As shown, step 430 above, reading multiple valid data elements from multiple storage cells located in the same storage row in multiple storage banks, includes: Step 1410: In response to the completion of writing the target storage row in the storage array, determine the row address of the target storage row; Step 1420: Based on the row address of the target storage row, read multiple valid data elements from multiple storage cells of the target storage row.

[0215] Figure 15 This is a flowchart illustrating a second data rearrangement method provided in this disclosure. For example... Figure 15 As shown, the second data rearrangement of the read multiple valid data elements in step 440 above includes: Step 1510: Select the target valid data element from the multiple valid data elements read.

[0216] Step 1520: Output the target valid data elements to the target position corresponding to the third multiplexer.

[0217] Figure 16 This is a flowchart illustrating a method for outputting target valid data elements provided in this disclosure. For example... Figure 16 As shown, step 1520 above, which involves outputting the target valid data element to the target position corresponding to the third multiplexer, includes: Step 1610: Based on the fourth selection signal and the second intermediate data block composed of the read valid data elements, determine the position of the second element of the target valid data element in the second intermediate data block.

[0218] Step 1620: Output the target valid data elements based on the position of the second element.

[0219] Figure 17 This is a flowchart illustrating a second data rearrangement method provided in this disclosure. For example... Figure 17 As shown, the second data rearrangement of the read multiple valid data elements in step 440 above includes: Step 1710: Determine the second spatial position mapping relationship between the second data arrangement method and the third data arrangement method based on the second data arrangement method and the third data arrangement method; Step 1720: Based on the second spatial location mapping relationship, perform a second data rearrangement on the multiple valid data elements read.

[0220] Figure 18 This is a flowchart illustrating a method for determining a second spatial location mapping relationship provided in this disclosure. For example... Figure 18 As shown, step 1710 above, determining the second spatial location mapping relationship between the second data arrangement method and the third data arrangement method based on the second data arrangement method and the third data arrangement method, includes: Step 1810: Based on the second data arrangement method and the third data arrangement method, determine the second correspondence between the element positions of multiple data elements in the read second intermediate data block and the element positions of multiple data elements in the output data block.

[0221] Step 1820: Determine the second spatial location mapping relationship based on the second correspondence.

[0222] Figure 19 This is a flowchart illustrating a method for determining an access address provided in this disclosure. For example... Figure 19 As shown, the method further includes: Step 1910: Based on the size of the data to be rearranged, the first data arrangement method, the first dimension step, the first access start address, and the first access range, generate a first access address for reading the input data block.

[0223] Step 1920: Based on the third data arrangement method, the second dimension step size, the second access start address, and the second access range, generate a second access address for writing the output data block.

[0224] Exemplary electronic devices Figure 20A structural diagram of an electronic device provided in an embodiment of this disclosure includes at least one processor 2001 and a memory 2002.

[0225] The processor 2001 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 2000 to perform desired functions.

[0226] The memory 2002 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 2001 may execute one or more computer program instructions to implement the data rearrangement methods and / or other desired functions of the various embodiments of this disclosure described above.

[0227] In one example, the electronic device 2000 may also include an input device 2003 and an output device 2004, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0228] The input device 2003 may also include, for example, a keyboard, a mouse, etc.

[0229] The output device 2004 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0230] Of course, for the sake of simplicity, Figure 20 Only some of the components of the electronic device 2000 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 2000 may include any other suitable components depending on the specific application.

[0231] Exemplary neural network processor Figure 21 This is a schematic diagram of the structure of a neural network processor provided in an embodiment of the present disclosure, such as... Figure 21 As shown, the neural network processor 2100 includes a data rearrangement device 2101 and a memory 2102.

[0232] The data rearrangement device 2101 can rearrange the obtained data to be concatenated and concatenate the valid data elements in the data to be concatenated to obtain the concatenated data elements. Therefore, the obtained data elements can be considered as the data elements corresponding to the data concatenation operator. The specific implementation of the data rearrangement device 2101 in concatenating the data to be concatenated can be referred to the corresponding content in the foregoing embodiments. This data rearrangement device 2101 can be the same as the data rearrangement device 100 in the foregoing embodiments.

[0233] The memory 2102 is used to store data during the data splicing process performed by the data rearrangement device 2101. For example, the memory 2102 is used to store data such as the data to be spliced, input data blocks, intermediate data blocks, output data blocks, and spliced ​​output data.

[0234] After acquiring the data to be spliced, the neural network processor 2100 can perform splicing processing on the data to be spliced ​​through the data rearrangement device 2101 in the neural network processor 2100 to obtain spliced ​​data elements. The obtained spliced ​​data elements can be stored in the memory 2102 for subsequent reading and retrieval or further processing.

[0235] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the data rearrangement methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0236] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0237] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the data rearrangement methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0238] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0239] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0240] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A data rearrangement apparatus, comprising: A storage array, comprising multiple storage units, wherein each storage unit comprises multiple storage cells; An input rearrangement circuit is configured to perform a first data rearrangement on multiple data elements in an input data block arranged according to a first data arrangement, to obtain a first intermediate data block arranged according to a second data arrangement. A data writing circuit is configured to write multiple valid data elements from the first intermediate data block into multiple memory cells; the valid data elements include data elements corresponding to non-hole positions in the input data block. A data reading circuit is configured to read a plurality of valid data elements from a plurality of storage cells located in the same storage row in a plurality of said storage banks; The output rearrangement circuit is configured to perform a second data rearrangement on the read multiple valid data elements to obtain an output data block arranged according to a third data arrangement.

2. The data rearrangement apparatus according to claim 1, wherein, The input rearrangement circuit includes a plurality of first multiplexers, and the plurality of first multiplexers are respectively configured to correspond to a plurality of memory banks; The first multiplexer is configured to select a first target data element from a plurality of data elements in the input data block and output the first target data element; wherein the element position of the first target data element in the first intermediate data block corresponds to the arrangement position of the first multiplexer in the input rearrangement circuit.

3. The data rearrangement apparatus according to claim 2, wherein, The first multiplexer is further configured as follows: Based on the first selection signal, determine the first element position of the first target data element in the input data block; Based on the position of the first element, the first target data element is output.

4. The data rearrangement apparatus according to claim 2, further comprising a controller, the controller being configured to: Determine the first data layout method, the size of the data to be spliced, and the second data layout method; wherein... The data to be spliced ​​includes the input data block; Based on the first data arrangement method, the size of the data to be spliced, and the second data arrangement method, determine the first spatial position mapping relationship between the first data arrangement method and the second data arrangement method, and the shift number corresponding to the multiple data elements; Based on the first spatial location mapping relationship and the shift number, the input rearrangement circuit is controlled to perform the first data rearrangement on multiple data elements in the input data block.

5. The data rearrangement apparatus according to claim 4, wherein the controller is further configured to: Based on the first data arrangement method, the size of the data to be spliced, and the second data arrangement method, a first correspondence between the element positions of multiple data elements in the input data block and the addresses of multiple storage bodies is determined; Based on the first correspondence, the first spatial location mapping relationship and the shift number are determined.

6. The data rearrangement apparatus according to any one of claims 1-5, wherein, The data writing circuit includes multiple data distributors, each of which is configured to correspond to a plurality of first multiplexers in the input rearrangement circuit; each data distributor is coupled to a plurality of memory cells in the memory bank. The data distributor is configured to write the second target data element into the target storage unit in the storage body corresponding to the data distributor in response to the output of the second target data element being a valid data element from the first multiplexer corresponding to the data distributor.

7. The data rearrangement apparatus according to claim 6, wherein, The data distributor is further configured to: Based on the second selection signal, the row address of the target storage unit in the storage bank corresponding to the data distributor is determined; Based on the write mask signal and the row address of the target storage unit in the storage bank corresponding to the data distributor, the second target data element is written to the target storage unit; the write mask signal is used to enable control of the write path of the second target data element when the second target data element is a valid data element.

8. The data rearrangement apparatus according to claim 1, wherein, The data rearrangement device further includes a controller, which is configured to: Based on the first data arrangement and the size of the data to be spliced, the location of the holes in the input data block is determined; wherein, the data to be spliced ​​includes the input data block; Based on the element positions of multiple valid data elements in the data to be spliced, the first spatial position mapping relationship, and the hole positions, the data writing circuit is controlled to write multiple valid data elements from the first intermediate data block into multiple storage units.

9. The data rearrangement apparatus according to claim 8, wherein, The controller is further configured to: Based on the mapping relationship between the element positions of multiple valid data elements in the data to be concatenated and the first spatial position, the row addresses of multiple storage volumes are determined. Based on the first spatial location mapping relationship, the hole location, and the row address of the multiple storage blocks, the data writing circuit is controlled to write multiple valid data elements from the first intermediate data block into the multiple storage blocks.

10. The data rearrangement apparatus according to any one of claims 1-5, wherein, The data reading circuit includes a plurality of second multiplexers; the plurality of second multiplexers are respectively configured corresponding to a plurality of the memory banks, and the second multiplexers are respectively coupled to a plurality of memory cells in the memory banks; The second multiplexer is configured to read the valid data element from the storage cell located in the target storage row in the storage bank corresponding to the second multiplexer and output it.

11. The data rearrangement apparatus according to claim 10, wherein, The second multiplexer is further configured as follows: Based on the third selection signal, the row address of the target storage row that has been written in the storage array is determined; Based on the row address of the target storage row, the storage cell located in the target storage row in the storage bank corresponding to the second multiplexer is selected; The valid data element is read from the storage cell located in the target storage row of the storage bank corresponding to the second multiplexer.

12. The data rearrangement apparatus according to claim 1, further comprising a controller, the controller being configured to: In response to the completion of writing to the target storage row in the storage array, the row address of the target storage row is determined; Based on the row address of the target storage row, the data reading circuit is controlled to read multiple valid data elements from multiple storage cells of the target storage row.

13. The data rearrangement apparatus according to any one of claims 1-5, wherein the output rearrangement circuit includes a plurality of third multiplexers, and the plurality of third multiplexers are respectively configured to correspond to the element positions of a plurality of data elements in the output data block; The third multiplexer is configured to select a target valid data element from the plurality of read valid data elements and output the target valid data element to the target position corresponding to the third multiplexer.

14. The data rearrangement apparatus according to claim 13, wherein the third multiplexer is further configured to: Based on the fourth selection signal and the second intermediate data block read by the data reading circuit, the position of the second element of the target valid data element in the second intermediate data block is determined; Based on the position of the second element, the target valid data element is output.

15. The data rearrangement apparatus according to claim 1, further comprising a controller, the controller being configured to: Based on the second data arrangement method and the third data arrangement method, a second spatial position mapping relationship between the second data arrangement method and the third data arrangement method is determined; Based on the second spatial location mapping relationship, the output rearrangement circuit is controlled to perform the second data rearrangement on the read multiple valid data elements.

16. The data rearrangement apparatus according to claim 15, wherein, The controller is further configured to: Based on the second data arrangement method and the third data arrangement method, a second correspondence relationship is determined between the element positions of multiple data elements in the second intermediate data block read by the data reading circuit and the element positions of multiple data elements in the output data block; Based on the second correspondence, the second spatial location mapping relationship is determined.

17. The data rearrangement apparatus according to claim 1, further comprising a controller, the controller comprising an input address generation unit and an output address generation unit; The input address generation unit is configured to generate a first access address for reading the input data block from the memory based on the size of the data to be rearranged, the first data arrangement method, the first dimension step size, the first access start address, and the first access range. The output address generation unit is configured to generate a second access address for writing the output data block into the memory based on the third data arrangement method, the second dimension step size, the second access start address, and the second access range.

18. A data rearrangement method, comprising: Based on the input data block arranged according to the first data arrangement method, multiple data elements in the input data block are rearranged in the first data arrangement to obtain a first intermediate data block arranged according to the second data arrangement method. Multiple valid data elements from the first intermediate data block are written into multiple storage units; the valid data elements include data elements corresponding to non-hole positions in the input data block. Read multiple valid data elements from multiple storage cells located in the same storage row in multiple storage banks; The multiple valid data elements read are rearranged in a second way to obtain an output data block arranged according to a third data arrangement method.

19. A neural network processor, comprising a data rearrangement device and a memory; wherein, The data rearrangement apparatus includes the data rearrangement apparatus as described in any one of claims 1-17.

20. A computer-readable storage medium storing a computer program for performing the data rearrangement method of claim 18.

21. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the data rearrangement method of claim 18.

Images