High-parallel three-dimensional array data recombination module and method supporting multiple operations

Abstract

The invention relates to a high-parallel three-dimensional array data recombination module and method supporting multiple operations. The data recombination module comprises an input processing module, a write address generation module, an RAM storage array, a read address generation module and an output processing module. And four three-dimensional array transformation operations of slicing, merging, compressing and transposing are realized. According to the module, a 512-bit wide data bus is adopted to be matched with 32 parallel RAM channels, and assembly line processing is achieved in combination with two ping-pong RAM sets. By means of rearrangement, displacement and address generation logic which are finely designed, efficient data reading and writing of 32-byte alignment are kept, and meanwhile flexible dimension transformation with one byte as the minimum unit is supported. The problems of low conversion efficiency and high power consumption of a traditional CPU are effectively solved, data conversion delay and power consumption are reduced by dozens of times, and an efficient data recombination solution is provided for high-performance computing application such as a three-dimensional convolutional neural network.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a highly parallel three-dimensional array data reassembly module and method that supports multiple operations. Background Technology

[0002] With the rapid development of the Internet and 5G communication technologies and the advent of the big data era, 3D convolutional neural networks have been widely used in video analysis, medical image processing, point cloud recognition, and other fields. Compared with 2D convolutional neural networks, 3D convolutional neural networks can directly process data with three dimensions: length, width, and number of channels. Through their 3D convolutional kernels, they directly and explicitly learn the spatiotemporal characteristics of 3D data simultaneously, thereby reflecting objective objects more realistically and comprehensively and providing a greater amount of information.

[0003] Three-dimensional convolutional neural networks are typically composed of multiple operators with different functions (such as convolution, pooling, and activation) strung together. Different operators, due to their varying algorithmic principles, have different requirements regarding the dimensional order and memory layout of the input data. For example, a convolutional layer might require data to be arranged in [height, width, channels] (HWC), while a fully connected layer might require data to be arranged in [channels, height, width] (CHW). Therefore, during network inference, data needs to undergo frequent dimensional transformations, i.e., data recombination, when transferred between different operators.

[0004] Currently, the main methods for reorganizing three-dimensional array data include: CPU transformation (low energy efficiency, slow processing speed, and high latency), dedicated software library transformation (additional software layer overhead), and module transformation (there is a contradiction between alignment requirements and flexible transformation).

[0005] In summary, there is an urgent need in this field for a three-dimensional array transformation module design that can effectively expand the parallelism of data processing while taking into account high bandwidth utilization and flexible transformation capabilities, so as to overcome the performance and energy efficiency bottlenecks of existing technologies in high-performance computing scenarios. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a highly parallel three-dimensional array data reassembly module that supports multiple operations. Through architectural design, it enables dimensional transformation of three-dimensional array data in 1-byte units while maintaining a 32-byte aligned data path between the input and output ends. The architecture includes:

[0007] The input processing module is used to receive a 512-bit input data stream and perform mode-related rearrangement and circular left shift on the input data according to the selected slicing, merging, compression or transposition operation mode.

[0008] The write address generation module is used to generate a set of 32 identical write addresses based on the operation mode;

[0009] The RAM storage array includes two sets of ping-pong RAM groups, each consisting of 32 RAM cells with a depth of 32 and a bit width of 16 bits. Under the write address control, the 512-bit data after input processing is written as a whole to the 32 RAM cells in parallel within a single clock cycle.

[0010] The read address generation module is used to generate a set of 32 read addresses based on the operation mode;

[0011] The output processing module is used to read 512 bits of data in parallel from the RAM storage array, perform a circular right shift and rearrangement operation according to the operation mode, and output the recombined data.

[0012] In one embodiment of the present invention, the data rearrangement operation of the input processing module varies depending on the different recombination modes: for the slicing operation, the 512-bit data is divided into two 256-bit parts and arranged in an interleaved manner in 8-bit units.

[0013] For some modes of transpose operation, the 512-bit data is divided into two 256-bit parts, and then rearranged by specified swapping in 16-bit units.

[0014] In one embodiment of the present invention, the cyclic left shift operation of the input processing module varies depending on the different recombination modes: for compression operations, the shift depends on a specific bit of the write address; for other operations, the shift amount is a linear function of the write address value.

[0015] In one embodiment of the present invention, the write address sequence generated by the write address generation module depends on the transformation operation type:

[0016] For transpose operations in length and width transpose mode, the write address is always 0x00;

[0017] For the width and number of channels of the transpose operation, the write address changes cyclically from 0x00 to 0x0F.

[0018] For slicing, merging, and compressing operations, the write address cycles through 0x00 to 0x1F.

[0019] In one embodiment of the present invention, the RAM storage array adopts a multi-port RAM, which supports concurrent read and write operations, and the two groups of RAM work in ping-pong mode to realize pipelined data processing.

[0020] In one embodiment of the present invention, the read address sequence generated by the read address generation module is determined according to the type of transformation operation: for the length and width transpose mode of the transpose operation, the read address is always 0x00; for other operations, the read addresses of the 32 RAMs are different.

[0021] In one embodiment of the present invention, the cyclic right shift operation of the output processing module is determined according to different recombination modes and the number of read operation sequences y, which increases cyclically with the read operation, and the shift amount is a function of y or depends on a specific bit of y.

[0022] In one embodiment of the present invention, the output processing module performs a rearrangement operation by dividing a 512-bit data block into four 128-bit data blocks and recombining them in a predetermined order of {data11,data01,data10,data00}.

[0023] In one embodiment of the present invention, the precision of the three-dimensional array data elements processed by the module is 8 bits, and the 512-bit data bus width matches 32 RAM channels, realizing a fully parallel operation of processing two 32-byte data per clock cycle.

[0024] The present invention also provides a three-dimensional array data recombination method, applied to the aforementioned data recombination module, comprising the following steps:

[0025] Step S1: Receive 512-bit input data aligned to 32 bytes;

[0026] Step S2: Rearrange and cyclically shift the input data to the left according to the specified recombination operation mode;

[0027] Step S3: Write the processed data into the Ping-Pong RAM group in parallel according to the generated write address;

[0028] Step S4: Read data from the RAM group in parallel according to the generated read address;

[0029] Step S5: Perform a circular right shift and rearrangement on the read data;

[0030] Step S6: Output 512-bit data that is aligned to 32 bytes and has undergone dimension reorganization.

[0031] Compared with the prior art, the above-mentioned technical solution of the present invention has the following advantages: The high-parallel three-dimensional array data reorganization module of the present invention achieves fully parallel operation of processing 64 bytes of data per clock cycle through precise matching of a 512-bit wide data bus and 32 parallel RAM channels, greatly improving data throughput. Simultaneously, it innovatively adopts a two-group ping-pong RAM architecture to realize pipelined processing of read and write operations, completely eliminating access conflicts and ensuring the continuity of data processing. Through meticulously designed rearrangement, shifting, and address generation logic, it successfully resolves the contradiction between 32-byte aligned read / write and 1-byte fine-grained transformation, achieving flexible dimensional reorganization while ensuring high bandwidth utilization. Furthermore, the dedicated hardware architecture concentrates energy consumption on effective computation, achieving a significant improvement of tens of times in power consumption and latency compared to CPU solutions, resulting in excellent energy efficiency. Attached Figure Description

[0032] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0033] Figure 1 This is a functional block diagram of the high-parallel three-dimensional array data reassembly module of the present invention;

[0034] Figure 2 This is a flowchart of the array data recombination and transformation of the present invention;

[0035] Figure 3 This is a schematic diagram illustrating the "slicing" and "merging" operations of the three-dimensional array described in this invention;

[0036] Figure 4 This is a schematic diagram of the three-dimensional array "compression" operation described in this invention;

[0037] Figure 5 This is a schematic diagram of the three-dimensional array "transpose" operation described in this invention;

[0038] Figure 6 This is a schematic diagram illustrating the address writing process for the "slicing" operation described in this invention.

[0039] Figure 7 This is a schematic diagram of the RAM data storage format for the "slicing" operation described in this invention;

[0040] Figure 8 This is a schematic diagram of the address reading operation of the "slice" described in this invention. Detailed Implementation

[0041] like Figure 1As shown, this embodiment provides a highly parallel three-dimensional array data reassembly module that supports multiple operations. The data reassembly module mainly includes five core parts: an input processing module, a write address generation module, a RAM storage array, a read address generation module, and an output processing module. The module's data path width is 512 bits, which corresponds exactly to two 32-byte arrays, totaling 64 bytes, to meet the 32-byte alignment requirements of external memory for reading and writing. The processed data element precision is 8 bits (1 byte), thus achieving transformation with 1 byte as the smallest unit.

[0042] Figure 2 This document presents the overall flowchart of the data recombination and transformation process in this module. The basic workflow is as follows: After preprocessing involving rearrangement and cyclic left shift, the input data is written to the RAM storage array according to its address. When the data accumulates to a certain level, it is read from the RAM according to the generated read address, and then post-processed by cyclic right shift and rearrangement to finally output the transformed data.

[0043] Specifically, the detailed working process of each module includes:

[0044] (a) Input processing module; such as Figure 2 As shown in the "Input Processing" section, this module is responsible for preprocessing the input 512-bit data. Specific operations include rearranging and circular left shifting, the specific modes of which are determined by the currently configured transformation operation (slicing, merging, compressing, transposing).

[0045] Rearrange (using "slice" and "transpose" operations as examples):

[0046] Combination Figure 7 The RAM data storage format diagram for the "slicing" operation: For the "slicing" operation, the input processing module rearranges a 512-bit input data (e.g., data = {1f..0100(01), 1f..0100(00)}) into {1f(01), 1f(00), ..., 00(01), 00(00)}. This process interleaves two 256-bit data blocks in 8-bit units to prepare for subsequent storage in RAM.

[0047] For certain patterns of the "transpose" operation, the rearrangement rules are different, involving the exchange of 16-bit units.

[0048] Circular left shift:

[0049] Shift operations are closely related to writing to address x. For example... Figure 6The diagram illustrates the write address for the "slice" operation, where the write address x cycles from 0x00 to 0x1F. For the "slice" operation, the data needs to be cyclically shifted left by x*16 bits. For example, when x = 0x1F, the data will undergo a corresponding shift, as shown below. Figure 7 As shown in the thirty-second input data, it becomes {00(3f),00(3e),...,01(3f),01(3e)}.

[0050] (ii) Write address generation module and RAM storage array; the processed input data is sent to the RAM storage array. For example... Figure 1 As shown, the RAM storage array consists of two sets of ping-pong RAM groups. This design ensures that read and write operations can be performed in parallel without conflict, enabling pipelined processing.

[0051] The write address generation module generates a set of 32 identical write addresses based on the transformation mode. For example... Figure 6 As shown, for the "slice" operation, the write address cycles from 0x00 to 0x1F. On the rising edge of the clock, the processed and shifted 512-bit data is split into 32 16-bit data segments and written in parallel to the corresponding addresses of the 32 currently active RAM segments. Figure 7 It clearly demonstrates how the rearranged and shifted data is distributed and stored in 32 RAMs according to the write address x.

[0052] (III) Read Address Generation Module and RAM Storage Array Reading; When the data stored in RAM meets specific conditions, the read address generation module starts working. For example... Figure 8 As shown in the diagram of the "slicing" operation read address, this module generates a set of 32 read addresses, each of which can be different. For the "slicing" operation, the read addresses of the 32 RAM cells are a differentiated sequence. On the rising edge of the clock, each of the 32 RAM cells simultaneously outputs a 16-bit data according to its respective read address. These data are then reassembled internally into a 512-bit data block. Figure 7 and Figure 8 As can be seen from the correspondence, only through this specific read address sequence can the correct combination of data that meets the requirements of the "slice" operation be read from RAM.

[0053] (iv) Output Processing Module; The 512-bit data read from RAM is sent to the output processing module for post-processing, such as... Figure 2 The "output processing" section includes circular right shift and rearrangement.

[0054] Circular Right Shift: According to the reassembly method, the output processing module maintains an independent read operation sequence counter y. Each time a 512-bit data read operation from the RAM bank is completed (i.e., the read address is applied once), the value of counter y is automatically incremented by 1. The value of y cyclically changes under different operating modes (e.g., incrementing cyclically from 0 to 31) and serves as a key parameter controlling the circular right shift operation. Taking the "slice" operation as an example, the data needs to be cyclically shifted right by y*16 bits. After the shift, the data is corrected to the correct byte alignment.

[0055] Rearrangement: This is a general rearrangement step. For a 512-bit data, the module divides it into four 128-bit data blocks and concatenates them in the order {data11, data01, data10, data00} to form output data that meets the target dimension requirements.

[0056] By combining Figures 3-8 The detailed schematic diagrams above fully illustrate the entire process of data from input, internal reorganization, storage, retrieval to final output. The high-parallelism three-dimensional array data reorganization module described in this embodiment, through its innovative 512-bit wide parallel architecture, ping-pong RAM buffering mechanism, and address generation and data rearrangement logic strongly correlated with the operation mode, successfully achieves flexible dimensional transformation of the three-dimensional array with 1-byte as the smallest unit while maintaining efficient 32-byte aligned read / write.

[0057] Therefore, with Figure 3 , Figure 6 , Figure 7 and Figure 8 Taking the "slicing" operation shown as an example, a three-dimensional array with dimensions W=64, H=2, and C=32 is divided into 32 planes. The module efficiently completes this transformation through the above process, particularly through input rearrangement / shifting, specific write / read address sequences in RAM, and output shifting. The final output data format is completely consistent with the theoretical "slicing" result.

[0058] Compared to CPU-executed transformations, this embodiment offers a revolutionary performance advantage:

[0059] Time performance: CPUs need to break down tasks into a large number of instructions and are limited by serial architecture and inefficient memory access. This invention generates efficient address sequences through dedicated hardware, enabling batch, continuous data access and fully parallel internal processing, reducing transformation latency by tens of times.

[0060] Power consumption performance: A significant portion of the CPU's power consumption is consumed by control logic and inefficient memory accesses. In this embodiment, almost all energy is concentrated on data transformation itself. The control logic is simple, and memory access efficiency is maximized through ping-pong RAM and a prefetch mechanism, resulting in a significant improvement in energy efficiency.

[0061] In summary, this invention, through its innovative hardware architecture and refined recombination method, successfully resolves the contradiction between high-bandwidth aligned read / write and fine-grained flexible transformation, providing key technical support for high-performance 3D convolutional neural network inference, and demonstrating significant advantages in large-scale and regularized array transformations.

[0062] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A highly parallel three-dimensional array data reassembly module supporting multiple operations, characterized in that, By designing an architecture, we can perform dimensional transformations on three-dimensional array data with 1 byte as the smallest unit, while maintaining a data path that is aligned to 32 bytes between the input and output ends. The architecture includes: The input processing module is used to receive a 512-bit input data stream and perform mode-related rearrangement and circular left shift on the input data according to the selected slicing, merging, compression or transposition operation mode. The write address generation module is used to generate a set of 32 identical write addresses based on the operation mode; The RAM storage array includes two sets of ping-pong RAM groups, each consisting of 32 RAM cells with a depth of 32 and a bit width of 16 bits. Under the write address control, the 512-bit data after input processing is written as a whole to the 32 RAM cells in parallel within a single clock cycle. The read address generation module is used to generate a set of 32 read addresses based on the operation mode; The output processing module is used to read 512 bits of data in parallel from the RAM storage array, perform a circular right shift and rearrangement operation according to the operation mode, and output the recombined data.

2. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The input processing module performs different data rearrangement operations depending on the different recombination modes: for slicing operations, the 512-bit data is divided into two 256-bit parts and arranged in an interleaved manner in 8-bit units; For some modes of transpose operation, the 512-bit data is divided into two 256-bit parts, and then rearranged by specified swapping in 16-bit units.

3. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The cyclic left shift operation of the input processing module varies depending on the different recombination modes: for compression operations, the shift depends on a specific bit of the write address; for other operations, the shift amount is a linear function of the write address value.

4. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The write address sequence generated by the write address generation module depends on the type of transformation operation: For transpose operations in length and width transpose mode, the write address is always 0x00; For the width and number of channels of the transpose operation, the write address changes cyclically from 0x00 to 0x0F. For slicing, merging, and compressing operations, the write address cycles through 0x00 to 0x1F.

5. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The RAM storage array uses multi-port RAM, supports concurrent read and write operations, and the two RAM groups work in ping-pong mode to realize pipelined data processing.

6. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The read address generation module generates a read address sequence that depends on the type of transformation operation: for the length and width transpose mode of the transpose operation, the read address is always 0x00; for other operations, the read addresses of the 32 RAMs are different.

7. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The cyclic right shift operation of the output processing module is determined based on different recombination modes and the number of read operation sequences y. y increases cyclically with the read operation, and the shift amount is a function of y or depends on a specific bit of y.

8. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The output processing module's rearrangement operation divides a 512-bit data block into four 128-bit data blocks and reassembles them in a predetermined order of {data11,data01,data10,data00}.

9. The high-parallel three-dimensional array data reassembly module according to claim 1, characterized in that: The module processes 3D array data elements with a precision of 8 bits. The 512-bit data bus width matches 32 RAM channels, enabling fully parallel operation of processing two 32-byte data units per clock cycle.

10. A three-dimensional array data reconstruction method, applied to the data reconstruction module as described in any one of claims 1 to 9, characterized in that, Includes the following steps: Step S1: Receive 512-bit input data aligned to 32 bytes; Step S2: Rearrange and cyclically shift the input data to the left according to the specified recombination operation mode; Step S3: Write the processed data into the Ping-Pong RAM group in parallel according to the generated write address; Step S4: Read data from the RAM group in parallel according to the generated read address; Step S5: Perform a circular right shift and rearrangement on the read data; Step S6: Output 512-bit data that is aligned to 32 bytes and has undergone dimension reorganization.

Images