A high-efficiency data stream transceiver controller suitable for multi-core particle computing system

Abstract

The application discloses a kind of high-efficiency data stream transceiver controller suitable for multi-core particle computing system, including two sending controllers and two receiving controllers, and sending controller and receiving controller are set in the periphery of core particle, for executing the packing sending of data and data stream judgment work;Wherein, sending controller includes configurable asynchronous FIFO, data packing unit and error correction code generation unit;Receiving controller includes error correction code decoding unit, data packet unpacking unit and flit detection unit;Flit detection unit is used to detect the information in the head flit to judge whether the data meets the computing needs of the current core particle, if it does not belong to the task range of the current core particle, then the data is redirected to the sending controller through the bypass transmission mechanism.This application improves the transmission speed of data stream between different core particles, and by constructing an efficient data transceiver controller, the data stream can be dynamically planned and transmitted according to the preset rules.

Description

Technical Field

[0001] This invention belongs to the field of chip design technology, and in particular relates to a high-efficiency data stream transceiver controller suitable for multi-chip in-memory computing systems. Background Technology

[0002] Multi-chip in-memory computing systems are a novel computing architecture that combines multi-chip architecture with in-memory computing technology. By integrating multiple chiplets into a single package and optimizing on-chip networking and interconnect architecture, it achieves efficient data transmission and collaborative computing and storage processing. This architecture not only increases computing density but also effectively reduces data movement overhead and memory access bottlenecks, making it particularly suitable for fields such as artificial intelligence, image processing, and high-performance computing. Through rational topology design, optimized AXI / PCIe interconnects, and the application of network structure decomposition techniques, multi-chip in-memory computing systems can overcome the limitations of traditional von Neumann architectures and achieve more efficient utilization of computing resources.

[0003] When mapping neural networks in in-memory computing architectures, the weight matrix is ​​directly stored in IMC crossbar arrays, leveraging the characteristics of in-memory computing. This allows computational tasks to be completed efficiently within the memory unit, avoiding high-power data movement and improving computational throughput. For data flow scheduling between computing units, the architecture employs on-chip networking (NoC) and encapsulated networking (NoP) for optimization. NoC handles data transmission between computing units within a single chip, while NoP enables cross-chip communication. In current common in-memory computing architectures, cross-chip data transmission relies on independent NoP transmitter (TX) and NoP receiver (RX) circuits, as well as a dedicated NoP router to perform packet scheduling. This router uses a pre-defined routing mechanism to determine the target chip for data packets and transmits data to the TX / RX module through a dedicated port, thus completing the data packet reception and transmission operations. In some current high-efficiency interconnect architectures, such as the Floret architecture based on space-filling curves, cross-core data transmission mode adopts unidirectional data flow. By reducing unnecessary router ports and interconnect links through dual-port design, the interconnect complexity of the encapsulation layer is reduced, resource consumption is reduced, and power consumption is optimized.

[0004] Traditional hardware-level bypass transmission is a design strategy to optimize data flow. It bypasses the core processing unit through a dedicated mechanism to achieve efficient data transmission and consists of three parts: bypass buffer, bypass switching, and bypass routing. The bypass buffer sets up an independent buffer on the data path, allowing data to be transmitted directly without passing through the main processing unit. The bypass switching operates within the on-chip network, allowing data to be transmitted directly from the input port to the destination port without passing through intermediate nodes. The bypass routing can bypass congested nodes and adjust the data flow path.

[0005] In the Floret in-memory architecture described above, when mapping neural networks, a problem arises where a single kernel cannot store the next convolutional node. Considering the maximum parameter payload that a kernel can store, where each kernel has 16 tiles, each tile has 40 physical parameters (PEs), and each PE has 128×128 ReRAM cells, let's consider mapping all parameters of a convolutional node with 512 input and 512 output channels and a 3×3 kernel size to this kernel. After quantization, each parameter is 8 bits. Calculations show that this node requires 1.8 kernels to store the data. While the data flow between kernels requires more bandwidth, it is still manageable. However, when mapping cross-layer networks (such as ResNet residual networks), the data flow within a single residual block is more complex. If unidirectional transmission is still used as in this architecture, some data will be transmitted excessively and unnecessarily in kernels that are not needed by the network.

[0006] In traditional bypass transmission, bypass buffers and bypass switching typically rely on predefined paths, lacking dynamic adjustment capabilities and struggling to adapt to complex traffic changes. Furthermore, under high load, they can still cause local bottlenecks, impacting overall throughput. In addition, while bypass mechanisms reduce data processing overhead, they still require additional hardware resources, such as dedicated buffers and switching logic. Summary of the Invention

[0007] The purpose of this invention is to solve the problem that the bypass buffer and bypass switching in traditional bypass transmission usually rely on predefined paths and lack dynamic adjustment capabilities, and to propose an efficient data stream transceiver controller suitable for multi-core memory computing systems.

[0008] This application provides a high-efficiency data stream transceiver controller suitable for multi-core in-memory computing systems, comprising multiple cores arranged in a chessboard pattern; each of the four sides of each core is respectively provided with a sending controller or a receiving controller; each core includes two sending controllers and two receiving controllers, and the sending controllers and receiving controllers are all located on the periphery of the corresponding core, used to perform data packaging and sending and data stream judgment.

[0009] A bypass transmission channel is set between adjacent transmitting or receiving controllers of the same core chip;

[0010] A bypass transmission channel is set between adjacent transmitting or receiving controllers of different core chips;

[0011] The transmitting controller includes a configurable asynchronous FIFO, a data packing unit, and an error correction code generation unit. The configurable asynchronous FIFO is used to store and buffer the transmitted data stream. The data packing unit is used to pack the stored data chips into data packets. The error correction code generation unit is used to calculate the checksum of the data stream.

[0012] The receiving controller includes an error correction code decoding unit, a data packet unpacking unit, and a chip detection unit. The error correction code decoding unit is used to detect and correct errors in the data stream transmission. The data packet unpacking unit is used to split the data packet into chip data streams. The chip detection unit is used to detect the information in the header chip to determine whether the data meets the computational requirements of the current chip. If it does not belong to the task scope of this chip, the data is redirected to the sending controller through a bypass transmission mechanism.

[0013] Furthermore, the microchip data stream includes a chip ID (CID), a current chip ID (NCID), a source node chip identifier (SCID), a destination node chip identifier (DCID), a source node router identifier (SRID), and a destination node router identifier (DRID).

[0014] Furthermore, the transmitting controller and the receiving controller exchange and forward data through a bypass transmission channel;

[0015] Specifically, the transmitting controller transmits data to the receiving controller adjacent to it in the same core through a bypass transmission channel, or to the receiving controller adjacent to it in an adjacent core that is also equipped with a bypass transmission channel.

[0016] Furthermore, when the input data stream data_in is transmitted to the first receiving controller, the first controller detects the relevant information of the received data packet according to the predefined parsing protocol, processes it according to the error correction code, and splits the data packet information that is verified to be transmitted without error after decoding into multiple micro data streams according to preset conditions; wherein, the first receiving controller is the controller that is currently receiving data among the two receiving controllers;

[0017] After detecting a head microchip, it determines whether the microchip information carried in the data is valid data required for the current chip to be calculated; if the microchip information in the data packet matches the calculation requirements of the corresponding chip, the first receiving controller directly transmits the valid data to the corresponding chip for calculation processing, until the next head microchip is detected;

[0018] If the microchip information contained in the data is not within the calculation range of the current chip, the first receiving controller will redirect it to the adjacent first transmitting controller for buffering through the bypass transmitter, and wait to send it.

[0019] Subsequently, the first transmitting controller continues to redistribute the data, packaging multiple microchip data streams into a complete data packet, then encrypting the data packet using ECC encoding, and transmitting it to the target chip.

[0020] Furthermore, if the receiving controller determines that the data does not belong to the scope of this core task, it redirects the data to the buffer unit of the transmitting controller for temporary storage, instead of storing it through a dedicated bypass buffer.

[0021] Furthermore, after parsing the data in the first receiving controller of the first core, it detects that the second data data_2 does not belong to the current core calculation task. It forwards the data to the adjacent first transmitting controller through the bypass transmitter. The first transmitting controller repackages the data stream according to predefined logic and transmits it to the second receiving controller of the second core; wherein the first transmitting controller of the first core and the second receiving controller of the second core are adjacent.

[0022] At the same time, the first data data_1 is detected to belong to the computing task of the chip. The first receiving controller of the first chip directly transmits the first data to the in-memory computing on-chip network for computing. After the computing is completed, the result is transmitted to the second transmitting controller of the first chip. After packaging, the output data stream data_out is transmitted to the next chip again to complete the overall computing and interaction of the data stream.

[0023] Furthermore, each chip independently performs data bypass transmission, achieving data forwarding through multiple bypasses.

[0024] Furthermore, the transmitting controller and the receiving controller correspond to different directions of the core, with the transmitting controller on the two adjacent sides of the core and the receiving controller on the other two sides.

[0025] The significant advancement of this invention compared to existing technologies lies in:

[0026] 1. Improved data transmission speed between different cores: By constructing an efficient data transceiver controller, the data stream can be dynamically planned and transmitted according to preset rules. This control mechanism, through the introduction of core ID identifiers, ensures continuous comparison and verification of the data stream during transmission, optimizing the data stream scheduling process and ensuring fast and accurate data flow between cores, thereby improving the overall system throughput. Furthermore, a reasonable scheduling strategy can reduce congestion during data transmission, enabling the data stream to remain efficient and stable in a multi-core architecture.

[0027] 2. Optimize invalid data transmission in traditional architectures by introducing a processing unit into the data receiving controller. This unit determines the validity of the data stream in real time, preventing useless data from entering the core for unnecessary storage and computation. Compared to traditional architectures, this improvement enables more accurate data transmission to the target module, avoiding performance degradation caused by redundant data entering the processing unit.

[0028] 3. By optimizing the bypass buffer resources, the resources occupied by the bypass buffer are reduced by reusing the buffer space in the data transmission controller. Furthermore, instead of using algorithms for path selection, data bypass transmission is performed separately in each core, achieving a simpler data forwarding function through multiple bypasses and avoiding the complex path calculations of traditional bypass transmission.

[0029] To more clearly illustrate the functional characteristics and structural parameters of the present invention, further explanation is provided below in conjunction with the accompanying drawings and specific embodiments. Attached Figure Description

[0030] Figure 1 This is a diagram illustrating the connection methods between NoP and NoC in a common in-memory computing architecture provided in the embodiments of this application;

[0031] Figure 2 This is a schematic diagram of the cross-chip interconnection method in the Floret scheme provided in the embodiments of this application;

[0032] Figure 3 This is a structural diagram of the data stream transceiver controller provided in an embodiment of this application;

[0033] Figure 4 This is a comparison diagram of the bypass transmission schemes provided in the embodiments of this application;

[0034] Figure 5 This is a data flow transmission diagram provided in the embodiments of this application;

[0035] Figure 6 This is an implementation flowchart provided in the embodiments of this application. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] This application replaces the NoP (No-Point) receive / transmit controller within the original core chip architecture. When data streams are transmitted to the core chip's receive controller, they are first detected. Data not processed by that core chip is sent to the data transmit controller via a bypass transmitter, ensuring that data streams not needed by that core chip are not transmitted to the core chip for useless transmission, thus achieving efficient data stream processing within the core chip. Furthermore, the traditional bypass transmission method is modified to better adapt to this data transceiver controller.

[0038] This invention provides a high-efficiency data stream transceiver controller suitable for in-memory computing systems. By adding a control architecture around the core, which performs data packetization and transmission as well as data stream judgment, it ensures that data streams not needed by the core are not transmitted to the core for useless transmission, but are instead delivered to the core where the data stream should be processed via an external bypass transmitter.

[0039] This transceiver controller is essentially an improvement on the NoP's TX / RX, therefore requiring a new definition of the transmitted data stream slices to ensure compatibility with the transceiver controller during slice data stream transmission. The transmitted data packets consist of a header slice, a body slice, and a tail slice. The header slice stores ID information related to data transmission and routing, the body slice stores the valid data payload, and the tail slice is used only to mark the end position of the data packet.

[0040] In this invention, data transmission across chiplets is included. Therefore, the single source node ID and destination node ID within the header chip in the traditional scheme cannot fully satisfy the start and end positions of data transmission. So the header chiplet includes a chiplet ID (CID) to represent different chiplets; and includes a current chiplet ID (NCID) to ensure the implementation of data transmission, which is used to represent the ID of the current chiplet where the data packet is located; SCID (Source Chiplet ID) and DCID (Destination Chiplet ID) are used to represent the chiplet identifiers of the source node and destination node of the data packet, and SRID (Source Route ID) and DRID (Destination Route ID) are used to represent the router identifiers of the source node and destination node of the data packet.

[0041] The transmitting controller is responsible for packaging and encoding error correction codes into the data within the core, facilitating subsequent data transmission between cores. The transmitting controller's functional components include: a first part is a configurable asynchronous FIFO used to store and buffer the transmitted data stream, enabling smooth data exchange during communication at different rates; a second part is a data packaging unit responsible for packaging the stored data chips into a data packet according to a preset format for subsequent transmission; and a third part is an error correction code generation unit that calculates checksums on the data stream to ensure data integrity and error detection capabilities during cross-core transmission.

[0042] The receiving controller is responsible for decoding error correction codes, unpacking data packets, and detecting data chips transmitted in the core, ensuring correct transmission of the data stream between cores. The functional components of the receiving controller include: first, an error correction code decoding unit, which detects and corrects errors during data transmission; second, a data packet unpacking unit, which splits transmitted data packets into multiple chip data streams, enabling them to be used correctly by subsequent modules; and third, a chip detection unit, which inspects the information in the header chip, comparing the destination core ID with the current core ID to determine if the data meets the computational requirements of the current core. If the corresponding chip data stream does not belong to the task scope of this core, the receiving controller redirects the data to the transmitting controller via a bypass transmission mechanism and stores it in a buffer unit, awaiting subsequent transmission and allocation.

[0043] In this embodiment, the North and West controllers are defined as data receiving controllers, and the South and East controllers are defined as data sending controllers; this is only one implementation. It should be noted that in actual implementation, it is sufficient to position the controllers on two adjacent sides as data receiving controllers and the other two sides as data sending controllers, and the specific directions need to be defined. Data interaction and forwarding between controllers are achieved through a bypass transmission channel to ensure better data flow transmission at the overall encapsulation level (such as in a mesh structure).

[0044] In traditional bypass transmission schemes, after the data stream undergoes decoding and unpacking, it is judged as a micro-data stream. Only after this judgment is made will bypass transmission be performed. During bypass transmission, data requiring bypass is stored in a bypass buffer. After buffering is complete, bypass routing judgment and data bypass transmission will then occur. In the bypass transmission design of this invention, the data stream no longer relies on a traditional dedicated bypass buffer for storage. Instead, after the micro-data stream is judged, it is directly sent to the next data transmission controller and temporarily stored in the transmission controller's buffer unit. Therefore, bypass transmission does not require additional storage units. This design reduces the resource consumption of the bypass buffer and optimizes data stream storage management. Compared to the traditional mechanism of bypass transmission throughout the entire network, this design adopts a strategy of independent data bypass transmission for each chip. This method allows data to undergo finer-grained bypass processing within the chip, achieving efficient data forwarding through multiple bypasses and simplifying data stream path planning.

[0045] The following section details a data stream transmission example from this application.

[0046] Taking the input of the data_in data stream as an example, the method of the present invention will be described in detail below.

[0047] When the input data stream `data_in` is transmitted to the western controller, the western controller detects relevant information in the received data packet according to a predefined parsing protocol, processes it using ECC (Error Correction Code), and splits the data packet information that has been verified to be transmitted without errors into multiple micro-data streams based on preset conditions. Subsequently, the header information of each micro-data stream is analyzed. If a header micro-data stream is detected, it is determined whether the micro-data stream information carried in the data is valid data required for the current core particle's calculation. If the micro-data stream information in the data packet matches the calculation requirements of the current core particle, the western controller will directly transmit the valid data to the corresponding core particle for calculation processing until the next header micro-data stream is detected.

[0048] If the micro-chip information contained in the data does not fall within the computational scope of the corresponding chip, the western controller will redirect it to the southern controller for buffering via a bypass transmitter, awaiting transmission. Subsequently, the southern controller continues to redistribute the data, packaging multiple micro-chip data streams into a complete data packet, then encoding and encrypting the data packet using ECC encoding before transmitting it to the target chip, ensuring that the data is delivered to the chip resource that actually needs to perform the computation.

[0049] Taking the input data stream `data_in` as an example, after parsing by the west controller of chip 1, it detects that `data_2` does not belong to the current chip's computing task. It is then precisely forwarded to the south controller via a bypass transmitter (the data stream corresponding to the black arrow and black tail wing in the diagram). The south controller repackages the data stream according to predefined logic and transmits it to the north controller of chip 2, achieving cross-chip data communication and collaborative computing. Simultaneously, it detects that `data_1` belongs to the chip's computing task. The west controller directly transmits this data to the in-memory computing on-chip network for computation (the data stream corresponding to the black arrow and white tail wing in the diagram). After computation, the result is transmitted to the east controller of chip 1. After packaging, the output data stream `data_out` is transmitted again to the next chip, completing the overall computation and interaction of the data stream.

[0050] The key point of this invention lies in achieving efficient data stream transmission across memory chips by constructing a data stream receiving and transmitting controller around the chip, thus realizing efficient data stream transmission across memory chips in the in-memory computing system. It has the following advantages:

[0051] An efficient data stream transmission and reception control mechanism optimizes data stream scheduling and improves the transmission speed between cores by constructing a dynamically planned data transmission and reception controller.

[0052] A valid data stream judgment unit is introduced into the data receiving controller to filter the data stream, prevent invalid data from entering the core, and thus reduce unnecessary storage and computing overhead.

[0053] The optimization of bypass transmission reduces the resource consumption of the bypass buffer by reusing the buffer space, and replaces complex path calculation with local bypass transmission, making the data forwarding process simpler and more efficient.

[0054] This optimization scheme improves the computational efficiency between cores and reduces resource consumption through key strategies such as data stream parsing and splitting, intelligent bypass transmission, and data stream redistribution. At the same time, it enhances the collaborative capability of cross-core data streams and builds an efficient data stream transceiver controller.

[0055] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A high-efficiency data stream transceiver controller suitable for multi-core in-memory computing systems, characterized in that, It includes multiple cores arranged in a chessboard pattern; each of the four sides of each core is respectively set with a sending controller or a receiving controller; each core includes two sending controllers and two receiving controllers, and the sending controllers and receiving controllers are set on the periphery of the corresponding core to perform data packaging and sending and data stream judgment. A bypass transmission channel is set between adjacent transmitting or receiving controllers of the same core chip; A bypass transmission channel is set between adjacent transmitting or receiving controllers of different core chips; The transmitting controller includes a configurable asynchronous FIFO, a data packing unit, and an error correction code generation unit. The configurable asynchronous FIFO is used to store and buffer the transmitted data stream. The data packing unit is used to pack the stored data chips into data packets. The error correction code generation unit is used to calculate the checksum of the data stream. The receiving controller includes an error correction code decoding unit, a data packet unpacking unit, and a chip detection unit. The error correction code decoding unit is used to detect and correct errors in the data stream transmission. The data packet unpacking unit is used to split the data packet into chip data streams. The chip detection unit is used to detect the information in the header chip to determine whether the data meets the computational requirements of the current chip. If it does not belong to the task scope of this chip, the data is redirected to the sending controller through a bypass transmission mechanism. If the information in the header chip matches the computational requirements of the current chip, the valid data is directly transmitted to the corresponding chip until the next header chip is detected.

2. The high-efficiency data stream transceiver controller according to claim 1, characterized in that, The microchip data stream includes chip ID (CID), current chip ID (NCID), source node chip identifier (SCID), destination node chip identifier (DCID), source node router identifier (SRID), and destination node router identifier (DRID).

3. The high-efficiency data stream transceiver controller according to claim 1, characterized in that, The transmitting controller and the receiving controller exchange and forward data through a bypass transmission channel; Specifically, the transmitting controller transmits data to the receiving controller adjacent to it in the same core through a bypass transmission channel, or to the receiving controller adjacent to it in an adjacent core that is also equipped with a bypass transmission channel.

4. The high-efficiency data stream transceiver controller according to claim 3, characterized in that, When the input data stream data_in is transmitted to the first receiving controller, the first controller detects the relevant information of the received data packet according to the predefined parsing protocol, processes it according to the error correction code, and splits the data packet information that is verified to be transmitted without error after decoding into multiple micro data streams according to preset conditions; among them, the first receiving controller is the controller that is currently receiving data among the two receiving controllers; After detecting a head microchip, it determines whether the microchip information carried in the data is valid data required for the current chip to be calculated; if the microchip information in the data packet matches the calculation requirements of the corresponding chip, the first receiving controller directly transmits the valid data to the corresponding chip for calculation processing, until the next head microchip is detected; If the microchip information contained in the data is not within the calculation range of the current chip, the first receiving controller will redirect the data to the adjacent first transmitting controller for buffering through the bypass transmitter, and wait to be transmitted. Subsequently, the first transmitting controller continues to redistribute the data, packaging multiple microchip data streams into a complete data packet, then encrypting the data packet using ECC encoding, and transmitting it to the target chip.

5. The high-efficiency data stream transceiver controller according to claim 1, characterized in that, If the receiving controller determines that the data does not belong to the scope of this core's task, it will redirect the data to the transmitting controller's buffer unit for temporary storage, instead of storing it through a dedicated bypass buffer.

6. The high-efficiency data stream transceiver controller according to claim 5, characterized in that, After parsing the data stream, the first receiving controller of the first core detects that the second data data_2 does not belong to the current core's computation task. It then forwards the data stream to the adjacent first transmitting controller via a bypass transmitter. The first transmitting controller repackages the data stream according to predefined logic and transmits it to the second receiving controller of the second core. The first transmitting controller of the first core and the second receiving controller of the second core are adjacent to each other. At the same time, the first data data_1 is detected to belong to the computing task of the chip. The first receiving controller of the first chip directly transmits the first data to the in-memory computing on-chip network for computing. After the computing is completed, the result is transmitted to the second transmitting controller of the first chip. After packaging, the output data stream data_out is transmitted to the next chip again to complete the overall computing and interaction of the data stream.

7. The high-efficiency data stream transceiver controller according to claim 1, characterized in that, Each chip independently performs data bypass transmission, and data forwarding is achieved through multiple bypasses.

8. The high-efficiency data stream transceiver controller according to claim 1, characterized in that, The transmitting controller and receiving controller correspond to different directions of the core, with the transmitting controller on the two adjacent sides of the core and the receiving controller on the other two sides.

Images