Data transmission device, data processing system, data processing method and medium

Abstract

According to an exemplary embodiment of the present disclosure, a data transmission device is provided. The data transmission device includes a plurality of first ports coupled to a processing unit; a plurality of second ports coupled to a plurality of memories; and a plurality of data channels arranged between the first ports and the second ports to form an interleaved network having a plurality of layers for transmitting data between the processing unit and the plurality of memories, such that each layer of the interleaved network includes at least one sub-interleaved network. By adopting the technology of multi-channel and hierarchical memory interleaving, the hardware overhead is effectively reduced, and the scalability and flexibility are improved. Since it is divided into a plurality of sub-interleaved networks, the data channels that need to take long paths are significantly reduced, and the physical implementation is easier. This is beneficial to power consumption, data transmission delay, and stability. The data transmission technology of the hierarchical interleaved network can more easily implement the layout of a symmetric structure, so that each module in the data processing system can be more easily arranged.

Description

Technical Field

[0001] Embodiments of the present disclosure mainly relate to the technical field of data transmission, and more specifically, to a data transmission device, a data processing system, and a method. Background Art

[0002] Currently, with the wide application of artificial intelligence (AI) technology, the computing power of graphics processing units (GPUs) or AI-specific chips is continuously increasing. This places an increasingly high demand on the bandwidth of memory access. For example, 512GB / s to 1TB / s has become very common, and even some chips have exceeded 1TB / s. However, the frequency and bus width of the SoC are limited by the process and timing and cannot be increased infinitely. For the currently common AXI interface, generally speaking, the frequency is about 1GHz, the data bit width is below 1024bit, and it is commonly 512bit. Its bandwidth is 64GB / s (=1GHz * 512bit / 8), which is equivalent to the bandwidth provided by single-channel GDDR, HBM, etc. storage. The existing memory multi-channel interleaving technology usually requires up to 16 (=1TB / 64GB) AXI interfaces to provide sufficient memory bandwidth to meet the requirements for the bandwidth of memory access.

[0003] However, the multi-channel interleaving of memory, especially above 8 channels, also brings huge challenges. On the one hand, it is very difficult to arrange such multi-channels on one side of the SoC chip, and they often need to be placed on both sides of the chip or even around the chip. This brings great difficulty to the planar layout and physical implementation of the chip. On the other hand, multi-channels also pose challenges to the on-chip interconnection (NoC) and system performance of the SoC, including not only how to connect modules that do not support interleaving to the system, but also carefully evaluating whether the performance bandwidth in various scenarios can meet the system requirements, etc. Summary of the Invention

[0004] According to an exemplary embodiment of the present disclosure, a solution for a data transmission device is provided to solve or at least partially solve the above problems and / or other potential problems.

[0005] In a first aspect of the present disclosure, a data transmission device is provided. The data transmission device includes a plurality of first ports coupled to a processing unit; a plurality of second ports coupled to a plurality of memories; and a plurality of data channels arranged between the first ports and the second ports to form an interleaved network having a plurality of layers for transmitting data between the processing unit and the plurality of memories, such that each layer of the interleaved network includes at least one sub-interleaved network.

[0006] By adopting the technology of multi-channel and hierarchical memory interleaving, the hardware overhead can be effectively reduced, and the scalability and flexibility can be effectively improved. Since at least one layer is scattered, that is, divided into multiple sub-interleaving networks, the data channels of long paths are significantly reduced. This is beneficial to power consumption, data transmission delay, and stability. In addition, the data transmission technology of the hierarchical interleaving network can more easily achieve the layout of a symmetric structure, so that each module in the data processing system can be more easily arranged.

[0007] In some embodiments, multiple data channels include interleaved data channels to interconnect adjacent layers among multiple layers of the interleaving network. This further improves the data transmission efficiency and scalability.

[0008] In some embodiments, the number of sub-interleaving networks in multiple layers of the interleaving network increases or decreases from the first port to the second port. By using various combinations of sub-interleaving networks between layers, the scalability and flexibility are further improved.

[0009] In some embodiments, the total bandwidth provided by the first port is not less than the total bandwidth provided by the second port.

[0010] In some embodiments, the second port is coupled to multiple memories via a memory controller.

[0011] In a second aspect of the present disclosure, a data processing method is proposed. The data processing method includes responding to receiving a read request for data in at least one of multiple memories; and obtaining data from at least one memory targeted by the read request via a data transmission device according to the first aspect of the present disclosure.

[0012] In a third aspect of the present disclosure, a data processing method is proposed. In response to receiving a write request to write data to at least one of multiple memories, the data is written to at least one memory via a data transmission device according to the first aspect of the present disclosure.

[0013] In a fourth aspect of the present disclosure, a data processing system is provided. The data processing system includes a processing unit, a data transmission device according to the first aspect of the present disclosure, and a storage device for storing one or more programs. When the one or more programs are executed by the processing unit, the processing unit implements the methods according to the second aspect and the third aspect of the present disclosure.

[0014] In some embodiments, the data transmission device is integrated in the processing unit.

[0015] In some embodiments, multiple memories are arranged at any position around the data processing system.

[0016] In a fifth aspect of the present disclosure, a computer-readable medium is provided, on which computer-readable instructions are stored, and when the computer-readable instructions are executed, they cause the device to execute the method according to the third or fourth aspect of the present disclosure.

[0017] It should be understood that the content described in the summary of the invention section is not intended to limit the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0019] Figure 1 FIG. shows a schematic diagram of data transmission using memory interleaving technology in the prior art;

[0020] Figure 2 FIG. shows a schematic diagram of data transmission using a multi-channel, hierarchical memory interleaving technology according to some embodiments of the present disclosure;

[0021] Figure 3 FIG. shows a schematic diagram of the physical implementation of a multi-channel, hierarchical memory interleaving technology according to some embodiments of the present disclosure;

[0022] Figure 4 FIG. shows a schematic diagram of data transmission using a multi-channel, hierarchical memory interleaving technology according to other embodiments of the present disclosure;

[0023] Figure 5 FIG. shows a schematic diagram of data transmission using a multi-channel, hierarchical memory interleaving technology according to other embodiments of the present disclosure;

[0024] Figure 6 FIG. shows a flowchart of a data processing method according to an exemplary embodiment of the present disclosure;

[0025] Figure 7 FIG. shows a flowchart of a data processing method according to an exemplary embodiment of the present disclosure; and

[0026] Figure 8 FIG. shows a block diagram of a computing device capable of implementing multiple embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0028] In the description of the embodiments of the present disclosure, the term "comprising" and its like should be understood as an open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0029] Memory interleaving technology is a technology used to improve memory performance. It can provide more transmission channels and higher memory bandwidth, enabling the memory to perform multiple write / read operations simultaneously at the same time, thereby effectively improving system performance. Figure 1 The figure shows a schematic diagram of data transmission using memory interleaving technology in the prior art. Taking a system-on-chip (SoC) as an example, as Figure 1 shown, for multi-channel (especially more than 8 channels) memory interleaving, the control side and the memory side usually use an N×N crossbar to interconnect.

[0030] The processing unit 101' of the SoC system is generally interconnected with the memory 200' via the interleaved data channel 103' and the memory controller and the physical layer (PHY). According to the needs of the SoC system chip layout, if the memory controller and the physical layer need to be distributed on both sides or more sides of the SoC system, for a common 500mm 2 AI chip, the implementation of a multi-channel crossbar is very difficult.

[0031] In particular, for a chip of such a size, when a symmetric structure is usually adopted and divided into multiple blocks for implementation, it is very difficult to achieve module division and the symmetric structure. The inventors have found through research that this is largely due to the fact that the multi-channel interleaved network in the prior art has only one layer.

[0032] Based on the above findings, the inventors have proposed a data transmission technology based on a multi-channel, hierarchical interleaved network to solve or at least partially solve the above and / or other potential problems. This data transmission technology can be embodied as a data transmission device. The data transmission device here can be a module applied in any suitable environment (such as an SoC).

[0033] Figure 2 FIG. shows a schematic diagram of data transmission by a data transmission device using a multi-channel, hierarchical memory interleaving technique according to some embodiments of the present disclosure. As Figure 2 shown, the data transmission device includes a plurality of first ports 101 coupled to a processing unit (e.g., a processing unit in a SoC system), a plurality of second ports 102 coupled to a plurality of memories (200), and a plurality of data channels 103. The plurality of data channels 103 are arranged between the first ports 101 and the second ports 102 for transmitting data between the processing unit and the plurality of memories 102.

[0034] The second ports 102 may be coupled to the plurality of memories via a memory controller and a physical layer. The number of the first ports 101 and the second ports 102 may be equal or unequal. Generally, the total bandwidth provided by the first ports 101 is not less than the total bandwidth provided by the second ports 102. For example, in some embodiments, when the bit widths are the same, the number of the first ports 101 may also be different from the number of the second ports 102. Hereinafter, the improvement according to the embodiments of the present disclosure will be mainly introduced by taking the case where the number of the first ports 101 and the second ports 102 is the same (e.g., N×N) as an example.

[0035] Different from the data channels using the traditional memory interleaving technique, the plurality of data channels according to the embodiments of the present disclosure adopt a hierarchical memory interleaving manner. On each layer of the interleaving network of the data channels, at least one sub-interleaving network 1031 is included. That is, the hierarchical interleaving network adopted by the plurality of data channels, for example Figure 2 shown, the plurality of data channels are divided into a two-layer structure, where there is one sub-interleaving network in the first layer and two sub-interleaving networks in the second layer.

[0036] In this case, the sub-interleaving network in the first layer only needs to adopt N×2 interleaving, and each sub-interleaving network in the second layer only needs to adopt (N / 2)×(N / 2) interleaving. First, it is obvious that adopting the hierarchical technique will make the hardware overhead much smaller. Specifically, for a single-layer N×N interleaving network, its overhead is much larger than that of the N×2 + 2×(N / 2)×(N / 2) interleaving network in the example shown in Figure 2 . Secondly, since at least one layer is arranged in a scattered manner, that is, divided into a plurality of sub-interleaving networks, the long-path data channels will be greatly reduced. This is beneficial to the power consumption, data transmission delay, and stability.

[0037] Adopting the hierarchical memory interleaving technique, each layer can form a new module alone or together with other existing modules, so that each sub-interleaving network does not occupy too much area and can be arranged more flexibly. As Figure 3 shown, adoptingFigure 2 The data transmission technology of the hierarchical interleaved network shown can more easily achieve the layout of a symmetric structure.

[0038] In some embodiments, during physical implementation, Figure 2 The second layer, the required memory controller, and the physical layer in the example can be merged into an artificial intelligence (AI) operation module for implementation. In this way, better module division and a symmetric structure can be achieved, enabling the hierarchical technology to have good scalability. For example, with reasonable module division and a symmetric structure, the number of channels can be easily expanded from 8 channels to 16 channels or even 32 channels or higher. This makes it easy to meet the high requirements for the memory access bandwidth.

[0039] In some embodiments, the sub-interleaved networks of at least adjacent layers among the multiple layers of the interleaved network can be interconnected using interleaved data channels. A bus transmission protocol such as the AXI interface can be used for data transmission between each adjacent layer. This further improves the data transmission efficiency and scalability.

[0040] In addition, Figure 2 FIG. only shows a schematic diagram of data transmission using a multi-channel, hierarchical memory interleaving technology according to some embodiments of the present disclosure. Of course, it should be understood that this is only exemplary and is not intended to limit the protection scope of the present disclosure. The data transmission device using hierarchical inner interleaving according to the embodiments of the present disclosure supports more flexible adjustment of each module and / or sub-interleaved network.

[0041] For example, as Figure 4 shown, in some embodiments, multiple data channels 103 also form a two-layer interleaved network. Different from the embodiment shown in Figure 2 , in the first-layer interleaved network, there can be one sub-interleaved network 1031, while in the second-layer interleaved network, there can be 4 sub-interleaved networks. In this way, the sub-interleaved network in the first layer can use N×4 interleaving, and each sub-interleaved network in the second layer can use (N / 4)×(N / 4) interleaving. This not only further reduces the hardware overhead but also supports more module division methods. In addition, this arrangement also allows the memory to be symmetrically arranged around the SoC chip.

[0042] Figure 2 and Figure 4 show the case where the data channels can be divided into two layers of the interleaved network. By Figure 2 and Figure 4As can be seen from the description of the embodiments, the number of sub-interleaving networks on each layer can be flexibly adjusted according to actual needs. In some embodiments, the number of sub-interleaving networks in multiple layers can gradually increase or decrease from the first interface 102 to the second interface. By using various combinations of sub-interleaving networks between layers, the scalability and flexibility of data transmission are further improved.

[0043] In addition to the number of sub-interleaving networks in each layer that can be flexibly adjusted according to actual needs, the number of layers of the interleaving network can also be adjusted, such as 3 layers or more. The more the number of channels, the more layers can be considered. For example, in some embodiments, as Figure 5 shown, the figure shows that the 16-channel interleaving network has three layers. The first layer can have one sub-interleaving network 1031, the second layer can have two sub-interleaving networks 1031, and the third layer can have 4 sub-interleaving networks 1031.

[0044] In this case, the sub-interleaving network 1031 in the first layer can adopt N×2 interleaving, each sub-interleaving network in the second layer can adopt (N / 2)×(N / 4) interleaving, and the sub-interleaving network in the third layer can adopt (N / 4)×(N / 4) interleaving. With this multi-layer structure, since the sub-interleaving network is further subdivided, it is more convenient for module division and symmetric layout, and the flexibility and scalability are further improved.

[0045] In addition to the number of layers and the number of sub-interleaving networks on each layer described above that can be flexibly adjusted, in order to further optimize the area, either the first port 101 or the second port 102 can also reduce the number of interfaces by increasing the data bit width.

[0046] For example, in some embodiments, the bit width of the first port 101 can be twice the bit width of the second port. In this way, the number of the first ports 101 can be half the number of the second ports 102. When using a two-layer structure interleaving network, on the one hand, the sub-interleaving network of the first layer can adopt (N / 2)×2 interleaving. This can significantly reduce the area and routing of the first layer. On the other hand, the reduction of routing makes the physical implementation more friendly. For example, when the channel is 16, only 8 first ports 101 are needed.

[0047] The above text describes a multi-channel interleaving method in which the interleaving network of the data channel can have more than two layers and each layer can have different numbers of sub-interleaving networks. This multi-channel interleaving method realizes various layout methods of the data channel, and realizes a multi-channel, hierarchical memory interleaving method with high scalability and good physical implementation.

[0048] According to an embodiment of the present disclosure, a read data processing method 600 is also provided. The read data processing method 600 can be executed by a processing unit in a data processing system. Figure 6 A flowchart of a data processing method according to an exemplary embodiment of the present disclosure is shown. As shown, at 610, it is determined whether a read request for data in at least one of the plurality of memories 200 is received. If the read request is received, then at 620, the method 600 obtains data from at least one of the memories 200 targeted by the read request via the data transmission device described above.

[0049] For example, if a user wants to process data stored in a memory or other storage device, the user can send a read request for the data to the processing unit through an appropriate interface. If the data is stored in other storage devices, the data can be transferred to the memory via appropriate operations. The read request may include the address of the data in the memory. In the case where a read request for the data is received, the processing unit can obtain the data from the memory 200 according to the address information. After the data is obtained, it can be sent to a module specified by the user through the interface for further processing after corresponding processing. For example, the data can be written into the memory 200 after being read and processed by the processing unit for further processing.

[0050] According to an embodiment of the present disclosure, a write data processing method 700 is also provided. The write data processing method 700 can be executed by a processing unit in a data processing system. Figure 7 A flowchart of a write data processing method according to an exemplary embodiment of the present disclosure is shown. As shown, at 710, it is determined whether a write request to write data into at least one of the plurality of memories 200 is received. If the write request is received, then at 720, the method 700 writes the data into at least one memory via the data transmission device described above.

[0051] For example, after the processing unit finishes processing the data and receives a request to write the processed data into the memory 200, the processing unit writes the data into the memory 200 via the data transmission device described above for further processing.

[0052] By adopting a multi-channel and hierarchical memory interleaving method, the above method can effectively reduce the number of data channels with long paths, and improve the data transmission efficiency and stability.

[0053] According to an embodiment of the present disclosure, a data processing system 300 is also provided, as Figure 3As shown. The data processing system may include a processing unit, a storage device, and the data transmission device 100 described above. The data processing system 300 may be a system-on-chip (SoC) in some embodiments. The storage device is used to store at least one program. The at least one program can be executed by the processing unit and, when executed, causes the processing unit to implement the method described above. The layout of the data processing system 300 using the data transmission device 100 described above is more reasonable, has higher scalability, and lower power consumption.

[0054] In some embodiments, the data transmission device 100 may be integrated in the processing unit. That is, in some embodiments, the system-on-chip (SoC) includes at least one processing unit, and the data transmission device 100 may be a data channel integrated in the processing unit of the system-on-chip. In some embodiments, multiple memories 200 may be arranged at any position of the data processing system (such as the system-on-chip (SoC)), for example, symmetrically arranged on both sides or around. The data processing unit with the above arrangement can be more easily physically implemented.

[0055] Figure 8 A schematic block diagram of an example device 600 that can be used to implement the embodiments of the present disclosure is shown. The device 800 can be used to implement Figure 7 and Figure 8 the methods shown therein. As shown, the device 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to computer program instructions stored in the read-only memory (ROM) 802 or computer program instructions loaded from the storage unit 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The CPU 801, ROM 802, and RAM 803 are connected to each other through a bus 804. The input / output (I / O) interface 805 is also connected to the bus 804.

[0056] Multiple components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, optical disc, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0057] The processing unit 801 executes the various methods and processes described above, such as processes 600 and 700. For example, in some embodiments, processes 600 and 700 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed onto device 800 via ROM 602 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by CPU 801, one or more steps of processes 600 and 700 described above may be executed. Alternatively, in other embodiments, CPU 801 may be configured to execute processes 600 and 700 by any other suitable means (e.g., by means of firmware).

[0058] The functions described above herein may be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that may be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0059] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0060] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be either a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0061] In addition, although the operations are depicted in a particular order, this should be understood as requiring that the operations be performed in the particular order shown or in sequential order, or that all of the illustrated operations be performed to achieve the desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments may also be implemented combinatorially in a single implementation. Conversely, the various features that are described in the context of a single implementation may also be implemented separately or in any suitable sub-combination in multiple implementations.

[0062] It should be understood that the above detailed embodiments of the present disclosure are merely for purposes of illustration or explanation of the principles of the present disclosure and are not intended to limit the present disclosure. Therefore, any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure should be included within the scope of protection of the present disclosure. At the same time, the appended claims of the present disclosure are intended to cover all variations and modifications that fall within the scope and boundaries of the equivalents of the scope and boundaries of the claims.

Claims

1. A data transmission device (100), comprising: A plurality of first ports (101), coupled to a processing unit; A plurality of second ports (102), coupled to a plurality of memories (200); And A plurality of data channels (103), arranged between the first ports (101) and the second ports (102) to form an interleaved network having a plurality of layers for transmitting data between the processing unit and the plurality of memories (200), such that each layer of the interleaved network includes at least one sub - interleaved network (1031), wherein the number of sub - interleaved networks (1031) in the plurality of layers of the interleaved network increases or decreases from the first ports (101) to the second ports (102), and the second ports (102) are coupled to the plurality of memories (200) via a memory controller (201).

2. The data transmission device (100) according to claim 1, wherein the plurality of data channels (103) include interleaved data channels to interconnect adjacent layers in the plurality of layers of the interleaved network.

3. The data transmission device (100) according to claim 1, wherein the total bandwidth provided by the first ports (101) is not less than the total bandwidth provided by the second ports (102).

4. A data processing method, comprising: In response to receiving a read request for data in at least one of a plurality of memories (200), obtaining the data from the at least one memory (200) targeted by the read request via the data transmission device (100) according to any one of claims 1 - 3.

5. A data processing method, comprising: In response to receiving a write request for writing data to at least one of a plurality of memories (200), writing the data to the at least one memory (200) via the data transmission device (100) according to any one of claims 1 - 3.

6. A data processing system (300), comprising: A processing unit; The data transmission device (100) according to any one of claims 1 - 3; And A storage device for storing one or more programs, which when executed by the processing unit cause the processing unit to implement the method according to claim 4 or claim 5.

7. The data processing system (300) according to claim 6, wherein the data transmission device (100) is integrated in the processing unit.

8. The data processing system (300) according to claim 6, wherein a plurality of memories (200) are arranged at any position around the data processing system.

9. A computer - readable storage medium, having computer - readable instructions stored thereon, which when executed cause a device to execute the method according to claim 4 or claim 5.

Images