Artificial intelligence processor, electronic device, and data transmission method

Abstract

The present disclosure provides an artificial intelligence processor, an electronic device and a data transmission method. The artificial intelligence processor comprises a data transmission assistance unit configured to transmit a data transmission request for target data; a communication device configured to transmit the data transmission request for the data transmission assistance unit, wherein the communication device is configured with a plurality of external communication channels for transmitting the target data through a communication network connected with the artificial intelligence processor, each of the plurality of external communication channels is configured to provide a plurality of virtual sub-channels, wherein the communication device comprises an arbitrator, and the communication device is configured to arbitrate the provided plurality of virtual sub-channels to obtain a target virtual sub-channel using the arbitrator, and transmit the target data using the target virtual sub-channel. The artificial intelligence processor can effectively improve the data transmission capacity between artificial intelligence processors and avoid congestion.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of artificial intelligence technology, and particularly to artificial intelligence processors, electronic devices, and data transmission methods. Background Technology

[0002] In AI scenarios such as data centers, different chips may need to interconnect to exchange data when performing complex calculations such as training and inference of large models. Specifically, a chip may include multiple AI processors, and data transfer may be required between different AI processors on different chips to improve overall computing power.

[0003] However, data exchange between different AI processors may require significant assistance from the central processing unit (CPU) and may be subject to congestion and other issues. Summary of the Invention

[0004] According to at least one embodiment of the present disclosure, an artificial intelligence processor is provided, comprising: a data transmission assistance unit configured to transmit a data transmission request for target data; and a communication device configured to transmit the data transmission request for the data transmission assistance unit, wherein the communication device is configured with a plurality of external communication channels for transmitting the target data through a communication network connected to the artificial intelligence processor, each of the plurality of external communication channels being configured to provide a plurality of virtual sub-channels, wherein the communication device includes an arbitrator, and the communication device is configured to: arbitrate the provided plurality of virtual sub-channels using the arbitrator to obtain a target virtual sub-channel, and use the target virtual sub-channel to transmit the target data.

[0005] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure further includes a plurality of computing cores and system direct memory access (SDMA) devices, wherein the data transfer assistance unit is included in at least one of the computing cores and SDMA devices of the artificial intelligence processor.

[0006] For example, in an artificial intelligence processor according to at least one embodiment of the present disclosure, the plurality of external communication channels include: a data request channel; and a data response channel.

[0007] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the communication device is configured with a plurality of transmission queues that respectively cache target data corresponding to the plurality of external communication channels, and the communication device is further configured to: use the arbitrator to arbitrate a plurality of virtual sub-channels provided by each of the plurality of external communication channels to determine a plurality of candidate virtual sub-channels respectively corresponding to the plurality of external communication channels; and transmit the target data cached in the transmission queue corresponding to the target virtual sub-channel among the plurality of candidate virtual sub-channels through the target virtual sub-channel.

[0008] For example, in an artificial intelligence processor according to at least one embodiment of the present disclosure, the communication device is further configured to: use the arbitrator to arbitrate the plurality of external communication channels to determine a target communication channel; and determine a candidate virtual subchannel corresponding to the target communication channel as the target virtual subchannel.

[0009] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the communication device is configured with a plurality of receiving queues that respectively cache target data corresponding to the plurality of external communication channels.

[0010] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure further includes an on-chip network, wherein the on-chip network is configured to communicatively connect the data transmission assistance unit and transmit the target data between the communication device and the data transmission assistance unit via a plurality of internal communication channels, wherein at least two of the plurality of internal communication channels are merged into one of the plurality of external communication channels, and the number of the plurality of external communication channels is less than the number of the plurality of internal communication channels.

[0011] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the plurality of internal communication channels include one or more of the following: a write address channel; a write data channel; a read address channel; a read data channel; and a write response channel.

[0012] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the plurality of external communication channels include: a first channel corresponding to the write address channel and the write data channel; a second channel corresponding to the read address channel; a third channel corresponding to the read data channel; and a fourth channel corresponding to the write response channel.

[0013] For example, in an artificial intelligence processor according to at least one embodiment of the present disclosure, the communication device further includes a routing device configured to route the plurality of internal communication channels to corresponding external communication channels.

[0014] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure further includes: a first storage device for the second channel, wherein the bit width of the first storage device matches the size of the data packets transmitted on the second channel; and a second storage device for the fourth channel, wherein the bit width of the second storage device matches the size of the data packets transmitted on the fourth channel.

[0015] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the communication device is configured with a buffer and a plurality of first linked lists corresponding to the plurality of virtual sub-channels respectively, each of the plurality of first linked lists including one or more first nodes, each of the one or more first nodes pointing to a storage area in the buffer for storing target data of the corresponding virtual sub-channel.

[0016] For example, according to at least one embodiment of the artificial intelligence processor of the present disclosure, the communication device is configured with a second linked list, the second linked list including one or more second nodes, each of the one or more second nodes pointing to a free storage area in the cache.

[0017] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, each of the plurality of first linked lists includes a plurality of nodes associated with the storage address of target data in a cache associated with the corresponding virtual sub-channel, wherein the communication device is configured to: release the storage area in the cache storing the confirmed target data in response to the confirmation of target data corresponding to at least one of the plurality of nodes through the communication network.

[0018] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the communication device is further configured to retransmit the unacknowledged target data to the communication network in response to determining that target data corresponding to at least one of the plurality of nodes has not been acknowledged, wherein the communication device determines that the target data has not been acknowledged by at least one of the following: receiving a negative acknowledgment for the target data through the communication network; and not receiving a positive acknowledgment for the target data through the communication network within a predetermined time period.

[0019] For example, in an artificial intelligence processor according to at least one embodiment of the present disclosure, the communication device is further configured to: obtain a target virtual subchannel by using an arbitrator to arbitrate the plurality of virtual subchannels in a polling manner and a fair arbitration manner.

[0020] An electronic device is provided according to at least one embodiment of the present disclosure, including an artificial intelligence processor as described in any of the embodiments above.

[0021] According to at least one embodiment of this disclosure, a data transmission method for an artificial intelligence processor is provided, wherein the artificial intelligence processor includes a data transmission assistance unit and a communication device. The method includes: transmitting a data transmission request for target data through the data transmission assistance unit; transmitting the data transmission request to the data transmission assistance unit through the communication device, wherein the communication device is configured with multiple external communication channels for transmitting the target data through a communication network connected to the artificial intelligence processor, each of the multiple external communication channels being configured to provide multiple virtual sub-channels, wherein the communication device includes an arbitrator; and arbitrating the provided multiple virtual sub-channels using the arbitrator to obtain a target virtual sub-channel, and transmitting the target data using the target virtual sub-channel.

[0022] The artificial intelligence processor, electronic device, and data transmission method provided according to at least one embodiment of this disclosure can effectively improve the data transmission capability between artificial intelligence processors to meet higher computing power requirements. Furthermore, the artificial intelligence processor provided according to at least one embodiment of this disclosure can avoid congestion while avoiding or reducing CPU involvement in the data transmission process of the artificial intelligence processor. Specifically, the artificial intelligence processor according to at least one embodiment of this disclosure can improve network throughput and avoid deadlock. By aggregating multiple internal communication channels into fewer external communication channels, the fixed protocol overhead of network transmission can be effectively reduced. More importantly, by introducing virtual sub-channels and arbitrators, it is ensured that even under high load, data streams of different priorities or destinations can obtain transmission opportunities fairly, avoiding overall transmission efficiency degradation or deadlock caused by congestion of a single data stream. The artificial intelligence processor according to at least one embodiment of this disclosure can reduce communication latency and CPU overhead. By integrating the aforementioned dedicated communication device within the artificial intelligence processor and introducing an arbitration mechanism (e.g., a two-level arbitration mechanism), data transmission requests can be directly processed at the hardware level without CPU access to the kernel protocol stack, thereby significantly reducing transmission latency. The artificial intelligence processor according to at least one embodiment of this disclosure can achieve efficient cache management and reliable retransmission. By employing dynamic cache management using a first linked list, such as a data linked list, and a second linked list, such as a free linked list, efficient reuse of cache free space is achieved, reducing storage resource waste. Furthermore, the AI ​​processor according to at least one embodiment of this disclosure can implement an end-to-end acknowledgment / retransmission mechanism based on virtual sub-channels, thereby improving the reliability of end-to-end transmission. For example, it ensures the reliability of data transmission over unreliable physical networks, providing a stable data communication foundation for large-scale AI clusters. Attached Figure Description

[0023] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, in which:

[0024] Figure 1 This demonstrates an artificial intelligence processor applied to scenarios such as data centers.

[0025] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0026] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0027] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0028] Figure 5 A schematic structural diagram of a communication device for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0029] Figure 6 A schematic structural diagram of an arbitrator according to at least one embodiment of the present disclosure is shown.

[0030] Figures 7A-7B A schematic flowchart illustrating the confirmation and negative confirmation of target data according to at least one embodiment of the present disclosure is shown.

[0031] Figure 8 A flowchart of a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0032] Figure 9 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown. Detailed Implementation

[0033] Before proceeding with the detailed description below, it may be advantageous to define certain words and phrases used throughout this disclosure. The terms “comprising” and “including” and their derivatives mean including but not limited to. The term “or” is inclusive, meaning and / or. The phrase “associated with” and its derivatives mean including, comprising, interconnecting, containing, contained within, connected or connected to, coupled or coupled to, communicating with, cooperating, intertwining, juxtaposing, proximate, binding or bound to, having, possessing attributes, having a relationship or being related to, etc. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware, or a combination of hardware and software and / or firmware. The functionality associated with any particular controller may be centralized or distributed, local or remote. The phrase “at least one,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and that only one item from the list may be required. For example, "at least one of A, B, and C" includes any one of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C.

[0034] Definitions of other specific words and phrases are provided throughout this disclosure. Those skilled in the art will understand that, in many, if not most, cases, such definitions apply to the prior and future use of the words and phrases thus defined.

[0035] The various embodiments of the principles of this disclosure described below with reference to the accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or device. In some cases, the actions described in the specification may be performed in a different order and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require a specific order or sequential sequence to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0036] Artificial intelligence processors can include parallel processors such as graphics processing units (GPUs) and general-purpose graphics processing units (GPGPUs). Figure 1 This demonstrates an artificial intelligence processor applied to scenarios such as data centers.

[0037] like Figure 1 As shown, a data center can deploy multiple servers, including Server 0 and Server 1. These servers can be used to perform tasks that require significant computing power, such as hosting websites and web applications, data storage and management, and running applications and software.

[0038] The following description uses server 0 as an example to illustrate the functionality of the components within the server. Server 1 can be the same as or similar to server 0, therefore, server 1 will not be described again to avoid redundancy. Server 0 may include a central processing unit (CPU) 0, a root complex 0, PCIe switches 00 and 01, GPU 00, GPU 01, GPU 02, and GPU 03.

[0039] CPU 0 can be used as the core computing and control component of server 0. CPU 0 can execute operating system and application instructions, perform logical operations and task scheduling by calling data stored in memory, and coordinate the collaborative work of various peripheral hardware components within the server 0 system to realize the general computing and overall control functions of server 0.

[0040] Root Complex 0 can be the core controller of a PCIe (Rapid Peripheral Component Interconnect) system architecture. Root Complex 0 can access the system via the uplink port ( Figure 1 (Not shown) is connected to the CPU and can be connected to PCIe switch 00 and PCIe switch 01 via downlink ports (e.g., root ports RP00 and RP01). Root complex 0 is responsible for enumerating, configuring, and managing the PCIe topology and acts as the root node for data communication, handling uplink and downlink data routing and protocol conversion.

[0041] PCIe switches 00 and 01 can be high-speed interconnect devices used to extend PCIe links. For example, PCIe switch 00 may include one uplink port USP00 and multiple downlink ports DSP01 and DSP02. PCIe switch 00 can achieve high-bandwidth, low-latency parallel data exchange between multiple downstream PCIe devices and the upstream root complex through internal crossbar switches, thereby effectively improving the system's peripheral connectivity. The function of PCIe switch 01 is similar to that of PCIe switch 00, and will not be described again here.

[0042] GPU00, GPU01, GPU02, and GPU03 can be dedicated coprocessors for parallel computing. GPU00, GPU01, GPU02, and GPU03 can employ a massively parallel stream processor architecture, suitable for handling highly parallel floating-point and integer operations in graphics rendering, deep learning, and scientific computing. GPU00, GPU01, GPU02, and GPU03 can take over specific computing tasks from CPU 0, thereby significantly improving the overall computing throughput and energy efficiency of server 0. GPU00, GPU01, GPU02, and GPU03 can communicate through their communication ports (such as...). Figure 1 (As shown in the slanted shaded area) is connected to the corresponding communication port of the Ethernet switch.

[0043] With the development of training large models, the demand for computing power in data centers is increasing, thus requiring the interconnection of multiple GPUs in data centers for data transmission. According to at least one embodiment of this disclosure, multiple GPUs can interconnect via Ethernet, for example, using their communication ports. Ethernet primarily follows the IEEE 802.3 standard. Ethernet can be a local area network (LAN) technology used for transmitting data within a local area network. As a widely used LAN technology, Ethernet allows multiple devices (e.g., computers such as servers, printers, switches, etc.) to communicate with each other.

[0044] Figure 1 The communication ports configured in each GPU (e.g., GPU00-GPU03 and GPU10-GPU13) are as follows: Figure 1 (As shown in the shaded area) can be connected to an Ethernet switch (e.g., via communication ports 0-7 of the Ethernet switch) via an Ethernet link. The following description uses access to other GPUs by GPU00 of server 0 as an example. GPU00 of server 0 can send Ethernet packets to the Ethernet switch, which can determine which communication port of the other GPU to forward the Ethernet packet to based on the destination address field in the packet header.

[0045] However, when multiple source GPUs (e.g., Figure 1 The GPUs shown (GPU00-GPU03) are directed to the same destination GPU (e.g., Figure 1 The GPU 10 shown in the diagram sends data packets. If the bandwidth of communication ports 0-7 in the Ethernet switch is 400Gbps unidirectionally, then multiple source GPUs sending data packets to a single destination GPU will cause congestion. When congestion occurs within the Ethernet switch, it not only reduces data transmission efficiency but also leads to packet loss within the Ethernet switch. For example, when multiple source processors send data to a single target processor simultaneously, congestion hotspots can easily form at the interface of the network switch (e.g., an Ethernet switch) or the target processor, causing packet loss and retransmission, triggering a "head-of-line blocking" effect. The validation involved in this process will slow down the overall training efficiency.

[0046] As mentioned above, existing technologies still face significant challenges in multi-chip interconnect scenarios. TCP / IP and Remote Direct Memory Access (RDMA) technologies can be used for data transfer between GPUs. However, TCP / IP requires implementation within the operating system kernel. During data transmission, data traverses the kernel protocol stack multiple times, resulting in high latency and consequently low performance. For example, traditional TCP / IP-based inter-processor communication introduces microsecond-level latency far exceeding the nanosecond-level data synchronization requirements of AI training, and CPU kernel interrupt handling and context switching consume substantial computational resources. Furthermore, both TCP / IP and RDMA require CPU assistance, thus consuming significant CPU resources and degrading CPU performance. Moreover, in large-scale artificial intelligence (AI) clusters, RDMA involves tens of thousands of dynamic, concurrent communication connections between processors, potentially leading to rapid exhaustion of RDMA connection management entries and difficulty in providing fine-grained, predictable quality of service.

[0047] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0048] Artificial intelligence processors may include parallel processors such as graphics processing units (GPUs) and general-purpose graphics processing units (GPGPUs), but this disclosure is not limited thereto. Figure 2 As shown, an artificial intelligence processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In an AI processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0049] like Figure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores (also called computational cores), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These Cores are used to execute specific computational tasks. Furthermore, a Computation Unit may also include registers (e.g., ...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0050] like Figure 2 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0051] like Figure 2 As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0052] In parallel computing, computational tasks are typically executed through multiple threads. These threads are divided into multiple thread blocks before execution in the artificial intelligence processor (or general-purpose graphics processor or parallel computing processor), and then dispatched via a thread block dispatch module. Figure 2 (Not shown in the diagram) Multiple thread blocks are distributed to various computing units. All threads in a thread block must be assigned to the same computing unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple threads from multiple thread blocks can execute on multiple computing cores within the same computing unit, or on multiple computing cores in different computing units, sharing registers, shared memory, and other data transfer mechanisms.

[0053] In each computing unit, the thread beam scheduling / distribution module ( Figure 2(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 2 It is used for reading and writing operations in high-bandwidth memory (HBM).

[0054] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 3 As shown, the artificial intelligence processor 3000 may include a data transmission assistance unit 3100 and a communication device 3200. Figure 3 The artificial intelligence processor in it can correspond to Figure 2 The computational unit will not be described again here for the sake of brevity.

[0055] According to at least one embodiment of this disclosure, the data transmission assistance unit 3100 can be configured to transmit a data transmission request for target data. The data transmission request for target data may include read requests and write requests for the target data, but this disclosure is not limited thereto. For example, the data transmission assistance unit 3100 can initiate a data transmission request for target data and can receive data transmission requests for target data. The target data may be data that a plurality of artificial intelligence processors, including the artificial intelligence processor 3000, wish to transmit to improve computing power.

[0056] According to at least one embodiment of this disclosure, the communication device 3200 can be configured to transmit data transmission requests to the data transmission assistance unit 3100. For example, the communication device 3200 can be connected to a communication network external to the artificial intelligence processor to perform data transmission. The communication device 3200 can send data transmission requests from the external communication network to the data transmission assistance unit 3100, and can also send data transmission requests from the data transmission assistance unit 3100 to the external communication network.

[0057] According to at least one embodiment of this disclosure, the communication device 3200 may be configured with multiple external communication channels for transmitting target data to an external communication network. For example, the multiple external communication channels may be physical channels of the communication device 3200. Each of the multiple external communication channels may be configured to provide multiple virtual sub-channels. That is, the data transmission assistance unit 3100 can transmit data with the external communication network through the multiple virtual sub-channels provided by the multiple external communication channels. Each of the multiple virtual sub-channels may have a different destination; for example, the multiple virtual sub-channels may point to or originate from multiple different artificial intelligence processors connected to the artificial intelligence processor 3000.

[0058] According to at least one embodiment of this disclosure, the communication device 3200 may include an arbitrator 3210. The communication device 3200 may use the arbitrator 3210 to arbitrate multiple virtual sub-channels provided to obtain a target virtual sub-channel. That is, depending on the outcome of the arbitration, each of the multiple virtual sub-channels has the opportunity to serve as the target virtual sub-channel. The communication device 3200 may use the target virtual sub-channel to transmit target data.

[0059] According to at least one embodiment of the present disclosure, the artificial intelligence processor 3000 can directly perform data transmission using a data transmission assistance unit 3100 and a communication device 3200 included within the artificial intelligence processor 3000, without the assistance of a CPU external to the artificial intelligence processor 3000. In this way, the workload of the CPU on data transmission to the artificial intelligence processor 3000 can be reduced, allowing the CPU to allocate computing and storage resources to other tasks, thus improving CPU performance. Therefore, the performance of computing devices (e.g., servers) deploying the CPU and the artificial intelligence processor 3000 can be enhanced.

[0060] An artificial intelligence processor 3000 according to at least one embodiment of the present disclosure provides multiple virtual sub-channels included in a plurality of channels. As described above, the multiple virtual sub-channels may point to or originate from multiple different artificial intelligence processors connected to the artificial intelligence processor 3000. Taking write requests as an example, suppose multiple write requests corresponding to the multiple virtual sub-channels need to access multiple destination artificial intelligence processors. The congestion levels of the multiple virtual sub-channels accessing the multiple destination artificial intelligence processors may be different. By arbitrating the aforementioned multiple virtual sub-channels using an arbitrator 3210, data transmission of virtual sub-channels can be prevented from being blocked due to congestion of one virtual sub-channel, thereby avoiding data transmission deadlock. The arbitrator 3210 can guarantee that no requester is "starved" when multiple active requesters compete for the same resource. "Starved" can refer to a requester's inability to obtain the required service or resource (e.g., data) for an extended period of time.

[0061] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 4 As shown, the artificial intelligence processor 4000 may include a computing core 4110, a system direct memory access (SDMA) device 4120, an on-chip network (NOC) 4200, and a communication device 4300. Figure 4 Zhongyu Figure 3 Components that are identical or similar will not be described again to avoid redundancy. Figure 4 The AI ​​processor 4000 in the middle can correspond to Figure 2 The computing unit, Figure 4 The computational kernel in the middle can correspond to Figure 2 The core of it will not be described again here for the sake of brevity.

[0062] According to at least one embodiment of this disclosure, an artificial intelligence processor 4000 may include at least one computing core 4110. Each computing core in the artificial intelligence processor 4000 may include one or more instruction processing units, such as, but not limited to, an instruction fetch unit, a decoding unit, an instruction issue unit, and an artificial intelligence unit, to perform processing operations on instructions. For example, instructions in a computing core may undergo instruction fetch, instruction decoding, instruction issue, instruction execution, and write-back. According to at least one embodiment of this disclosure, the instruction fetch unit may be configured to fetch instructions to be executed subsequently. The decoding unit may be configured to decode the fetched instructions into control signals executable by the underlying hardware. The issue unit may monitor the execution status of a thread or thread bundle (e.g., waiting for operands or hardware resources to be ready) and issue instructions that meet the issue conditions to the execution unit, such as the artificial intelligence computing unit. The artificial intelligence computing unit may be configured to perform artificial intelligence-related calculations based on the issued instructions. The write-back unit may write the data output by the artificial intelligence computing unit back to memory or cache.

[0063] According to at least one embodiment of this disclosure, the SDMA device 4120 can be a dedicated engine for implementing system direct memory access. The SDMA device 4120 can independently handle data transfer tasks between the video memory and system memory of the AI ​​processor 4000 without CPU assistance or intervention. The SDMA device 4120 separates data transfer from the operation of the AI ​​processor 4000, effectively eliminating the adverse effects of data transfer latency on computing tasks, thereby significantly improving the overall throughput efficiency and system energy efficiency of the AI ​​processor 4000.

[0064] According to at least one embodiment of the present disclosure, a data transmission assistance unit may be included in at least one computing core and an SDMA device. For example, data transmission assistance unit 4111 may be disposed in computing core 4110 and data transmission assistance unit 4121 may be disposed in SDMA device 4120, but the present disclosure is not limited thereto, and the data transmission assistance unit may be disposed in any device in the artificial intelligence processor 4000 that can initiate or receive data transmission requests.

[0065] According to at least one embodiment of this disclosure, the NOC 4200 can provide a high-bandwidth, low-latency, and scalable data exchange path for multiple parallel processing units (e.g., computing cores 4110 and SDMA devices 4120, etc.) and communication devices 4300 through a mesh topology of nodes and routers and a configured communication protocol.

[0066] According to at least one embodiment of this disclosure, the communication device 4300 may include an arbitrator 4310. Figure 4 The Media Access Control (MAC) layer is not shown in the image. Figure 4 The serializer / deserializer (SerDes) is not shown in the diagram.

[0067] The MAC layer can be the core controller of the data link layer. The MAC unit can assemble and parse data frames, execute access control protocols such as Carrier Sense Multiple Access with Collision Detection (CSMA / CD), manage frame check sequences, and perform flow control. The MAC unit can process data link packets, providing a standardized data interface to upper layers.

[0068] SerDes enables high-speed serial communication. It converts parallel data output from the MAC layer into a serial bit stream for transmission and reconstructs received serial signals into parallel data. Through clock recovery and channel coding technologies, SerDes overcomes transmission losses from PCB traces and cables, enabling high-speed, long-distance data exchange between the AI ​​Processor 4000 and external communication networks.

[0069] The target channel obtained by the communication device 4300 through the arbitrator 4310 can be connected to an external communication network sequentially through the MAC layer and SerDes. The external communication network may include Ethernet, but this disclosure is not limited to this; other communication networks are also possible. For example, the communication device 4300 can be connected to an Ethernet switch to enable data transmission between different artificial intelligence processors.

[0070] According to at least one embodiment of this disclosure, as referred to Figure 3 As described, the communication device 4300 can be configured with multiple external communication channels for transmitting target data to an external communication network. According to at least one embodiment of this disclosure, such as... Figure 4 As shown, NOC 4200 can be configured to communicate with data transmission assistance units 4111 and 4121. NOC 4200 can transmit target data between communication device 4300 and data transmission assistance units 4111 and 4121 via multiple internal communication channels. The number of external communication channels can be less than the number of internal communication channels. For example, at least two of the internal communication channels can be combined into one external communication channel. In this way, the differences between the external communication channels used by communication device 4300 to communicate with an external communication network and the internal communication channels used by communication device 4300 to communicate with NOC 4200, data transmission assistance units 4111 and 4121, etc., can be minimized to the point of incompatibility, thereby achieving efficient and lossless communication interconnection between multiple artificial intelligence processors.

[0071] The artificial intelligence processor according to at least one embodiment of this disclosure can improve network throughput and avoid deadlock. For example, network throughput can be increased because the number of external communication channels is reduced compared to the number of internal communication channels. When using external communication channels for data transmission over Ethernet, the more external communication channels there are, the greater the fixed overhead. By merging a portion of the internal communication channels into a single external communication channel, the fixed overhead of data transmission over Ethernet can be reduced. Specifically, by aggregating multiple internal communication channels (e.g., 5 channels of the AXI protocol) into fewer external communication channels (e.g., 4 channels), the fixed protocol overhead of network transmission can be effectively reduced. More importantly, by introducing virtual sub-channels and arbitrators, it is ensured that data streams of different priorities or destinations can obtain transmission opportunities fairly, even under high load, avoiding overall transmission efficiency degradation or deadlock caused by congestion of a single data stream.

[0072] According to at least one embodiment of this disclosure, multiple external communication channels may include a data request channel and a data response channel. By separating the data request channel and the data response channel, deadlocks in data transmission between different artificial intelligence processors can be avoided to some extent.

[0073] According to at least one embodiment of this disclosure, each of a plurality of external communication channels can be configured to provide a plurality of virtual sub-channels. For example, each of the plurality of external communication channels can provide 1024 virtual sub-channels, but this disclosure is not limited thereto, and other numbers of virtual sub-channels are also possible.

[0074] According to at least one embodiment of this disclosure, the communication device 3200 may include an arbitrator 3210. The communication device 3200 can use the arbitrator 3210 to arbitrate multiple virtual sub-channels to obtain a target virtual sub-channel. The communication device 3200 can use the arbitrator 3210 to arbitrate multiple virtual sub-channels in at least one of the following methods: round-robin and fair arbitration, to obtain the target virtual sub-channel. However, this disclosure is not limited to this, and other arbitration methods for obtaining the target virtual sub-channel are also possible. The round-robin method can ensure fairness by sequentially and cyclically allocating access permissions among all active requesters when multiple active requesters compete for the same resource. Furthermore, the round-robin method has lower hardware costs and implementation complexity, thus reducing the manufacturing cost of artificial intelligence processors. According to at least one embodiment of this disclosure, when the communication device 3200 uses the arbitrator 3210 to arbitrate in a fair arbitration manner, the weights of the fair arbitration process do not need to be fixed. For example, the weights of the fair arbitration process can be dynamically updated based on the bandwidth requirements of the data transmission request, service priority, etc. However, this disclosure is not limited to this, and other methods for determining the weights of the fair arbitration process are also possible. In this way, resources can be allocated precisely according to a preset ratio, making it easy to manage and predict the allocation of communication resources used for data transmission.

[0075] Figure 5 A schematic structural diagram of a communication device for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 5 As shown, the communication device 5001 of the artificial intelligence processor may include a receiving unit 5100, a sending unit 5200, a receiving queue 5300, an arbitrator 5400, a sending queue 5500, a routing device 5600, a master device 5700, and a slave device 5800. Figure 5 China Communication Device 5001 and Figure 2 and Figure 3 Components that are identical or similar to the communication devices described herein will not be described again to avoid redundancy.

[0076] According to at least one embodiment of this disclosure, the communication device 5001 can receive target data and / or data transmission requests from an external communication network through the receiving unit 5100, and can send target data and / or data transmission requests to the external communication network through the sending unit 5200.

[0077] According to at least one embodiment of this disclosure, NOC 5002 can be configured to communicate using the Advanced Extensible Interface (AXI) protocol, and NOC 5002 can have five internal communication channels. However, this disclosure is not limited to this; NOC 5002 can use other protocols and can have other numbers and types of internal communication channels. Specifically, NOC 5002 can be configured with multiple internal communication channels. These multiple internal channels may include one or more of the following: a write address channel (AW), a write data channel (W), a read address channel (AR), a read data channel (R), and a write response channel (B). The master device 5700 and slave device 5800 included in the communication device 5001 transmit data with NOC 5002 using the aforementioned five internal communication channels respectively.

[0078] According to at least one embodiment of this disclosure, the master device 5700 acts as the initiator of a transaction from an external communication network. The master device 5700 is configured to initiate read / write operations by sending addresses (e.g., read addresses and write addresses) to a data transmission assistance unit via NOC 5002. For example, the master device 5700 can send a write address and write data to the data transmission assistance unit via NOC 5002 through a write address channel (AW) and a write data channel (W), and receive a write response from the data transmission assistance unit via NOC 5002 through a write response channel (B). For example, the master device 5700 can send a read address to the data transmission assistance unit via NOC 5002 through a read address channel (AR), and receive read data from the data transmission assistance unit via NOC 5002 through a read data channel (R).

[0079] According to at least one embodiment of this disclosure, slave device 5800 refers to the initiator of a transaction to an external communication network. Slave device 5800 is configured to initiate read / write operations by receiving addresses (e.g., read address and write address) from a data transmission assistance unit via NOC 5002. For example, slave device 5800 can receive write address and write data from the data transmission assistance unit via NOC 5002 through a write address channel (AW) and a write data channel (W), and send a write response to the data transmission assistance unit via NOC 5002 through a write response channel (B). For example, slave device 5800 can receive a read address from the data transmission assistance unit via NOC 5002 through a read address channel (AR), and send read data to the data transmission assistance unit via NOC 5002 through a read data channel (R).

[0080] According to at least one embodiment of this disclosure, as referred to Figure 4The number of external communication channels may be less than the number of internal communication channels. For example, at least two of the internal communication channels may be merged into one of the external communication channels. The routing device 5600 may be configured to route the internal communication channels to their corresponding external communication channels. For example, the routing device 5600 may route the write address channel and the write data channel to a first channel corresponding to the write address channel and the write data channel, respectively; route the read address channel to a second channel corresponding to the read address channel; route the read data channel to a third channel corresponding to the read data channel; and route the write response channel to a fourth channel corresponding to the write response channel. According to at least one embodiment of this disclosure, the routing device 5600 may be implemented as an AXI bridge, but this disclosure is not limited thereto.

[0081] According to at least one embodiment of this disclosure, the communication device 5001 may be configured with a plurality of receiving queues 5300, each corresponding to a plurality of external communication channels. The plurality of receiving queues 5300 may include queues 5301-5304 to respectively buffer target data corresponding to the plurality of external communication channels. For example, the plurality of receiving queues 5300 may receive data transmission requests for target data and the target data from the receiving unit 5100. The plurality of receiving queues 5300 may be first-in-first-out (FIFO) queues.

[0082] According to at least one embodiment of this disclosure, the communication device 5001 may be configured with a plurality of transmission queues 5500, each corresponding to a plurality of external communication channels. The plurality of transmission queues 5500 may include queues 5501-5504 to respectively buffer target data corresponding to the plurality of external communication channels. For example, the plurality of transmission queues 5500 may send a data transmission request for the target data and the target data from the data transmission assistance unit to the transmission unit 5200.

[0083] According to at least one embodiment of this disclosure, the arbitrator 5400 can select a target virtual subchannel in the form of two-level arbitration. Figure 6 A schematic structural diagram of an arbitrator according to at least one embodiment of the present disclosure is shown. Figure 6As shown, arbitrator 5400 may include arbitrator 5410 and multiple arbitrators 5421, 5422, 5423, and 5424. Each of the multiple arbitrators 5421, 5422, 5423, and 5424 may correspond to multiple external communication channels (e.g., a first channel through a fourth channel). Arbitrator 5400 can be used to arbitrate multiple virtual sub-channels provided by each of the multiple external communication channels to determine multiple candidate virtual sub-channels corresponding to each of the multiple external communication channels. For example, each of the multiple arbitrators 5421, 5422, 5423, and 5424 can arbitrate multiple virtual sub-channels provided by the first channel through the fourth channel to determine candidate virtual sub-channels corresponding to each of the first channel through the fourth channel. For example, each of arbitrators 5421, 5422, 5423, and 5424 can arbitrate multiple virtual sub-channels in at least one of a round-robin manner and a fair arbitration manner to determine candidate virtual sub-channels. The target data cached in the transmission queue corresponding to the target virtual subchannel among multiple candidate virtual subchannels can be transmitted through the target virtual subchannel. The communication device can use arbitrator 5400 to arbitrate multiple external communication channels to determine the target communication channel. For example, arbitrator 5410 can arbitrate multiple external communication channels to determine the target communication channel, and can transmit the target data cached in multiple transmission queues 5500 corresponding to the target communication channel. The communication device can determine the candidate virtual subchannel corresponding to the target communication channel as the target virtual subchannel. For example, arbitrator 5410 can arbitrate multiple external communication channels using at least one of polling and fair arbitration methods to determine the target communication channel. According to at least one embodiment of this disclosure, the weights of arbitrator 5410 and arbitrators 5421, 5422, 5423, and 5424 in performing the fair arbitration process may not be fixed. For example, the weights may be dynamically updated based on the bandwidth requirements of the data transmission request, service priority, etc., but this disclosure is not limited to this, and other methods for determining the weights of the fair arbitration process are also possible.

[0084] The artificial intelligence processor according to at least one embodiment of this disclosure can reduce communication latency and CPU overhead. By integrating the aforementioned dedicated communication device within the artificial intelligence processor and introducing a two-level arbitration mechanism, data transmission requests can be processed directly at the hardware level without the CPU accessing the kernel protocol stack, thereby significantly reducing transmission latency.

[0085] According to at least one embodiment of this disclosure, multiple virtual subchannels can share a buffer, and linked lists can be used to manage the data of each virtual subchannel. For example, the communication device 5001 is configured with a buffer and multiple first linked lists corresponding to the multiple virtual subchannels respectively. In one embodiment, the first linked list may be a data linked list. Each of the multiple first linked lists may include one or more first nodes. Each of the one or more first nodes included in each first linked list may point to a storage area in the buffer used to store the target data of the corresponding virtual subchannel. For example, each of the multiple first linked lists may be configured to track the storage address of the target data associated with the corresponding virtual subchannel in the storage area (e.g., data slot) in the buffer during the arbitration of the multiple virtual subchannels provided by the arbitrator 5400.

[0086] Specifically, each of the multiple first linked lists includes multiple first nodes associated with the storage address of the target data in the buffer corresponding to the virtual subchannel. The first node in the first linked list can store a pointer to the next first node, that is, the storage address of the next storage area (e.g., a data slot) in the buffer. Each virtual subchannel can record the head and tail of the corresponding first linked list so that data is quickly appended to the tail when writing data to the virtual subchannel and quickly read from the head when reading data. Furthermore, a second linked list can be configured in the communication device 5001. The second linked list can include one or more second nodes, and each of the one or more second nodes can point to a free storage area in the buffer. The second linked list can be a global linked list, such as a free list. The second linked list can be configured to record the storage addresses of free storage areas (e.g., data slots) in the buffer associated with the multiple virtual subchannels during arbitration using the arbitrator 5400.

[0087] The artificial intelligence processor according to at least one embodiment of this disclosure can achieve efficient cache management and reliable retransmission. By employing dynamic cache management using a first linked list such as a data linked list and a second linked list such as a free linked list, efficient reuse of cache free space is achieved, reducing storage resource waste.

[0088] According to at least one embodiment of this disclosure, in response to target data corresponding to at least one of a plurality of nodes being acknowledged (Ack) via an external communication network, the communication device 5001 can release the storage area in the buffer where the acknowledged target data is stored. In response to determining that target data corresponding to at least one of the plurality of nodes has not been acknowledged, the communication device 5001 can retransmit the unacknowledged target data to the external communication network. In this way, an end-to-end acknowledgment / retransmission mechanism based on virtual subchannels can be implemented, thereby improving the reliability of end-to-end transmission. For example, it ensures the reliability of data transmission over unreliable physical networks, providing a stable data communication foundation for large-scale AI clusters.

[0089] Reference Figures 7A-7B Describe the process of confirming and rejecting target data. Figures 7A-7B A schematic flowchart illustrating the confirmation and negative confirmation of target data according to at least one embodiment of the present disclosure is shown.

[0090] For each virtual sub-channel, the requesting end can send a message through the communication device. Only after the responding end replies with an Ack will it clear the corresponding message backup identified in the buffer.

[0091] According to at least one embodiment of this disclosure, determining that target data has not been acknowledged by the communication device may include receiving a negative acknowledgment for the target data via an external network. Specifically, see... Figure 7A AI processor A, acting as the requesting end, can send two packets with sequence numbers PSN1 and PSN2 to AI processor B, acting as the responding end. When these two packets pass through the Ethernet switch, the packet with sequence number PSN1 is lost. Since the expected packet with sequence number PSN1 was not received, AI processor B can return a Nak1 response to AI processor A for the packet with sequence number PSN1. Upon receiving Nak1, AI processor A can retransmit the packet with sequence number PSN1 to AI processor B.

[0092] According to at least one embodiment of this disclosure, the communication device determining that target data has not been acknowledged may include not receiving positive acknowledgment for the target data via an external network within a predetermined time period. Specifically, see... Figure 7B AI processor A, acting as the requesting end, can send two packets with sequence numbers PSN1 and PSN2 to AI processor B, acting as the responding end. When these two packets pass through the Ethernet switch, the packet with sequence number PSN2 is lost. Since the expected Ack2 has not been received for an extended period, AI processor A can retransmit the packet with sequence number PSN2.

[0093] According to at least one embodiment of this disclosure, when an artificial intelligence processor uses a communication device to transmit data, it can store target data received through an external communication channel and target data to be transmitted through the external communication channel in corresponding storage devices. For example, target data received through a first channel-fourth channel and target data to be transmitted through the first channel-fourth channel can be stored in corresponding storage devices respectively. As mentioned above, data received or transmitted using the first channel may include write addresses and write data, and data received or transmitted using the third channel may include read data. Since the sizes of write data and read data are variable, the bit width of data received or transmitted using the first and third channels may be indeterminate. As mentioned above, data received or transmitted using the second channel may include read addresses, and data received or transmitted using the fourth channel may include write responses. Since the sizes of read addresses and write responses are fixed, the bit width of data received or transmitted using the second and fourth channels may be determined. Separate storage devices can be configured for data received or transmitted using the second and fourth channels. For example, a first storage device can be configured for the second channel, and the bit width of the first storage device can be matched with the size of the data packets transmitted on the second channel. For example, a second storage device can be configured for the fourth channel, and the bit width of the second storage device can be matched with the size of the data packets transmitted on the fourth channel. In this way, a data packet received using the second and fourth channels can be written or a data packet transmitted using the second and fourth channels can be read in each clock cycle (e.g., each clock cycle), thereby improving the data transmission rate. Furthermore, since the bit width of the first storage device matches the size of the data packets transmitted on the second channel and the bit width of the second storage device matches the size of the data packets transmitted on the fourth channel, wasted storage space in both the first and second storage devices can be avoided. In this way, the utilization rate of the first and second storage devices can be improved and their footprint reduced. According to at least one embodiment of this disclosure, the corresponding storage devices for storing target data received through an external communication channel and target data to be transmitted through the external communication channel can be implemented in static random access memory (SRAM). That is, the first and second storage devices can be implemented in SRAM.

[0094] Figure 8 A flowchart illustrating a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 8 As shown, method 8000 includes steps S8100-S8300. The execution is as follows: Figure 8 The artificial intelligence processor of the method shown may include a data transmission assistance unit and a communication device.

[0095] In step S8100, a data transmission request for the target data can be transmitted through the data transmission assistance unit.

[0096] In step S8200, a data transmission request can be transmitted to the data transmission assistance unit via a communication device. The communication device is configured with multiple external communication channels for transmitting target data via a communication network connected to the artificial intelligence processor. Each of the multiple external communication channels is configured to provide multiple virtual sub-channels. The communication device includes an arbitrator.

[0097] In step S8300, an arbitrator can be used to arbitrate the multiple provided virtual sub-channels to obtain a target virtual sub-channel, and the target virtual sub-channel can be used to transmit target data.

[0098] A data transfer method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further includes a plurality of computing cores and a system direct memory access (SDMA) device, wherein a data transfer assistance unit is included in at least one of the computing cores and the SDMA device of the artificial intelligence processor.

[0099] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein a plurality of external communication channels include: a data request channel; and a data response channel.

[0100] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor is provided, wherein a communication device is configured with a plurality of transmission queues that respectively cache target data corresponding to a plurality of external communication channels, and the method further includes using an arbitrator through the communication device to arbitrate a plurality of virtual sub-channels provided by each of the plurality of external communication channels to determine a target communication channel; and transmitting the target data cached in the transmission queue corresponding to the target virtual sub-channel among the plurality of candidate virtual sub-channels through the target virtual sub-channel.

[0101] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes arbitrating the plurality of external communication channels using the arbitrator via a communication device to determine a target communication channel; and determining a candidate virtual subchannel corresponding to the target communication channel as the target virtual subchannel.

[0102] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor is provided, wherein the communication device is configured with a plurality of receiving queues that respectively cache target data corresponding to a plurality of external communication channels.

[0103] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further includes an on-chip network, wherein the on-chip network is configured to communicate with a data transmission assistance unit, the method further includes transmitting target data between a communication device and a data transmission assistance unit via a plurality of internal communication channels, wherein at least two of the plurality of internal communication channels are merged into one of a plurality of external communication channels, and the number of the plurality of external communication channels is less than the number of the plurality of internal communication channels.

[0104] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein a plurality of internal communication channels include one or more of the following: a write address channel; a write data channel; a read address channel; a read data channel; and a write response channel.

[0105] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor includes a plurality of external communication channels: a first channel corresponding to a write address channel and a write data channel; a second channel corresponding to a read address channel; a third channel corresponding to a read data channel; and a fourth channel corresponding to a write response channel.

[0106] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the communication apparatus further includes a routing device, and the method further includes routing multiple internal communication channels to corresponding external communication channels via the routing device.

[0107] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further includes a first storage device for a second channel, wherein the bit width of the first storage device matches the size of the data packets transmitted on the second channel; and a second storage device for a fourth channel, wherein the bit width of the second storage device matches the size of the data packets transmitted on the fourth channel.

[0108] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor is provided, wherein the communication device is configured with a buffer and a plurality of first linked lists corresponding to a plurality of virtual sub-channels, wherein each of the plurality of first linked lists includes one or more first nodes, and the method further includes pointing to a storage area in the buffer for storing target data of the corresponding virtual sub-channel through each of the one or more first nodes.

[0109] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein a second linked list is configured in the communication device, wherein the second linked list includes one or more second nodes, and the method further includes pointing to a free storage area in the cache through each of the one or more second nodes.

[0110] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein each of a plurality of first linked lists includes a plurality of nodes associated with the storage address of target data in a cache associated with the corresponding virtual subchannel, wherein the method further includes releasing the storage area in the cache storing the confirmed target data in response to the confirmation of the target data corresponding to at least one of the plurality of nodes via an external network using a communication device.

[0111] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes, in response to determining that target data corresponding to at least one of a plurality of nodes has not been acknowledged, retransmitting the unacknowledged target data to an external communication network using a communication device, wherein determining that the target data has not been acknowledged by the communication device includes at least one of the following: receiving a negative acknowledgment for the target data through the external network; and not receiving a positive acknowledgment for the target data through the external network within a predetermined time period.

[0112] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes obtaining a target virtual subchannel by using an arbitrator provided by a communication device in at least one of polling and fair arbitration methods.

[0113] Figure 9 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown.

[0114] like Figure 9 As shown, the electronic device 300 is, for example, suitable for implementing the data transmission method for an artificial intelligence processor provided in embodiments of this disclosure. It should be noted that... Figure 9 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0115] For example, such as Figure 9 As shown, in some examples, electronic device 300 includes a processing device (e.g., including as referenced). Figures 3-4The AI ​​processor 301 shown is a parallel processor such as a graphics processor or a general-purpose graphics processor. It can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303 to achieve various functions, such as performing a data transmission method for an AI processor provided in at least one embodiment of this disclosure.

[0116] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the data transmission method for an artificial intelligence processor according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the data transmission method for an artificial intelligence processor can be found in the relevant descriptions in the embodiments of the data transmission method for an artificial intelligence processor described above; repeated details will not be repeated here.

[0117] RAM 303 also stores various programs and data required for the operation of the computer system. Processing device 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0118] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0119] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage device 308 as needed. Although Figure 8 An electronic device 300 including various devices is shown; however, it should be understood that implementation or inclusion of all shown devices is not required. More or fewer devices may be implemented or included alternatively. For example, a processor 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a parallel processor, an artificial intelligence processor, a tensor processor (TPU), or a graphics processing unit (GPU)—a device with data processing and / or program execution capabilities. The CPU may be based on an x86 or ARM architecture, etc. The GPU may be integrated directly onto the motherboard or built into the motherboard's northbridge chip. The GPU may also be built into the CPU.

[0120] For example, the electronic device 300 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device 309 can communicate wirelessly with a network and other devices, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0121] For example, electronic device 300 can be any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, or navigator, or any combination of electronic devices and hardware. The embodiments disclosed herein do not limit this.

[0122] For example, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, the data transmission method for an artificial intelligence processor as defined in the embodiments of this disclosure is performed.

[0123] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0125] The artificial intelligence processor, electronic device, and instruction processing method provided according to at least one embodiment of this disclosure can obtain the execution trajectory of instructions in a program, thereby accurately and sufficiently determining instruction latency. The artificial intelligence processor, electronic device, and instruction processing method provided according to at least one embodiment of this disclosure can accurately and sufficiently determine instruction latency while consuming relatively low memory and transmission costs.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0127] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0128] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0129] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0130] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0131] The following points should be noted regarding this disclosure:

[0132] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0133] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0134] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An artificial intelligence processor, comprising: The data transmission assistance unit is configured to transmit a data transmission request for the target data; A communication device is configured to transmit the data transmission request to the data transmission assistance unit, wherein the communication device is configured with multiple external communication channels for transmitting target data through a communication network connected to the artificial intelligence processor, and each of the multiple external communication channels is configured to provide multiple virtual sub-channels. The communication device includes an arbitrator, and the communication device is configured to: An arbitrator is used to arbitrate the provided multiple virtual sub-channels to obtain a target virtual sub-channel, and the target data is transmitted using the target virtual sub-channel. Specifically, the process of arbitrating the provided multiple virtual sub-channels using an arbitrator to obtain a target virtual sub-channel, and then using the target virtual sub-channel to transmit the target data, includes: Using the arbitrator, arbitrate multiple virtual sub-channels provided by each of the plurality of external communication channels to determine multiple candidate virtual sub-channels corresponding to each of the plurality of external communication channels; and The target data corresponding to the target virtual sub-channel among the plurality of candidate virtual sub-channels is sent through the target virtual sub-channel.

2. The artificial intelligence processor according to claim 1 further includes multiple computing cores and a system direct memory access device, wherein, The data transmission assistance unit is included in at least one of the computing cores of the artificial intelligence processor and at least one of the system direct memory access devices.

3. The artificial intelligence processor according to claim 1, wherein, The plurality of external communication channels include: Data request channel; and Data response channel.

4. The artificial intelligence processor according to claim 1, wherein, The communication device is configured with multiple transmission queues that respectively buffer target data corresponding to the multiple external communication channels, and the communication device is further configured to: The target data cached in the transmission queue corresponding to the target virtual sub-channel among the plurality of candidate virtual sub-channels is transmitted through the target virtual sub-channel.

5. The artificial intelligence processor according to claim 4, wherein, The communication device is also configured to: The arbitrator is used to arbitrate the plurality of external communication channels to determine the target communication channel. as well as The candidate virtual sub-channels corresponding to the target communication channel are determined as the target virtual sub-channels.

6. The artificial intelligence processor according to claim 1, wherein, The communication device is configured with multiple receiving queues that respectively cache target data corresponding to the multiple external communication channels.

7. The artificial intelligence processor according to claim 1, wherein, It also includes an on-chip network, wherein the on-chip network is configured to communicatively connect the data transmission assistance unit and transmit the target data between the communication device and the data transmission assistance unit via multiple internal communication channels. In this configuration, at least two of the multiple internal communication channels are merged into one of the multiple external communication channels, and the number of the multiple external communication channels is less than the number of the multiple internal communication channels.

8. The artificial intelligence processor according to claim 7, wherein, The plurality of internal communication channels includes one or more of the following: Write address channel; Write data channel; Read address channel; Read data channel; as well as Write the response channel.

9. The artificial intelligence processor according to claim 8, wherein, The plurality of external communication channels include: The first channel corresponding to the write address channel and the write data channel; The second channel corresponding to the read address channel; The third channel corresponding to the read data channel; and The fourth channel corresponding to the write response channel.

10. The artificial intelligence processor according to claim 9, wherein, The communication device further includes a routing device, wherein the routing device is configured to route the plurality of internal communication channels to corresponding external communication channels.

11. The artificial intelligence processor according to claim 9, further comprising: A first storage device is used for the second channel, wherein the bit width of the first storage device matches the size of the data packets transmitted on the second channel; and A second storage device is used for the fourth channel, wherein the bit width of the second storage device is matched with the size of the data packets transmitted on the fourth channel.

12. The artificial intelligence processor according to any one of claims 1-11, wherein, The communication device is configured with a buffer and multiple first linked lists corresponding to the multiple virtual sub-channels. Each of the multiple first linked lists includes one or more first nodes, and each of the one or more first nodes points to a storage area in the buffer used to store the target data of the corresponding virtual sub-channel.

13. The artificial intelligence processor according to claim 12, wherein, The communication device is configured with a second linked list, which includes one or more second nodes, each of which points to a free storage area in the cache.

14. The artificial intelligence processor according to claim 12, wherein, Each of the plurality of first linked lists includes multiple nodes associated with the storage address in the cache of the target data associated with the corresponding virtual sub-channel, wherein the communication device is configured to: In response to the confirmation of target data corresponding to at least one of the plurality of nodes via the communication network, the storage area in the buffer storing the confirmed target data is released.

15. The artificial intelligence processor according to claim 14, wherein, The communication device is further configured to retransmit the unacknowledged target data to the communication network in response to determining that target data corresponding to at least one of the plurality of nodes has not been acknowledged, wherein the communication device determines that the target data has not been acknowledged by at least one of the following: Receive a negative acknowledgment for the target data through the communication network; and No positive confirmation for the target data was received through the communication network within the predetermined time period.

16. The artificial intelligence processor according to any one of claims 1-11, wherein, The communication device is also configured to: The target virtual subchannel is obtained by using an arbitrator to provide the multiple virtual subchannels in at least one of the following arbitration methods: polling and fair arbitration.

17. An electronic device comprising an artificial intelligence processor according to any one of claims 1-16.

18. A data transmission method for an artificial intelligence processor, wherein, The artificial intelligence processor includes a data transmission assistance unit and a communication device, and the method includes: The data transmission assistance unit transmits a data transmission request for the target data. The communication device transmits the data transmission request to the data transmission assistance unit, wherein the communication device is configured with multiple external communication channels for transmitting target data through a communication network connected to the artificial intelligence processor, each of the multiple external communication channels is configured to provide multiple virtual sub-channels, and the communication device includes an arbitrator; The arbitrator is used to arbitrate the provided plurality of virtual sub-channels to obtain a target virtual sub-channel, and the target data is transmitted using the target virtual sub-channel. Specifically, the arbitrator is used to arbitrate the provided plurality of virtual sub-channels to obtain a target virtual sub-channel, and the target data is transmitted using the target virtual sub-channel. Using the arbitrator, arbitration is performed on multiple virtual sub-channels provided by each of the plurality of external communication channels to determine multiple candidate virtual sub-channels corresponding to each of the plurality of external communication channels, and The target data corresponding to the target virtual sub-channel among the plurality of candidate virtual sub-channels is transmitted through the target virtual sub-channel via the communication device.

Images