Artificial intelligence processors, electronic devices, and data transmission methods

Abstract

This disclosure provides an artificial intelligence processor, an electronic device, and a data transmission method. The artificial intelligence processor includes: a first sub-communication device configured to receive, via a communication network, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from a first other artificial intelligence processor, wherein the second data packet instructs the first other artificial intelligence processor to continuously transmit at least one first data packet until the second data packet is received; and a data operation unit configured to receive the first type of data packet and the second type of data packet from the first sub-communication device, wherein the first sub-communication device is further configured to: in response to the data operation unit receiving all at least one first data packet from the first sub-communication device, send the second data packet to the data operation unit. This artificial intelligence processor can conveniently confirm whether data transmission is complete.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of artificial intelligence technology, and particularly to artificial intelligence processors, electronic devices, and data transmission methods. Background Technology

[0002] In AI scenarios such as data centers, different chips may need to interconnect to exchange data when performing complex calculations such as training and inference of large models. Specifically, a chip may include multiple AI processors, and data transfer may be required between different AI processors on different chips to improve overall computing power.

[0003] However, data exchange between different AI processors, especially in scenarios where data transfer needs to be confirmed between different AI processors, may require significant assistance from the central processing unit (CPU) or frequent memory access. Summary of the Invention

[0004] An artificial intelligence processor is provided according to at least one embodiment of the present disclosure, comprising: a first sub-communication device configured to receive, via a communication network, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from the first other artificial intelligence processor, wherein the second data packet instructs the first other artificial intelligence processor to continuously send the at least one first data packet to the second data packet; and a data operation unit configured to receive the first type of data packet and the second type of data packet from the first sub-communication device, wherein the first sub-communication device is further configured to: in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, send the second data packet to the data operation unit.

[0005] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a first queue for data packets of the first type and a second queue for data packets of the second type; and the first sub-communication device is further configured to: store the at least one first data packet in the first queue for transmission to the data operation unit, receive response messages corresponding to the at least one first data packet from the data operation unit, store the second data packet in the second queue, and in response to receiving response messages corresponding to all of the at least one first data packet from the data operation unit, transmit the second data packet from the second queue to the data operation unit.

[0006] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a set of counters for the first other artificial intelligence processor, wherein the first sub-communication device is further configured to: increment the count value of a first counter in the set of counters in response to storing the at least one first data packet into the first queue respectively; decrement the count value of the first counter in response to receiving a response message corresponding to the at least one first data packet respectively from the data operation unit; and send the second data packet to the data operation unit in response to the count value of the first counter decreasing to zero.

[0007] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a third queue for first type data packets and second type data packets, and a multiplexer, wherein the first sub-communication device is further configured to: store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet in the third queue; and send at least one first data packet and the second data packet stored in the third queue to the first queue and the second queue respectively via the multiplexer in the order of reception through the communication network.

[0008] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a buffer and a first linked list and a second linked list for second type data packets, wherein the first linked list includes one or more first nodes, each of the one or more first nodes pointing to a storage area in the buffer for storing the corresponding second type data packets; and the second linked list includes one or more second nodes, each of the one or more second nodes pointing to a free storage area in the buffer.

[0009] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device is further configured to: in response to the existence of an idle counter in the counter set, send a data packet of the second type from the buffer to the second queue, and release the storage area corresponding to the sent data packet of the second type; and in response to the absence of an idle counter in the counter set, not send a data packet of the second type from the buffer to the second queue.

[0010] For example, according to at least one embodiment of the present disclosure, an artificial intelligence processor wherein the first sub-communication device is configured to receive a first type of data packet and a second type of data packet from a plurality of other artificial intelligence processors, including the first other artificial intelligence processor, via the communication network; and wherein there are a plurality of first linked lists, and the plurality of first linked lists respectively correspond to the plurality of other artificial intelligence processors.

[0011] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device is configured to receive a first type of data packet and a second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor via the communication network; wherein there are a plurality of second queues, and the plurality of second queues respectively correspond to the plurality of other artificial intelligence processors; and wherein there are a plurality of counter sets, and the plurality of counter sets respectively correspond to the plurality of other artificial intelligence processors.

[0012] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a first arbitrator, and the first arbitrator is configured to arbitrate second type data packets sent by a plurality of second queues to determine a first target data packet of the second type.

[0013] For example, in an artificial intelligence processor according to at least one embodiment of the present disclosure, the first sub-communication device further includes a routing device, and the routing device is configured to: receive a first type of data packet from the first queue and receive a second type of data packet from the second queue; send the first type of data packet and the second type of data packet to the data operation unit; and receive a response message for the first type of data packet from the data operation unit.

[0014] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first type of data includes a first identification field indicating a plurality of other artificial intelligence processors, including the first other artificial intelligence processor, that send the first type of data, and each counter in the counter set corresponds to a second identification field; and wherein the routing device is further configured to: split the first type of data packets into write address packets and write data packets for transmission to the data operation unit, and determine a third identification field based on the first identification field and the second identification field, and allocate the third identification field to the write address packets and the write data packets.

[0015] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the routing device is further configured to: identify a third identification field included in a received response message; determine a first identification field and a second identification field based on the third identification field; and determine a counter in the counter set whose count value is to be decremented based on the first identification field and the second identification field.

[0016] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a second arbitrator, and the second arbitrator is configured to: arbitrate a first type of data packet sent by the first queue and a second type of data packet sent by the second queue to determine a second target data packet in the first type of data packet and the second type of data packet; and send the second target data packet to the routing device.

[0017] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device further includes a second arbitrator, and the second arbitrator is configured to arbitrate the first type of data packets and the second type of data packets in such a way that the priority of the first type of data packets is lower than the priority of the second type of data packets.

[0018] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device is configured to receive a first type of data packets and a second type of data packets from a plurality of other artificial intelligence processors, including the first other artificial intelligence processor, through the communication network; and wherein the depth of the second queue for the first other artificial intelligence processor is the same as the number of counters in the counter set for the first other artificial intelligence processor.

[0019] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device is configured to receive, via a communication network, at least one third data packet of a first type following the second data packet from the first other artificial intelligence processor for the data operation unit; and wherein the first sub-communication device is configured to: in response to storing the second data packet in a second queue, determine that there is an idle second counter in the counter set, store the at least one third data packet in a first queue for transmission to the data operation unit, and in response to storing the at least one third data packet in the first queue respectively, stop incrementing the count value of the first counter and increment the count value of the second counter.

[0020] For example, according to at least one embodiment of the artificial intelligence processor of this disclosure, the first sub-communication device is configured to receive, via a communication network, at least one third data packet of a first type following the second data packet from the first other artificial intelligence processor for the data operation unit; and wherein the first sub-communication device is configured to: in response to storing the second data packet in a second queue, determine that there is no idle counter in the counter set, store the at least one third data packet in a first queue for transmission to the data operation unit, and in response to storing the at least one third data packet in the first queue respectively, continue to increment the count value of the first counter.

[0021] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure further includes a second sub-communication device configured to send at least one fourth data packet of a first type and a fifth data packet of a second type associated with the at least one fourth data packet to the communication network for the data operation unit, wherein the fifth data packet instructs the first other artificial intelligence processor to continuously send the at least one fourth data packet until the fifth data packet is sent; and wherein the second sub-communication device is further configured to: in response to receiving at least one fourth data packet from the data operation unit, send a response message to the data operation unit corresponding to the at least one fourth data packet respectively; in response to sending the response message corresponding to the at least one fourth data packet respectively, receive a fifth data packet from the data operation unit; and in response to receiving the fifth data packet from the data operation unit, send a response message corresponding to the fifth data packet to the data operation unit.

[0022] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure further includes a plurality of computing cores and system direct memory access means; and wherein the data manipulation unit is included in at least one of the computing cores and system direct memory access means of the artificial intelligence processor.

[0023] An electronic device is provided according to at least one embodiment of the present disclosure, including an artificial intelligence processor as described in any of the embodiments above.

[0024] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor is provided, wherein the artificial intelligence processor includes a data operation unit and a first sub-communication device, and the data operation unit is configured to receive data packets of a first type and data packets of a second type from the first sub-communication device. The method includes: receiving, via a communication network, at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet from the first other artificial intelligence processor, wherein the second data packet indicates that the first other artificial intelligence processor continuously transmits the at least one first data packet to the second data packet; and, in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, transmitting the second data packet to the data operation unit via the first sub-communication device.

[0025] The artificial intelligence processor, electronic device, and data transmission method provided according to at least one embodiment of this disclosure can conveniently confirm whether data transmission is complete while avoiding or reducing CPU involvement in the data transmission process of the artificial intelligence processor. Specifically, the artificial intelligence processor according to at least one embodiment of this disclosure can receive a first type of data packet and a second type of data packet associated with the first type of data packet. The second type of data packet can be received by the data operation unit of the artificial intelligence processor acting as the receiving end only after ensuring that all the first type of data packets associated with the second type of data packet have been received (e.g., written) by the artificial intelligence processor acting as the receiving end. In this way, the artificial intelligence processor acting as the receiving end can be indicated through the second type of data packet that all the first type of data packets associated with the second type of data packet have been received.

[0026] According to at least one embodiment of the artificial intelligence processor, electronic device, and data transmission method provided in this disclosure, while waiting for a first-type data packet associated with a certain second-type data packet to be received by an artificial intelligence processor acting as the receiving end, it is not prevented from allowing a first-type data packet not associated with that second-type data packet to be received by an artificial intelligence processor acting as the receiving end. For example, while waiting for a first-type data packet associated with a certain second-type data packet to be received, first-type data packets not associated with that second-type data packet can continue to be sent to the artificial intelligence processor acting as the receiving end. In this way, the data transmission efficiency between artificial intelligence processors can be increased, and congestion can be reduced.

[0027] According to at least one embodiment of the artificial intelligence processor, electronic device, and data transmission method provided in this disclosure, first-type data packets from different artificial intelligence processors acting as transmitters can be transmitted to a data processing unit without hindering second-type data packets from other artificial intelligence processors acting as transmitters. In this way, data transmission efficiency between artificial intelligence processors can be increased, and congestion can be reduced.

[0028] According to at least one embodiment of the artificial intelligence processor, electronic device, and data transmission method provided in this disclosure, after all first data packets associated with a certain second type of data packet from the artificial intelligence processor (as the sender) have completed their reception operations, the second type of data packet from the artificial intelligence processor (as the sender) can be sent to the data operation unit as quickly as possible. This avoids the second type of data packet being affected for a long time by first type of data packets not associated with the second type of data packet (e.g., first type of data packets following the second type of data packet). In this way, the data transmission efficiency between artificial intelligence processors can be increased, and congestion can be reduced. Attached Figure Description

[0029] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become clearer from the following description taken in conjunction with the accompanying drawings, in which:

[0030] Figure 1 This demonstrates an artificial intelligence processor applied to scenarios such as data centers.

[0031] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0032] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0033] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0034] Figure 5 A schematic structural diagram of a first sub-communication device of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0035] Figure 6 A flowchart of a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0036] Figure 7 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown. Detailed Implementation

[0037] Before proceeding with the detailed description below, it may be advantageous to define certain words and phrases used throughout this disclosure. The terms “comprising” and “including” and their derivatives mean including but not limited to. The term “or” is inclusive, meaning and / or. The phrase “associated with” and its derivatives mean including, comprising, interconnecting, containing, contained within, connected or connected to, coupled or coupled to, communicating with, cooperating, intertwining, juxtaposing, proximate, binding or bound to, having, possessing attributes, having a relationship or being related to, etc. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware, or a combination of hardware and software and / or firmware. The functionality associated with any particular controller may be centralized or distributed, local or remote. The phrase “at least one,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and that only one item from the list may be required. For example, "at least one of A, B, and C" includes any one of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C.

[0038] Definitions of other specific words and phrases are provided throughout this disclosure. Those skilled in the art will understand that, in many, if not most, cases, such definitions apply to the prior and future use of the words and phrases thus defined.

[0039] The various embodiments of the principles of this disclosure described below with reference to the accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or device. In some cases, the actions described in the specification may be performed in a different order and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require a specific order or sequential sequence to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0040] Artificial intelligence processors can include parallel processors such as graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), and tensor processors (TPUs). Figure 1 This demonstrates an artificial intelligence processor applied to scenarios such as data centers.

[0041] like Figure 1 As shown, a data center can deploy multiple servers, including Server 0 and Server 1. These servers can be used to perform tasks such as hosting websites and web applications with high computing power requirements, data storage and management, and running applications and software.

[0042] The following description uses server 0 as an example to illustrate the functionality of the components within the server. Server 1 can be the same as or similar to server 0; therefore, server 1 will not be described again to avoid redundancy. Server 0 may include a Central Processing Unit (CPU) 0, a root complex 0, PCIe switches 00 and 01, and GPUs 00, 01, 02, and 03. Server 1 may include a CPU 1, a root complex 1, PCIe switches 10 and 11, and GPUs 10, 11, 12, and 13. Root complex 1 may include root ports RP10 and RP11. PCIe switch 10 may include one uplink port USP10 and multiple downlink ports DSP11 and DSP12, and PCIe switch 11 may include one uplink port USP11 and multiple downlink ports DSP13 and DSP14. GPU10 may include uplink port USP12, GPU11 may include uplink port USP13, GPU12 may include uplink port USP14, and GPU13 may include uplink port USP15, thereby communicating with the corresponding PCIe switch.

[0043] CPU 0 can be used as the core computing and control component of server 0. CPU 0 can execute operating system and application instructions, perform logical operations and task scheduling by calling data stored in memory, and coordinate the collaborative work of various peripheral hardware components within the server 0 system to realize the general computing and overall control functions of server 0.

[0044] Root Complex 0 can be the core controller of a PCIe (Rapid Peripheral Component Interconnect) system architecture. Root Complex 0 can access the system via the uplink port ( Figure 1 (Not shown) is connected to the CPU and can be connected to PCIe switch 00 and PCIe switch 01 via downlink ports (e.g., root ports RP00 and RP01). Root complex 0 is responsible for enumerating, configuring, and managing the PCIe topology and acts as the root node for data communication, handling uplink and downlink data routing and protocol conversion.

[0045] PCIe switches 00 and 01 can be high-speed interconnect devices used to extend PCIe links. For example, PCIe switch 00 may include one uplink port USP00 and multiple downlink ports DSP01 and DSP02. PCIe switch 00 can achieve high-bandwidth, low-latency parallel data exchange between multiple downstream PCIe devices and the upstream root complex through internal crossbar switches, thereby effectively improving the system's peripheral connectivity. PCIe switch 01 may include one uplink port USP01 and multiple downlink ports DSP03 and DSP04. The function of PCIe switch 01 is similar to that of PCIe switch 00, and will not be described again here.

[0046] GPU00, GPU01, GPU02, and GPU03 can be dedicated coprocessors for parallel computing. GPU00 may include uplink port USP02, GPU01 may include uplink port USP03, GPU02 may include uplink port USP04, and GPU03 may include uplink port USP05, thereby communicating with the corresponding PCIe switch. GPU00, GPU01, GPU02, and GPU03 can adopt a massively parallel stream processor architecture, suitable for handling highly parallel floating-point and integer operations in graphics rendering, deep learning, and scientific computing. GPU00, GPU01, GPU02, and GPU03 can take over specific computing tasks from CPU 0, thereby significantly improving the overall computing throughput and energy efficiency of server 0. GPU00, GPU01, GPU02, and GPU03 can communicate through their communication ports (e.g., ... Figure 1 (As shown in the slanted shaded area) is connected to the corresponding communication port of the Ethernet switch.

[0047] With the development of training large models, the demand for computing power in data centers is increasing, thus requiring the interconnection of multiple GPUs in data centers for data transmission. According to at least one embodiment of this disclosure, multiple GPUs can interconnect via Ethernet, for example, using their communication ports. Ethernet primarily follows the IEEE 802.3 standard. Ethernet can be a local area network (LAN) technology used for transmitting data within a local area network. As a widely used LAN technology, Ethernet allows multiple devices (e.g., computers such as servers, printers, switches, etc.) to communicate with each other.

[0048] Figure 1 The communication ports configured in each GPU (e.g., GPU00-GPU03 and GPU10-GPU13) are as follows: Figure 1(As shown in the shaded area) can be connected to an Ethernet switch (e.g., via communication ports 0-7 of the Ethernet switch) via an Ethernet link. The following description uses access to other GPUs by GPU00 of server 0 as an example. GPU00 of server 0 can send Ethernet packets to the Ethernet switch, which can determine which communication port of the other GPU to forward the Ethernet packet to based on the destination address field in the packet header.

[0049] In multi-chip interconnect scenarios, existing technologies still face significant challenges. TCP / IP and Remote Direct Memory Access (RDMA) technologies can be used for data transfer between GPUs. However, TCP / IP requires implementation within the operating system kernel. During data transmission, data traverses the kernel protocol stack multiple times, resulting in high latency and consequently low performance. For example, traditional TCP / IP-based inter-processor communication introduces microsecond-level latency far exceeding the nanosecond-level data synchronization requirements of AI training, and CPU kernel interrupt handling and context switching consume substantial computational resources. Furthermore, both TCP / IP and RDMA require CPU assistance, thus consuming significant CPU resources and degrading CPU performance.

[0050] Furthermore, in large-scale artificial intelligence (AI) clusters, RDMA technology involves dynamic, concurrent communication connections between tens of thousands of processors. This can lead to rapid exhaustion of RDMA connection management entries and difficulty in providing fine-grained, predictable quality of service. In addition, RDMA technology implements an end-to-end retransmission mechanism to achieve lossless data transmission. For example, an AI processor acting as the sender can send a message, and an AI processor acting as the receiver can reply with an acknowledgment (ACK) to indicate that it has received the message.

[0051] Specifically, GPU 00, acting as the sender, can establish an RDMA communication channel with GPU 10, acting as the receiver, via a socket. For example, a user at the sender end can send instructions to GPU 00, such as causing the GPU 00's computation core to write the computed data into memory. The user at the sender end can also send instructions (i.e., work queue elements (WQEs)) to the RDMA device on GPU 00. Furthermore, the user at the sender end can send instructions to the RDMA device on GPU 00 to trigger it to read the aforementioned WQE. The RDMA device on GPU 00 can then read the written data from memory based on the WQE and encapsulate the data into an Ethernet packet, sending it to GPU 10. After receiving the data, GPU 10 verifies it and replies with an ACK packet to GPU 00, writing the data to a designated location on GPU 10. Upon receiving the ACK, GPU 00 can generate a completion queue element (CQE) and place it at the designated location. The user at the sender end can then obtain the CQE.

[0052] During the process of GPU 00 sending data to GPU 10, GPU 00 can act as a data producer in a producer-consumer model, and GPU 10 can act as a data consumer in the same model. After sending data, GPU 00 needs to notify GPU 10 that the data write is complete. When the RDMA device uses the InfiniBand (IB) protocol, an RDMA write operation with immediate data can be used for this notification. For example, the RDMA transmit queue (SQ) on GPU 00, the sending end, can have two write queues (WQEs), such as WQE1 and WQE2. WQE1 can be used to perform an RDMA write operation, writing data to a specified location on GPU 10, the receiving end. WQE2 can be used to perform an RDMA write operation with immediate data, informing GPU 10, the receiving end, that the data has been written. However, the aforementioned RDMA write operation with immediate data is based on memory semantics, not control or protocol semantics. In other words, the immediate value is still stored in a designated area of ​​memory to inform the GPU 10, which is the receiving end, that the above data reception is complete.

[0053] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0054] Artificial intelligence processors may include parallel processors such as graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), and tensor processors (TPUs), but this disclosure is not limited thereto. Figure 2 As shown, an artificial intelligence processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a Streaming Processor Cluster (SPC), which includes... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In an AI processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0055] like Figure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores (also called computational cores), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These Cores are used to execute specific computational tasks. Furthermore, a Computation Unit may also include registers (e.g., ...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0056] like Figure 2 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0057] like Figure 2 As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0058] In parallel computing, computational tasks are typically executed through multiple threads. These threads are divided into multiple thread blocks before execution in the artificial intelligence processor (or general-purpose graphics processor or parallel computing processor), and then dispatched via a thread block dispatch module. Figure 2(Not shown in the diagram) Multiple thread blocks are distributed to various computing units. All threads in a thread block must be assigned to the same computing unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple threads from multiple thread blocks can execute on multiple computing cores within the same computing unit, or on multiple computing cores in different computing units, sharing registers, shared memory, and other data transfer mechanisms.

[0059] In each computing unit, the thread beam scheduling / distribution module ( Figure 2 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., ...). Figure 2 It is used for reading and writing operations in high-bandwidth memory (HBM).

[0060] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 3 As shown, the artificial intelligence processor 3000 may include a first sub-communication device 3100 and a data operation unit 3200. Figure 3 The artificial intelligence processor in it can correspond to Figure 2 The computational unit will not be described again here for the sake of brevity.

[0061] According to at least one embodiment of this disclosure, the data operation unit 3200 can be configured to receive data sent from other artificial intelligence processors and to other artificial intelligence processors. For example, other artificial intelligence processors can write data to artificial intelligence processor 3000 through the data operation unit 3200, and artificial intelligence processor 3000 can write data to other artificial intelligence processors through the data operation unit 3200, but this disclosure is not limited thereto. The data that artificial intelligence processor 3000 wants to write to other artificial intelligence processors through the data operation unit 3200 and the data that other artificial intelligence processors want to write to artificial intelligence processor 3000 through the data operation unit 3200 can be data that multiple artificial intelligence processors, including artificial intelligence processor 3000, wish to transmit to improve computing power. The data operation unit 3200 can receive first type data packets and second type data packets from the first sub-communication device 3100.

[0062] According to at least one embodiment of this disclosure, the first sub-communication device 3100 can be configured to transmit data packets to the data operation unit 3200. For example, the first sub-communication device 3100 can be connected to a communication network external to the artificial intelligence processor for data transmission. The first sub-communication device 3100 can send first-type and second-type data packets sent by other artificial intelligence processors through the communication network to the data operation unit 3200. According to at least one embodiment of this disclosure, the first sub-communication device 3100 can be a receiving device in the Ethernet port of the artificial intelligence processor 3000.

[0063] According to at least one embodiment of this disclosure, a first sub-communication device 3100 can receive at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from a first other artificial intelligence processor via a communication network. That is, the second data packet and the at least one first data packet can be sent from the first other artificial intelligence processor to the artificial intelligence processor 3000 as a batch. The second data packet can instruct the first other artificial intelligence processor to continuously send at least one first data packet until the second data packet is sent. For example, the first other artificial intelligence processor can first send at least one first data packet, and then immediately send the second data packet after sending the at least one first data packet. The first sub-communication device 3100 can receive the at least one first data packet and the second data packet in the sending order of the first other artificial intelligence processor. For example, for the first other artificial intelligence processor, the first sub-communication device 3100 can first receive at least one first data packet, and then immediately receive the second data packet after receiving the at least one first data packet.

[0064] According to at least one embodiment of this disclosure, the first sub-communication device 3100 may also be configured to send a second data packet to the data operation unit 3200 in response to the data operation unit 3200 receiving all at least one first data packet from the first sub-communication device 3100. According to at least one embodiment of this disclosure, the first sub-communication device 3100 may determine that the data operation unit 3200 has received all at least one first data packet from the first sub-communication device 3100 by using a counter or a queue pointer; however, this disclosure is not limited to this, and other possible methods are also possible. That is, the first sub-communication device 3100 may synchronize the ready state by sending a second data packet to the data operation unit 3200 when at least one first data packet has been received by the data operation unit 3200 (i.e., at least one first data packet is ready). The second data packet may be based on control semantics or protocol semantics, rather than memory semantics.

[0065] According to at least one embodiment of the present disclosure, the artificial intelligence processor 3000 can directly perform data transmission synchronization using a data operation unit 3200 and a first sub-communication device 3100 included within the artificial intelligence processor 3000, without the assistance of a CPU external to the artificial intelligence processor 3000. In this way, the workload of the CPU on data transmission to the artificial intelligence processor 3000 can be reduced, allowing the CPU to allocate computing and storage resources to other tasks, thus improving CPU performance. Therefore, the performance of computing devices (e.g., servers) deploying the CPU and the artificial intelligence processor 3000 can be enhanced.

[0066] According to at least one embodiment of the AI ​​processor 3000 of this disclosure, a second type of data packet may be received by the AI ​​processor 3000's data operation unit 3200 only after ensuring that all first type of data packets associated with the second type of data packet have been received (e.g., written) by the data operation unit 3200 of the AI ​​processor 3000 acting as the receiving end. The second type of data packet can be used to indicate to the data operation unit 3200 of the AI ​​processor 3000, acting as the receiving end, that all first type of data packets associated with the second type of data packet have been received. In this way, the data transmission capability between AI processors can be increased.

[0067] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 4 As shown, the artificial intelligence processor 4000 may include a computing core 4110, a system direct memory access (SDMA) device 4120, an on-chip network (NOC) 4200, and a first sub-communication device 4300. Figure 4 Zhongyu Figure 3 Components that are identical or similar will not be described again to avoid redundancy. Figure 4 The AI ​​processor 4000 in the middle can correspond to Figure 2 The computing unit, Figure 4 The computational kernel in the middle can correspond to Figure 2 The core of it will not be described again here for the sake of brevity.

[0068] According to at least one embodiment of this disclosure, an artificial intelligence processor 4000 may include at least one computing core 4110. Each computing core in the artificial intelligence processor 4000 may include one or more instruction processing units, such as, but not limited to, an instruction fetch unit, a decoding unit, an instruction issue unit, and an artificial intelligence unit, to perform processing operations on instructions. For example, instructions in a computing core may undergo instruction fetch, instruction decoding, instruction issue, instruction execution, and write-back. According to at least one embodiment of this disclosure, the instruction fetch unit may be configured to fetch instructions to be executed subsequently. The decoding unit may be configured to decode the fetched instructions into control signals executable by the underlying hardware. The issue unit may monitor the execution status of a thread or thread bundle (e.g., waiting for operands or hardware resources to be ready) and issue instructions that meet the issue conditions to the execution unit, such as the artificial intelligence computing unit. The artificial intelligence computing unit may be configured to perform artificial intelligence-related calculations based on the issued instructions. The write-back unit may write the data output by the artificial intelligence computing unit back to memory or cache.

[0069] According to at least one embodiment of this disclosure, the SDMA device 4120 can be a dedicated engine for implementing system direct memory access. The SDMA device 4120 can independently handle data transfer tasks between the video memory and system memory of the AI ​​processor 4000 without CPU assistance or intervention. The SDMA device 4120 separates data transfer from the operation of the AI ​​processor 4000, effectively eliminating the adverse effects of data transfer latency on computing tasks, thereby significantly improving the overall throughput efficiency and system energy efficiency of the AI ​​processor 4000.

[0070] According to at least one embodiment of this disclosure, a data operation unit may be included in at least one computing core and an SDMA device. For example, data operation unit 4111 may be disposed in computing core 4110 and data operation unit 4121 may be disposed in SDMA device 4120, but this disclosure is not limited thereto. The data operation unit may be disposed in any device of the artificial intelligence processor 4000 that can send or receive data packets of the first type and data packets of the second type.

[0071] According to at least one embodiment of the present disclosure, the NOC 4200 may be a mesh topology and provide a high-bandwidth, low-latency and scalable data exchange path for multiple parallel processing units (e.g., computing core 4110 and SDMA device 4120, etc.) and a first sub-communication device 4300 through a configured communication protocol (e.g., Advanced Extensible Interface (AXI) protocol, but the present disclosure is not limited thereto).

[0072] According to at least one embodiment of this disclosure, the first sub-communication device 4300 may include a first queue 4310, a second queue 4320, and a counter set 4330. Figure 4 The Media Access Control (MAC) layer, not shown in the diagram, and Figure 4 The serializer / deserializer (SerDes) is not shown. According to at least one embodiment of this disclosure, the first queue 4310, the second queue 4320, and the counter set 4330 may be configured for a first other artificial intelligence processor. Additionally or alternatively, additional first queues, second queues, and counter sets may be configured for other artificial intelligence processors besides the first other artificial intelligence processor.

[0073] According to at least one embodiment of this disclosure, the MAC layer can be a data link layer core controller. The MAC unit can assemble and parse data frames, execute access control protocols such as Carrier Sense Multiple Access with Collision Detection (CSMA / CD), manage frame check sequences, and perform flow control. The MAC unit can process data link packets, providing a standardized data interface to upper layers.

[0074] According to at least one embodiment of this disclosure, SerDes can realize high-speed serial communication. SerDes can convert parallel data output from the MAC layer into a serial bit stream for transmission, and reconstruct the received serial signal into parallel data. SerDes overcomes the transmission loss of PCB traces and cables through technologies such as clock recovery and channel coding, enabling high-speed, long-distance data exchange between the AI ​​processor 4000 and external communication networks.

[0075] According to at least one embodiment of this disclosure, a first sub-communication device 4300 may first receive at least one first data packet from a first other artificial intelligence processor, and immediately after receiving the at least one first data packet, receive a second data packet. A first queue 4310 of the first sub-communication device 4300 may be for first-type data packets and a second queue 4320 may be for second-type data packets.

[0076] According to at least one embodiment of the present disclosure, the first sub-communication device 4300 can store at least one first data packet in a first queue 4310 for transmission to a data operation unit. For example, at least one first data packet stored in the first queue 4310 can be transmitted via NOC 4200 to at least one of data operation units 4111 and 4121. According to at least one embodiment of the present disclosure, the first queue 4310 can be a first-in-first-out (FIFO) queue. In response to storing at least one first data packet in the first queue 4310, the count value of a first counter in the counter set 4330 can be incremented. For example, in response to storing a first data packet in the first queue 4310, an idle counter in the counter set 4330 can be selected as the first counter, and the first counter can be used to count the first data packets entering the first queue 4310. According to at least one embodiment of the present disclosure, the bit width of the counter can be 9 bits, but the present disclosure is not limited thereto.

[0077] According to at least one embodiment of this disclosure, after sending all at least one first data packet to the first queue 4310, the first sub-communication device 4300 can store the second data packet in the second queue 4320. According to at least one embodiment of this disclosure, the second queue 4320 can be a first-in-first-out (FIFO) queue. Furthermore, after sending at least one first data packet to at least one of the data operation unit 4111 and the data operation unit 4121, the first sub-communication device 4300 can receive response messages corresponding to the at least one first data packet from at least one of the data operation unit 4111 and the data operation unit 4121.

[0078] According to at least one embodiment of this disclosure, in response to receiving a response message corresponding to all at least one first data packet from at least one of data operation unit 4111 and data operation unit 4121, the first sub-communication device 4300 can send a second data packet from the second queue 4320 to at least one of data operation unit 4111 and data operation unit 4121. For example, the second data packet stored in the second queue 4320 can be sent to at least one of data operation unit 4111 and data operation unit 4121 via NOC 4200. Specifically, in response to receiving a response message corresponding to at least one first data packet from at least one of data operation unit 4111 and data operation unit 4121, the first sub-communication device 4300 can decrement the count value of a first counter. In response to the count value of the first counter decreasing to zero, the first sub-communication device 4300 can send the second data packet to at least one of data operation unit 4111 and data operation unit 4121.

[0079] Figure 5A schematic structural diagram of a first sub-communication device of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 5 As shown, the first sub-communication device 5001 of the artificial intelligence processor may include a first queue 5100, a second queue 5200, a counter set 5300, a third queue 5400, a first arbitrator 5410, a buffer 5600, a second arbitrator 5700, and a routing device 5800. Figure 5 China's first sub-communication device 5001 and Figure 2 and Figure 3 Components that are identical or similar to the communication devices described herein will not be described again to avoid redundancy.

[0080] According to at least one embodiment of this disclosure, the third queue 5400 can be used for first type data packets and second type data packets. For a first other artificial intelligence processor, the first sub-communication device 5001 can store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet in the third queue 5400. Further, the first sub-communication device 5001 can send at least one first data packet and the second data packet stored in the third queue 5400 to the first queue 5100 and the second queue 5200 respectively via a multiplexer 5500, according to the receiving order through the communication network. According to at least one embodiment of this disclosure, the third queue 5400 can be a first-in-first-out queue, and the first sub-communication device 5001 can store at least one first data packet of the first type in the third queue 5400 first, and then store the second data packet of the second type associated with the at least one first data packet in the third queue 5400 afterward, according to the receiving order.

[0081] According to at least one embodiment of this disclosure, a first sub-communication device 5001 can receive data packets of a first type and a second type from a communication network via a third queue 5400, which includes a first other artificial intelligence processor. The first sub-communication device 5001 can store the data packets of the first type and the second type from the multiple artificial intelligence processors in the third queue 5400 in the order they are received.

[0082] According to at least one embodiment of this disclosure, the first type of data may include a first identification field indicating a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, that sent the first type of data. In response to at least one first data packet of the first type from the first other artificial intelligence processor entering the first queue 5100, the first data packet can be identified as originating from the first other artificial intelligence processor based on the first identification field in the first data packet. As previously described, a plurality of counter sets can be configured; for example, a counter set can be configured for each of the first other artificial intelligence processors including the first other artificial intelligence processor. The at least one first data packet can be determined to originate from the first other artificial intelligence processor based on the first identification field included in the at least one first data packet, and an idle counter can be selected from the plurality of counter sets 5300 corresponding to the counter set of the first other artificial intelligence processor to count the at least one first data packet entering the first queue 5100. The first queue 5100 can be used to ensure the timing requirements of the first type of data packets, and therefore the depth of the first queue 5100 can be 1, but this disclosure is not limited thereto.

[0083] According to at least one embodiment of this disclosure, the buffer 5600 may store second-type data packets received from the multiplexer 5500. For example, for a first other artificial intelligence processor, after the third queue 5400 has sent all of at least one first data packet to the first queue 5100 via the multiplexer 5500, the third queue 5400 may send second data packets associated with at least one first data packet to the buffer 5600 for caching via the multiplexer 5500. The first sub-communication device 5001 may also be configured with a first linked list. For example, the first linked list may be a data linked list. The first linked list may include one or more first nodes, each of which points to a storage area in the buffer 5600 for storing the corresponding second data packet.

[0084] According to at least one embodiment of this disclosure, the first sub-communication device 5001 can receive data packets of a first type and data packets of a second type from a plurality of other artificial intelligence processors. Therefore, a corresponding first linked list can be configured for each of the plurality of other artificial intelligence processors. Each of one or more first nodes in the first linked list can point to a storage area in the cache 5600 for storing the corresponding second type of data packets.

[0085] According to at least one embodiment of this disclosure, each of the plurality of first linked lists can be configured to track the storage addresses of storage areas (e.g., data slots) of a second type of data packets associated with corresponding other artificial intelligence processors in a cache.

[0086] Specifically, each of the multiple first linked lists includes multiple first nodes associated with the storage address in the cache of a second type of data packet associated with a corresponding other AI processor. A first node in a first linked list can store a pointer to the next first node, i.e., the storage address of the next storage area (e.g., a data slot) in the cache. The head and tail of the corresponding first linked list can be recorded for each corresponding other AI processor to allow for quick appending to the tail when writing data and quick reading from the head when reading data. Furthermore, a second linked list can be configured in the first sub-communication device 5001. The second linked list can include one or more second nodes, and each of the one or more second nodes can point to a free storage area in the cache 5600. The second linked list can be a global linked list, such as a free list. The second linked list can be configured to record the storage addresses of free storage areas (e.g., data slots) in the cache associated with multiple other AI processors. For example, when a second type of data packet is read from the cache 5600 (e.g., sent to the second queue 5200), the corresponding data slot can be released.

[0087] The artificial intelligence processor according to at least one embodiment of this disclosure can achieve efficient cache management. By employing dynamic cache management using a first linked list such as a data linked list and a second linked list such as a free linked list, efficient reuse of cache free space is achieved, reducing storage resource waste.

[0088] According to at least one embodiment of the artificial intelligence processor disclosed herein, by using a buffer 5600, while waiting for a first-type data packet associated with a certain second-type data packet to be received by the receiving AI processor, the receiving operation of a first-type data packet not associated with that second-type data packet can be continued without hindering the receiving operation of a first-type data packet not associated with that second-type data packet by the receiving AI processor. For example, while waiting for a first-type data packet associated with a certain second-type data packet to be received, first-type data packets not associated with that second-type data packet can continue to be sent to the receiving AI processor. In this way, the data transmission efficiency between AI processors can be increased, and congestion can be reduced.

[0089] According to at least one embodiment of this disclosure, in response to the presence of an idle counter in the counter set, a second type of data packet can be sent from the cache 5600 to the second queue 5200, and in response to the absence of an idle counter in the counter set, the second type of data packet can be left unsent from the cache 5600 to the second queue 5200. For example, for a first other artificial intelligence processor, in response to the presence of an idle counter in the counter set corresponding to the first other artificial intelligence processor, the aforementioned second data packet can be sent from the cache 5600 to the second queue 5200, and in response to the absence of an idle counter in the counter set corresponding to the first other artificial intelligence processor, the aforementioned second data packet can be left unsent from the cache 5600 to the second queue 5200.

[0090] According to at least one embodiment of this disclosure, since the first sub-communication device 5001 can receive first-type data packets and second-type data packets from a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, a corresponding second queue can be configured for each other artificial intelligence processor. For example, the second queue 5200 may include N queues 5201 to 520N. As an example, N can be 1024, that is, the first sub-communication device 5001 can receive first-type data packets and second-type data packets from 1024 other artificial intelligence processors, but this disclosure is not limited thereto, and N can be more or less.

[0091] According to at least one embodiment of this disclosure, for each other AI processor, the depth of the corresponding second queue can be the same as the number of counters in the corresponding counter set. For example, the depth of the second queue for the first other AI processor is the same as the number of counters in the counter set for the first other AI processor. According to at least one embodiment of this disclosure, the depth of the corresponding second queue and the number of counters in the corresponding counter set can be set to at least 2. In this way, after all first data packets associated with a certain second type of data packet from the AI ​​processor acting as the sender have completed their reception operations, the second type of data packet from the AI ​​processor acting as the sender can be sent to the data operation unit as quickly as possible, thereby avoiding the second type of data packet being affected for a long time by first type of data packets not associated with the second type of data packet (e.g., first type of data packets following the second type of data packet).

[0092] According to at least one embodiment of this disclosure, in response to the count value of the counter corresponding to the first (e.g., the data packet about to be sent) of the second type in the queues 5201-520N included in the second queue 5200 being reduced to zero by a corresponding response message, the corresponding second type of data packet can be sent.

[0093] According to at least one embodiment of this disclosure, a first arbitrator 5410 may be configured to arbitrate second-type data packets sent by multiple queues 5201 to 520N included in a second queue 5200 to determine a first target data packet of the second type. For example, the first arbitrator 5410 may arbitrate second-type data packets sent by multiple queues 5201 to 520N in at least one of a polling manner and a fair arbitration manner. According to at least one embodiment of this disclosure, the weights used by the first arbitrator 5410 in performing the fair arbitration process may not be fixed. For example, the weights may be dynamically updated based on bandwidth requirements, service priorities, etc., but this disclosure is not limited to this, and other methods for determining the weights of the fair arbitration process are also possible.

[0094] According to at least one embodiment of this disclosure, a second arbitrator 5700 can be configured to arbitrate data packets of a first type sent by a first queue 5100 and data packets of a second type sent by a second queue 5200 to determine a second target data packet among the first and second type data packets. The second arbitrator 5700 can then send the second target data packet determined through arbitration to a routing device 5800. The second arbitrator 5700 can be further configured to arbitrate the first and second type data packets in such a way that the first type data packets have a lower priority than the second type data packets. This allows the second type data packets to be sent as quickly as possible, preventing them from being affected by subsequent first type data packets.

[0095] According to at least one embodiment of this disclosure, routing device 5800 can be configured to receive data packets of a first type from a first queue 5100 and data packets of a second type from a second queue 5200 (e.g., via a first arbitrator 5410 and a second arbitrator 5700). Routing device 5800 can transmit the first and second type data packets to a data operation unit, for example, via NOC 5002. Further, response messages for the first type data packets can be received from the data operation unit, for example, via NOC 5002.

[0096] According to at least one embodiment of this disclosure, the first type of data includes a first identification field indicating other artificial intelligence processors, including the first other artificial intelligence processor, that send the first type of data, and each counter in the counter set corresponds to a second identification field. For example, for the first other artificial intelligence device, the routing device 5800 can split the first type of data packets into write address packets and write data packets for transmission to the data operation unit via NOC 5002, and the routing device 5800 can determine a third identification field based on the first and second identification fields, and assign the third identification field to the write address packets and write data packets. As mentioned above, the NOC 5002 may be configured with the AXI protocol, and by splitting the first type of data packets into write address packets and write data packets in the routing device 5800, whole packet scheduling can be used before converting the first type of data packets into AXI requests. In this way, premature separation of the AXI AW (write address) channel and the write data (W) channel can be avoided, thereby simplifying the design and reducing the probability of design errors. According to at least one embodiment of this disclosure, the routing device 5800 can similarly split the second type of data packets, or assign them to the corresponding channels in the AXI protocol.

[0097] According to at least one embodiment of this disclosure, after the data operation unit completes receiving a data packet of the first type having a third identifier field, it can return a response message (e.g., a write response message) having a third identifier field. Since the third identifier fields corresponding to the write data packets and write channel packets split from different data packets can be different, the data operation unit can return response messages out of order. The routing device 5800 can also be configured to identify the third identifier field included in the received response message and determine a first identifier field and a second identifier field based on the third identifier field. The routing device 5800 can determine the counters in the counter set whose count values ​​need to be decremented based on the first identifier field and the second identifier field using a reverse lookup method.

[0098] According to at least one embodiment of this disclosure, a first sub-communication device 5001 can be configured to receive at least one third data packet of a first type from a first other artificial intelligence processor via a communication network for a data operation unit. According to at least one embodiment of this disclosure, the first sub-communication device 5001 can determine whether there is an idle second counter in the counter set for the first other artificial intelligence processor in response to storing the second data packet in a second queue. For example, in response to determining that there is an idle second counter, at least one third data packet is stored in a first queue 5100 for transmission to the data operation unit. Further, in response to storing at least one third data packet in the first queue 5100 respectively, the increment of the first counter can be stopped and the increment of the second counter can be increased. That is, in response to the second data packet entering the second queue, the increment of the first counter corresponding to the second data packet can be stopped, and the count of at least one third data packet following the second data packet entering the first queue 5100 can be switched to an idle second counter.

[0099] According to at least one embodiment of this disclosure, a first sub-communication device 5001 can be configured to store at least one third data packet in a first queue for transmission to a data operation unit in response to determining that there are no idle counters in the counter set (e.g., all counters are being used for counting, and all corresponding count values ​​have not yet decreased to zero). In response to storing at least one third data packet in the first queue, the count value of the first counter can continue to increment. That is, when a second data packet from a first other artificial intelligence processor enters the second queue, if there are no idle counters in the counter set used by the first other artificial intelligence processor, the current first counter can continue to be used to count subsequent at least one third data packet. Further, the second data packet can be transmitted from the second queue in response to the count value of the corresponding first counter decreasing to zero via a response message.

[0100] According to at least one embodiment of this disclosure, the artificial intelligence processor may further include a second sub-communication device. For example, the second sub-communication device may be a transmitting device in an Ethernet port of the artificial intelligence processor. The second sub-communication device may be configured to transmit at least one fourth data packet of a first type and a fifth data packet of a second type associated with the at least one fourth data packet to a communication network for the data operation unit. The fifth data packet may instruct a first other artificial intelligence processor to continuously transmit at least one fourth data packet until the fifth data packet. The second sub-communication device may be configured to send a response message (e.g., b response) corresponding to each of the at least one fourth data packet to the data operation unit in response to receiving the at least one fourth data packet from the data operation unit. In response to sending the response message corresponding to each of the at least one fourth data packet, the second sub-communication device may receive the fifth data packet from the data operation unit. In response to receiving the fifth data packet from the data operation unit, the second sub-communication device sends a response message corresponding to the fifth data packet to the data operation unit. In this way, latency can be reduced, allowing the data operation unit to continuously transmit the first type of data packets and the second type of data packets.

[0101] Figure 6 A flowchart illustrating a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 6 As shown, method 6000 includes steps S6100-S6200. The execution is as follows: Figure 6 The artificial intelligence processor of the illustrated method may include a data operation unit and a first sub-communication device. The data operation unit may be configured to receive data packets of a first type and a second type from the first sub-communication device.

[0102] In step S6100, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet can be received from a first other artificial intelligence processor via a communication network using the first sub-communication device, wherein the second data packet instructs the first other artificial intelligence processor to continuously send the at least one first data packet to the second data packet.

[0103] In step S6200, in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, the second data packet may be sent to the data operation unit using the first sub-communication device.

[0104] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor, wherein the first sub-communication device further includes a first queue for data packets of a first type and a second queue for data packets of a second type; and the method further includes, through the first sub-communication device: storing at least one first data packet in the first queue for transmission to a data operation unit, receiving response messages corresponding to at least one first data packet from the data operation unit, storing a second data packet in the second queue, and, in response to receiving response messages corresponding to all at least one first data packet from the data operation unit, transmitting the second data packet from the second queue to the data operation unit.

[0105] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein a first sub-communication device further includes a set of counters for a first other artificial intelligence processor, wherein the method further includes, via the first sub-communication device: in response to storing at least one first data packet into a first queue, incrementing the count value of a first counter in the set of counters; in response to receiving a response message corresponding to at least one first data packet from a data operation unit, decrementing the count value of the first counter; and in response to the count value of the first counter decreasing to zero, sending a second data packet to the data operation unit.

[0106] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor, wherein a first sub-communication device further includes a third queue for first type data packets and second type data packets, and a multiplexer, wherein the method further includes, through the first sub-communication device: storing at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue; and, according to the receiving order through the communication network, sending at least one first data packet and the second data packet stored in the third queue to the first queue and the second queue respectively through the multiplexer.

[0107] According to at least one embodiment of the data transmission method for an artificial intelligence processor disclosed herein, the first sub-communication device further includes a buffer and a first linked list and a second linked list for second type data packets, wherein the first linked list includes one or more first nodes, each of the one or more first nodes pointing to a storage area in the buffer for storing the corresponding second type data packets; and the second linked list includes one or more second nodes, each of the one or more second nodes pointing to a free storage area in the buffer.

[0108] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes, via a first sub-communication device: in response to the presence of an idle counter in the counter set, sending a second type of data packet from a buffer to a second queue and releasing the storage area corresponding to the sent second type of data packet; and in response to the absence of an idle counter in the counter set, not sending the second type of data packet from the buffer to the second queue.

[0109] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor receives data packets of a first type and a second type from a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, via a communication network through a first sub-communication device; wherein there are multiple first linked lists, and the multiple first linked lists respectively correspond to the multiple other artificial intelligence processors.

[0110] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes receiving, via a first sub-communication device, data packets of a first type and data packets of a second type from a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, via a communication network; wherein there are multiple second queues, and the multiple second queues correspond to the multiple other artificial intelligence processors respectively; and wherein there are multiple counter sets, and the multiple counter sets correspond to the multiple other artificial intelligence processors respectively.

[0111] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further includes a first arbitrator, and the method further includes arbitrating data packets of a second type sent by a plurality of second queues using the first arbitrator to determine a first target data packet of the second type.

[0112] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further includes a routing device, and wherein the method further includes, via the routing device: receiving a first type of data packet from a first queue and receiving a second type of data packet from a second queue; sending the first type of data packet and the second type of data packet to a data operation unit; and receiving a response message for the first type of data packet from the data operation unit.

[0113] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein a first type of data includes a first identification field indicating a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, to send the first type of data, and each counter in a counter set corresponds to a second identification field; and wherein the method further includes, via a routing device: splitting the first type of data packets into write address packets and write data packets for transmission to a data operation unit, and determining a third identification field based on the first identification field and the second identification field, and allocating the third identification field to the write address packets and the write data packets.

[0114] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes, via a routing device: identifying a third identification field included in a received response message; determining a first identification field and a second identification field based on the third identification field; and determining a counter in a counter set whose count value is to be reduced based on the first identification field and the second identification field.

[0115] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further includes a second arbitrator, and wherein the method further includes, through the second arbitrator, arbitrating data packets of a first type sent by a first queue and data packets of a second type sent by a second queue to determine a second target data packet in the first type data packets and the second type data packets; and sending the second target data packet to a routing device.

[0116] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further includes a second arbitrator, and wherein the method further includes arbitrating the first type of data packets and the second type of data packets through the second arbitrator in such a way that the priority of the first type of data packets is lower than the priority of the second type of data packets.

[0117] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes receiving a first type of data packets and a second type of data packets from a plurality of other artificial intelligence processors, including a first other artificial intelligence processor, via a communication network using a first sub-communication device; and wherein the depth of the second queue for the first other artificial intelligence processor is the same as the number of counters in the counter set for the first other artificial intelligence processor.

[0118] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes receiving, via a communication network, at least one third data packet of a first type from a first other artificial intelligence processor after a second data packet for a data operation unit; and wherein the method further includes, via the first sub-communication device: in response to storing the second data packet in a second queue, determining that there is an idle second counter in a counter set, storing at least one third data packet in a first queue for transmission to the data operation unit, and in response to storing at least one third data packet in the first queue respectively, stopping the increment of the count value of the first counter and incrementing the count value of the second counter.

[0119] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further includes receiving, via a communication network, at least one third data packet of a first type from a first other artificial intelligence processor after a second data packet for a data operation unit; and wherein the method further includes, via the first sub-communication device: in response to storing the second data packet in a second queue, determining that there is no idle counter in the counter set, storing at least one third data packet in a first queue for transmission to the data operation unit, and in response to storing at least one third data packet in the first queue respectively, continuing to increment the count value of a first counter.

[0120] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further includes a second sub-communication device, the method further including, via the second sub-communication device: sending at least one fourth data packet of a first type and a fifth data packet of a second type associated with the at least one fourth data packet to a communication network, wherein the fifth data packet instructs a first other artificial intelligence processor to continuously send at least one fourth data packet until the fifth data packet; and wherein, via the second sub-communication device: in response to receiving at least one fourth data packet from the data operation unit, sending a response message corresponding to the at least one fourth data packet to the data operation unit; in response to sending the response messages corresponding to the at least one fourth data packet, receiving the fifth data packet from the data operation unit; and in response to receiving the fifth data packet from the data operation unit, sending a response message corresponding to the fifth data packet to the data operation unit.

[0121] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further includes a plurality of computing cores and system direct memory access means; and wherein a data operation unit is included in at least one of the computing cores and system direct memory access means of the artificial intelligence processor.

[0122] Figure 7 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown.

[0123] like Figure 7 As shown, the electronic device 300 is, for example, suitable for implementing the data transmission method for an artificial intelligence processor provided in embodiments of this disclosure. It should be noted that... Figure 7 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0124] For example, such as Figure 7 As shown, in some examples, electronic device 300 includes a processing device (e.g., including as referenced). Figures 3-4 The parallel processor 301 shown, such as a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processor (TPU), etc., can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303 to achieve various functions, such as performing a data transmission method for an artificial intelligence processor provided in at least one embodiment of this disclosure.

[0125] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the data transmission method for an artificial intelligence processor according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the data transmission method for an artificial intelligence processor can be found in the relevant descriptions in the embodiments of the data transmission method for an artificial intelligence processor described above; repeated details will not be repeated here.

[0126] RAM 303 also stores various programs and data required for the operation of the computer system. Processing device 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0127] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as style images, and various data used and / or generated by the applications, may also be stored in the computer-readable storage medium.

[0128] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage device 308 as needed. Although Figure 7 An electronic device 300 including various devices is shown; however, it should be understood that implementation or inclusion of all shown devices is not required. More or fewer devices may be implemented or included alternatively. For example, a processing device 301 may control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing and / or program execution capabilities, such as a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a parallel processor, an artificial intelligence processor, a tensor processor (TPU), or a graphics processing unit (GPU). The CPU may be based on an x86 or ARM architecture, etc. The GPU may be integrated directly onto the motherboard or built into the motherboard's northbridge chip. Alternatively, the GPU may be built into the CPU.

[0129] For example, the electronic device 300 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device 309 can communicate wirelessly with a network and other devices, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0130] For example, electronic device 300 can be any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, or navigator, or any combination of electronic devices and hardware. The embodiments disclosed herein do not limit this.

[0131] For example, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For instance, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, the data transmission method for an artificial intelligence processor as defined in the embodiments of this disclosure is performed.

[0132] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0134] The artificial intelligence processor, electronic device, and data transmission method provided according to at least one embodiment of this disclosure can conveniently confirm whether data transmission is complete while avoiding or reducing CPU involvement in the data transmission process of the artificial intelligence processor. Specifically, the artificial intelligence processor according to at least one embodiment of this disclosure can receive a first type of data packet and a second type of data packet associated with the first type of data packet. The second type of data packet can be received by the data operation unit of the artificial intelligence processor acting as the receiving end only after ensuring that all the first type of data packets associated with the second type of data packet have been received (e.g., written) by the artificial intelligence processor acting as the receiving end. In this way, the artificial intelligence processor acting as the receiving end can be indicated through the second type of data packet that all the first type of data packets associated with the second type of data packet have been received.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0137] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0138] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0139] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0140] The following points should be noted regarding this disclosure:

[0141] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0142] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0143] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. An artificial intelligence processor, comprising: Comprising: a first sub-communication device configured to receive, from a first other artificial intelligence processor over a communication network, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet, wherein the second data packet indicates that the first other artificial intelligence processor continuously sends the at least one first data packet until the second data packet; a data operation unit configured to receive, from the first sub-communication device, data packets of the first type and data packets of the second type, wherein the first sub-communication device is further configured to, in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, send the second data packet to the data operation unit, wherein the first sub-communication device further comprises a second queue for data packets of the second type, and the first sub-communication device further comprises a set of counters for the first other artificial intelligence processor, wherein a depth of the second queue for the first other artificial intelligence processor and a number of counters in the set of counters for the first other artificial intelligence processor are the same.

2. The artificial intelligence processor of claim 1, wherein, the first sub-communication device further comprises a first queue for data packets of the first type; and the first sub-communication device is further configured to: store the at least one first data packet to the first queue for sending to the data operation unit, receive, from the data operation unit, response messages corresponding to the at least one first data packet respectively, store the second data packet to the second queue, and in response to receiving, from the data operation unit, response messages corresponding to all of the at least one first data packet, send the second data packet from the second queue to the data operation unit.

3. The artificial intelligence processor of claim 2, wherein, the first sub-communication device is further configured to: in response to storing the at least one first data packet to the first queue respectively, increase a count value of a first counter in the set of counters; in response to receiving, from the data operation unit, response messages corresponding to the at least one first data packet respectively, decrease the count value of the first counter; and in response to the count value of the first counter decreasing to zero, send the second data packet to the data operation unit. the first sub-communication device further comprises a third queue for data packets of the first type and data packets of the second type and a multiplexer, wherein the first sub-communication device is further configured to:

4. The artificial intelligence processor of claim 2, wherein, store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue; and send, to the first queue and the second queue respectively via the multiplexer, the at least one first data packet and the second data packet stored in the third queue in an order of reception over the communication network. the first sub-communication device further comprises a buffer and a first linked list and a second linked list for data packets of the second type, wherein, 5. The artificial intelligence processor of claim 3, wherein, ​ The first chain table includes one or more first nodes, each of the one or more first nodes points to a storage area in the buffer for storing a corresponding second type of data packet; and The second chain table includes one or more second nodes, each of the one or more second nodes points to a free storage area in the buffer.

6. The artificial intelligence processor of claim 5, wherein, The first sub-communication device is further configured to: in response to there being a free counter in the counter set, send the second type of data packet from the buffer to the second queue, and release a storage area corresponding to the sent second type of data packet; and in response to there being no free counter in the counter set, not send the second type of data packet from the buffer to the second queue.

7. The artificial intelligence processor of claim 5, wherein, The first sub-communication device is configured to receive first type of data packets and second type of data packets from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through the communication network; and wherein the first chain table is a plurality, and the plurality of first chain tables respectively correspond to the plurality of other artificial intelligence processors.

8. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive first type of data packets and second type of data packets from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through the communication network, wherein the second queue is a plurality, and the plurality of second queues respectively correspond to the plurality of other artificial intelligence processors, and wherein the counter set is a plurality, and the plurality of counter sets respectively correspond to the plurality of other artificial intelligence processors.

9. The artificial intelligence processor of claim 8, wherein, The first sub-communication device further includes a first arbitrator; and wherein the first arbitrator is configured to arbitrate second type of data packets sent by the plurality of second queues to determine a first target data packet of the second type.

10. The artificial intelligence processor of claim 3, wherein, The first sub-communication device further includes a routing device, and wherein the routing device is configured to: receive first type of data packets from the first queue and second type of data packets from the second queue; send the first type of data packets and the second type of data packets to the data operation unit; and receive a response message to the first type of data packets from the data operation unit.

11. The artificial intelligence processor of claim 10, wherein, The first type of data includes a first identification field indicating a plurality of other artificial intelligence processors including the first other artificial intelligence processor that sends the first type of data, and the counters in the counter set each correspond to a second identification field; and wherein the routing device is further configured to: split the first type of data packet into a write address packet and a write data packet for sending to the data operation unit, and determine a third identification field according to the first identification field and the second identification field, and assign the third identification field to the write address packet and the write data packet.

12. The artificial intelligence processor of claim 11, wherein, The routing device is further configured to: identify the third identification field included in the received response message; determining the first identification field and the second identification field based on the third identification field; and determining a counter of the set of counters to decrease a count value based on the first identification field and the second identification field.

13. The artificial intelligence processor of claim 10, wherein, The first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate between a first type of data packet sent by the first queue and a second type of data packet sent by the second queue to determine a second target data packet of the first type of data packet and the second type of data packet; and send the second target data packet to the routing device.

14. The artificial intelligence processor of claim 13, wherein, The first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate between the first type of data packet and the second type of data packet in a manner that a priority of the first type of data packet is lower than a priority of the second type of data packet.

15. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor over the communication network.

16. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive, for the data operation unit, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor over the communication network; and wherein the first sub-communication device is configured to: determine, in response to storing the second data packet to the second queue, that there is a free second counter of the set of counters, store the at least one third data packet to the first queue for sending to the data operation unit, and stop increasing the count value of the first counter and increase the count value of the second counter in response to storing the at least one third data packet to the first queue, respectively.

17. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive, for the data operation unit, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor over the communication network; and wherein the first sub-communication device is configured to: determine, in response to storing the second data packet to the second queue, that there is no free counter of the set of counters, store the at least one third data packet to the first queue for sending to the data operation unit, and continue increasing the count value of the first counter in response to storing the at least one third data packet to the first queue, respectively.

18. The artificial intelligence processor of any one of claims 1 to 17, wherein, The first sub-communication device is configured to receive, for the data operation unit, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor over the communication network; and wherein the first sub-communication device is configured to: determine, in response to storing the second data packet to the second queue, that there is a free second counter of the set of counters, store the at least one third data packet to the first queue for sending to the data operation unit, and continue increasing the count value of the first counter in response to storing the at least one third data packet to the first queue, respectively. The first sub-communication device is configured to receive, for the data operation unit, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor over the communication network; and wherein the first sub-communication device is configured to: determine, in response to storing the second data packet to the second queue, that there is a free second counter of the set of counters, store the at least one third data packet to the first queue for sending to the data operation unit, and continue increasing the count value of the first counter in response to storing the at least one third data packet to the first queue, respectively. in response to receiving the at least one fourth data packet from the data operation unit, sending a response message corresponding to the at least one fourth data packet to the data operation unit, in response to sending the response message corresponding to the at least one fourth data packet, receiving a fifth data packet from the data operation unit, and in response to receiving the fifth data packet from the data operation unit, sending a response message corresponding to the fifth data packet to the data operation unit.

19. The artificial intelligence processor of any one of claims 1 to 17, wherein, further comprising a plurality of compute cores and system direct memory access devices; and wherein the data operation unit is comprised in at least one of the at least one compute core and the system direct memory access device of the artificial intelligence processor.

20. An electronic device, comprising: An artificial intelligence processor according to any one of claims 1-19.

21. A data transfer method for an artificial intelligence processor, the method comprising: The artificial intelligence processor comprises a data operation unit and a first sub-communication device, and the data operation unit is configured to receive first type data packets and second type data packets from the first sub-communication device, and the method comprises: receiving, by using the first sub-communication device, at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet from a first other artificial intelligence processor over a communication network, wherein the second data packet indicates that the first other artificial intelligence processor continues to send the at least one first data packet until the second data packet; and in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, sending, by using the first sub-communication device, the second data packet to the data operation unit, wherein the first sub-communication device further comprises a second queue for the second type of data packets, and the first sub-communication device further comprises a set of counters for the first other artificial intelligence processor, wherein a depth of the second queue for the first other artificial intelligence processor and a number of counters in the set of counters for the first other artificial intelligence processor are the same.

Images