Artificial intelligence processor, electronic device and data transmission method

Abstract

The invention provides an artificial intelligence processor, an electronic device and a data transmission method. The artificial intelligence processor comprises a first sub-communication device configured to receive at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from a first other artificial intelligence processor through a communication network, the second data packet instructs the first other artificial intelligence processor to continuously send at least one first data packet until the second data packet; a data operation unit configured to receive data packets of a first type and data packets of a second type from the first sub-communication device, the first sub-communication device further configured to respond to the data operation unit to receive all of the at least one first data packet from the first sub-communication device; the second data packet is sent to the data operation unit. The artificial intelligence processor can confirm whether data transmission is completed in a convenient manner.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence technology, and in particular, to an artificial intelligence processor, an electronic device, and a data transmission method. BACKGROUND

[0002] In an artificial intelligence scenario such as a data center, when performing complex calculations such as large model training inference, interconnection between different chips may be needed for data exchange. Specifically, the chip can include multiple artificial intelligence processors, and data transmission between different artificial intelligence processors of different chips can be needed to improve the overall computing power.

[0003] However, data exchange between different artificial intelligence processors, especially in the scenario of confirming whether data transmission is completed between different artificial intelligence processors, can require a large amount of assistance of a central processing unit (CPU) or frequent access to memory. SUMMARY

[0004] According to at least one embodiment of the present disclosure, an artificial intelligence processor is provided, comprising: a first sub-communication device configured to receive at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from a first other artificial intelligence processor through a communication network, wherein the second data packet indicates that the first other artificial intelligence processor continues to send the at least one first data packet until the second data packet; a data operation unit configured to receive data packets of the first type and data packets of the second type from the first sub-communication device, wherein the first sub-communication device is further configured to send the second data packet to the data operation unit in response to the data operation unit receiving all the at least one first data packet from the first sub-communication device.

[0005] For example, according to the artificial intelligence processor of at least one embodiment of the present disclosure, the first sub-communication device further includes a first queue for data packets of the first type and a second queue for data packets of the second type; and the first sub-communication device is further configured to store the at least one first data packet to the first queue for sending to the data operation unit, receive response messages corresponding to the at least one first data packet respectively from the data operation unit, store the second data packet to the second queue, and send the second data packet from the second queue to the data operation unit in response to receiving response messages corresponding to all the at least one first data packet from the data operation unit.

[0006] For example, the artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a counter set for the first other artificial intelligence processor, and the first sub-communication device is further configured to: in response to storing the at least one first data packet into the first queue respectively, increase a count value of a first counter in the counter set; in response to receiving a response message corresponding to the at least one first data packet respectively from the data operation unit, decrease the count value of the first counter; and in response to the count value of the first counter being decreased to zero, send the second data packet to the data operation unit.

[0007] For example, the artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a third queue for first type data packets and second type data packets and a multiplexer, and the first sub-communication device is further configured to: store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue; and send the at least one first data packet and the second data packet stored in the third queue to the first queue and the second queue respectively through the multiplexer in the order of receiving through the communication network.

[0008] For example, the artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a buffer, and a first linked list and a second linked list for second type data packets, wherein the first linked list comprises one or more first nodes, each of the one or more first nodes points to a storage area in the buffer for storing a corresponding second type data packet; and the second linked list comprises one or more second nodes, each of the one or more second nodes points to a free storage area in the buffer.

[0009] For example, the artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is further configured to: in response to there being a free counter in the counter set, send the second type data packet from the buffer to the second queue, and release the storage area corresponding to the sent second type data packet; and in response to there being no free counter in the counter set, not send the second type data packet from the buffer to the second queue.

[0010] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is configured to receive, from a plurality of other artificial intelligence processors including the first other artificial intelligence processor, first type data packets and second type data packets through the communication network; and wherein the first linked list is a plurality, and the plurality of first linked lists respectively correspond to the plurality of other artificial intelligence processors.

[0011] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is configured to receive, from a plurality of other artificial intelligence processors including the first other artificial intelligence processor, first type data packets and second type data packets through the communication network; wherein the second queue is a plurality, and the plurality of second queues respectively correspond to the plurality of other artificial intelligence processors; and wherein the counter set is a plurality, and the plurality of counter sets respectively correspond to the plurality of other artificial intelligence processors.

[0012] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a first arbitrator, and the first arbitrator is configured to arbitrate the second type data packets sent by the plurality of second queues to determine a first target data packet of the second type.

[0013] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a routing device, and wherein the routing device is configured to: receive the first type data packets from the first queue and the second type data packets from the second queue; send the first type data packets and the second type data packets to the data operation unit; and receive a response message to the first type data packets from the data operation unit.

[0014] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first type data comprises a first identification field indicating a plurality of other artificial intelligence processors including the first other artificial intelligence processor that sends the first type data, and the counters in the counter set each correspond to a second identification field; and wherein the routing device is further configured to: split the first type data packet into a write address packet and a write data packet for sending to the data operation unit, and determine a third identification field according to the first identification field and the second identification field, and assign the third identification field to the write address packet and the write data packet.

[0015] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the routing device is further configured to: identify a third identification field included in the received response message; determine the first identification field and the second identification field based on the third identification field; and determine the counter in the set of counters to decrease the count value based on the first identification field and the second identification field.

[0016] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate the first type of data packet sent by the first queue and the second type of data packet sent by the second queue to determine a second target data packet from among the first type of data packet and the second type of data packet; and send the second target data packet to the routing device.

[0017] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate the first type of data packet and the second type of data packet in a manner that the priority of the first type of data packet is lower than the priority of the second type of data packet.

[0018] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is configured to receive the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through the communication network; and wherein the depth of the second queue for the first other artificial intelligence processor and the number of counters in the set of counters for the first other artificial intelligence processor are the same.

[0019] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is configured to receive at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor for the data operation unit through the communication network; and wherein the first sub-communication device is configured to: in response to storing the second data packet to the second queue, determine that there is a free second counter in the set of counters, store the at least one third data packet to the first queue to send to the data operation unit, and in response to storing the at least one third data packet to the first queue respectively, stop increasing the count value of the first counter and increase the count value of the second counter.

[0020] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device is configured to receive, for the data operation unit, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor through the communication network; and wherein the first sub-communication device is configured to, in response to storing the second data packet to the second queue, determine that there is no free counter in the set of counters, store the at least one third data packet to the first queue for sending to the data operation unit, and in response to storing the at least one third data packet to the first queue respectively, continue to increase the count value of the first counter.

[0021] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, further comprising a second sub-communication device configured to send, for the data operation unit, at least one fourth data packet of the first type and a fifth data packet of the second type associated with the at least one fourth data packet to the communication network, wherein the fifth data packet indicates that the first other artificial intelligence processor continues to send the at least one fourth data packet to the fifth data packet; and wherein the second sub-communication device is further configured to, in response to receiving at least one fourth data packet from the data operation unit, send a response message corresponding to the at least one fourth data packet respectively to the data operation unit, in response to sending the response message corresponding to the at least one fourth data packet respectively, receive the fifth data packet from the data operation unit, and in response to receiving the fifth data packet from the data operation unit, send a response message corresponding to the fifth data packet to the data operation unit.

[0022] For example, an artificial intelligence processor according to at least one embodiment of the present disclosure, further comprising a plurality of computing cores and a system direct memory access device; and wherein the data operation unit is included in at least one of the at least one computing core and the system direct memory access device of the artificial intelligence processor.

[0023] According to at least one embodiment of the present disclosure, an electronic device is provided, comprising an artificial intelligence processor of any of the above embodiments.

[0024] According to at least one embodiment of the present disclosure, a data transmission method for an artificial intelligence processor is provided, wherein the artificial intelligence processor comprises a data operation unit and a first sub-communication device, and the data operation unit is configured to receive a first type of data packet and a second type of data packet from the first sub-communication device, the method comprising: receiving, by using the first sub-communication device, at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet from a first other artificial intelligence processor through a communication network, wherein the second data packet indicates that the first other artificial intelligence processor continues to send the at least one first data packet until the second data packet; and in response to the data operation unit receiving all the at least one first data packet from the first sub-communication device, sending, by using the first sub-communication device, the second data packet to the data operation unit.

[0025] The artificial intelligence processor, electronic device and data transmission method according to at least one embodiment of the present disclosure can confirm whether the data transmission is completed in a convenient manner while avoiding or reducing the involvement of the CPU in the data transmission process of the artificial intelligence processor. Specifically, the artificial intelligence processor according to at least one embodiment of the present disclosure can receive a first type of data packet and a second type of data packet associated with the first type of data packet. The second type of data packet can be received by the data operation unit of the artificial intelligence processor as the receiving end after ensuring that the first type of data packet associated with the second type of data packet has been completely received (e.g., written) by the artificial intelligence processor as the receiving end. In this way, the artificial intelligence processor as the receiving end can be indicated by the second type of data packet that the first type of data packet associated with the second type of data packet has been completely received.

[0026] The artificial intelligence processor, electronic device and data transmission method according to at least one embodiment of the present disclosure can avoid hindering the first type of data packet not associated with the second type of data packet from being executed by the artificial intelligence processor as the receiving end during the process of waiting for the first type of data packet associated with the second type of data packet to be executed by the artificial intelligence processor as the receiving end. For example, during the process of waiting for the first type of data packet associated with the second type of data packet to be executed, the first type of data packet not associated with the second type of data packet can continue to be sent to the artificial intelligence processor as the receiving end. In this way, the data transmission efficiency between artificial intelligence processors can be improved, and the occurrence of congestion can be reduced.

[0027] According to at least one embodiment of the present disclosure, the artificial intelligence processor, the electronic device and the data transmission method provided can prevent the data packets of the second type from the artificial intelligence processor as the sending end from being sent to the data operation unit for a long time, thereby avoiding the data packets of the second type being affected by the data packets of the first type (for example, the data packets of the first type after the data packets of the second type) which are not associated with the data packets of the second type for a long time. In this way, the data transmission efficiency between the artificial intelligence processors can be increased, and the congestion situation can be reduced.

[0028] According to at least one embodiment of the present disclosure, the artificial intelligence processor, the electronic device and the data transmission method provided can prevent the data packets of the second type from the artificial intelligence processor as the sending end from being sent to the data operation unit for a long time, thereby avoiding the data packets of the second type being affected by the data packets of the first type (for example, the data packets of the first type after the data packets of the second type) which are not associated with the data packets of the second type for a long time. In this way, the data transmission efficiency between the artificial intelligence processors can be increased, and the congestion situation can be reduced. BRIEF DESCRIPTION OF DRAWINGS

[0029] The above aspects and other aspects, features and advantages of certain embodiments of the present disclosure will be more clearly understood from the following description taken in conjunction with the accompanying drawings, in which:

[0030] Figure 1 An artificial intelligence processor is applied to a scenario such as a data center.

[0031] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0032] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0033] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0034] Figure 5 A schematic structural diagram of a first sub-communication device of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0035] Figure 6 A flowchart of a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown.

[0036] Figure 7 A schematic block diagram of an electronic device provided by at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Before proceeding with the detailed description below, it may be advantageous to define certain words and phrases used throughout this disclosure. The terms “comprising” and “including” and their derivatives mean including but not limited to. The term “or” is inclusive, meaning and / or. The phrase “associated with” and its derivatives mean including, comprising, interconnecting, containing, contained within, connected or connected to, coupled or coupled to, communicating with, cooperating, intertwining, juxtaposing, proximate, binding or bound to, having, possessing attributes, having a relationship or being related to, etc. The term “controller” means any device, system, or part thereof that controls at least one operation. Such a controller may be implemented in hardware, or a combination of hardware and software and / or firmware. The functionality associated with any particular controller may be centralized or distributed, local or remote. The phrase “at least one,” when used with a list of items, means that different combinations of one or more of the listed items may be used, and that only one item from the list may be required. For example, "at least one of A, B, and C" includes any one of the following combinations: A, B, C, A and B, A and C, B and C, A and B and C.

[0038] Definitions of other specific words and phrases are provided throughout this disclosure. Those skilled in the art will understand that, in many, if not most, cases, such definitions apply to the prior and future use of the words and phrases thus defined.

[0039] The various embodiments of the principles of this disclosure described below with reference to the accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged system or device. In some cases, the actions described in the specification may be performed in a different order and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily require a specific order or sequential sequence to achieve the desired result. In certain embodiments, multitasking and parallel processing may be advantageous.

[0040] Artificial intelligence processors can include parallel processors such as graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), and tensor processors (TPUs). Figure 1 This demonstrates an artificial intelligence processor applied to scenarios such as data centers.

[0041] like Figure 1 As shown, a data center can deploy multiple servers, including Server 0 and Server 1. These servers can be used to perform tasks such as hosting websites and web applications with high computing power requirements, data storage and management, and running applications and software.

[0042] The following description uses server 0 as an example to illustrate the functionality of the components within the server. Server 1 can be the same as or similar to server 0; therefore, server 1 will not be described again to avoid redundancy. Server 0 may include a Central Processing Unit (CPU) 0, a root complex 0, PCIe switches 00 and 01, and GPUs 00, 01, 02, and 03. Server 1 may include a CPU 1, a root complex 1, PCIe switches 10 and 11, and GPUs 10, 11, 12, and 13. Root complex 1 may include root ports RP10 and RP11. PCIe switch 10 may include one uplink port USP10 and multiple downlink ports DSP11 and DSP12, and PCIe switch 11 may include one uplink port USP11 and multiple downlink ports DSP13 and DSP14. GPU10 may include uplink port USP12, GPU11 may include uplink port USP13, GPU12 may include uplink port USP14, and GPU13 may include uplink port USP15, thereby communicating with the corresponding PCIe switch.

[0043] CPU 0 can be used as the core computing and control component of server 0. CPU 0 can execute operating system and application instructions, perform logical operations and task scheduling by calling data stored in memory, and coordinate the collaborative work of various peripheral hardware components within the server 0 system to realize the general computing and overall control functions of server 0.

[0044] Root Complex 0 can be the core controller of a PCIe (Rapid Peripheral Component Interconnect) system architecture. Root Complex 0 can access the system via the uplink port ( Figure 1 (Not shown) is connected to the CPU and can be connected to PCIe switch 00 and PCIe switch 01 via downlink ports (e.g., root ports RP00 and RP01). Root complex 0 is responsible for enumerating, configuring, and managing the PCIe topology and acts as the root node for data communication, handling uplink and downlink data routing and protocol conversion.

[0045] PCIe switch 00 and PCIe switch 01 can be high-speed interconnection devices for extending PCIe links. For example, PCIe switch 00 can include one upstream port USP00 and multiple downstream ports DSP01 and DSP02. PCIe switch 00 can implement high-bandwidth, low-latency parallel data exchange between multiple downstream PCIe devices and an upstream root complex through an internal crossbar, thereby effectively improving the peripheral connection capability of the system. PCIe switch 01 can include one upstream port USP01 and multiple downstream ports DSP03 and DSP04. The function of PCIe switch 01 is similar to that of PCIe switch 00, which will not be repeated here.

[0046] GPU00, GPU01, GPU02, GPU03 can be coprocessors dedicated to parallel computing. GPU00 can include an upstream port USP02, GPU01 can include an upstream port USP03, GPU02 can include an upstream port USP04, and GPU03 can include an upstream port USP05, thereby communicating with the corresponding PCIe switch. GPU00, GPU01, GPU02, GPU03 can adopt a large-scale parallel stream processor architecture suitable for processing highly parallel floating-point and integer operations in graphics rendering, deep learning, and scientific computing. GPU00, GPU01, GPU02, GPU03 can take over certain computing tasks of CPU 0, thereby significantly improving the overall computing throughput and energy efficiency of server 0. GPU00, GPU01, GPU02, GPU03 can be connected to the corresponding communication port of the Ethernet switch through their communication ports (such as Figure 1 indicated by the diagonal shading in FIG. 1).

[0047] With the development of training of large models, the demand for computing power of data centers is increasing, and therefore it is required to implement communication interconnection of multiple GPUs in the data center for data transmission. According to at least one embodiment of the present disclosure, multiple GPUs can be interconnected for communication, for example, using their communication ports through Ethernet. The main protocol followed by Ethernet is the IEEE 802.3 standard. Ethernet can be a local area network technology used for transmitting data within a local area network. As a widely used local area network technology, Ethernet can allow multiple devices (e.g., computers such as servers, printers, switches, etc.) to communicate with each other.

[0048] Figure 1 The communication port (such as Figure 1The GPUs of the server 0 (e.g., GPU00, GPU01, and GPU02) can be connected to the Ethernet switch (e.g., through the communication ports 0-7 of the Ethernet switch) through the Ethernet link (as shown in the oblique shading of the left side of FIG. 1). The description is made by taking the GPU00 of the server 0 accessing other GPUs as an example. The GPU00 of the server 0 can send an Ethernet message to the Ethernet switch. The Ethernet switch can determine to forward the Ethernet message to a certain communication port of other GPUs based on the destination address field in the message header.

[0049] In the scenario of multi-chip interconnection, the prior art still faces severe challenges. It can be tried to use TCP / IP technology and remote memory direct access (RDMA) technology to perform data transmission between GPUs. However, the above-mentioned TCP / IP technology needs to be implemented in the operating system kernel. When transmitting data, the data needs to pass through the kernel protocol stack multiple times, resulting in large delay and thus low performance. For example, the traditional inter-processor communication based on TCP / IP protocol requires the data packet to pass through the operating system kernel protocol stack multiple times, which introduces a microsecond-level delay much higher than the nanosecond-level data synchronization requirement in AI training, and the interruption processing and context switching of the CPU kernel consume a large amount of computing resources. In addition, the above-mentioned TCP / IP technology and RDMA technology need the assistance of the CPU, so a large amount of CPU resources will be occupied, and thus the performance of the CPU will be reduced.

[0050] Further, in a large-scale artificial intelligence (AI) cluster, the RDMA technology involves thousands of dynamic and concurrent inter-processor communication connections, which can cause the RDMA connection management table items to be quickly exhausted and it is difficult to provide fine and predictable quality of service. In addition, in order to achieve lossless data transmission, the RDMA technology implements an end-to-end retransmission mechanism. For example, an artificial intelligence processor as a sending end can send a message, and an artificial intelligence processor as a receiving end can inform the artificial intelligence processor as the sending end that the artificial intelligence processor as the receiving end has received the message by replying an acknowledgement (ACK).

[0051] In particular, the GPU 00 as a sender can establish an RDMA communication channel with the GPU 10 as a receiver via a socket manner. For example, a user of the sender can send an instruction to the GPU 00 to cause, for example, a compute core of the GPU 00 to write the computed data to the memory. The user of the sender can send the instruction (i.e., a work queue element (WQE)) to the RDMA device on the GPU 00. Further, the user of the sender can send an instruction to the RDMA device on the GPU 00 to trigger the RDMA device on the GPU 00 to read the WQE. The RDMA device on the GPU 00 can read the written data from the memory based on the WQE and encapsulate the data into an Ethernet packet to send to the GPU 10. After the GPU 10 receives the data, an ACK packet is sent back to the GPU 00 after a check and the data is written to a designated location of the GPU 10. After the GPU 00 receives the ACK, a completion queue element (CQE) can be generated and placed at the designated location. The user of the sender can further obtain the CQE.

[0052] In the process of sending data from the GPU 00 to the GPU 10, the GPU 00 can act as a producer of data in a producer and consumer model and the GPU 10 can act as a consumer of data in the producer and consumer model. After the GPU 00 sends the data, the GPU 00 needs to notify the GPU 10 that the data is written. In the case that the RDMA device uses an InfiniBand (IB) protocol, a write with immediate data RDMA operation can be used to achieve the notification purpose. For example, there can be 2 WQEs, such as WQE1 and WQE2, in a send queue (SQ) on the GPU 00 as a sender. WQE1 can be used to perform a write operation to write data to a designated location of the GPU 10 as a receiver. WQE2 can be used to perform a write with immediate data operation to notify the GPU 10 as a receiver that the data is written. However, the write with immediate data operation is based on memory semantics, not control or protocol semantics. That is, the immediate data is still stored in a designated area of the memory to notify the GPU 10 as a receiver that the data is received.

[0053] Figure 2 A schematic structural diagram of an artificial intelligence processor is shown.

[0054] The artificial intelligence processor can include parallel processors such as a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), etc., but the present disclosure is not limited thereto. As shown in FIG. 1, the artificial intelligence processor can include a plurality of processing units 100, a memory 200, a cache 300, a bus 400, a host interface 500, and a network interface 600.Figure 2 As shown, an artificial intelligence processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a Streaming Processor Cluster (SPC), which includes... Figure 2 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In an AI processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0055] like Figure 2 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 2 The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores (also called computational cores), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These Cores are used to execute specific computational tasks. Furthermore, a Computation Unit may also include registers (e.g., ...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0056] like Figure 2 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0057] like Figure 2 As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0058] In parallel computing, computational tasks are typically executed through multiple threads. These threads are divided into multiple thread blocks before execution in the artificial intelligence processor (or general-purpose graphics processor or parallel computing processor), and then dispatched via a thread block dispatch module. Figure 2(Not shown in the diagram) Multiple thread blocks are distributed to various computing units. All threads in a thread block must be assigned to the same computing unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple threads from multiple thread blocks can execute on multiple computing cores within the same computing unit, or on multiple computing cores in different computing units, sharing registers, shared memory, and other data transfer mechanisms.

[0059] In each computing unit, the thread beam scheduling / distribution module ( Figure 2 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 2 It is used for reading and writing operations in high-bandwidth memory (HBM).

[0060] Figure 3 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. Figure 3 As shown, the artificial intelligence processor 3000 may include a first sub-communication device 3100 and a data operation unit 3200. Figure 3 The artificial intelligence processor in it can correspond to Figure 2 The computational unit will not be described again here for the sake of brevity.

[0061] According to at least one embodiment of this disclosure, the data operation unit 3200 can be configured to receive data sent from other artificial intelligence processors and to other artificial intelligence processors. For example, other artificial intelligence processors can write data to artificial intelligence processor 3000 through the data operation unit 3200, and artificial intelligence processor 3000 can write data to other artificial intelligence processors through the data operation unit 3200, but this disclosure is not limited thereto. The data that artificial intelligence processor 3000 wants to write to other artificial intelligence processors through the data operation unit 3200 and the data that other artificial intelligence processors want to write to artificial intelligence processor 3000 through the data operation unit 3200 can be data that multiple artificial intelligence processors, including artificial intelligence processor 3000, wish to transmit to improve computing power. The data operation unit 3200 can receive first type data packets and second type data packets from the first sub-communication device 3100.

[0062] According to at least one embodiment of the present disclosure, the first sub-communication device 3100 can be configured to transmit data packets for the data operation unit 3200. For example, the first sub-communication device 3100 can be connected with a communication network outside the artificial intelligence processor to perform data transmission. The first sub-communication device 3100 can transmit the first type of data packets and the second type of data packets sent by other artificial intelligence processors through the communication network to the data operation unit 3200. According to at least one embodiment of the present disclosure, the first sub-communication device 3100 can be a receiving device in an Ethernet port of the artificial intelligence processor 3000.

[0063] According to at least one embodiment of the present disclosure, the first sub-communication device 3100 can receive at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet from the first other artificial intelligence processor through the communication network. That is, the above-mentioned second data packet and the above-mentioned at least one first data packet can be sent as a batch from the first other artificial intelligence processor to the artificial intelligence processor 3000. The second data packet can indicate that the first other artificial intelligence processor continues to send at least one first data packet until the second data packet. For example, the first other artificial intelligence processor can first send at least one first data packet, and immediately after sending the at least one first data packet, send the second data packet. The first sub-communication device 3100 can receive the above-mentioned at least one first data packet and the second data packet in the order of sending by the first other artificial intelligence processor. For example, for the first other artificial intelligence processor, the first sub-communication device 3100 can first receive at least one first data packet, and immediately after receiving the at least one first data packet, receive the second data packet.

[0064] According to at least one embodiment of the present disclosure, the first sub-communication device 3100 can also be configured to, in response to the data operation unit 3200 receiving all the at least one first data packet from the first sub-communication device 3100, send the second data packet to the data operation unit. According to at least one embodiment of the present disclosure, the first sub-communication device 3100 can determine that the data operation unit 3200 receives all the at least one first data packet from the first sub-communication device 3100 by using a counter or a queue pointer, but the present disclosure is not limited thereto, and other possible ways are also possible. That is, the first sub-communication device 3100 can synchronize the ready state by sending the second data packet to the data operation unit 3200 in the case that the at least one first data packet has been received by the data operation unit 3200 (i.e., the at least one first data packet is ready). The second data packet can be based on control semantics or protocol semantics, rather than memory semantics.

[0065] The artificial intelligence processor 3000 according to at least one embodiment of the present disclosure can directly perform data transmission synchronization using the data operation unit 3200 and the first sub-communication device 3100 included in the artificial intelligence processor 3000 without the assistance of a CPU outside the artificial intelligence processor 3000. In this way, the amount of task processing of the CPU for data transmission of the artificial intelligence processor 3000 can be reduced, so that the CPU can use computing resources and storage resources for other task processing to improve the CPU. Therefore, the performance of a computing device (for example, a server) in which the CPU and the artificial intelligence processor 3000 are deployed can be improved.

[0066] The artificial intelligence processor 3000 according to at least one embodiment of the present disclosure can receive the second type of data packet by the data operation unit 3200 of the artificial intelligence processor 3000 as a receiving end after ensuring that the first type of data packet associated with the second type of data packet has been completely received by the data operation unit 3200 of the artificial intelligence processor 3000 as a receiving end (for example, a write operation). The first type of data packet associated with the second type of data packet can be indicated to the data operation unit 3200 of the artificial intelligence processor 3000 as a receiving end by the second type of data packet in a control semantic or protocol semantic manner. In this way, the data transmission capability between artificial intelligence processors can be increased.

[0067] Figure 4 A schematic structural diagram of an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. As shown in Figure 4 The artificial intelligence processor 4000 can include a computing core 4110, a system direct memory access (SDMA) device 4120, an on-chip network (NOC) 4200, and a first sub-communication device 4300. Figure 4 The artificial intelligence processor 4000 in Figure 3 The same or similar components are not described again to avoid redundancy. Figure 4 The artificial intelligence processor 4000 in Figure 2 The computing unit of Figure 4 The computing core in Figure 2 The core of

[0068] According to at least one embodiment of the present disclosure, the artificial intelligence processor 4000 can include at least one computing core 4110. Each computing core in the artificial intelligence processor 4000 can include one or more instruction processing units, such as but not limited to an instruction fetch unit, a decode unit, an issue unit, an artificial intelligence unit, to perform processing operations on instructions. For example, an instruction in a computing core goes through Instruction Fetch, Decode, Issue, Execute, Write, etc. According to at least one embodiment of the present disclosure, the instruction fetch unit can be configured to fetch instructions to be executed subsequently. The decode unit can be configured to decode the fetched instructions into control signals that the underlying hardware can execute. The issue unit can monitor the execution status of a thread or a thread bundle (e.g., waiting for operands or hardware resources to be ready), and issue instructions that meet the issue conditions to the execution unit, such as an artificial intelligence computing unit. The artificial intelligence computing unit can be configured to perform computations for artificial intelligence aspects based on the issued instructions. The write back unit can write back data output by the artificial intelligence computing unit to memory or cache.

[0069] According to at least one embodiment of the present disclosure, the SDMA device 4120 can be a dedicated engine to implement system direct memory access. The SDMA device 4120 can independently undertake the data transfer task between the video memory of the artificial intelligence processor 4000 and the system memory without the assistance or intervention of the CPU. The SDMA device 4120 can separate data transfer from the work of the artificial intelligence processor 4000, effectively eliminating the adverse effects of data transfer delay on computing tasks, thereby significantly improving the overall throughput efficiency and system energy efficiency of the artificial intelligence processor 4000.

[0070] According to at least one embodiment of the present disclosure, a data operation unit can be included in at least one of the at least one computing core and the SDMA device. For example, the data operation unit 4111 can be disposed in the computing core 4110, and the data operation unit 4121 can be disposed in the SDMA device 4120, but the present disclosure is not limited thereto, and the data operation unit can be disposed in any device in the artificial intelligence processor 4000 that can send or receive the first type of data packet and the second type of data packet.

[0071] According to at least one embodiment of the present disclosure, the NOC 4200 can be a mesh topology, and provide high-bandwidth, low-latency, and scalable data exchange paths for a plurality of parallel processing units (e.g., the computing core 4110 and the SDMA device 4120, etc.) and the first sub-communication device 4300 through a configured communication protocol (e.g., an Advanced eXtensible Interface (AXI) protocol, but the present disclosure is not limited thereto).

[0072] According to at least one embodiment of the present disclosure, the first sub-communication device 4300 can include a first queue 4310, a second queue 4320, a set of counters 4330, Figure 4 a medium access control (MAC) layer not shown in FIG. 4, and Figure 4 a serializer / deserializer (SerDes) not shown in FIG. 4. According to at least one embodiment of the present disclosure, the first queue 4310, the second queue 4320, and the set of counters 4330 can be configured for the first other artificial intelligence processor. Additionally or alternatively, another first queue, another second queue, and another set of counters can be configured for another artificial intelligence processor other than the first other artificial intelligence processor.

[0073] According to at least one embodiment of the present disclosure, the MAC layer can be a data link layer core controller. The MAC layer can implement assembly and parsing of data frames, perform access control protocols such as carrier sense multiple access with collision detection (CSMA / CD) with collision detection, manage frame check sequences, and conduct flow control. The MAC layer can process data link packets and provide standardized data interfaces for upper layers.

[0074] According to at least one embodiment of the present disclosure, the SerDes can implement high-speed serial communication. The SerDes can convert parallel data output by the MAC layer into a serial bit stream for transmission and reconstruct a received serial signal into parallel data. The SerDes overcomes transmission loss of PCB traces and cables through clock recovery, channel coding, and other techniques to enable high-speed, long-distance data exchange between the artificial intelligence processor 4000 and an external communication network.

[0075] According to at least one embodiment of the present disclosure, the first sub-communication device 4300 can first receive at least one first data packet from the first other artificial intelligence processor and immediately after receiving the at least one first data packet, receive a second data packet. The first queue 4310 of the first sub-communication device 4300 can be for a first type of data packet and the second queue 4320 can be for a second type of data packet.

[0076] According to at least one embodiment of the present disclosure, the first sub-communication device 4300 can store at least one first data packet to the first queue 4310 for sending to the data operation units. For example, the at least one first data packet stored to the first queue 4310 can be sent to at least one of the data operation unit 4111 and the data operation unit 4121 through the NOC 4200. According to at least one embodiment of the present disclosure, the first queue 4310 can be a first-in-first-out (FIFO) queue. In response to storing the at least one first data packet to the first queue 4310, respectively, a count value of a first counter in a counter set 4330 can be increased. For example, in response to the first data packet being stored to the first queue 4310, one free counter in the counter set 4330 can be selected as the first counter, and the first data packet entering the first queue 4310 can be counted using the first counter. According to at least one embodiment of the present disclosure, a bit width of the counter can be 9 bits, but the present disclosure is not limited thereto.

[0077] According to at least one embodiment of the present disclosure, the first sub-communication device 4300 can store a second data packet to a second queue 4320 after sending all the at least one first data packet to the first queue 4310. According to at least one embodiment of the present disclosure, the second queue 4320 can be a first-in-first-out (FIFO) queue. In addition, the first sub-communication device 4300 can receive a response message corresponding to the at least one first data packet from at least one of the data operation unit 4111 and the data operation unit 4121 after sending the at least one first data packet to at least one of the data operation unit 4111 and the data operation unit 4121.

[0078] According to at least one embodiment of the present disclosure, in response to receiving the response message corresponding to all the at least one first data packet from at least one of the data operation unit 4111 and the data operation unit 4121, the first sub-communication device 4300 can send the second data packet from the second queue 4320 to at least one of the data operation unit 4111 and the data operation unit 4121. For example, the second data packet stored to the second queue 4320 can be sent to at least one of the data operation unit 4111 and the data operation unit 4121 through the NOC 4200. Specifically, in response to receiving the response message corresponding to the at least one first data packet from at least one of the data operation unit 4111 and the data operation unit 4121, respectively, the first sub-communication device 4300 can decrease the count value of the first counter. In response to the count value of the first counter being decreased to zero, the first sub-communication device 4300 can send the second data packet to at least one of the data operation unit 4111 and the data operation unit 4121.

[0079] Figure 5A schematic structural diagram of a first sub-communication device of an artificial intelligence processor is shown according to at least one embodiment of the present disclosure. As shown in Figure 5 The first sub-communication device 5001 of the artificial intelligence processor can include a first queue 5100, a second queue 5200, a counter set 5300, a third queue 5400, a first arbiter 5410, a buffer 5600, a second arbiter 5700, and a routing device 5800, as shown. Figure 5 The first sub-communication device 5001 of the artificial intelligence processor can include a first queue 5100, a second queue 5200, a counter set 5300, a third queue 5400, a first arbiter 5410, a buffer 5600, a second arbiter 5700, and a routing device 5800, as shown. Figure 2 The first sub-communication device 5001 of the artificial intelligence processor can include a first queue 5100, a second queue 5200, a counter set 5300, a third queue 5400, a first arbiter 5410, a buffer 5600, a second arbiter 5700, and a routing device 5800, as shown. Figure 3 The components in the first sub-communication device 5001 of the artificial intelligence processor that are the same as or similar to those in the communication device in

[0080] According to at least one embodiment of the present disclosure, the third queue 5400 can be used for the first type of data packets and the second type of data packets. For the first other artificial intelligence processor, the first sub-communication device 5001 can store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue 5400. Further, the first sub-communication device 5001 can send the at least one first data packet and the second data packet stored in the third queue 5400 to the first queue 5100 and the second queue 5200, respectively, through the multiplexer 5500 in the order of reception through the communication network. According to at least one embodiment of the present disclosure, the third queue 5400 can be a first-in-first-out queue, and the first sub-communication device 5001 can store at least one first data packet of the first type into the third queue 5400 first and store a second data packet of the second type associated with the at least one first data packet into the third queue 5400 later, in the order of reception.

[0081] According to at least one embodiment of the present disclosure, the first sub-communication device 5001 can receive, through the third queue 5400, the first type of data packets and the second type of data packets sent by a plurality of artificial intelligence processors including the first other artificial intelligence processor from the communication network. The first sub-communication device 5001 can store the first type of data packets and the second type of data packets from the plurality of artificial intelligence processors into the third queue 5400 in the order of reception.

[0082] According to at least one embodiment of the present disclosure, the first type of data can comprise a first identification field indicating a plurality of other artificial intelligence processors including a first other artificial intelligence processor that sends the first type of data. In response to at least one first data packet of the first type from the first other artificial intelligence processor entering the first queue 5100, the first data packet can be identified as coming from the first other artificial intelligence processor according to the first identification field in the first data packet. As previously described, a plurality of counter sets can be configured, for example, a counter set can be configured for each of the first other artificial intelligence processors including the first other artificial intelligence processor. The at least one first data packet can be determined to come from the first other artificial intelligence processor according to the first identification field included in the at least one first data packet, and a free counter pair can be selected from a counter set corresponding to the first other artificial intelligence processor in the plurality of counter sets 5300 to count the at least one first data packet entering the first queue 5100. The first queue 5100 can be used to ensure the timing requirement of the first type of data packet, and therefore the depth of the first queue 5100 can be 1, but the present disclosure is not limited thereto.

[0083] According to at least one embodiment of the present disclosure, the buffer 5600 can store data packets of the second type received from the multiplexer 5500. For example, for the first other artificial intelligence processor, after the third queue 5400 sends at least one first data packet to the first queue 5100 through the multiplexer 5500, the third queue 5400 can send a second data packet associated with the at least one first data packet to the buffer 5600 for storage through the multiplexer 5500. The first sub-communication device 5001 can also be configured with a first linked list. For example, the first linked list can be a data linked list. The first linked list can comprise one or more first nodes, and each of the one or more first nodes can point to a storage area in the buffer 5600 for storing a corresponding second data packet.

[0084] According to at least one embodiment of the present disclosure, the first sub-communication device 5001 can receive data packets of the first type and data packets of the second type from a plurality of other artificial intelligence processors. Therefore, a corresponding first linked list can be configured for each of the plurality of other artificial intelligence processors. Each of the one or more first nodes included in the first linked list can point to a storage area in the buffer 5600 for storing a corresponding data packet of the second type.

[0085] According to at least one embodiment of the present disclosure, each of the plurality of first linked lists can be configured to track the storage address of the storage area (e.g., data slot) in the buffer associated with the data packet of the second type associated with the corresponding other artificial intelligence processor.

[0086] Specifically, each of the plurality of first linked lists includes a plurality of first nodes associated with storage addresses of the second type of data packets associated with the corresponding other artificial intelligence processor in the cache. The first node in the first linked list can store a pointer to the next first node, that is, the storage address of the next storage area (e.g., data slot) in the cache. The head (Head) and tail (Tail) of the corresponding first linked list can be recorded for each corresponding other artificial intelligence processor, so as to quickly append to the tail when writing data and quickly read out from the head when reading data. In addition, the second linked list can be configured in the first sub-communication device 5001. The second linked list can include one or more second nodes, and each of the one or more second nodes can point to a free storage area in the cache 5600. The second linked list can be a global linked list, for example, a free linked list. The second linked list can be configured to record the storage addresses of the free storage areas (e.g., data slots) in the cache associated with the plurality of other artificial intelligence processors. For example, after a certain second type of data packet is read out from the cache 5600 (e.g., sent to the second queue 5200), the corresponding data slot can be released.

[0087] The artificial intelligence processor according to at least one embodiment of the present disclosure can implement efficient cache management. By adopting dynamic cache management such as the first linked list of data linked list and the second linked list such as free linked list, efficient reuse of cache free is achieved, and waste of storage resources is reduced.

[0088] According to the artificial intelligence processor of at least one embodiment of the present disclosure, by using the cache 5600, in the process of waiting for the first type of data packet associated with a certain second type of data packet to be executed as a receiving end artificial intelligence processor receiving operation, the first type of data packet not associated with the second type of data packet is not hindered to be executed as a receiving end artificial intelligence processor receiving operation. For example, in the process of waiting for the first type of data packet associated with a certain second type of data packet to be executed as a receiving end artificial intelligence processor receiving operation, the first type of data packet not associated with the second type of data packet can continue to be sent to the artificial intelligence processor as a receiving end. In this way, the data transmission efficiency between artificial intelligence processors can be increased, and the congestion situation can be reduced.

[0089] According to at least one embodiment of the present disclosure, in response to the presence of a free counter in the counter set, the second type of data packet can be sent from the buffer 5600 to the second queue 5200, and in response to the absence of a free counter in the counter set, the second type of data packet can not be sent from the buffer 5600 to the second queue 5200. For example, in response to the presence of a free counter in the counter set corresponding to the first other artificial intelligence processor, the above-mentioned second data packet can be sent from the buffer 5600 to the second queue 5200, and in response to the absence of a free counter in the counter set corresponding to the first other artificial intelligence processor, the above-mentioned second data packet can not be sent from the buffer 5600 to the second queue 5200.

[0090] According to at least one embodiment of the present disclosure, since the first sub-communication device 5001 can receive the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor, a corresponding second queue can be configured for each other artificial intelligence processor. For example, the second queue 5200 can include N queues 5201-520N. As an example, N can be 1024, that is, the first sub-communication device 5001 can receive the first type of data packet and the second type of data packet from 1024 other artificial intelligence processors, but the present disclosure is not limited thereto, and N can be more or less.

[0091] According to at least one embodiment of the present disclosure, for each other artificial intelligence processor, the depth of the corresponding second queue can be the same as the number of counters in the corresponding counter set. For example, the depth of the second queue for the first other artificial intelligence processor is the same as the number of counters in the counter set for the first other artificial intelligence processor. According to at least one embodiment of the present disclosure, the depth of the corresponding second queue and the number of counters in the corresponding counter set can be set to at least 2. In this way, after all first data packets associated with a certain second type of data packet from an artificial intelligence processor as a sending end are received, the second type of data packet from the artificial intelligence processor as the sending end can be sent to the data operation unit as soon as possible, thereby avoiding the second type of data packet being affected for a long time by the first type of data packet not associated with the second type of data packet (for example, the first type of data packet after the second type of data packet).

[0092] According to at least one embodiment of the present disclosure, in response to the count value of the counter corresponding to the second type of data packet located at the front end (for example, to be sent) in the queues 5201-520N included in the second queue 5200 being reduced to zero through the corresponding response message, the corresponding second type of data packet can be sent.

[0093] According to at least one embodiment of the present disclosure, the first arbiter 5410 can be configured to arbitrate the second type of data packets transmitted by the plurality of queues 5201-520N included in the second queue 5200 to determine a first target data packet of the second type. For example, the first arbiter 5410 can arbitrate the second type of data packets transmitted by the plurality of queues 5201-520N in at least one of a round-robin manner and a fair arbitration manner. According to at least one embodiment of the present disclosure, the weight of the fair arbitration process performed by the first arbiter 5410 can not be fixed. For example, the weight can be dynamically updated according to the bandwidth requirement of the data transmission request, the service priority, etc., but the present disclosure is not limited thereto, and other ways of determining the weight of the fair arbitration process are also possible.

[0094] According to at least one embodiment of the present disclosure, the second arbiter 5700 can be configured to arbitrate the first type of data packets transmitted by the first queue 5100 and the second type of data packets transmitted by the second queue 5200 to determine a second target data packet from among the first type of data packets and the second type of data packets. The second arbiter 5700 can transmit the second target data packet determined by arbitration to the routing device 5800. The second arbiter 5700 can be further configured to arbitrate the first type of data packets and the second type of data packets in a manner in which the priority of the first type of data packets is lower than the priority of the second type of data packets. In this way, the second type of data packets can be transmitted as soon as possible, preventing the second type of data packets from being affected by subsequent first type of data packets.

[0095] According to at least one embodiment of the present disclosure, the routing device 5800 can be configured to receive the first type of data packets from the first queue 5100 and the second type of data packets from the second queue 5200 (e.g., through the first arbiter 5410 and the second arbiter 5700). The routing device 5800 can transmit the first type of data packets and the second type of data packets to the data operation unit, e.g., through the NOC 5002. Further, a response message to the first type of data packets can be received from the data operation unit, e.g., through the NOC 5002.

[0096] According to at least one embodiment of the present disclosure, the first type of data includes a first identification field indicating a first other artificial intelligence processor including the other artificial intelligence processor sending the first type of data, and the counters in the counter set each correspond to a second identification field. For example, for the first other artificial intelligence device, the routing device 5800 can split the first type of data packet into a write address packet and a write data packet to send to the data operation unit through the NOC 5002, and the routing device 5800 can determine a third identification field according to the first identification field and the second identification field, and assign the third identification field to the write address packet and the write data packet. As previously described, the AXI protocol can be configured in the NOC 5002, by splitting the first type of data packet into a write address packet and a write data packet at the routing device 5800, whole packet scheduling can be employed before converting the first type of data packet into an AXI request. In this way, the AXI AW (write address) channel and the write data (W) channel can be prevented from being prematurely separated, thereby simplifying the design and reducing the probability of design errors. According to at least one embodiment of the present disclosure, the routing device 5800 can perform similar splitting on the second type of data packet, or assign to the corresponding channel in the AXI protocol.

[0097] According to at least one embodiment of the present disclosure, after the data operation unit completes receiving the first type of data packet with the third identification field, a response message (e.g., a write response (b response) message) with the third identification field can be returned. Since the third identification fields corresponding to the write data packets and the write channel packets split from different data packets can be different, the data operation unit can return the response messages out of order. The routing device 5800 can also be configured to identify the third identification field included in the received response message, and determine the first identification field and the second identification field based on the third identification field. The routing device 5800 can determine the counter in the counter set whose count value is to be reduced in a reverse lookup manner based on the first identification field and the second identification field.

[0098] According to at least one embodiment of the present disclosure, the first sub-communication device 5001 can be configured to receive, by the communication network, at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor for the data operation unit. According to at least one embodiment of the present disclosure, the first sub-communication device 5001 can determine whether there is a free second counter in the counter set for the first other artificial intelligence processor in response to storing the second data packet to the second queue. For example, in response to determining that there is a free second counter, the at least one third data packet is stored to the first queue 5100 for sending to the data operation unit. Further, in response to storing the at least one third data packet to the first queue 5100 respectively, the count value of the first counter can be stopped increasing and the count value of the second counter can be increased. That is, in response to the second data packet entering the second queue, the growth of the first counter corresponding to the second data packet can be stopped, and the free second counter is switched to count the at least one third data packet after the second data packet entering the first queue 5100.

[0099] According to at least one embodiment of the present disclosure, the first sub-communication device 5001 can be configured to store the at least one third data packet to the first queue for sending to the data operation unit in response to determining that there is no free counter in the counter set (for example, all counters are used for counting, and all corresponding count values have not been reduced to zero). In response to storing the at least one third data packet to the first queue respectively, the count value of the first counter can continue to increase. That is, when the second data packet from the first other artificial intelligence processor enters the second queue, there is no free counter in the counter set for the first other artificial intelligence processor, and the current first counter can continue to be used to count the subsequent at least one third data packet. Further, the second data packet can be sent from the second queue in response to the count value of the corresponding first counter being reduced to zero by the response message.

[0100] According to at least one embodiment of the present disclosure, the artificial intelligence processor can further include a second sub-communication device. For example, the second sub-communication device can be a transmitting device in an Ethernet port of the artificial intelligence processor. The second sub-communication device can be configured to transmit, for the data operation unit, at least one fourth data packet of the first type and a fifth data packet of the second type associated with the at least one fourth data packet to the communication network. The fifth data packet can indicate that the first other artificial intelligence processor continues to transmit the at least one fourth data packet until the fifth data packet. The second sub-communication device can be configured to, in response to receiving the at least one fourth data packet from the data operation unit, transmit a response message (e.g., b response) corresponding to the at least one fourth data packet, respectively, to the data operation unit. In response to transmitting the response message corresponding to the at least one fourth data packet, respectively, the second sub-communication device can receive the fifth data packet from the data operation unit. In response to receiving the fifth data packet from the data operation unit, the second sub-communication device transmits a response message corresponding to the fifth data packet to the data operation unit. In this way, latency can be reduced so that the data operation unit can continuously transmit data packets of the first type and data packets of the second type.

[0101] Figure 6 A flowchart of a data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure is shown. As shown in FIG. 6, the method 6000 includes steps S6100-S6200. An artificial intelligence processor performing the method shown in FIG. 6 can include a data operation unit and a first sub-communication device. The data operation unit can be configured to receive data packets of a first type and data packets of a second type from the first sub-communication device. Figure 6 Figure 6 The artificial intelligence processor performing the method shown in FIG. 6 can include a data operation unit and a first sub-communication device. The data operation unit can be configured to receive data packets of a first type and data packets of a second type from the first sub-communication device.

[0102] At step S6100, at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet can be received from a first other artificial intelligence processor through a communication network by using the first sub-communication device, wherein the second data packet indicates that the first other artificial intelligence processor continues to transmit the at least one first data packet until the second data packet.

[0103] At step S6200, in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, the second data packet can be transmitted to the data operation unit by using the first sub-communication device.

[0104] ​A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a first queue for data packets of a first type and a second queue for data packets of a second type; and the method further comprises, by the first sub-communication device: storing at least one first data packet to the first queue for sending to the data operation unit, receiving response messages corresponding to the at least one first data packet respectively from the data operation unit, storing a second data packet to the second queue, and sending the second data packet from the second queue to the data operation unit in response to receiving the response messages corresponding to all of the at least one first data packet from the data operation unit.

[0105] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a set of counters for a first other artificial intelligence processor, wherein the method further comprises, by the first sub-communication device: increasing a count value of a first counter in the set of counters in response to storing the at least one first data packet to the first queue respectively; decreasing the count value of the first counter in response to receiving the response messages corresponding to the at least one first data packet respectively from the data operation unit; and sending the second data packet to the data operation unit in response to the count value of the first counter being decreased to zero.

[0106] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a third queue for data packets of the first type and the second type and a multiplexer, wherein the method further comprises, by the first sub-communication device: storing at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue; and sending the at least one first data packet and the second data packet stored in the third queue to the first queue and the second queue respectively through the multiplexer in an order of receiving through the communication network.

[0107] A data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a buffer and a first linked list and a second linked list for data packets of the second type, wherein the first linked list comprises one or more first nodes, each of the one or more first nodes points to a storage area in the buffer for storing a corresponding data packet of the second type; and the second linked list comprises one or more second nodes, each of the one or more second nodes points to a free storage area in the buffer.

[0108] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further comprises, by the first sub-communication device: in response to the presence of a free counter in the counter set, sending the second type of data packet from the buffer to the second queue, and releasing the storage area corresponding to the sent second type of data packet; and in response to the absence of a free counter in the counter set, not sending the second type of data packet from the buffer to the second queue.

[0109] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, by the first sub-communication device, receives the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through a communication network; and wherein the first linked list is a plurality, and the plurality of first linked lists correspond to the plurality of other artificial intelligence processors respectively.

[0110] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further comprises, by the first sub-communication device: receiving the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through a communication network; wherein the second queue is a plurality, and the plurality of second queues correspond to the plurality of other artificial intelligence processors respectively; and wherein the counter set is a plurality, and the plurality of counter sets correspond to the plurality of other artificial intelligence processors respectively.

[0111] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a first arbitrator, and the method further comprises arbitrating the second type of data packet sent by the plurality of second queues using the first arbitrator to determine the first target data packet of the second type.

[0112] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a routing device, and wherein the method further comprises, by the routing device: receiving the first type of data packet from the first queue and the second type of data packet from the second queue; sending the first type of data packet and the second type of data packet to the data operation unit; and receiving a response message to the first type of data packet from the data operation unit.

[0113] A data transmission method for artificial intelligence processors according to at least one embodiment of the present disclosure, wherein the first type of data includes a first identification field indicating a plurality of other artificial intelligence processors including the first other artificial intelligence processor that sends the first type of data, and the counters in the counter set each correspond to a second identification field; and wherein the method further comprises, by the routing device: splitting the first type of data packet into a write address packet and a write data packet for sending to the data operation unit, and determining a third identification field according to the first identification field and the second identification field, and assigning the third identification field to the write address packet and the write data packet.

[0114] A data transmission method for artificial intelligence processors according to at least one embodiment of the present disclosure, further comprising, by the routing device: identifying the third identification field included in the received response message; determining the first identification field and the second identification field based on the third identification field; and determining the counter in the counter set to reduce the count value based on the first identification field and the second identification field.

[0115] A data transmission method for artificial intelligence processors according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a second arbiter, and wherein the method further comprises, by the second arbiter: arbitrating the first type of data packet sent by the first queue and the second type of data packet sent by the second queue to determine a second target data packet in the first type of data packet and the second type of data packet; and sending the second target data packet to the routing device.

[0116] A data transmission method for artificial intelligence processors according to at least one embodiment of the present disclosure, wherein the first sub-communication device further comprises a second arbiter, and wherein the method further comprises, by the second arbiter: arbitrating the first type of data packet and the second type of data packet in a manner that the priority of the first type of data packet is lower than the priority of the second type of data packet.

[0117] A data transmission method for artificial intelligence processors according to at least one embodiment of the present disclosure, further comprising, by using the first sub-communication device, receiving the first type of data packet and the second type of data packet from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through a communication network; and wherein the depth of the second queue for the first other artificial intelligence processor and the number of counters in the counter set for the first other artificial intelligence processor are the same.

[0118] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further comprises receiving, by the first sub-communication device, at least one third data packet of the first type after the second data packet for the data operation unit from the first other artificial intelligence processor over the communication network; and wherein the method further comprises, by the first sub-communication device: in response to storing the second data packet to the second queue, determining that there is a free second counter in the set of counters, storing the at least one third data packet to the first queue for sending to the data operation unit, and in response to storing the at least one third data packet to the first queue respectively, stopping increasing the count value of the first counter and increasing the count value of the second counter.

[0119] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure further comprises receiving, by the first sub-communication device, at least one third data packet of the first type after the second data packet for the data operation unit from the first other artificial intelligence processor over the communication network; and wherein the method further comprises, by the first sub-communication device: in response to storing the second data packet to the second queue, determining that there is no free counter in the set of counters, storing the at least one third data packet to the first queue for sending to the data operation unit, and in response to storing the at least one third data packet to the first queue respectively, continuing to increase the count value of the first counter.

[0120] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further comprises a second sub-communication device, the method further comprises, by the second sub-communication device: sending, for the data operation unit, at least one fourth data packet of the first type and a fifth data packet of the second type associated with the at least one fourth data packet to the communication network, wherein the fifth data packet indicates that the first other artificial intelligence processor continues to send the at least one fourth data packet to the fifth data packet; and wherein the method further comprises, by the second sub-communication device: in response to receiving the at least one fourth data packet from the data operation unit, sending a response message corresponding to the at least one fourth data packet respectively to the data operation unit, in response to sending the response message corresponding to the at least one fourth data packet respectively, receiving the fifth data packet from the data operation unit, and in response to receiving the fifth data packet from the data operation unit, sending a response message corresponding to the fifth data packet to the data operation unit.

[0121] The data transmission method for an artificial intelligence processor according to at least one embodiment of the present disclosure, wherein the artificial intelligence processor further comprises a plurality of computing cores and a system direct memory access device; and wherein the data operation unit is included in at least one of the at least one computing core and the system direct memory access device of the artificial intelligence processor.

[0122] Figure 7 A schematic block diagram of an electronic device is shown, which is provided by at least one embodiment of the present disclosure.

[0123] As Figure 7 shown, the electronic device 300 is suitable for implementing the data transmission method for artificial intelligence processor provided by the embodiments of the present disclosure, for example. It should be noted that Figure 7 The components of the electronic device 300 shown are only exemplary and are not limiting, and the electronic device 300 can also have other components according to actual application needs.

[0124] For example, as Figure 7 shown, in some examples, the electronic device 300 includes a processing device (for example, including a parallel processor such as a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), etc. as shown with reference to Figures 3-4 , which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 302 or programs loaded into a random access memory (RAM) 303 from a storage device 308 to implement various functions, such as performing the data transmission method for artificial intelligence processor provided by at least one embodiment of the present disclosure.

[0125] For example, when computer readable instructions are executed by the processing device 301, one or more steps of the data transmission method for artificial intelligence processor according to any of the above embodiments can be performed. It should be noted that the detailed description of the processing process of the data transmission method for artificial intelligence processor can refer to the related description in the above embodiments of the data transmission method for artificial intelligence processor, and the repeated parts will not be described again.

[0126] In the RAM 303, various programs and data required for computer system operation are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0127] For example, the memory can include any combination of one or more computer program products which can include various forms of computer-readable storage media, for example, volatile and / or non-volatile memory. Volatile memory, for example, can include random access memory (RAM) 303 and / or cache, etc. For example, computer-readable instructions can be loaded into random access memory (RAM) 303 from storage 308 to run computer-readable instructions. Non-volatile memory, for example, can include read only memory (ROM) 302, hard disk, erasable programmable read only memory (EPROM), compact disc read only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various data, for example, style images, and various data used and / or generated by the applications, etc. can also be stored in the computer-readable storage media.

[0128] Generally, the following devices can be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other electronic devices to exchange data. Drivers 310 are also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, are mounted on the driver 310 as needed, so that computer programs read therefrom are installed in the storage device 308 as needed. Although Figure 7 The electronic device 300 including various devices is shown, but it is understood that all of the shown devices are not required to be implemented or included. More or less devices can alternatively be implemented or included. For example, the processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 can be a central processing unit (CPU), a general purpose graphics processor (GPGPU), a parallel processor, an artificial intelligence processor, a tensor processor (TPU), or a graphics processor (GPU), etc. device having data processing capability and / or program execution capability. The central processing unit can be X86 or ARM architecture, etc. The graphics processor can be directly integrated onto the mainboard alone or built into the north bridge chip of the mainboard. The graphics processor can also be built into the central processing unit.

[0129] For example, the electronic device 300 can further include a peripheral interface (not shown in the figure) and the like. The peripheral interface can be various types of interfaces, such as a USB interface, a lighting interface, and the like. The communication device 309 can communicate with networks and other devices through wireless communication, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communication can use any of a plurality of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0130] For example, the electronic device 300 can be any device such as a mobile phone, a tablet computer, a notebook computer, an e-book, a game console, a television, a digital photo frame, a navigator, and the like, or any combination of electronic devices and hardware, and embodiments of the present disclosure are not limited thereto.

[0131] For example, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the data transmission method for an artificial intelligence processor defined in the methods of embodiments of the present disclosure is executed.

[0132] Note that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium may, for example, be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In an embodiment of the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used or used in conjunction with an instruction execution system, apparatus or device. In an embodiment of the present disclosure, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium that can send, propagate or transmit a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to a wire, an optical fiber, an RF (radio frequency) or the like, or any suitable combination thereof.

[0133] The computer readable medium described above can be included in the electronic device described above; or can exist separately without being assembled into the electronic device.

[0134] The artificial intelligence processor, the electronic device and the data transmission method according to at least one embodiment of the present disclosure can confirm whether the data transmission is completed in a convenient manner while avoiding or reducing the participation of the CPU in the data transmission process of the artificial intelligence processor. Specifically, the artificial intelligence processor according to at least one embodiment of the present disclosure can receive a first type of data packet and a second type of data packet associated with the first type of data packet. The second type of data packet can be received by the data operation unit of the artificial intelligence processor as a receiving end after ensuring that the first type of data packet associated with the second type of data packet has been completely received by the artificial intelligence processor as a receiving end (e.g., a write operation). In this way, the artificial intelligence processor as a receiving end can be indicated by the second type of data packet that the first type of data packet associated with the second type of data packet has been completely received.

[0135] The computer program product of the present disclosure can be a computer program including a plurality of instructions for executing the methods according to the embodiments of the present disclosure on various data processing apparatuses. More specifically, the computer program product, physical medium or article of manufacture including the plurality of instructions can be deployed / employed to implement processes, methods, functions, etc. according to the embodiments of the present disclosure.

[0136] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0137] The above description is merely illustrative of the exemplary embodiments of the present disclosure and the principles of the technology involved. It is understood that the disclosed scope of the present disclosure is not limited to the technical solutions formed by the specific combinations of the above technical features, and also includes other technical solutions formed by the combinations of the above technical features or equivalent features thereof without departing from the above disclosed concept. For example, the technical solutions formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0138] In addition, although each operation is depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing can be advantageous. Similarly, although several specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of separate embodiments can also be combined in a single embodiment. Conversely, various features described in the context of a single embodiment can also be separated and implemented in multiple embodiments.

[0139] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0140] For the present disclosure, the following points also need to be explained:

[0141] (1) The drawings of the embodiments of the present disclosure only involve the structures related to the embodiments of the present disclosure, and other structures can be referred to the general design.

[0142] (2) In the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined to obtain new embodiments.

[0143] The above description is merely a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. An artificial intelligence processor, comprising: Comprising: a first sub-communication device configured to receive, from a first other artificial intelligence processor over a communication network, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet, wherein the second data packet indicates that the first other artificial intelligence processor continuously sends the at least one first data packet until the second data packet; a data operation unit configured to receive, from the first sub-communication device, data packets of the first type and data packets of the second type, wherein the first sub-communication device is further configured to, in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, send the second data packet to the data operation unit.

2. The artificial intelligence processor of claim 1, wherein, the first sub-communication device further comprises a first queue for data packets of the first type, a second queue for data packets of the second type; and the first sub-communication device is further configured to: store the at least one first data packet to the first queue for sending to the data operation unit, receive, from the data operation unit, response messages corresponding to the at least one first data packet respectively, store the second data packet to the second queue, and in response to receiving, from the data operation unit, response messages corresponding to all of the at least one first data packet, send the second data packet from the second queue to the data operation unit.

3. The artificial intelligence processor of claim 2, wherein, the first sub-communication device further comprises a set of counters for the first other artificial intelligence processor, wherein the first sub-communication device is further configured to: in response to storing the at least one first data packet to the first queue respectively, increase a count value of a first counter in the set of counters; in response to receiving, from the data operation unit, response messages corresponding to the at least one first data packet respectively, decrease the count value of the first counter; and in response to the count value of the first counter decreasing to zero, send the second data packet to the data operation unit.

4. The artificial intelligence processor of claim 2, wherein, the first sub-communication device further comprises a third queue for data packets of the first type and data packets of the second type and a multiplexer, wherein the first sub-communication device is further configured to: store at least one first data packet of the first type and a second data packet of the second type associated with the at least one first data packet into the third queue; and send, to the first queue and the second queue respectively, at least one first data packet and a second data packet stored in the third queue through the multiplexer in an order of reception over the communication network.

5. The artificial intelligence processor of claim 3, wherein, the first sub-communication device further comprises a buffer and a first linked list and a second linked list for data packets of the second type, wherein, the first linked list comprises one or more first nodes, each of the one or more first nodes points to a storage area in the buffer for storing a corresponding data packet of the second type; and the second linked list comprises one or more second nodes, each of the one or more second nodes points to a storage area in the buffer for storing a corresponding data packet of the second type. The second chain list comprises one or more second nodes, each of the one or more second nodes points to a free storage area in the buffer.

6. The artificial intelligence processor of claim 5, wherein, The first sub-communication device is further configured to: in response to there being a free counter in the counter set, send the second type of data packet from the buffer to the second queue, and release the storage area corresponding to the sent second type of data packet; and in response to there being no free counter in the counter set, not send the second type of data packet from the buffer to the second queue.

7. The artificial intelligence processor of claim 5, wherein, The first sub-communication device is configured to receive, through the communication network, first type of data packets and second type of data packets from a plurality of other artificial intelligence processors including the first other artificial intelligence processor; and wherein the first chain list is a plurality, and the plurality of first chain lists respectively correspond to the plurality of other artificial intelligence processors.

8. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive, through the communication network, first type of data packets and second type of data packets from a plurality of other artificial intelligence processors including the first other artificial intelligence processor, wherein the second queue is a plurality, and the plurality of second queues respectively correspond to the plurality of other artificial intelligence processors, and wherein the counter set is a plurality, and the plurality of counter sets respectively correspond to the plurality of other artificial intelligence processors.

9. The artificial intelligence processor of claim 8, wherein, The first sub-communication device further comprises a first arbitrator; and wherein the first arbitrator is configured to arbitrate the second type of data packets sent by the plurality of second queues to determine a first target data packet of the second type.

10. The artificial intelligence processor of claim 3, wherein, The first sub-communication device further comprises a routing device, and wherein the routing device is configured to: receive first type of data packets from the first queue and second type of data packets from the second queue; send the first type of data packets and the second type of data packets to the data operation unit; and receive a response message to the first type of data packets from the data operation unit.

11. The artificial intelligence processor of claim 10, wherein, The first type of data comprises a first identification field indicating a plurality of other artificial intelligence processors including the first other artificial intelligence processor that sends the first type of data, and the counters in the counter set each correspond to a second identification field; and wherein the routing device is further configured to: split the first type of data packet into a write address packet and a write data packet for sending to the data operation unit, and determine a third identification field according to the first identification field and the second identification field, and assign the third identification field to the write address packet and the write data packet.

12. The artificial intelligence processor of claim 11, wherein, The routing device is further configured to: identify the third identification field included in the received response message; determine the first identification field and the second identification field based on the third identification field; and determine the counter in the counter set to reduce the count value based on the first identification field and the second identification field.

13. The artificial intelligence processor of claim 10, wherein, The first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate between the first type of data packets sent by the first queue and the second type of data packets sent by the second queue to determine a second target data packet among the first type of data packets and the second type of data packets; and send the second target data packet to the routing device.

14. The artificial intelligence processor of claim 13, wherein, The first sub-communication device further comprises a second arbiter, and wherein the second arbiter is configured to: arbitrate between the first type of data packets and the second type of data packets in a manner that the priority of the first type of data packets is lower than the priority of the second type of data packets.

15. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive the first type of data packets and the second type of data packets from a plurality of other artificial intelligence processors including the first other artificial intelligence processor through the communication network; and wherein the depth of the second queue for the first other artificial intelligence processor and the number of counters in the set of counters for the first other artificial intelligence processor are the same.

16. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor for the data operation unit through the communication network; and wherein the first sub-communication device is configured to: determine that there is a free second counter in the set of counters in response to storing the second data packet to the second queue, store the at least one third data packet to the first queue for sending to the data operation unit, and stop increasing the count value of the first counter and increase the count value of the second counter in response to storing the at least one third data packet to the first queue, respectively.

17. The artificial intelligence processor of claim 3, wherein, The first sub-communication device is configured to receive at least one third data packet of the first type after the second data packet from the first other artificial intelligence processor for the data operation unit through the communication network; and wherein the first sub-communication device is configured to: determine that there is no free counter in the set of counters in response to storing the second data packet to the second queue, store the at least one third data packet to the first queue for sending to the data operation unit, and continue to increase the count value of the first counter in response to storing the at least one third data packet to the first queue, respectively.

18. The artificial intelligence processor of any one of claims 1 to 17, wherein, The second sub-communication device is further configured to send at least one fourth data packet of the first type and a fifth data packet of the second type associated with the at least one fourth data packet to the communication network for the data operation unit, wherein the fifth data packet indicates that the first other artificial intelligence processor continues to send the at least one fourth data packet to the fifth data packet; and wherein the second sub-communication device is further configured to: in response to receiving the at least one fourth data packet from the data operation unit, sending a response message corresponding to the at least one fourth data packet, respectively, to the data operation unit, in response to sending the response message corresponding to the at least one fourth data packet, respectively, receiving a fifth data packet from the data operation unit, and in response to receiving the fifth data packet from the data operation unit, sending a response message corresponding to the fifth data packet to the data operation unit.

19. The artificial intelligence processor of any one of claims 1 to 17, wherein, further comprising a plurality of compute cores and system direct memory access devices; and wherein the data operation unit is comprised in at least one of the at least one compute core and the system direct memory access device of the artificial intelligence processor.

20. An electronic device, comprising: An artificial intelligence processor according to any one of claims 1-19.

21. A data transfer method for an artificial intelligence processor, the method comprising: The artificial intelligence processor comprises a data operation unit and a first sub-communication device, and the data operation unit is configured to receive first type data packets and second type data packets from the first sub-communication device, and the method comprises: receiving, by using the first sub-communication device, at least one first data packet of a first type and a second data packet of a second type associated with the at least one first data packet from a first other artificial intelligence processor over a communication network, wherein the second data packet indicates that the first other artificial intelligence processor continues to send the at least one first data packet until the second data packet; and in response to the data operation unit receiving all of the at least one first data packet from the first sub-communication device, sending, by using the first sub-communication device, the second data packet to the data operation unit.

Images