Artificial intelligence chip and computing system

Abstract

Embodiments of the present disclosure relate to the field of processor hardware architecture, and provide an artificial intelligence chip and a computing system. The artificial intelligence chip comprises a first computing core, a shared memory device corresponding to the first computing core, and a data synchronization device. The first computing core is configured to issue a first data request instruction. The data synchronization device is configured to perform a read operation on first data, a flag detection operation, and a synchronization operation according to the first data request instruction. The first data is stored in a global memory device coupled to the artificial intelligence chip. The synchronization operation comprises sending the first data to the shared memory device. The shared memory device is configured to store the first data sent by the data synchronization device for reading by the first computing core. Embodiments of the present disclosure hardwareize data synchronization and retry, significantly reduce communication delay, and release core computing resources of the artificial intelligence chip.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of processor hardware architecture, and more specifically to an artificial intelligence chip and computing system. Background Technology

[0002] Modern GPU hardware (such as NVLink) offers extremely high theoretical bandwidth, but in practical applications, the performance of inter-GPU communication is still constrained by a variety of factors, one of the main bottlenecks being the overhead of data synchronization. In collective communication operations, the sender and receiver need to coordinate the rhythm of data transmission to ensure that data is not overwritten before the receiver is ready, and that the receiver does not begin processing until the data has fully arrived. This synchronization is typically achieved through software-level flag polling or memory barriers. Summary of the Invention

[0003] At least one embodiment of this disclosure provides an artificial intelligence chip, which includes a first computing core, a shared memory device corresponding to the first computing core, and a data synchronization device. The first computing core is configured to issue a first data request instruction. The data synchronization device is configured to perform a reading operation on the first data, a flag bit detection operation on the first data, and a synchronization operation according to the first data request instruction. The first data is stored in a global memory device coupled to the artificial intelligence chip, and the synchronization operation includes sending the first data to the shared memory device. The shared memory device is configured to store the first data sent by the data synchronization device for the first computing core to read and use.

[0004] For example, in an artificial intelligence chip according to at least one embodiment of this disclosure, the data synchronization device includes an instruction parsing and scheduling unit, a memory access control unit, a flag bit detection and comparison unit, and a data writing unit. The read operation includes an instruction parsing and scheduling operation for a first data request instruction and a memory access control operation for the first data. The instruction parsing and scheduling unit is configured to perform the instruction parsing and scheduling operation for the first data request instruction, the memory access control unit is configured to perform the memory access control operation for the first data, the flag bit detection and comparison unit is configured to perform the flag bit detection operation for the first data, and the data writing unit is configured to perform the synchronization operation for the first data.

[0005] For example, in an artificial intelligence chip according to at least one embodiment of this disclosure, the instruction parsing and scheduling unit is further configured to receive and parse a first data request instruction to obtain instruction configuration parameters, and send the instruction configuration parameters to the memory access control unit and the flag bit detection and comparison unit. The instruction configuration parameters include the source address of the first data in the global memory device, the target shared memory address where the first data needs to be stored in the shared memory device, and the expected flag bit corresponding to the first data. The memory access control unit is further configured to generate and send a memory read request to the global memory device controller based on the source address, and in response to receiving the pending data located at the source address returned by the global memory device controller, forward the pending data to the flag bit detection and comparison unit. The flag bit detection and comparison unit is further configured to extract and compare the flag bits in the pending data with the expected flag bits to obtain a flag bit comparison result, and send the flag bit comparison result to the data writing unit. The data writing unit is further configured to send the pending data and the target shared memory address to the shared memory device in response to a matching flag bit comparison result, so that the write control unit of the shared memory device writes the pending data to the shared memory device based on the target shared memory address.

[0006] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, the instruction configuration parameters further include an abort signal address, the data synchronization device further includes an abort unit, and the memory access control unit is further configured to send the abort signal address to the global memory device controller; the abort unit is configured to suspend the instruction parsing and scheduling unit from receiving the first data request instruction in response to the receipt of an abort signal at the abort signal address returned by the global memory device controller as valid.

[0007] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, the flag bit detection and comparison unit is further configured to temporarily store the data to be processed in the buffer unit of the artificial intelligence chip before extracting and comparing the flag bit of the data to be processed with the expected flag bit to obtain the flag bit comparison result.

[0008] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, the data writing unit is further configured to extract the remaining valid data other than the flag bit from the data to be processed in the buffer unit, and generate at least one data write request according to the target shared memory address to send the remaining valid data to the shared memory device.

[0009] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, at least one data write request includes multiple sub-write requests, each sub-write request corresponding to at least a portion of the remaining valid data. Sending the remaining valid data to the shared memory device through the data write unit includes: using write merging technology to merge the multiple sub-write requests into a large-width write request aligned with the target shared memory address, and sending the remaining valid data as a whole to the shared memory device based on the large-width write request.

[0010] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, the data synchronization device is further configured to perform a rereading operation on the first data according to the first data request instruction. The data synchronization device further includes a retry unit configured to send a retry signal to the global memory device controller in response to a mismatch in the flag comparison result, so that the global memory device controller reads the data at the source address again.

[0011] For example, in an artificial intelligence chip according to at least one embodiment of this disclosure, the retry unit includes a retry counter, and the retry unit is further configured to: after the global memory device controller reads the data at the source address again, update the retry counter and detect whether the updated number of retries exceeds a first threshold, or detect whether the retry time exceeds a second threshold; and in response to the updated number of retries exceeding the first threshold or the retry time exceeding the second threshold, terminate the retry operation and send an error message to the first computing core.

[0012] For example, in an artificial intelligence chip according to at least one embodiment of this disclosure, the length of the data to be processed is equal to 128 bytes, the flag bit is the lower 8 bytes of the data to be processed, and the remaining valid data is the upper 120 bytes of the data to be processed.

[0013] For example, in an artificial intelligence chip according to at least one embodiment of this disclosure, the artificial intelligence chip further includes a tensor acceleration engine, a data synchronization device is integrated into the tensor acceleration engine, the tensor acceleration engine is configured to receive and forward a first data request instruction to an instruction parsing and scheduling unit; the data writing unit is further configured to reuse the data transport engine in the tensor acceleration engine; the tensor acceleration engine is further configured to set a status register or send a lightweight interrupt signal to the first computing core to report that the data to be processed has been transferred to the shared memory device after the data writing unit sends the data to be processed and the target shared memory address to the shared memory device.

[0014] For example, in an artificial intelligence chip according to at least one embodiment of the present disclosure, the artificial intelligence chip includes a secondary cache, a data synchronization device is integrated in the secondary cache, the secondary cache is configured to receive and route a first data request instruction to an instruction parsing and scheduling unit, and route the data to be processed located at the source address returned by the global memory device controller to the memory access control unit; the secondary cache is also configured to report to the first computing core that the data to be processed is ready in response to a flag comparison result matching.

[0015] At least one embodiment of this disclosure also provides a computing system including a plurality of artificial intelligence chips according to the above-described at least one embodiment, wherein the plurality of artificial intelligence chips are interconnected with each other. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0017] Figure 1 This is a schematic diagram of a structure for inter-GPU communication interconnection;

[0018] Figure 2 This is an exemplary execution flowchart for data synchronization based on the LL128 protocol;

[0019] Figure 3 This is a schematic diagram of the structure of an artificial intelligence chip provided in at least one embodiment of the present disclosure;

[0020] Figure 4 A schematic diagram of the structure of a data synchronization device provided in at least one embodiment of this disclosure;

[0021] Figure 5 A schematic diagram of the structure of another artificial intelligence chip provided in at least one embodiment of this disclosure;

[0022] Figure 6 A schematic diagram of the structure of another artificial intelligence chip provided in at least one embodiment of the present disclosure;

[0023] Figure 7 This is a schematic diagram of the structure of a computing system provided in at least one embodiment of the present disclosure;

[0024] Figure 8 This is a schematic diagram of an electronic device provided for at least one embodiment of the present disclosure. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0026] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0027] The NVIDIA Collective Communication Library (NCCL) is a highly optimized communication library designed specifically for NVIDIA GPUs. It provides a set of standard collective communication primitives, such as AllReduce, Broadcast, Reduce, AllGather, and ReduceScatter. NCCL plays a crucial role in multi-GPU parallel computing, especially in deep learning. When training a large neural network, it is typically necessary to distribute the model or data across multiple GPUs, which need to exchange gradient information after each iteration to update the model parameters.

[0028] Figure 1 This is a schematic diagram of a structure for inter-GPU communication interconnection.

[0029] like Figure 1As shown, GPU 0 needs to load the data to be transmitted from the send buffer of memory unit 0 into stream processor 0, and then stream processor 0 of GPU 0 writes it to the receive buffer in the memory of GPU 1 through a physical transmission path (e.g., PCIe). Stream processor 1 of GPU 1 ensures the write is complete and performs the corresponding reduction operation. The core of this process is collective communication operation. NCCL greatly simplifies the complexity of multi-GPU programming and significantly improves communication efficiency by providing highly optimized, scalable communication algorithms and protocols. It not only fully utilizes the high-speed interconnects between GPUs (such as NVLink), but also intelligently selects the optimal communication algorithm (such as Ring or Tree) and protocol (such as Simple, LL, or LL128) to adapt to different message sizes and system topologies. Therefore, the performance of NCCL directly determines the scalability and efficiency of multi-GPU parallel computing and is an indispensable software foundation for building large-scale, high-performance computing systems.

[0030] Despite the extremely high theoretical bandwidth offered by modern GPU hardware (such as NVLink), inter-GPU communication performance remains constrained by various factors in practical applications. One of the main bottlenecks is synchronization overhead. In collective communication operations, the sender and receiver need to coordinate the pace of data transmission to ensure that data is not overwritten before the receiver is ready and that the receiver does not begin processing until the data has fully arrived. This synchronization is typically achieved through software-level flag polling or memory barriers.

[0031] Software polling means that the GPU core needs to continuously check a flag in memory until its value changes. This process consumes valuable computing core resources, resulting in wasted computing power. While memory barriers ensure the sequentiality of memory operations, they also introduce significant latency overhead, especially in high-frequency, small-message communication scenarios. Another bottleneck is the overhead of the communication protocol itself. For example, in the LL protocol, the flags added for synchronization occupy half of the data packet space, resulting in an effective bandwidth utilization of only about 50%.

[0032] The LL128 (Low Latency 128) protocol is a key communication protocol in NCCL, designed to achieve higher bandwidth utilization than the LL protocol while maintaining low latency. In the NCCL protocol architecture, the Simple protocol focuses on maximizing bandwidth for large message transmissions but incurs significant synchronization overhead; the LL protocol sacrifices bandwidth for extremely low latency in small message transmissions. The LL128 protocol cleverly balances these two aspects by using larger data packets (128 bytes) to amortize synchronization overhead while retaining a lightweight flag-based synchronization mechanism, avoiding costly memory barriers.

[0033] The core feature of the LL128 protocol is its fixed 128-byte data packet structure. Each data packet is precisely divided into two parts: a 120-byte payload and an 8-byte flag. This structure is designed to maximize bandwidth efficiency. Compared to the LL protocol's 4-byte data plus 4-byte flag (50% effective bandwidth) structure, the LL128 protocol increases the effective data ratio to 120 / 128, or approximately 93.75%. This means that when transmitting the same amount of data, the LL128 protocol needs to transmit less metadata (flags), thus making fuller use of the underlying physical link bandwidth.

[0034] Specifically, the LL128 protocol employs a flag-based synchronization mechanism to coordinate the operations of the sender and receiver. After writing 120 bytes of data to the target memory, the sender immediately writes a specific flag value to the end of the same 128-byte data packet (the last 8 bytes). This flag update serves as a clear signal to the receiver that the entire 128-byte data packet (including 120 bytes of valid data) has arrived completely and been written to memory. The receiver then polls this flag to determine if the data is ready. In the software implementation, a dedicated thread (flagThread) is typically responsible for checking this flag. When the receiver detects that the flag value matches the expected value, it knows that the data has safely arrived and can begin reading and processing the 120 bytes. Compared to the Simple protocol, which uses a global memory barrier (threadfence_system), this mechanism has significantly lower synchronization overhead because it avoids system-wide synchronization operations, limiting synchronization to the read and write operations of a single data packet, thus achieving lower communication latency.

[0035] In the software implementation of the NCCL LL128 protocol, its architecture primarily relies on thread blocks or warps on the GPU to collaboratively send and receive data. For example, firstly, one or more GPU threads (typically a portion of a warp) are assigned as "worker threads" responsible for performing data loading operations. Among these worker threads, a few are designated as "flag threads." The core responsibility of these flag threads is to poll and check the flags in memory. The entire synchronization process is controlled by software logic. The worker thread initiates a read request to High Bandwidth Memory (HBM) to obtain a 128-byte data block containing data and flags. Subsequently, the flag thread extracts the last 8 bytes of the flag from the read data block and compares it with an expected value. This comparison operation and subsequent retry decisions are entirely executed by software instructions (such as CUDA code). If the flags do not match, the software logic controls the thread to retry, i.e., re-initiate the read request until the flags match. The essence of this architecture is "software-defined synchronization," meaning that the control flow, decision logic, and retry mechanism of synchronization are all implemented through software code running on the GPU core.

[0036] Figure 2 This is an exemplary execution flowchart for data synchronization based on the LL128 protocol. Figure 2 As shown, the data synchronization execution flow includes steps S100-S106. Among them, the worker thread (including flagThread) is mainly used to execute steps S102-S105.

[0037] For example, the thread `flagThread` continuously reads 128-byte data packets from High Bandwidth Memory (HBM) and checks the last 8 bytes of flag bits. This process is typically performed in a loop, with the thread repeatedly executing load instructions until a specific condition is met. Each time data is read from HBM, `flagThread` compares the flag bit value in the data packet with a pre-calculated expected value, which is usually determined by the communication context and protocol state. If the flag bit matches the expected value, `flagThread` sends a signal (e.g., by setting a shared variable or using synchronization primitives within a thread bundle warp) to notify other worker threads in the same group that the data is ready and can be safely processed. If the flag bit does not match the expected value, it indicates that the data has not fully arrived or there is another problem. In this case, `flagThread` triggers retry logic. In software implementations, this typically means that control flow jumps to the beginning of the loop (i.e., "No" in step S105), re-initiating the memory read operation. This process repeats until the flag bit matches or polling is interrupted.

[0038] However, to achieve synchronization, a designated software thread (specifically the flagThread) must execute a continuous polling loop, constantly reading data from the HBM and checking the flag bits. During this polling process, the thread remains in a "busy waiting" or "spinning" state. This means the thread is bound to a single streaming processor (SP) on the GPU (i.e., a single computational core), continuously executing load and comparison instructions until the flag bits match. If there are multiple such communication warps, the computation threads may be blocked until the communication warps complete, due to the limited number of warps that each streaming multiprocessor (SM) can execute simultaneously. Before multiple communication warps complete, the GPU may be unable to load computation warps or the number of computation warps that can be loaded may be limited, resulting in idle computational units, such as matrix multiplication or convolution operations in neural networks. This resource idleness is particularly pronounced in communication-intensive applications; when a large number of threads are in a polling waiting state, a significant amount of GPU computational core resources are occupied, leading to a decrease in overall computational throughput.

[0039] Furthermore, the software polling mechanism itself introduces an inherent latency that cannot be ignored. This latency mainly consists of two parts: the overhead of executing the polling instructions and the memory access latency from initiating a memory read request to the return of data. In each polling iteration, the flagThread needs to execute at least one memory load instruction and one comparison instruction. The execution of these instructions themselves consumes multiple clock cycles. More importantly, each load instruction requires accessing the HBM, and the access latency of the HBM is typically on the order of hundreds of clock cycles. Even if the data is eventually valid, the receiver still needs to go through a complete software process of "initiating a request → waiting for data to return → checking the flag." This latency introduced by the software execution path is a significant disadvantage in low-latency communication scenarios that require microsecond or even nanosecond-level response times.

[0040] In view of at least one of the above-mentioned problems, at least one embodiment of this disclosure provides an artificial intelligence chip, which includes a first computing core, a shared memory device corresponding to the first computing core, and a data synchronization device. The first computing core is configured to issue a first data request instruction; the data synchronization device is configured to perform a reading operation on the first data, a flag bit detection operation on the first data, and a synchronization operation according to the first data request instruction. The first data is stored in a global memory device coupled to the artificial intelligence chip, and the synchronization operation includes sending the first data to the shared memory device. The shared memory device is configured to store the first data sent by the data synchronization device for the first computing core to read and use.

[0041] At least one embodiment of this disclosure also provides a computing system including multiple of the above-described artificial intelligence chips.

[0042] The artificial intelligence chip provided in at least one embodiment of this disclosure implements, for example, the data synchronization and retry mechanism of the LL128 protocol in hardware, freeing up the core computing resources of the AI ​​chip. Furthermore, it achieves nanosecond-level flag detection and decision-making through dedicated circuitry, with a response speed far faster than software instructions. In addition, the AI ​​chip provided in at least one embodiment of this disclosure greatly enhances the scalability of computing systems in large-scale cluster environments, making it possible to build efficient and stable communication systems on clusters of thousands or even tens of thousands of AI chips.

[0043] The artificial intelligence chip of this disclosure will be described below with reference to specific embodiments.

[0044] It should be noted that the artificial intelligence chip of at least one embodiment of this disclosure can be applied to large-scale distributed AI training and high-performance computing scenarios with extremely demanding communication performance requirements. In these scenarios, multiple artificial intelligence chips (e.g., GPUs) form a computing cluster through high-speed interconnect technology to collaboratively complete complex computing tasks. For example, when training large language models (LMs) or performing large-scale scientific simulations, GPUs need to frequently exchange and synchronize data, such as through aggregate communication operations like AllReduce and AllGather.

[0045] Figure 3 This is a schematic diagram of the structure of an artificial intelligence chip provided in at least one embodiment of the present disclosure.

[0046] like Figure 3 As shown, the artificial intelligence chip 200 includes a first computing core 210, a data synchronization device 220, and a shared memory device 230 corresponding to the first computing core 210. The artificial intelligence chip 200 is also coupled to an external global memory device 240 to realize data interaction.

[0047] Here, the artificial intelligence chip 200 is, for example, a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a neural processing unit (NPU), an intelligence processing unit (IPU), or a language processing unit (LPU). Unless otherwise specified, the following embodiments will use the artificial intelligence chip 200 as a GPU for description.

[0048] For example, the global memory device 240 is the high-bandwidth memory HBM mentioned above. This high-bandwidth memory HBM is an advanced dynamic random-access memory (DRAM) technology that vertically integrates multiple DRAM chips together in a 3D stacking manner and packages them on the same substrate as the GPU.

[0049] It should be noted that a GPU may include multiple streaming multiprocessors (SMs) as mentioned above, and a streaming multiprocessor SM may include multiple streaming processors (SPs). Each streaming multiprocessor SM has its own shared memory within the GPU. In this shared memory, all streaming processors (SPs) within each streaming multiprocessor SM can access the data stored therein. For example, this shared memory is a type of high-speed on-chip memory on the GPU.

[0050] For example, the first computing core 210 can be any SP in any SM in the GPU, and the shared memory device 230 is the shared memory corresponding to that SM.

[0051] Here, the first computing core 210 is configured to issue a first data request instruction to the data synchronization device 220. For example, the first data can be data stored at any address in HBM.

[0052] Accordingly, the data synchronization device 220 is configured to perform a read operation on the first data, a flag detection operation on the first data, and a synchronization operation according to the first data request instruction, wherein the synchronization operation includes sending the first data to the shared memory device 230. For example, the data synchronization device 220 implements the read, flag detection, and synchronization operations on the first data entirely through internal hardware circuitry.

[0053] The shared memory device 230 is configured to store the first data sent by the data synchronization device 220 for the first computing core 210 to read and use. For example, if the data synchronization device 220 performs a flag check and shows a flag match, it will send the first data to the shared memory device 230, and then the write control unit of the shared memory device 230 will write the first data to the corresponding address in the shared memory for the first computing core 210 to read and use.

[0054] For example, in one possible implementation, the data synchronization device includes an instruction parsing and scheduling unit, a memory access control unit, a flag bit detection and comparison unit, and a data writing unit. The read operation includes an instruction parsing and scheduling operation for a first data request instruction and a memory access control operation for the first data. The instruction parsing and scheduling unit is configured to perform the instruction parsing and scheduling operation for the first data request instruction, the memory access control unit is configured to perform the memory access control operation for the first data, the flag bit detection and comparison unit is configured to perform the flag bit detection operation for the first data, and the data writing unit is configured to perform the synchronization operation for the first data.

[0055] Figure 4 This is a schematic diagram of the structure of a data synchronization device provided in at least one embodiment of the present disclosure.

[0056] like Figure 4 As shown, the data synchronization device 220 includes an instruction parsing and scheduling unit 2201, a memory access control unit 2202, a flag detection and comparison unit 2203, and a data writing unit 2204. These functional units are implemented, for example, at least in part, through hardware, firmware, or software. The embodiments of this disclosure do not limit this.

[0057] For example, the above-mentioned read operation of the first data includes instruction parsing and scheduling operations for the first data request instruction and memory access control operations for the first data.

[0058] Here, the instruction parsing and scheduling unit 2201 is configured to perform parsing and scheduling operations on the first data request instruction in order to send the parsed relevant instruction parameters to subsequent related units.

[0059] The memory access control unit 2202 is configured to perform memory access control operations on the first data. For example, the memory access control unit 2202 can interact with a global shared device controller, thereby enabling the global shared device controller to access, for example... Figure 3 The first data stored in the global memory device 240 is read.

[0060] Flag detection and comparison unit 2203 is configured to perform flag detection operation on the first data, and data writing unit 2204 is configured to perform synchronization operation on the first data to send the first data to, for example, if the flag detection operation matches. Figure 3 In the shared memory device 230.

[0061] For example, in one possible implementation, the instruction parsing and scheduling unit 2201 is further configured to receive and parse a first data request instruction to obtain instruction configuration parameters, and send the instruction configuration parameters to the memory access control unit 2202 and the flag bit detection and comparison unit 2203. The instruction configuration parameters include the source address of the first data in the global memory device, the target shared memory address where the first data needs to be stored in the shared memory device, and the expected flag bit corresponding to the first data. The memory access control unit 2202 is further configured to generate and send a memory read request to the global memory device controller based on the source address, and in response to receiving the pending data located at the source address returned by the global memory device controller, forward the pending data to the flag bit detection and comparison unit 2203. The flag bit detection and comparison unit 2203 is further configured to extract and compare the flag bits in the pending data with the expected flag bits to obtain a flag bit comparison result, and send the flag bit comparison result to the data writing unit 2204. The data writing unit 2204 is further configured to send the data to be processed and the target shared memory address to the shared memory device in response to a matching flag comparison result, so that the write control unit of the shared memory device writes the data to be processed to the shared memory device based on the target shared memory address.

[0062] It should be noted that the "expected flag bit corresponding to the first data" here is a reference value used to compare with the flag bit in the "data to be processed". This reference value is usually determined by the communication context and protocol state. After the instruction parsing and scheduling unit 2201 sends the parsed instruction configuration parameters to the memory access control unit 2202 and the flag bit detection and comparison unit 2203, it means that the synchronization process of the data synchronization device 220 has been started.

[0063] For example, the global memory device controller is an HBM controller, which resides inside the GPU and is connected to the HBM memory stack via, for example, a silicon interposer. It manages multiple pending data read requests to support pipelined operations, thereby maximizing memory bandwidth utilization, for example. For instance, the HBM controller translates GPU memory requests (i.e., memory read requests generated by the memory access control unit based on the source address) into HBM read commands, reads the corresponding data (i.e., the aforementioned "pending data") from the source address in the HBM, and returns it to the memory access control unit 2202.

[0064] For example, after receiving the "data to be processed", the memory access control unit 2202 will pass it through to the flag bit detection and comparison unit 2203 for flag bit detection operation. The flag bit detection and comparison unit 2203 will accurately extract the flag bit from a specific position (e.g., the last bit) of the "data to be processed" according to a fixed data packet format and compare it with the expected flag bit. The obtained flag bit comparison result (e.g., match) will be immediately output to the data writing unit 2204.

[0065] For example, in one possible implementation, before extracting and comparing the flag bits of the data to be processed with the expected flag bits to obtain the flag bit comparison result, the aforementioned flag bit detection and comparison unit 2203 is further configured to temporarily store the data to be processed in a buffer unit of the GPU. For example, the buffer unit may be a register or a small FIFO inside the GPU, and the embodiments of this disclosure are not limited thereto.

[0066] For example, in one possible implementation, the data writing unit 2204 is further configured to extract the remaining valid data (excluding the flag bit) from the data to be processed in the buffer unit, and generate at least one data write request based on the target shared memory address to send the remaining valid data to the shared memory device. For example, when the flag bit comparison result is a match, the data writing unit 2204 extracts the remaining valid payload data (excluding the flag bit) from the "data to be processed" from, for example, an internal register or a small FIFO, and generates at least one data write request to send the remaining valid payload data and the target shared memory address together to the shared memory device so that the write control unit of the shared memory device itself writes the data to be processed to the shared memory device based on the target shared memory address.

[0067] It should be noted that the function of the data writing unit 2204 here is limited to sending the data to be processed (i.e., the first data) and the target shared memory address where the data to be processed needs to be stored in the shared memory device to the shared memory device. The actual writing operation is performed by the write control unit of the shared memory device itself.

[0068] It should also be noted that the data written to the shared memory device by the data writing unit 2204 is mainly the remaining valid data except for the flag bit. However, those skilled in the art can also write the entire packet of data including the flag bit into the shared memory device as needed for processing in specific situations. The embodiments of this disclosure do not limit this.

[0069] For example, in one possible implementation, at least one data write request includes multiple sub-write requests, each sub-write request corresponding to at least a portion of the remaining valid data. Sending the remaining valid data to the shared memory device through the data write unit includes: employing write merging technology to merge the multiple sub-write requests into a large-width write request aligned to the target shared memory address, and sending the remaining valid data as a whole to the shared memory device based on the large-width write request.

[0070] Write merging is a performance optimization technique designed to combine multiple small, consecutive write memory requests into a larger, more efficient write operation, thereby reducing bus usage, improving storage system bandwidth utilization, and reducing power consumption. For example, since the interface between the data write unit 2204 and the shared memory device requires high efficiency and low latency, write merging can be used to further improve efficiency by merging multiple small write requests into a large-width request aligned with the target shared memory address. Based on this large-width request, the remaining payload data is then sent to the shared memory device all at once to fully utilize the high bandwidth advantage of shared memory.

[0071] For example, after the data writing unit 2204 obtains a set of 128 bytes (128B) of data with a successfully matched flag bit in the first polling cycle, the 128 bytes of data will not be directly sent to the shared memory device at this time, but will be temporarily stored in, for example, a buffer unit. At this time, the data writing unit 2204 will continue to poll the subsequent data. After, for example, the 2nd to the 99th clock cycles (for example, the correct data is obtained in all 99 cycles, that is, a total of 12544B of data, and all of them are temporarily stored in the buffer unit in order), in the 100th clock cycle, the new data 128B obtained in the 100th clock cycle (for example, the flag bit is matched) will be combined with the 12672B of data temporarily stored in the buffer unit from the 1st to the 99th clock cycles to form a large bit width (12800B) write request. The data writing unit 2204 will send the 12800B of data to the shared memory device as a whole, with the target shared memory address aligned, based on the large bit width write request of the 12800B.

[0072] For example, in one possible implementation, the data synchronization device is further configured to perform a reread operation on the first data according to the first data request instruction. The data synchronization device also includes a retry unit configured to send a retry signal to the global memory device controller in response to a mismatch in the flag comparison result, so that the global memory device controller reads the data at the source address again.

[0073] For example, in addition to outputting the flag comparison result (e.g., match) to the data writing unit 2204, the aforementioned flag bit detection and comparison unit 2203 also outputs the flag comparison result (mismatch) to the retry unit. Figure 4 (Not shown in the image). For example, if the flag comparison results do not match, the retry unit sends a retry signal to the HBM controller, causing the HBM controller to read the data at the source address carried by the first data instruction again, either immediately or after a short delay.

[0074] For example, in one possible implementation, the retry unit includes a retry counter, and the retry unit is further configured to: after the global memory device controller reads the data at the source address again, update the retry counter and detect whether the updated number of retries exceeds a first threshold, or detect whether the retry time exceeds a second threshold; and in response to the updated number of retries exceeding the first threshold or the retry time exceeding the second threshold, terminate the retry operation and send an error message to the first computing core.

[0075] Here, while sending a retry signal, the retry unit updates its internal retry counter and checks whether the updated number of retries exceeds a first threshold (depending on the situation) or whether the retry time exceeds a second threshold (i.e., a predetermined duration). If the updated number of retries exceeds the first threshold or the retry time exceeds the second threshold, it means that the retries have reached the limit. At this time, the retry unit will stop the retry operation and send an error message to the GPU.

[0076] For example, in one possible implementation, the instruction configuration parameters further include an abort signal address, the data synchronization device further includes an abort unit, and the memory access control unit is further configured to send the abort signal address to the global memory device controller; the abort unit is configured to suspend the instruction parsing and scheduling unit from receiving the first data request instruction in response to the abort signal at the abort signal address returned by the global memory device controller being valid.

[0077] It should be noted that the data synchronization process mentioned in at least one of the above embodiments is only a loop process of data synchronization. In fact, during the data synchronization process, the "first data" may include multiple "data to be processed", and data synchronization may include multiple polling processes, depending on the size of the "first data".

[0078] For example, during multiple data synchronization polling operations, if any system anomaly occurs, the address bit in the HBM used to store the abort signal will be set to valid (e.g., logic 1). The abort unit, upon receiving a valid abort signal from the HBM controller, will suspend the instruction parsing and scheduling unit's reception of the first data request instruction, thereby aborting the data synchronization process. For instance, typically, after multiple requests to the target address (i.e., the source address), an access to the abort address is initiated to determine whether abort is necessary; for example, an abort address access is initiated after every 1000 target address accesses. It should also be noted that this abort signal address can be shared by multiple compute cores (SPs) of the GPU. If the abort signal is valid, all compute cores (SPs) sharing this abort signal will stop data synchronization operations at the target address.

[0079] For example, in one possible implementation, the length of the data to be processed is equal to 128 bytes, the flag bit is the lower 8 bytes of the data to be processed, and the remaining valid data is the higher 120 bytes of the data to be processed. For example, the above-mentioned "data to be processed" refers to data synchronization for the LL128 protocol.

[0080] The artificial intelligence chip provided in at least one embodiment of this disclosure implements, for example, the data synchronization and retry mechanism of the LL128 protocol in hardware, freeing up the core computing resources of the AI ​​chip. Furthermore, it achieves nanosecond-level flag detection and decision-making through dedicated circuitry, with a response speed far faster than software instructions. In addition, the AI ​​chip provided in at least one embodiment of this disclosure greatly enhances the scalability of computing systems in large-scale cluster environments, making it possible to build efficient and stable communication systems on clusters of thousands or even tens of thousands of AI chips.

[0081] For example, in one possible implementation, the AI ​​chip further includes a tensor acceleration engine, in which a data synchronization device is integrated. The tensor acceleration engine is configured to receive and forward a first data request instruction to an instruction parsing and scheduling unit. The data writing unit is further configured to reuse the data transfer engine in the tensor acceleration engine. Furthermore, the tensor acceleration engine is also configured to set a status register or send a lightweight interrupt signal to the first computing core to report that the data to be processed has been transferred to the shared memory device after the data writing unit sends the data to be processed and the target shared memory address to the shared memory device.

[0082] Figure 5 This is a schematic diagram of the structure of another artificial intelligence chip provided in at least one embodiment of the present disclosure.

[0083] like Figure 5 As shown, compared to Figure 3The AI ​​chip 200 also includes a tensor acceleration engine 250, and the data synchronization device 220 is integrated into the tensor acceleration engine 250. The data synchronization device 220 is tightly coupled with the core logic of the tensor acceleration engine 250. By adding one or more data synchronization devices 220 disclosed herein, the hardware synchronization and retry functions of the LL128 protocol can be efficiently realized.

[0084] It's important to note that while both the Tensor Acceleration Engine 250 and the First Computation Core 210 belong to the same streaming multiprocessor (SM), they differ in that the First Computation Core 210 (e.g., a streaming processor SP) handles general-purpose scalar computations, while the Tensor Acceleration Engine 250 is a dedicated hardware circuit unit, such as in a GPU, specifically designed to perform tensor / matrix operations. It is the core computing component of modern GPUs (especially AI computing GPUs), used to accelerate deep learning training and inference. For example, an SM can include 128 First Computation Cores (i.e., streaming processors SP) and 4 Tensor Acceleration Engines 250 (also called Tensor Cores).

[0085] For example, when the GPU's first computing core 210 needs to perform an LL128 communication, it can send a special "synchronization move" instruction to the tensor acceleration engine 250, which integrates synchronization functionality. This instruction can optionally include information about the source and target tensors, and it also includes the expected flag values. Upon receiving the instruction, the tensor acceleration engine 250's internal data synchronization device 220 first configures flag detection and retry logic. After receiving data returned by the global memory device controller, the tensor acceleration engine 250 will only efficiently move the 120 bytes of payload data to the shared memory device 230 if the flags match successfully. The specific interaction process is as follows.

[0086] (1) Initiating a request: The first computing core 210 of the GPU executes a new synchronous transfer instruction with the LL128 flag bit and submits it to the tensor acceleration engine 250.

[0087] (2) Tensor acceleration engine takeover: After receiving the above-mentioned synchronization transmission instruction, the tensor acceleration engine 250 forwards the synchronization transmission instruction to the instruction parsing and scheduling unit in the data synchronization device 220 to start data synchronization, and autonomously polls the flag bits in HBM.

[0088] (3) Data transfer: Once the flag bit in the received data matches, the data transfer engine of the tensor acceleration engine 250 is activated, and the data writing unit of the data synchronization device 220 will reuse the data transfer engine to automatically transfer the data and the corresponding target shared memory address to the shared memory device 230. The whole process does not require the intervention of the GPU computing core.

[0089] (4) Notification of completion: After the data transfer is completed, the Tensor Acceleration Engine 250 notifies that the data has been transferred to the shared memory device by setting, for example, a status register or sending a lightweight interrupt to the first computing core 210 of the GPU.

[0090] (5) GPU core continues to compute: After submitting the data synchronization task, the first computing core 210 can return to perform other computations (such as convolution operations of neural networks) until it receives a completion notification, thereby achieving efficient overlap of computation and communication.

[0091] For example, in another possible implementation, the artificial intelligence chip includes a secondary cache, in which a data synchronization device is integrated. The secondary cache is configured to receive and route a first data request instruction to an instruction parsing and scheduling unit, and to route pending data at a source address returned by a global memory device controller to a memory access control unit. The secondary cache is also configured to report to a first computing core that the pending data is ready in response to a flag comparison result matching.

[0092] Figure 6 This is a schematic diagram of the structure of another artificial intelligence chip provided in at least one embodiment of the present disclosure.

[0093] like Figure 6 As shown, compared to Figure 3 The AI ​​chip 200 also includes a secondary cache 260, and the data synchronization device 220 is integrated into the secondary cache 260. The core logic of the data synchronization device 220 and the secondary cache 260 are tightly coupled. By adding one or more data synchronization devices 220 disclosed herein, the hardware synchronization and retry functions of the LL128 protocol can be efficiently realized.

[0094] For example, when the GPU's first compute core 210 initiates an LL128 data read request, the request first reaches the L2 cache 260. The L2 cache 260 is configured to route data (or at least its flag portion) returned by the global memory device controller to a newly added data synchronization device 220 for such special requests. The data synchronization device 220 checks the flags in parallel within the L2 cache 260. If the flags do not match, it can immediately initiate a retry at the L2 cache level, avoiding returning invalid data to the first compute core 210, thus saving bandwidth. Only when the flags match will it return the data to the first compute core 210. The interaction flow is as follows.

[0095] (1) Initiating a request: The first computing core 210 executes a new synchronous transmission instruction with the LL128 flag bit and submits it to the second-level cache 260.

[0096] (2) Secondary cache takeover: After receiving the above-mentioned synchronization transmission instruction, the secondary cache 260 routes the instruction to the instruction parsing and scheduling unit in one or more data synchronization devices 220 inside it to start data synchronization, and autonomously polls the flag bits in HBM.

[0097] (3) Flag check: The data returned by the HBM controller will be routed to the memory access control unit in the data synchronization device 220 in the secondary cache, and the flags of the data will be checked by the subsequent flag detection and comparison unit. If they do not match, the polling will continue.

[0098] (4) Data writing: If the flag bit matches, the second-level cache 260 notifies the first computing core 210 that the data has arrived, and only the valid part of the data is moved to the shared memory device 230 (specifically the write control unit of the shared memory device 230).

[0099] (5) GPU computing core processes the data returned by the L2 cache: After receiving the data returned by the L2 cache 260, the first computing core 210 writes the data into the shared memory device 230 through the write control unit of the shared memory device 230.

[0100] The artificial intelligence chip provided in at least one embodiment of this disclosure integrates a data synchronization device into a tensor acceleration engine or a secondary cache. When LL128 communication is required, the GPU computing core only needs to send a request to the data synchronization device once, specifying the data source (the source address of HBM), the target shared memory address, and the expected flag value, and then immediately return to execute other computing tasks. The data synchronization device takes over all subsequent data synchronization and retry work, thereby freeing the GPU computing core from inefficient communication waiting. At the same time, due to the use of a hardware solution, communication latency is also significantly reduced.

[0101] At least one embodiment of this disclosure also provides a computing system including a plurality of artificial intelligence chips according to the above-described at least one embodiment, wherein the plurality of artificial intelligence chips are interconnected with each other.

[0102] Figure 7 This is a schematic diagram of the structure of a computing system provided in at least one embodiment of the present disclosure.

[0103] like Figure 7 As shown, the computing system 300 includes artificial intelligence chips 1, ..., and N, which can be interconnected, for example, via a PCIe bus. Furthermore, the computing system provided by at least one embodiment of this disclosure enables the construction of an efficient and stable communication system on a cluster of thousands or even tens of thousands of artificial intelligence chips.

[0104] It should be noted that, for clarity and brevity, the embodiments of this disclosure do not provide all the constituent units of the data synchronization device 220 or computing system 300 described above. To achieve the necessary functions of the data synchronization device 220 or computing system 300, those skilled in the art can provide or set other constituent units (not shown) according to specific needs, and the embodiments of this disclosure do not impose any limitations on this.

[0105] At least one embodiment of this disclosure also provides an electronic device that includes the computing system described in the above embodiments.

[0106] Figure 8 This is a schematic diagram of an electronic device provided for at least one embodiment of the present disclosure. Figure 8 The illustrated electronic device 400 is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0107] like Figure 8 As shown, in some examples, electronic device 400 includes a processing unit (e.g., central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the computer system. The processing unit 401, ROM 402, and RAM 403 are connected via bus 404. Input / output (I / O) interface 405 is also connected to bus 404.

[0108] For example, the following components can be connected to I / O interface 405: input devices 406 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409, such as network interface cards like LAN cards and modems, etc. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data and perform communication processing via networks such as the Internet. Drive 410 is also connected to I / O interface 405 as needed. Removable media 411, such as disks, optical disks, magneto-optical disks, semiconductor memories, etc., are installed on drive 410 as needed so that computer programs read from them can be installed into storage device 408 as needed.

[0109] Although Figure 8 An electronic device 400 including various devices is shown; however, it should be understood that implementation or inclusion of all shown devices is not required. More or fewer devices may be implemented or included alternatively.

[0110] For example, the electronic device 400 may further include a peripheral interface (not shown in the figure). This peripheral interface can be various types of interfaces, such as a USB interface, a Lightning interface, etc. The communication device 409 can communicate wirelessly with a network and other devices, such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). Wireless communication can use any of a variety of communication standards, protocols, and technologies, including but not limited to Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wi-Fi (e.g., based on IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n standards), Voice over Internet Protocol (VoIP), Wi-MAX, protocols for email, instant messaging, and / or Short Message Service (SMS), or any other suitable communication protocol.

[0111] For example, the electronic device 400 may include any device such as a mobile phone, tablet computer, laptop computer, e-book, game console, television, digital photo frame, navigator, server, etc., or any combination of hardware. The embodiments disclosed herein do not limit this.

[0112] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0113] The following points should be noted regarding this disclosure:

[0114] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0115] (2) For clarity, the thickness of layers or regions in the drawings used to describe embodiments of the present disclosure is enlarged or reduced, i.e., these drawings are not drawn to actual scale.

[0116] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0117] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. An artificial intelligence chip, characterized in that, It includes a first computing core, a shared memory device corresponding to the first computing core, and a data synchronization device, wherein, The first computing core is configured to issue a first data request command; The data synchronization device is configured to perform a reading operation on the first data, a flag bit detection operation on the first data, and a synchronization operation according to the first data request instruction, wherein the first data is stored in a global memory device coupled to the artificial intelligence chip, and the synchronization operation includes sending the first data to the shared memory device. The shared memory device is configured to store the first data sent by the data synchronization device for the first computing core to read and use. The data synchronization device includes an instruction parsing and scheduling unit, a memory access control unit, a flag detection and comparison unit, and a data writing unit. The read operation includes instruction parsing and scheduling of the first data request instruction and memory access control of the first data. The instruction parsing and scheduling unit is configured to perform the instruction parsing and scheduling operation on the first data request instruction. The memory access control unit is configured to perform the memory access control operation on the first data. The flag detection and comparison unit is configured to perform the flag detection operation on the first data. The data writing unit is configured to perform the synchronization operation on the first data.

2. The artificial intelligence chip according to claim 1, characterized in that, The instruction parsing and scheduling unit is further configured to receive and parse the first data request instruction to obtain instruction configuration parameters, and send the instruction configuration parameters to the memory access control unit and the flag bit detection and comparison unit, wherein the instruction configuration parameters include the source address of the first data in the global memory device, the target shared memory address where the first data needs to be stored in the shared memory device, and the expected flag bit corresponding to the first data. The memory access control unit is further configured to generate and send a memory read request to the global memory device controller based on the source address, and to forward the data to be processed to the flag detection and comparison unit in response to receiving the data to be processed located at the source address returned by the global memory device controller. The flag bit detection and comparison unit is further configured to extract and compare the flag bit in the data to be processed with the expected flag bit to obtain a flag bit comparison result, and send the flag bit comparison result to the data writing unit; The data writing unit is further configured to send the data to be processed and the target shared memory address to the shared memory device in response to a matching flag comparison result, so that the write control unit of the shared memory device writes the data to be processed to the shared memory device based on the target shared memory address.

3. The artificial intelligence chip according to claim 2, characterized in that, The instruction configuration parameters also include an abort signal address, and the data synchronization device further includes an abort unit. The memory access control unit is also configured to send the abort signal address to the global memory device controller; The abort unit is configured to suspend the instruction parsing and scheduling unit's reception of the first data request instruction in response to receiving a valid abort signal at the abort signal address returned by the global memory device controller.

4. The artificial intelligence chip according to claim 2, characterized in that, The flag bit detection and comparison unit is further configured to temporarily store the data to be processed in the buffer unit of the artificial intelligence chip before extracting and comparing the flag bit of the data to be processed with the expected flag bit to obtain the flag bit comparison result.

5. The artificial intelligence chip according to claim 4, characterized in that, The data writing unit is further configured to extract the remaining valid data, excluding the flag bit, from the data to be processed in the buffer unit, and generate at least one data write request based on the target shared memory address to send the remaining valid data to the shared memory device.

6. The artificial intelligence chip according to claim 5, characterized in that, The at least one data write request includes multiple sub-write requests, each sub-write request corresponding to at least a portion of the remaining valid data. Sending the remaining valid data to the shared memory device includes: A write merging technique is used to merge the multiple sub-write requests into a large-width write request aligned with the target shared memory address, and the remaining valid data is sent to the shared memory device as a whole based on the large-width write request.

7. The artificial intelligence chip according to any one of claims 2-6, characterized in that, The data synchronization device is also configured to perform a re-reading operation on the first data according to the first data request instruction. The data synchronization device further includes: The retry unit is configured to send a retry signal to the global memory device controller in response to a mismatch in the flag comparison result, so that the global memory device controller reads the data at the source address again.

8. The artificial intelligence chip according to claim 7, characterized in that, The retry unit includes a retry counter. The retry unit is also configured to: After the global memory device controller reads the data at the source address again, it updates the retry counter and checks whether the updated number of retries exceeds a first threshold, or whether the retry time exceeds a second threshold. as well as If the updated number of retries exceeds the first threshold or the retry time exceeds the second threshold, the retry operation is terminated and an error message is sent to the first computing core.

9. The artificial intelligence chip according to claim 5 or 6, characterized in that, The length of the data to be processed is 128 bytes, the flag bit is the lower 8 bytes of the data to be processed, and the remaining valid data is the upper 120 bytes of the data to be processed.

10. The artificial intelligence chip according to any one of claims 2-6, characterized in that, The AI ​​chip also includes a tensor acceleration engine, and the data synchronization device is integrated into the tensor acceleration engine. The tensor acceleration engine is configured to receive and forward the first data request instruction to the instruction parsing and scheduling unit. The data writing unit is also configured to reuse the data transfer engine in the tensor acceleration engine; The tensor acceleration engine is also configured to, after the data writing unit sends the data to be processed and the target shared memory address to the shared memory device, set a status register or send a lightweight interrupt signal to the first computing core to report that the data to be processed has been transferred to the shared memory device.

11. The artificial intelligence chip according to any one of claims 2-6, characterized in that, The artificial intelligence chip includes a secondary cache, and the data synchronization device is integrated into the secondary cache. The secondary cache is configured to receive and route the first data request instruction to the instruction parsing and scheduling unit, and to route the data to be processed located at the source address returned by the global memory device controller to the memory access control unit. The secondary cache is also configured to report to the first computing core that the data to be processed is ready in response to a matching flag comparison result.

12. A computing system, characterized in that, It includes a plurality of artificial intelligence chips according to any one of claims 1-11, wherein the plurality of artificial intelligence chips are interconnected with each other.

Images