Data transmission method, broadcast engine, electronic device, and storage medium

Abstract

The present disclosure provides a data transmission method, a broadcast engine, an electronic device and a storage medium. First, the intra-cluster broadcast engine acquires broadcast data and a broadcast mask in response to a broadcast request reported by the computing unit. Then, the intra-cluster broadcast engine performs intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask. Then, the intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engine of other block processing clusters through the inter-cluster interconnection network, so as to trigger other intra-cluster broadcast engines to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask. The embodiment of the present disclosure can break through the data broadcast limitation of a single block processing cluster, realize efficient cross-block processing cluster data broadcast, and improve the data transmission efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular to a data transmission method, a broadcast engine, an electronic device and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence, high-performance computing and other fields, the demand for efficient data synchronization is increasingly urgent in distributed training and inference processes. Current graphics processors and AI accelerator architectures usually take block processing clusters as the basic scheduling unit, integrate multiple computing units in each block processing cluster, and build communication links through intra-cluster interconnection networks, so that data can be quickly distributed to all computing units in the same block processing cluster, effectively improving local communication efficiency. However, with the expansion of artificial intelligence model size (such as large language model parallel training), a large amount of data needs to be globally synchronized to all computing units in the device, which may be distributed in different block processing clusters. Current technology can only implement data broadcast within a single block processing cluster, and it is difficult to directly implement global broadcast operations across clusters, which limits the parallel expansion capability of artificial intelligence models. SUMMARY

[0003] Embodiments of the present disclosure provide a data transmission method, a broadcast engine, an electronic device and a storage medium, which can break through the data broadcast limitation of a single block processing cluster, implement efficient cross-block processing cluster data broadcast, and improve data transmission efficiency.

[0004] In a first aspect, embodiments of the present disclosure provide a data transmission method applied to a data processor, the data processor including a plurality of block processing clusters, and an inter-cluster interconnection network for communication transmission between the plurality of block processing clusters, each block processing cluster including an intra-cluster broadcast engine, a plurality of computing units, and an intra-cluster interconnection network for communication transmission between the plurality of computing units, each computing unit being configured with a meta-thread; the method comprising:

[0005] The intra-cluster broadcast engine acquires broadcast data and a broadcast mask in response to a broadcast request initiated by the meta-thread in the computing unit;

[0006] The intra-cluster broadcast engine performs intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask;

[0007] The intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engine of other block processing clusters through the inter-cluster interconnection network to trigger other intra-cluster broadcast engines to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0008] In a second aspect, the present disclosure provides a broadcast engine applied to a data processor, the data processor comprising a plurality of block processing clusters, and an inter-cluster interconnect network for communication transmission between the plurality of block processing clusters, each of the block processing clusters comprising the broadcast engine, a plurality of compute units, and an intra-cluster interconnect network for communication transmission between the plurality of compute units, each of the compute units being configured with a meta-thread, the broadcast engine further comprising:

[0009] a data obtaining unit, configured to obtain broadcast data and a broadcast mask in response to a broadcast request initiated by the meta-thread in the compute unit;

[0010] an intra-cluster broadcast unit, configured to perform an intra-cluster broadcast process in the current block processing cluster based on the broadcast data and the broadcast mask;

[0011] an inter-cluster broadcast unit, configured to forward the broadcast data and the broadcast mask to an intra-cluster broadcast engine of another block processing cluster through the inter-cluster interconnect network, so as to trigger the intra-cluster broadcast engine of another block processing cluster to perform an intra-cluster broadcast process based on the broadcast data and the broadcast mask.

[0012] Further, the inter-cluster broadcast unit is further configured to:

[0013] determine a target compute unit in the current block processing cluster based on the broadcast mask;

[0014] broadcast the broadcast data to the target compute unit through the intra-cluster interconnect network.

[0015] Further, the inter-cluster broadcast unit is further configured to:

[0016] determine a target block processing cluster containing a target compute unit based on the broadcast mask;

[0017] forward the broadcast data and the broadcast mask to an intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnect network.

[0018] Further, the inter-cluster broadcast unit is further configured to:

[0019] determine a target block processing cluster containing a target compute unit based on each of the independent mask segments;

[0020] take the independent mask segment corresponding to the target block processing cluster as a target mask;

[0021] forward the broadcast data and the target mask to an intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnect network.

[0022] Further, the intra-cluster broadcast unit is further configured to perform, in response to the broadcast data and the broadcast mask broadcasted by other intra-cluster broadcast engines, intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask.

[0023] Further, the inter-cluster broadcast unit is further configured to:

[0024] upload the broadcast data and the broadcast mask to the inter-cluster broadcast engine, so as to cause the inter-cluster broadcast engine to distribute the broadcast data and the broadcast mask to intra-cluster broadcast engines of all the block processing clusters through the inter-cluster interconnect network.

[0025] Further, the inter-cluster broadcast unit is further configured to, in response to the broadcast range indicated by the broadcast request being inter-die broadcast, broadcast the broadcast data and the broadcast mask to intra-cluster broadcast engines in other computing dies through the inter-die interconnect network, so as to trigger the intra-cluster broadcast engines in other computing dies to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0026] Further, the intra-cluster broadcast unit is further configured to:

[0027] perform, in response to receiving the broadcast data and the broadcast mask broadcasted by other intra-cluster broadcast engines, intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask,

[0028] the inter-cluster broadcast unit is further configured to:

[0029] broadcast the broadcast data and the broadcast mask to other broadcast engines in the same computing die, so as to trigger the broadcast engines in the same computing die to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0030] Further, the intra-cluster broadcast unit is further configured to upload the broadcast data and the broadcast mask to other broadcast engines;

[0031] the inter-cluster broadcast unit is further configured to broadcast the broadcast data and the broadcast mask to broadcast engines of other computing dies, so as to cause each of the broadcast engines to perform inter-cluster broadcast processing;

[0032] wherein the inter-cluster broadcast processing comprises that the broadcast engine distributes the broadcast data and the broadcast mask to broadcast engines in the same computing die, so as to cause the broadcast engines to perform intra-cluster broadcast processing.

[0033] Further, the inter-cluster broadcast unit is further configured to:

[0034] determine, based on the broadcast mask, a target block processing cluster containing a target computing unit in the current computing die;

[0035] The broadcast data is delivered to the broadcast engine of the target block processing cluster.

[0036] Further, the intra-cluster broadcast unit is further configured to broadcast the broadcast data and the broadcast mask to all the broadcast engines of the same computing die in response to the broadcast data and the broadcast mask broadcast by other broadcast engines.

[0037] The inter-cluster broadcast unit is further configured to determine a plurality of target computing dies containing the target computing unit based on the broadcast mask, and determine an adaptive computing die matching the computing die currently belonging to from the plurality of target computing dies based on a preset broadcast rule, and broadcast the broadcast mask and the broadcast data to the broadcast engine of the adaptive computing die.

[0038] Further, the inter-cluster broadcast unit is further configured to:

[0039] upload the broadcast data and the broadcast mask to the broadcast engine through the inter-die interconnection network;

[0040] Or, deliver the broadcast data and the broadcast mask to the broadcast engine in other computing dies through the inter-die interconnection network.

[0041] Further, the data acquisition unit is further configured to:

[0042] In response to a broadcast request initiated by the meta-thread in the computing unit, parse a broadcast mask and a data storage address from the broadcast request;

[0043] read the broadcast data from the local memory of the computing unit based on the data storage address.

[0044] Further, the intra-cluster broadcast unit is further configured to:

[0045] The intra-cluster broadcast engine determines the target computing unit in the block processing cluster currently based on the broadcast mask;

[0046] broadcast the broadcast data to the target computing unit.

[0047] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which comprises a memory, a processor, a program stored in the memory and capable of running on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program is run by the processor to realize the data transmission method in the first aspect.

[0048] In a fourth aspect, the disclosure also provides a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the data transmission method according to the first aspect.

[0049] The data transmission method, broadcast engine, electronic device and storage medium provided by the disclosure first receive a broadcast request reported by a meta thread in a computing unit, obtain broadcast data and a broadcast mask, and parse the broadcast mask to locate a target computing unit range. The target computing unit range is expanded to all block processing clusters. The target computing unit in the current block processing cluster is determined based on the broadcast mask. The broadcast data is broadcast to the target computing unit through an intra-cluster interconnection network. Local data distribution is completed by multiplexing an intra-cluster broadcast mechanism in the current block processing cluster. On this basis, the broadcast data and the broadcast mask are concurrently forwarded to the intra-cluster broadcast engines of all block processing clusters through an inter-cluster interconnection network. The intra-cluster broadcast engines of other clusters are triggered to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask. The broadcast data is broadcast to the target computing units in each block processing cluster. This can reduce the delay of cross-cluster communication and multiplex the broadcast mask to avoid the overhead of repeated data transmission. Therefore, the data transmission method provided by the disclosure can realize automatic forwarding and intra-cluster distribution of broadcast data between clusters through the intra-cluster broadcast engine processing of a broadcast request. The data can be broadcast to any computing unit in a die synchronously. This achieves efficient cross-block processing cluster data broadcast and reduces communication delay and overhead.

[0050] Other features and advantages of the disclosure will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the disclosure. The objects and other advantages of the disclosure can be achieved and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings are intended to provide a further understanding of the technical solutions of the disclosure, and constitute a part of the specification, and are used together with the embodiments of the disclosure to explain the technical solutions of the disclosure, and do not constitute a limitation on the technical solutions of the disclosure.

[0052] Figure 1 is a hardware hierarchical architecture diagram of a data processor provided by an embodiment of the disclosure;

[0053] Figure 2 is a hardware hierarchical architecture diagram of a data processor provided by another embodiment of the disclosure;

[0054] Figure 3 is a hardware hierarchical architecture diagram of a data processor provided by another embodiment of the disclosure;

[0055] Figure 4is a hardware hierarchical architecture diagram of a data processor provided by another embodiment of the disclosure;

[0056] Figure 5 is a hardware hierarchical architecture diagram of a data processor provided by another embodiment of the disclosure;

[0057] Figure 6 is an optional flowchart of a data transmission method provided by an embodiment of the disclosure;

[0058] Figure 7 is a schematic diagram of data transmission in a copy mode provided by an embodiment of the disclosure;

[0059] Figure 8 is a schematic diagram of data transmission across block processing clusters in an arbitrary mode provided by an embodiment of the disclosure;

[0060] Figure 9 is a flowchart of data transmission when broadcasting between clusters provided by an embodiment of the disclosure;

[0061] Figure 10 is an effect diagram of broadcasting across block processing clusters provided by an embodiment of the disclosure;

[0062] Figure 11 is an effect diagram of broadcasting between clusters provided by an embodiment of the disclosure;

[0063] Figure 12 is a flowchart of data transmission when broadcasting between clusters provided by another embodiment of the disclosure;

[0064] Figure 13 is an effect diagram of broadcasting between clusters provided by another embodiment of the disclosure;

[0065] Figure 14 is a flowchart of data transmission when broadcasting between dies provided by an embodiment of the disclosure;

[0066] Figure 15 is an effect diagram of broadcasting across computing dies provided by an embodiment of the disclosure;

[0067] Figure 16 is a schematic diagram of data transmission across computing dies in an arbitrary mode provided by an embodiment of the disclosure;

[0068] Figure 17 is an effect diagram of broadcasting between dies provided by an embodiment of the disclosure;

[0069] Figure 18 is a flowchart of data transmission when broadcasting between dies provided by another embodiment of the disclosure;

[0070] Figure 19 is an effect diagram of broadcasting between dies provided by another embodiment of the disclosure;

[0071] Figure 20 is a flow diagram of data transmission in inter-die broadcasting provided by another embodiment of the present disclosure;

[0072] Figure 21 is a schematic diagram of a broadcasting flow provided by another embodiment of the present disclosure;

[0073] Figure 22 is an optional structural schematic diagram of a broadcasting engine provided by an embodiment of the present disclosure;

[0074] Figure 23 is an optional structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present disclosure clearer, the present disclosure is further described in detail below in combination with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and do not limit the present disclosure.

[0076] In an embodiment of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the function of the module or unit.

[0077] In order to facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, some key terms used in the embodiments of the present disclosure are explained first:

[0078] Graphics Processing Unit (GPU): A microprocessor designed specifically for handling graphics and image computations; GPUs were originally designed to accelerate the rendering of 2D and 3D graphics, improving the performance of games and professional graphics software. With thousands of cores, GPUs can handle large amounts of data simultaneously, making them ideal for parallel computing tasks. As technology has evolved, GPUs are no longer limited to graphics processing; they are now used for a variety of general-purpose computing tasks. GPUs play an important role in deep learning, machine learning, and artificial intelligence, as they can quickly process large amounts of data, accelerating the training and inference of neural networks. In scientific computing and data analysis, GPUs are used to accelerate complex numerical simulations and data analysis tasks. In cloud computing and data centers, GPUs are used to provide high-performance computing resources, supporting various compute-intensive applications. In professional applications such as video editing, 3D modeling, scientific visualization, and more, GPUs can provide real-time high-performance rendering.

[0079] Compute Unit (CU): A processing module in a graphics processor that can implement multi-threaded concurrent execution, processing different data segments such as matrix multiplication block operations. Compute units can not only perform multiplication and addition, logical judgment, and other operations independently, but also can cooperate with other compute units through intra-cluster interconnection networks, such as participating in local data sharing and task slicing; in a broadcast scenario, on the one hand, it can receive intra-cluster meta-thread instructions and report broadcast requests, initiating data synchronization requirements; on the other hand, it can also act as a target node for data distribution, receiving broadcast data.

[0080] Meta Thread: A hardware control flow integrated within a compute unit that can perceive broadcast requirements and initiate broadcast requirements, sending broadcast requests to the broadcast engine through embedded compute units. Meta threads are deployed inside each compute unit and share local memory and computing resources with compute units. Unlike ordinary execution threads responsible for data computation, meta threads can be control flow threads, focusing on the perception, decision-making, and coordination of broadcast requirements, serving as broadcast management units.

[0081] Block Process Cluster: A cluster of multiple compute units in a single graphics processing unit, which can also be referred to as a Graphics Processing Cluster (GPC) in some graphics processing unit architectures. In a graphics processing unit, a block process cluster can assign computing tasks to internal compute units and enable communication between the internal compute units through an intra-cluster interconnection network. In a broadcast scenario, the broadcast engine of a block process cluster can analyze the broadcast request of a meta-thread, perform local data distribution, and coordinate global data synchronization through an inter-cluster interconnection network.

[0082] Compute Die: A core integrated chip in a graphics processing unit and an AI accelerator, which is a basic computing unit in a chiplet architecture and integrates multiple block process clusters, a global inter-cluster interconnection network, and a cluster broadcast engine. A compute die usually contains multiple block process clusters, each of which integrates multiple compute units. The compute die can support data synchronization of a large number of compute units through an inter-die interconnection network, an inter-cluster interconnection network, and an intra-cluster interconnection network.

[0083] Interconnect Network: A signal transmission structure for connecting different functional modules on a compute die, which can be divided into an intra-cluster interconnection network, an inter-cluster interconnection network, and an inter-die interconnection network. The intra-cluster interconnection network is dedicated to the high-speed interconnection of compute units, broadcast engines, and shared storage within the same block process cluster and can be implemented by a local crossbar switch. The inter-cluster interconnection network connects different block process clusters and undertakes the task of cross-cluster data forwarding, which can be implemented by a hierarchical switch architecture. The inter-die interconnection network is an architecture integrated in the same data processor device for communication transmission between multiple compute dies, which can establish a cross-die physical path and support the transmission of broadcast data and control signals.

[0084] Broadcast Engine: a hardware unit dedicated to data distribution in a processor architecture, the broadcast engine has different data distribution ranges based on the arranged location and connection relationship, for example, the broadcast engine integrated in each block processing cluster can be called intra-cluster broadcast engine, which can refer to block processing cluster (BPC) level broadcast engine, i.e. BPC-BCE, which can analyze the broadcast request reported by the computing unit and drive the intra-cluster interconnection network to distribute data to the corresponding computing unit, realizing intra-cluster data synchronization; for example, the broadcast engine independent of each block processing cluster and connected to each block processing cluster through an inter-cluster interconnection network can be called inter-cluster broadcast engine, which can also be called die-level broadcast engine, i.e. Die-BCE, the broadcast range of the inter-cluster broadcast engine covers all computing units of all block processing clusters in the computing die, which can realize global broadcast on the computing die; and a broadcast engine independent of each computing die and connected to each computing die through an inter-die interconnection network can be called inter-die broadcast engine, which can also be called device-level broadcast engine, i.e. Device-BCE.

[0085] With the rapid development of artificial intelligence, high-performance computing and other fields, in the process of distributed training and inference, the block processing cluster architecture of the data processor has a communication bottleneck in cross-cluster global broadcast. Due to the expansion of artificial intelligence model size (such as large language model parallel training), a large amount of data needs to be globally synchronized to all computing units in the die (whether the target synchronized computing units belong to the same block processing cluster), the current technology only supports intra-cluster broadcast mechanism, which cannot directly realize data synchronization across multiple block processing clusters, resulting in low communication efficiency in the global synchronization scenario, the parallel expansion capability of the system is restricted, and the data throughput performance is also difficult to meet the demand of high-performance computing.

[0086] For example, when training a large-scale deep learning model, the data processor includes multiple block processing clusters, each block processing cluster integrates an intra-cluster interconnection network and multiple computing units. When a computing unit completes local computation and needs to broadcast data to all computing units, due to the lack of cross-cluster broadcast mechanism, the broadcast request is decomposed into multiple point-to-point communication operations, and in this process, the data must be transmitted through the intra-cluster interconnection network one by one, causing increased communication delay and network congestion, prolonging the data synchronization process and reducing the utilization of computing resources.

[0087] Based on this, the embodiment of the disclosure proposes a data transmission method, a broadcast engine, an electronic device and a storage medium. First, a broadcast request reported by a computing unit is received, broadcast data and a broadcast mask are obtained, and the broadcast mask is parsed to locate a target computing unit range. The target computing unit range is expanded to all block processing clusters. The target computing unit in the current block processing cluster is determined based on the broadcast mask. The broadcast data is broadcast to the target computing unit through an intra-cluster interconnection network. Local data distribution is completed by multiplexing an intra-cluster broadcast mechanism in the current block processing cluster. On this basis, the broadcast data and the broadcast mask are concurrently forwarded to the intra-cluster broadcast engines of all block processing clusters through an inter-cluster interconnection network. The intra-cluster broadcast engines of other clusters are triggered to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask. The broadcast data is broadcast to the target computing units in each block processing cluster. This can reduce the delay of cross-cluster communication and multiplex the broadcast mask to avoid the overhead of repeated data transmission. Therefore, the data transmission method proposed in the embodiment of the disclosure can realize automatic forwarding and intra-cluster distribution of broadcast data between clusters through the intra-cluster broadcast engine processing of the broadcast request. The data can be broadcast to any computing unit in the die synchronously. This can achieve efficient cross-block processing cluster data broadcast effect and reduce communication delay and overhead.

[0088] In a first aspect, with reference to Figure 1 , Figure 1Figure 1 is a hardware level architecture diagram of a data processor provided by an embodiment of the present disclosure. As shown in the figure, the hardware architecture of the data processor provided by the embodiment of the present disclosure is built on a computing die, and is integrated with a plurality of block processing clusters. The block processing cluster serves as a main scheduling unit. Each block processing cluster is integrated with a plurality of computing units. The computing unit is configured with a meta-thread. The computing unit can execute instructions and process data. The meta-thread can perceive the data broadcast requirement of the computing unit in the process of executing a task, and automatically generate a broadcast request. The broadcast request includes broadcast data, a broadcast mask, and a broadcast range. Each computing unit is also equipped with a corresponding local memory, which can realize data caching, and realize data exchange between computing units in the same block processing cluster through an intra-cluster interconnection network. Specifically, the computing unit can be integrated with a static random access memory (SRAM). The computing unit can use the SRAM to cache high-frequency computing data, reduce data delay, and pre-load the broadcast data to be distributed to the SRAM during broadcasting, so as to facilitate the intra-cluster broadcast engine to directly read. In addition, the local memory can also be a dynamic random access memory (DRAM) arranged next to the computing unit and connected to the computing unit through a dedicated interface (rather than integrated internally). The DRAM can share data across computing units, and can also act as a relay station for broadcast data. The intra-cluster broadcast engine can read the broadcast data from the DRAM for distribution. The meta-thread driving the computing unit task execution in the computing unit can initiate a broadcast request. The computing unit reports the broadcast request to the intra-cluster broadcast engine in the block processing cluster. The intra-cluster broadcast engine analyzes the broadcast request, and uses the intra-cluster interconnection network to distribute the requested data broadcast to the target computing unit in the block processing cluster, thereby realizing local data synchronization.

[0089] The inter-cluster interconnection network connects the intra-cluster broadcast engines of all block processing clusters to form a topology directly connected by inter-cluster broadcast engines. When the intra-cluster broadcast engine receives a broadcast request reported by a thread of a computing unit indicating a global broadcast (i.e., a cross-cluster broadcast), the intra-cluster broadcast engine can forward the broadcast request to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnection network to trigger the intra-cluster broadcast engines of each block processing cluster to perform local broadcasting. At this time, the request reporting path of the cross-cluster global broadcast is from the computing unit where the initiating thread is located to the intra-cluster broadcast engine of the local block processing cluster, and then forwarded to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnection network. When the broadcast request received by the intra-cluster broadcast engine contains a local broadcast range and a cross-cluster broadcast range, the intra-cluster broadcast engine can first process the local intra-cluster broadcast and then forward it to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnection network. It can be understood that each block processing cluster can be regarded as the current boundary of resource scheduling, which internally includes computing resources, an intra-cluster interconnection network, and a dedicated intra-cluster broadcast engine, and externally cooperates with other block processing clusters through the inter-cluster interconnection network.

[0090] It is worth noting that the cross-cluster broadcast can also be implemented through the inter-cluster interconnection network in combination with the inter-cluster broadcast engine. For details, refer to Figure 2 , Figure 2 is a hardware-level architecture diagram of a data processor provided by another embodiment of the present disclosure. It can be understood that the data processor is also provided with an inter-cluster broadcast engine. The inter-cluster broadcast engine can be directly integrated into the topology of the inter-cluster interconnection network, can be implemented by embedding the broadcast engine into a network switch chip or as a component of a network interface controller, and can be directly integrated into a crossbar switch as a hardware expansion module of a routing node of the inter-cluster interconnection network. Thus, the additional jump link between the broadcast engine and the interconnection network interface can be eliminated, the data transmission path can be shortened, the routing table of the inter-cluster interconnection network can be directly accessed, and zero-copy broadcasting can be implemented, such as single-cycle copying of data to multiple ports. Alternatively, the inter-cluster broadcast engine can be independently deployed, i.e., the physical location of the inter-cluster broadcast engine is arranged adjacent to the interface point of the inter-cluster interconnection network. The inter-cluster broadcast engine can be arranged in the circuit board area adjacent to the network switch or connected through a short-distance interconnection line. Thus, the inter-cluster broadcast engine can interact with the inter-cluster interconnection network through a dedicated link, and the propagation distance and potential interference of data in the transmission medium can be reduced.

[0091] It can be understood that the inter-cluster broadcast engine can be implemented as a logical module embedded in the inter-cluster interconnection network switch chip. Alternatively, in another embodiment, the inter-cluster broadcast engine can be arranged on the substrate of the data processor, and its physical location is adjacent to the interface port of the inter-cluster interconnection network.

[0092] When the intra-cluster broadcast engine receives a broadcast request indicating a global broadcast (i.e., a cross-cluster broadcast), the intra-cluster broadcast engine can report the broadcast request to the inter-cluster broadcast engine over the inter-cluster interconnect network, and the inter-cluster broadcast engine can distribute the broadcast request to the intra-cluster broadcast engines of other block processing clusters using the inter-cluster interconnect network.

[0093] Referring to Figure 3 , Figure 3 is a hardware hierarchical architecture diagram of a data processor provided by another embodiment of the present disclosure. The hardware architecture of the data processor provided by the embodiment of the present disclosure has a multi-layer architecture, which is composed of a plurality of computing dies connected in parallel through an inter-die interconnect network. Each computing die internally includes an inter-cluster interconnect network and a plurality of block processing clusters. The inter-cluster interconnect network is responsible for data communication between the block processing clusters. Each block processing cluster internally includes an intra-cluster interconnect network, an intra-cluster broadcast engine (BPC-BCE), a plurality of computing units, and meta threads in the computing units.

[0094] The broadcast across the block processing clusters is implemented through the inter-cluster interconnect network. The inter-cluster interconnect network connects the intra-cluster broadcast engines of all block processing clusters to form a topology in which the inter-cluster broadcast engines are directly connected. When the intra-cluster broadcast engine receives a broadcast request reported by a computing unit in the block processing cluster, the broadcast request indicating a global broadcast (i.e., a cross-cluster broadcast) within the block processing cluster, the intra-cluster broadcast engine can broadcast the broadcast request to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network to trigger the intra-cluster broadcast engines of each block processing cluster to perform local broadcast. At this time, the reporting path of the request for cross-cluster global broadcast is the computing unit where the initiating meta thread is located to the intra-cluster broadcast engine of the local block processing cluster, and then broadcast to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network. When the broadcast request received by the intra-cluster broadcast engine includes a local broadcast range and a cross-cluster broadcast range, the local intra-cluster broadcast can be processed first, and then broadcast to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network. It can be understood that each block processing cluster can be regarded as a boundary of current resource scheduling, which internally includes computing resources, an intra-cluster interconnect network, and a dedicated intra-cluster broadcast engine. Externally, it cooperates with other block processing clusters through the inter-cluster interconnect network.

[0095] Wherein, the broadcast across the computing dies can be implemented through the inter-die interconnection network, the inter-die interconnection network can connect the intra-cluster broadcast engines within all computing dies, so that the intra-cluster broadcast engines establish a communication link through the inter-die interconnection network, when the intra-cluster broadcast engine within a certain computing die receives a cross-die range broadcast request initiated by a meta-thread, the broadcast request (including broadcast data, broadcast mask) can be forwarded to the intra-cluster broadcast engines of other computing dies through the inter-die interconnection network. After the intra-cluster broadcast engines of each computing die receive the broadcast request, the broadcast request is transmitted to the intra-cluster broadcast engines of all block processing clusters within the die through the intra-cluster interconnection network, and then the target computing units are located according to the broadcast mask, the data broadcast within the cluster is completed, efficient data synchronization from local to global is realized, for example, in the distributed training of large-scale deep learning models, when the gradient data needs to be synchronized to all computing units in the device, the broadcast data can be broadcast to all target computing units of the computing dies at one time through the data transmission method provided by the present disclosure, avoiding the multiple forwarding delays of traditional point-to-point transmission, and effectively improving the global synchronization efficiency.

[0096] It is worth noting that the broadcast across the block processing clusters and the computing dies can also be implemented through the inter-cluster interconnection network, the inter-die interconnection network, and the inter-die broadcast engine (Die-BCE). For details, please refer to Figure 4 , Figure 4 is a hardware level architecture diagram of a data processor provided by another embodiment of the present disclosure. It can be understood that the data processor is also provided with an inter-cluster broadcast engine. The inter-cluster broadcast engine can be directly integrated into the topology of the inter-cluster interconnection network, can be implemented by embedding the broadcast engine into a network switch chip or as a component of a network interface controller, and can be directly integrated into a crossbar switch as a hardware expansion module of a routing node of the inter-cluster interconnection network. Thus, the additional jump link between the broadcast engine and the interconnection network interface can be eliminated, the data transmission path can be shortened, the routing table of the inter-cluster interconnection network can be directly accessed, and zero-copy broadcast can be realized, such as single-cycle copying of data to multiple ports. Alternatively, the inter-cluster broadcast engine can be independently deployed, that is, the physical position of the inter-cluster broadcast engine is arranged adjacent to the interface point of the inter-cluster interconnection network. The inter-cluster broadcast engine can be arranged in the circuit board area adjacent to the network switch or connected through a short-distance interconnection line. Thus, the inter-cluster broadcast engine can interact with the inter-cluster interconnection network through a dedicated link, and the propagation distance and potential interference of data in the transmission medium can be reduced.

[0097] Wherein, the inter-die interconnection network is located outside the computing dies and serves as a global communication backbone across the dies. The inter-die interconnection network can connect the inter-cluster broadcast engines of all computing dies and implement data packet routing and broadcast across the dies. The inter-cluster broadcast engine can receive the broadcast request uploaded by the intra-cluster broadcast engine within the same die through the inter-cluster interconnection network, and forward the request to the inter-cluster broadcast engines of other computing dies through the inter-die interconnection network.

[0098] It can be understood that the inter-cluster broadcast engine can be implemented as a logic module embedded in an inter-cluster interconnection network switch chip; or in another embodiment, the inter-cluster broadcast engine can be arranged on the substrate of the data processor, and its physical position is adjacent to the interface port of the inter-cluster interconnection network.

[0099] When the intra-cluster broadcast engine receives a broadcast request indicating a global broadcast (i.e., a cross-cluster broadcast), the intra-cluster broadcast engine can report the broadcast request to the inter-cluster broadcast engine through the inter-cluster interconnection network, and the inter-cluster broadcast engine will use the inter-cluster interconnection network to distribute the broadcast request to the intra-cluster broadcast engines of other block processing clusters in the same die, and will use the inter-die interconnection network to forward the broadcast request to the inter-cluster broadcast engines of other dies, so that the other inter-cluster broadcast engines distribute the broadcast request to the intra-cluster broadcast engines in the corresponding dies.

[0100] It can be understood that the broadcast across the computing dies can also be implemented through the inter-die interconnection network in combination with the inter-die broadcast engine, referring to Figure 5 , Figure 5 is a hardware level architecture diagram of a data processor provided by another embodiment of the present disclosure. The inter-die broadcast engine (Device-BCE) serves as the coordination core of cross-die broadcast, and establishes a communication link with the inter-cluster broadcast engines of each computing die through the inter-die interconnection network. When the inter-cluster broadcast engine in a certain computing die receives a request containing a cross-die broadcast range, it forwards the request (containing broadcast data and a broadcast mask) to the inter-die broadcast engine. After the inter-die broadcast engine analyzes the request, it uses the high-bandwidth characteristics of the inter-die interconnection network to concurrently distribute the broadcast data and the mask to the inter-cluster broadcast engines of all computing dies. After the inter-cluster broadcast engines of each computing die receive the data, they pass the request to the intra-cluster broadcast engines of all block processing clusters in the die through the inter-cluster interconnection network, and the intra-cluster broadcast engines locate the target computing units according to the broadcast mask to complete the intra-cluster data distribution, thereby realizing efficient data synchronization from the local to the global through the block processing cluster (intra-cluster broadcast engine)-computing die (inter-cluster broadcast engine)-data processor (inter-die broadcast engine) hierarchical broadcast mechanism. Through this mechanism, the data can be broadcast to all target computing units of the computing dies at one time. In addition, the inter-die interconnection network supports multi-path transmission and fault tolerance mechanism. When a certain link is congested or fails, the inter-die broadcast engine can dynamically adjust the data distribution path (such as replacing the link or using the inter-cluster broadcast engine and the inter-cluster interconnection network for forwarding), to ensure the reliability and real-time performance of cross-die broadcast, and meet the efficiency requirements of global data synchronization in scenarios such as artificial intelligence model training.

[0101] Based on the above data processor architecture, the embodiment of the present disclosure proposes a new data transmission method, a broadcast engine, an electronic device and a storage medium. The broadcast data is automatically forwarded between clusters and distributed within the cluster by processing the broadcast request by the intra-cluster broadcast engine. The data can be broadcasted to any computing unit within the die synchronously, which can achieve efficient cross-block processing cluster data broadcast effect and reduce communication delay and overhead.

[0102] Specifically, the embodiment of the present disclosure proposes a data transmission method which can be applied to the intra-cluster broadcast engine of the data processor as shown in Figure 1 or Figure 2 or Figure 3 or Figure 4 or Figure 5 The data transmission method provided by the embodiment of the present disclosure is described in detail with reference to Figure 6 , Figure 6 An optional flow diagram of the data transmission method provided by the embodiment of the present disclosure is shown in the following figure. The data transmission method includes but is not limited to the following steps 601 to 603.

[0103] Step 601: The intra-cluster broadcast engine acquires broadcast data and broadcast mask in response to the broadcast request reported by the computing unit.

[0104] Step 602: The intra-cluster broadcast engine performs intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask.

[0105] Step 603: The intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engines of all block processing clusters through the inter-cluster interconnection network.

[0106] The broadcast request can be a communication instruction initiated by a meta-thread in the computing unit, which requests to broadcast the broadcast data to the target computing unit. Whether the target computing unit belongs to the same block processing cluster or the same computing die, the target computing unit can be the receiving node of the broadcast data. It is worth noting that the broadcast request can carry the storage address of the broadcast data, as well as the broadcast mode and the broadcast range. In the case of global synchronization of the meta-thread execution, such as model parameter distribution, the broadcast request is generated and reported to the local intra-cluster broadcast engine by the computing unit, and the local intra-cluster broadcast engine performs local intra-cluster broadcast and cross-block processing cluster broadcast cooperation.

[0107] The broadcast data can be data content that needs to be synchronized to the target computing unit, as a carrier of the broadcast content, and is distributed to the target computing unit by the broadcast engine. The broadcast data can be shared parameters of model computing, such as gradient data and activation values. The broadcast data can be sourced from local memory within the block processing cluster, such as SRAM within the computing unit or DRAM within the block processing cluster. Specifically, the meta-thread can initiate a broadcast request by sending a control message containing the broadcast data and the broadcast mask to the broadcast engine within the cluster. For example, after the computing unit completes a certain local computing task, the result needs to be broadcast to other computing units. At this time, the meta-thread can take the computing result as the broadcast data, and encode the identification information of the target computing unit into the broadcast mask, and send them to the broadcast engine together.

[0108] It is worth noting that the broadcast data can be obtained by parsing the data storage address in the broadcast request and reading from the local memory, but other ways such as directly receiving an externally input data stream can also be used. For example, the meta-thread can send a simple trigger signal to the broadcast engine within the cluster, and the broadcast engine within the cluster reads the broadcast data from a preset shared storage area after receiving the signal. That is, the broadcast data can be pre-stored in the local memory or shared cache of the computing unit, and the broadcast engine within the cluster accesses these information through the memory address.

[0109] The broadcast mask is a bitmap data structure used to identify the location of the target computing unit, and identifies whether each computing unit is a receiving target of the broadcast data, i.e., a target computing unit, through a binary bit (0 or 1). Specifically, each computing unit corresponds to a flag bit in the broadcast mask, and each binary bit is directly mapped to the address coordinates of a computing unit. When the flag bit is 0, the corresponding computing unit is a non-receiving target, i.e., a non-target computing unit; when the flag bit is 1, the corresponding computing unit is a receiving target, i.e., a target computing unit.

[0110] In a possible implementation, the broadcast request can specify a broadcast mode. The broadcast mode can be a distribution strategy of the broadcast data, and different broadcast modes switch distribution logic to adapt to different task scenarios, thereby improving data transmission efficiency and optimizing latency. The broadcast mode can include an arbitrary mode and a copy mode, and the broadcast mask formats corresponding to different broadcast modes are different.

[0111] Specifically, the copy mode refers to a data distribution scenario in which all block processing clusters to be distributed adopt the same mask bitmap, i.e., a broadcast mask. It can be understood as a communication mechanism for selectively copying broadcast data to the same computing unit in each block processing cluster. Each flag bit corresponds to a computing unit in the block processing cluster. At this time, the number of flag bits in the broadcast mask is equal to the number of computing units in the block processing cluster. The broadcast mask in the copy mode only defines the relative position of the target computing unit in a single block processing cluster. The intra-cluster broadcast engine in each block processing cluster applies the same broadcast mask to all local computing units. That is, the broadcast mask is applicable to all block processing clusters without the need for individual configuration for each block processing cluster. The receiving state and behavior of the computing units with the same sequence number (or position) in different block processing clusters are consistent. It should be noted that the broadcast mask can index the inherent sequence number of the computing unit in the block processing cluster. For example, flag bit 0 of the broadcast mask corresponds to computing unit 0 in the block processing cluster, and flag bit 1 corresponds to computing unit 1 in the block processing cluster.

[0112] Specifically, the present disclosure accurately specifies the target computing unit in the copy mode by precisely matching the structure of the broadcast mask with the hardware configuration of the block processing cluster. When the broadcast request specifies the broadcast mode as the copy mode, the broadcast mask is configured to contain M flag bits, where M is equal to the number of computing units in the current block processing cluster. Each flag bit in the broadcast mask establishes a one-to-one mapping relationship with a specific computing unit in the block processing cluster, so that the intra-cluster broadcast engine can accurately identify the target computing unit based on the mask and send the broadcast data to the specified unit through the intra-cluster interconnection network, thereby ensuring the integrity and efficiency of the broadcast operation.

[0113] In one example, referring to Figure 7 , Figure 7 is a schematic diagram of data transmission in the copy mode provided by the embodiment of the present disclosure. A block processing cluster in the data processor contains four computing units, i.e., M = 4. When the broadcast mode is specified as the copy mode in the broadcast request, the broadcast mask can be set to four flag bits. Assuming that the broadcast mask is 1101, computing unit 0, computing unit 1, and computing unit 3 in all block processing clusters can receive the broadcast data, and only computing unit 2 does not receive the broadcast data.

[0114] Specifically, the arbitrary mode refers to that the target compute unit position of each block processing cluster can be independently configured, realizing on-demand allocation to partial compute units, at this time, the broadcast mask contains NXM flag bits, N represents the total number of block processing clusters in the current compute die, at the same time, the broadcast mask is divided into N independent mask segments, each independent mask segment corresponds to a block processing cluster, M represents the number of compute units in a block processing cluster, and each independent mask segment also includes M flag bits, each flag bit corresponds to each compute unit in the current compute die one by one, for example, the jth independent mask segment in the broadcast mask independently determines whether the compute unit in the jth block processing cluster in the compute die receives the broadcast data.

[0115] When the broadcast request specifies the broadcast mode as the arbitrary mode, the broadcast mask is organized into N independent mask segments, each independent mask segment corresponding to a block processing cluster. After receiving the broadcast request reported by the compute unit, the intra-cluster broadcast engine first parses the broadcast mask and the broadcast data, and then determines the target compute unit based on the independent mask segment corresponding to the current block processing cluster, and transmits the broadcast data to the target compute unit through the intra-cluster interconnection network. At the same time, the intra-cluster broadcast engine that initiates the broadcast forwards the broadcast data and the complete broadcast mask to the intra-cluster broadcast engine of other block processing clusters through the inter-cluster interconnection network, triggering other clusters to perform corresponding intra-cluster broadcast processing. In the forwarding process, the N independent mask segments of the broadcast mask ensure that each target block processing cluster only receives the mask segment content related to it, thereby realizing accurate routing of inter-cluster broadcast, so that cross-cluster broadcast can independently specify a target compute unit set for each block processing cluster, avoiding the limitations of traditional single-mask structure in the cross-cluster scenario, and effectively improving the flexibility and execution efficiency of global data synchronization.

[0116] In one example, with reference to Figure 8 , Figure 8is a schematic diagram of data transmission in any mode provided by the embodiments of the present disclosure. Similarly, a computing unit included in one block processing cluster in the data processor is 4, that is, M=4, and one computing die includes 3 block processing clusters, that is, N=3, and the broadcast mask also includes 3 independent mask segments, each of which corresponds to one block processing cluster. When the broadcast mode specified in the broadcast request is any mode, the broadcast mask sets 12 flag bits. Assuming that the independent mask segment 0 of the broadcast mask is 0011, the independent mask segment 1 is 1100, and the independent mask segment 2 is 1100, it means that the computing unit 2 and the computing unit 3 in the block processing cluster 0 receive broadcast data, the computing unit 0 and the computing unit 1 in the block processing cluster 1 receive broadcast data, the computing unit 0 and the computing unit 1 in the block processing cluster 2 receive broadcast data, and the remaining computing units do not receive broadcast data. It is worth noting that in any mode, the flag bits of some independent mask segments can be the same or different. Therefore, the intra-cluster broadcast engine transmits the broadcast data to the specified target computing unit according to the received corresponding independent mask segment content, thereby realizing accurate cross-cluster data synchronization.

[0117] In combination with Figure 7 and Figure 8 , the broadcast mask adopts a bitmap form to realize the positioning of the target computing unit, and the flag bits of 0 and 1 control the data distribution range, which can effectively realize the control logic of global broadcast across block processing clusters, and the structure of the broadcast mask can be flexibly designed according to actual needs to support synchronization operations of different granularities. During the global broadcast across clusters, different broadcast modes can be selected to switch the data distribution strategy according to different task scenarios. In the copy mode, efficient global data distribution is realized by using a unified broadcast mask, which is suitable for the task scenario of global synchronization. In the any mode, accurate data distribution is realized by using segmented independent mask segments, which realizes accurate control of the data distribution range, avoids invalid communication overhead, and is suitable for the task scenario of sparse gradient update. Therefore, the present disclosure accurately controls the data distribution range by the mask mechanism, effectively reduces the invalid communication overhead, and effectively utilizes the high bandwidth of the intra-cluster interconnection network, which can avoid intra-cluster redundant transmission.

[0118] It can be understood that the intra-cluster broadcast engine can be a logic unit for performing an intra-cluster broadcast operation. The intra-cluster broadcast engine can realize the following functions such as parsing a broadcast request, generating a broadcast control signal, and driving an intra-cluster (inter-cluster) interconnection network to forward data by a hardware logic circuit or a firmware program. Specifically, the intra-cluster broadcast engine can be integrated in the control unit of the block processing cluster and directly connected with the interface of the intra-cluster interconnection network, so as to quickly respond to the broadcast request of the local computing unit and efficiently schedule the intra-cluster resources.

[0119] In the in-cluster broadcast process, the in-cluster broadcast engine can start internal logic to prepare data distribution after receiving the broadcast data and the broadcast mask. Specifically, the in-cluster broadcast engine can perform necessary format conversion or encapsulation on the broadcast data to adapt to the transmission protocol of the intra-cluster interconnection network and the inter-chip interconnection network.

[0120] It can be understood that the in-cluster broadcast process includes the following steps: the in-cluster broadcast engine determines the target computing unit of the current processing cluster based on the broadcast mask, and broadcasts the broadcast data to the target computing unit through the intra-cluster interconnection network. Specifically, the in-cluster broadcast engine can check the broadcast mask bit by bit, and send the broadcast data to the corresponding computing unit for each flag bit set to 1 in the broadcast mask. For example, if the broadcast mask is a bitmap, each bit of which corresponds to a computing unit in the cluster, the in-cluster broadcast engine will traverse the bitmap and send the broadcast data to all computing units corresponding to the bits of "1". In addition, the in-cluster broadcast engine can directly provide the broadcast mask to the intra-cluster interconnection network, which will distribute the broadcast data to the target computing unit according to the indication of the broadcast mask. For example, the intra-cluster interconnection network can include an address resolution module that can dynamically generate a target address list according to the broadcast mask and send the broadcast data to these addresses in parallel.

[0121] It can be understood that in the distributed training and inference scenario, efficient data synchronization is crucial for artificial intelligence and high-performance computing tasks, and the existing technology faces significant communication bottlenecks when implementing global broadcasting across block processing clusters. The data processor is composed of multiple block processing clusters, which are connected to each other through an inter-cluster interconnection network. Each block processing cluster integrates an in-cluster broadcast engine, multiple computing units, and an intra-cluster interconnection network.

[0122] The inter-cluster interconnection network can be a hardware structure for providing a communication path between different block processing clusters within the same computing die. The inter-cluster interconnection network can enable efficient data exchange between block processing clusters, reducing the delay and overhead caused by indirect communication through higher-level inter-chip interconnection networks. The inter-cluster interconnection network can be implemented based on a Network-on-Chip (NoC) architecture, which includes routing nodes and communication links to allow data packets to be routed and forwarded between different block processing clusters. In addition, the inter-cluster interconnection network can also adopt a shared bus or ring bus structure, allowing each block processing cluster to exchange data through a unified interface; or a crossbar switch structure, providing point-to-point direct connections between block processing clusters to achieve high-bandwidth and low-latency communication.

[0123] It can be understood that the intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to other block processing clusters in the same computing die through the inter-cluster interconnection network, and triggers other intra-cluster broadcast engines in the intra-cluster to perform the intra-cluster broadcast processing based on the broadcast data and the broadcast mask, which can expand the broadcast range of the intra-cluster broadcast engine, so that the intra-cluster broadcast engine can efficiently deliver data and masks to other block processing clusters in the same computing die, and trigger the intra-cluster broadcast engines of these clusters to perform the intra-cluster broadcast processing. Specifically, when an intra-cluster broadcast engine receives a broadcast request, in addition to performing intra-cluster broadcast in the current block processing cluster, it can also encapsulate the broadcast data and the broadcast mask according to the range indicated by the broadcast mask through the inter-cluster interconnection network, and send the broadcast data and the broadcast mask to the intra-cluster broadcast engines of other target block processing clusters in the same computing die. The target intra-cluster broadcast engines receiving these data and masks will parse the received information and perform the intra-cluster broadcast processing procedure, thereby ensuring that the data is correctly distributed in the entire target range.

[0124] It can be understood that the inter-cluster interconnection network can provide a direct communication channel between multiple block processing clusters in the same computing die. When a meta-thread in a computing unit initiates a broadcast request, the corresponding intra-cluster broadcast engine responds to the request, obtains the broadcast data and the broadcast mask, and performs the intra-cluster broadcast processing in the current block processing cluster. On this basis, if the broadcast mask indicates that the broadcast range needs to be expanded to other block processing clusters in the same computing die, the intra-cluster broadcast engine can use the inter-cluster interconnection network to directly forward the broadcast data and the broadcast mask to the intra-cluster broadcast engines in the target block processing clusters, thereby reducing indirect communication across die-level through the inter-die interconnection network, significantly reducing communication delay. The target intra-cluster broadcast engines receiving the forwarded data can perform intra-cluster broadcast processing in their respective block processing clusters based on the received broadcast data and broadcast mask, thereby ensuring that the data is uniformly distributed in all related computing units in the entire target computing die, and achieving efficient data transmission according to the hierarchical nature of the broadcast range.

[0125] Referring to Figure 9 , Figure 9 is a flowchart of data transmission during inter-cluster broadcast provided by the embodiments of the present disclosure. When a meta-thread in the computing unit A of the block processing cluster A performs a task that requires cross-cluster data synchronization, such as gradient aggregation in deep neural network training, feature fusion in multi-modal data processing, etc., it will actively initiate a broadcast request. The broadcast request not only contains the broadcast data to be transmitted itself, but also carries a broadcast mask for accurately indicating the target computing unit range, and related control information required for data transmission, such as data length, priority identifier, etc. After the computing unit generates the broadcast request, it will quickly report the request to the intra-cluster broadcast engine A of the block processing cluster A through its internal preset communication interface.

[0126] When the intra-cluster broadcast engine A receives the broadcast request reported by the computing unit, it first extracts the storage address (such as the address of SRAM or DRAM) of the broadcast data and the broadcast mask from the broadcast request, and then reads the broadcast data from the local memory (such as the SRAM of the computing unit or the shared DRAM in the block processing cluster) according to the storage address. After reading is completed, the intra-cluster broadcast engine A parses the broadcast mask, where the broadcast mask is used to accurately identify the target range to avoid redundant transmission caused by full-network broadcast. By combining the identification information of the current block processing cluster with the broadcast mask, the target computing unit (such as computing unit B) in the current block processing cluster that needs to receive the broadcast data is determined. For example, in the copy mode, the broadcast mask directly indicates whether each computing unit in the current cluster is a target computing unit; in the arbitrary mode, the intra-cluster broadcast engine locates the independent mask segment corresponding to the block processing cluster in the broadcast mask, and then determines the target computing unit according to the mask segment. After determining the target computing unit, the intra-cluster broadcast engine A sends the broadcast data to these target computing units in parallel or in series through the intra-cluster interconnection network (such as a crossbar network, a ring network, or a bus network with a topology structure). For the intra-cluster interconnection network, if a crossbar network is used, the intra-cluster broadcast engine can simultaneously establish connections with multiple target computing units to achieve parallel broadcast of data, thereby effectively reducing the intra-cluster broadcast delay; if a ring network is used, the data can be sent to each target computing unit in turn through the transmission of broadcast packets on the ring. In this process, after receiving the broadcast data, the target computing unit stores the broadcast data in the local designated memory space for subsequent access and processing by the meta-thread.

[0127] After completing the intra-cluster broadcast in the current block processing cluster A, the intra-cluster broadcast engine A starts the inter-cluster broadcast process. At this time, the intra-cluster broadcast engine A concurrently forwards the broadcast data and the complete broadcast mask to the intra-cluster broadcast engines of all other block processing clusters B in the computing die through the interface of the inter-cluster interconnection network. Here, “concurrently forwarding” means that the local intra-cluster broadcast engine can actively push out the broadcast data and the broadcast mask without waiting for the response of other block processing clusters while processing the intra-cluster broadcast of itself or after completing the intra-cluster broadcast of itself. The inter-cluster interconnection network has efficient multicast or broadcast capability. For example, the inter-cluster interconnection network can adopt a tree topology, a mesh topology, or a full-connection topology, and can also have a hardware-level multicast routing function to ensure that the broadcast data and the broadcast mask can quickly and low-delay reach the intra-cluster broadcast engines of all target block processing clusters.

[0128] When the cluster-intra broadcast engine B of the other block processing cluster receives the broadcast data and the broadcast mask from the inter-cluster interconnection network, the same cluster-intra broadcast processing flow as the cluster-intra broadcast engine A that initiates the broadcast is triggered immediately: the broadcast mask is parsed, the target computing units (computing units C or D) in the cluster are determined, and then the broadcast data is distributed to the target computing units through the cluster-intra interconnection network of the cluster-intra broadcast engine B. In this way, all target computing units in the entire computing die (regardless of the target computing units located in which block processing cluster) can receive the broadcast data, thereby realizing efficient data broadcast in a global range.

[0129] Referring to Figure 10 , Figure 10 is an effect diagram of the broadcast range across the block processing clusters provided by the embodiment of the present disclosure. It is assumed that a computing die includes two block processing clusters, and each block processing cluster includes four computing units. In the related art, the data broadcast range is limited to all computing units in the same block processing cluster, as shown by the shaded area in Figure 10 . The data broadcast range of the computing units is strictly limited within the block processing cluster to which the computing units belong, and cannot directly reach the computing units of other block processing clusters (i.e., the white area shown in Figure 10 ). However, by using the data transmission method provided by the embodiment of the present disclosure, the data broadcast range of the computing units is extended to all computing units of all block processing clusters in the computing die, realizing cross-cluster data broadcast, as shown by the white area in Figure 10 . The computing unit that initiates the broadcast can accurately send the broadcast data to the specified computing units of all target block processing clusters in the computing die by configuring the broadcast mask (such as the uniform broadcast mask in the copy mode or the independent mask segment in the arbitrary mode), and the data broadcast range is extended from the same block processing cluster to the data broadcast range of all block processing clusters in the same computing die.

[0130] In summary, by using the broadcast mask mechanism and the hierarchical broadcast architecture, the embodiment of the present disclosure can dynamically specify the target range according to the broadcast mask, rather than directly broadcast the entire network, and at the same time, in combination with the low-delay high-rate transmission characteristics of the cluster-intra interconnection network and the distributed forwarding capability of the inter-cluster interconnection network, the global broadcast operation is decomposed into multiple parallel cluster-intra broadcast processing. The cluster-intra broadcast engine of each block processing cluster can independently parse the mask and perform local cluster-intra broadcast processing operation. Therefore, the embodiment of the present disclosure effectively alleviates the cross-cluster communication bottleneck, realizes efficient data synchronization, and avoids the redundant transmission problem caused by blind broadcast in the traditional technology.

[0131] In some possible implementation manners, in the process of forwarding the broadcast data and the broadcast mask to the intra-cluster broadcast engines of all block processing clusters through the inter-cluster interconnection network, when the broadcast range specified in the broadcast request is inter-cluster broadcast, target block processing clusters containing target computing units are determined based on the broadcast mask; the broadcast data and the broadcast mask are forwarded to the intra-cluster broadcast engines of the target block processing clusters through the inter-cluster interconnection network.

[0132] In the broadcast request, the identification field corresponding to the broadcast range is carried, which is a control parameter for identifying the broadcast data distribution boundary. The broadcast range includes two levels. One is the block processing cluster range, that is, the broadcast data is only broadcast and distributed to the computing units in the current block processing cluster. The other is the computing die range, that is, the broadcast data is broadcast and distributed to the computing units in all block processing clusters in the computing die. The intra-cluster broadcast corresponds to the block processing cluster range in the broadcast range, and the inter-cluster broadcast corresponds to the computing die range in the broadcast range. In the broadcast request message, a specific bit identification bit or field value can be reserved to realize the identification field of the broadcast range. Therefore, the intra-cluster broadcast engine can quickly identify the data distribution range after receiving the broadcast request, and trigger the corresponding broadcast operation. For example, when the intra-cluster broadcast engine receives the broadcast request, the broadcast request is parsed, and the identification bit representing the broadcast range is identified (wherein the identification bit = 0 represents the block processing cluster range, and the identification bit = 1 represents the computing die range). If the broadcast range is inter-cluster broadcast, that is, the computing die range, the operation of cross-cluster forwarding is performed. The local intra-cluster broadcast engine triggers the intra-cluster broadcast engines of other block processing clusters to cooperate through the inter-cluster interconnection network, to realize global data synchronization.

[0133] In the broadcast request, the identification field corresponding to the broadcast range is carried, which is a control parameter for identifying the broadcast data distribution boundary. The broadcast range includes two levels. One is the block processing cluster range, that is, the broadcast data is only broadcast and distributed to the computing units in the current block processing cluster. The other is the computing die range, that is, the broadcast data is broadcast and distributed to the computing units in all block processing clusters in the computing die. The intra-cluster broadcast corresponds to the block processing cluster range in the broadcast range, and the inter-cluster broadcast corresponds to the computing die range in the broadcast range. In the broadcast request message, a specific bit identification bit or field value can be reserved to realize the identification field of the broadcast range. Therefore, the intra-cluster broadcast engine can quickly identify the data distribution range after receiving the broadcast request, and trigger the corresponding broadcast operation. For example, when the intra-cluster broadcast engine receives the broadcast request, the broadcast request is parsed, and the identification bit representing the broadcast range is identified (wherein the identification bit = 0 represents the block processing cluster range, and the identification bit = 1 represents the computing die range). If the broadcast range is inter-cluster broadcast, that is, the computing die range, the operation of cross-cluster forwarding is performed. The local intra-cluster broadcast engine triggers the intra-cluster broadcast engines of other block processing clusters to cooperate through the inter-cluster interconnection network, to realize global data synchronization.

[0134] In an example, in the data processor, when the broadcast request indicates that the broadcast range is inter-cluster broadcast, the inter-cluster broadcast engine parses the broadcast mask, for example, in the case where the broadcast request indicates that the broadcast mode is the arbitrary mode, the target block processing cluster can be determined by checking whether each block processing cluster corresponding to an independent mask segment in the broadcast mask has a non-zero value; for another example, in the case where the broadcast request indicates that the broadcast mode is the replication mode, all block processing clusters can be directly determined as the target block processing cluster; then, the inter-cluster interconnection network only forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engine of the target block processing cluster, for example, in a data processor containing multiple block processing clusters, if the broadcast mask indicates that part of the block processing clusters contain target computing units, only the data is forwarded to these block processing clusters, and other block processing clusters are ignored, thereby effectively reducing the redundant data transmission in the inter-cluster interconnection network, saving network bandwidth resources, and at the same time reducing the communication delay and improving the efficiency of data synchronization.

[0135] In an example, assuming that the computing die contains 8 block processing clusters, when the gradient aggregation of a certain layer is performed in the model training, if only 4 computing units of the block processing clusters participate in the computation of the layer, the intra-cluster broadcast engine forwards the gradient data to only the 4 target clusters through the inter-cluster interconnection network, compared with the full-network broadcast, the data transmission amount can be effectively reduced, and the communication delay is reduced. In addition, the broadcast mask can also be dynamically adjusted, for example, when multiple tasks are concurrent, if the computing units of the block processing cluster switch tasks, the computing units in the block processing cluster no longer process related data, the broadcast mask in the broadcast request can be updated in real time to remove the block processing cluster from the multicast group list of the target block processing cluster, thereby avoiding invalid transmission.

[0136] In an example, when the broadcast range indicated by the broadcast request is intra-cluster broadcast, that is, the target computing units specified by the broadcast request are only distributed in the local block processing cluster, the intra-cluster broadcast engine automatically skips the inter-cluster forwarding process and directly performs the intra-cluster broadcast processing, only completes the intra-cluster data synchronization, simplifies the data transmission process, and can adapt to the flexible data synchronization requirements of the intra-cluster broadcast and the global broadcast.

[0137] In some possible implementation manners, in the process of forwarding the broadcast data and the broadcast mask to the intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnection network, the intra-cluster broadcast engine initiating the broadcast determines the target block processing cluster containing the target computing unit through an independent mask segment in the broadcast mask, extracts the independent mask segment corresponding to the target block processing cluster from the broadcast mask as a target mask; then, the broadcast data and the target mask are forwarded to the intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnection network.

[0138] The independent mask segment is a segment of the broadcast mask corresponding to the number of block processing clusters. Each independent mask segment corresponds to a block processing cluster and can be implemented by a bit vector or a register file, so that independent broadcast control can be performed for different block processing clusters. The independent mask segment corresponding to a specific target block processing cluster can be located and separated from the broadcast mask. The specific target block processing cluster can be obtained by using a lookup or bit shift operation based on the index offset of the block processing cluster, so that a target mask containing only the broadcast information of the target block processing cluster can be obtained, and the amount of data to be transmitted subsequently can be reduced.

[0139] It can be understood that the data processor includes a plurality of block processing clusters, and each block processing cluster includes a plurality of computing units. The broadcast mask is composed of a plurality of independent mask segments, and each segment includes a flag bit corresponding to the number of computing units. For example, each block processing cluster includes M computing units, and each independent mask segment includes M flag bits. After the target block processing cluster is determined, M bits can be cut from the broadcast mask by a hardware shifter according to the index offset of the target block processing cluster. Then, the independent mask segment corresponding to the target block processing cluster is encapsulated as a target mask. The target mask is a sub-mask dedicated to the target block processing cluster, rather than a complete broadcast mask. A label of the corresponding serial number of the block processing cluster can be attached to the target mask. The broadcast data and the target mask are transmitted to the intra-cluster broadcast engine of the target block processing cluster, so that the total amount of data to be transmitted can be reduced, irrelevant mask information can be avoided to be transmitted, and the resource utilization rate can be optimized.

[0140] In some possible implementation manners, when the local block processing cluster is a block processing cluster that does not initiate the broadcast, the local intra-cluster broadcast engine receives a broadcast request sent by another intra-cluster broadcast engine. At this time, the local intra-cluster broadcast engine can extract the broadcast data and the broadcast mask from the broadcast request. The intra-cluster broadcast processing is performed in the current block processing cluster based on the broadcast data and the broadcast mask.

[0141] The broadcast request can be understood as a cross-cluster communication signal, which can be implemented in the form of a data packet or a message frame. The broadcast request is used to identify the broadcast source (the computing unit of the initiating block processing cluster) and the target broadcast range. Specifically, the current block processing cluster, as the target block processing cluster of the broadcast process, can receive a broadcast request sent by another intra-cluster broadcast engine through an inter-cluster interconnection network. The broadcast request can include the storage address of the broadcast data, the broadcast mask, the broadcast range, the serial number of the block processing cluster that initiates the broadcast, and the like. The current block processing cluster, that is, the target block processing cluster, can parse the broadcast request by using a hardware parsing circuit or a software parsing module, extract the broadcast mask based on the parsed information, and receive the forwarded broadcast data. Then, the current block processing cluster performs the intra-cluster broadcast processing according to the broadcast data and the broadcast mask.

[0142] Reference Figure 11 ,Figure 11 is an effect schematic diagram of inter-cluster broadcast provided by the embodiments of the present disclosure, assuming that there are 2 computing dies (such as computing die 0 and computing die 1) in the device, each computing die includes 3 block processing clusters (such as block processing cluster 0 to block processing cluster 5), and each block processing cluster also includes 4 computing units. In the computing die 0, the intra-cluster broadcast engine 0 of the block processing cluster 0 receives the broadcast request reported by the local computing unit, and needs to broadcast the broadcast data to other block processing clusters in the computing die 0, that is, the block processing cluster 1 and the block processing cluster 2. At this time, the intra-cluster broadcast engine 0 can synchronously forward the broadcast data and the broadcast mask to the intra-cluster broadcast engine 1 corresponding to the block processing cluster 1 and the intra-cluster broadcast engine 2 corresponding to the block processing cluster 2 while performing the intra-cluster broadcast processing according to the broadcast request. The intra-cluster broadcast engine 1 performs the intra-cluster broadcast processing in the block processing cluster 1, and the intra-cluster broadcast engine 2 performs the broadcast processing in the block processing cluster 2. It is worth noting that the intra-cluster broadcast engine 0 as the intra-cluster broadcast engine initiating the broadcast can forward the broadcast request to the intra-cluster broadcast engine 1 and the intra-cluster broadcast engine 2 through the inter-cluster interconnection network at the same time. At this time, the intra-cluster broadcast engine 1 and the intra-cluster broadcast engine 2 only need to perform the intra-cluster broadcast processing according to the broadcast request forwarded by the intra-cluster broadcast engine 0. In the computing die 1, the intra-cluster broadcast engine 3 corresponding to the block processing cluster 3 receives the broadcast request reported by the local computing unit, and needs to broadcast the broadcast data to all block processing clusters in the computing die 1, that is, the block processing cluster 4 and the block processing cluster 5. At this time, the intra-cluster broadcast engine 3 can also perform the intra-cluster broadcast processing according to the corresponding broadcast request, and then forward the broadcast request to the intra-cluster broadcast engine 4 through the inter-cluster interconnection network. When the intra-cluster broadcast engine 4 receives the broadcast request forwarded by the intra-cluster broadcast engine 3 and obtains the broadcast range from the broadcast request, it will perform the intra-cluster broadcast processing locally based on the broadcast request, and then forward the broadcast request to the intra-cluster broadcast engine 5 through the inter-cluster interconnection network again. The intra-cluster broadcast engine 5 performs the local intra-cluster broadcast processing in response to the broadcast request. Specifically, by responding to the broadcast request sent by other intra-cluster broadcast engines and extracting key information, the current block processing cluster can correctly identify the external broadcast request. Then, based on the extracted broadcast data and broadcast mask, the intra-cluster broadcast processing is performed locally to ensure that the broadcast data is accurately distributed to the target computing unit according to the broadcast mask. In addition, the original broadcast request can also be forwarded to other block processing clusters through the inter-cluster interconnection network to form a cascade propagation chain, so as to realize the step-by-step transmission and coverage of the broadcast request. This mechanism avoids that the broadcast request is ignored or incorrectly processed at the receiving end, and ensures the integrity and efficiency of cross-cluster communication.

[0143] It can be understood that when the intra-cluster broadcast engine of the block processing cluster receives the broadcast request sent from other block processing clusters, the broadcast data and the broadcast mask are first parsed by the hardware logic circuit; then the target computing units in the current block processing cluster are determined according to the broadcast mask, and the broadcast data is broadcast to these computing units through the intra-cluster interconnection network; then, the original broadcast request can be forwarded to the intra-cluster broadcast engines of the remaining block processing clusters through the inter-cluster interconnection network, triggering them to perform the same parsing, local broadcast and forwarding operations. Therefore, the current block processing cluster can effectively respond to and process the broadcast request from other block processing clusters, guarantee the integrity of the cross-cluster global broadcast, and effectively improve the data synchronization efficiency and parallel expansion capability.

[0144] In some possible implementation manners, referring to Figure 2 , the intra-cluster broadcast engine initiating the broadcast uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine, and the inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines of all block processing clusters through the inter-cluster interconnection network. Then, each intra-cluster broadcast engine performs the intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask distributed by the inter-cluster broadcast engine.

[0145] The inter-cluster broadcast engine can be a centralized data distribution management module independent of the block processing cluster (intra-cluster broadcast engine), which is used to separate the forwarding responsibility of the inter-cluster broadcast from the intra-cluster broadcast engine, so as to avoid resource competition and delay accumulation caused by the double tasks of local intra-cluster broadcast processing and inter-cluster broadcast forwarding. The uploading operation can be understood as a directional transmission process of the broadcast data and the mask from the intra-cluster broadcast engine to the inter-cluster broadcast engine through the inter-cluster interconnection network, which can be implemented by using a reserved bandwidth channel in the inter-cluster interconnection network, and is helpful to realize the centralized collection of the broadcast request and reduce the frequency of dispersed access of the inter-cluster interconnection network. The distribution operation is a unified scheduling process of the inter-cluster broadcast engine to distribute data to the intra-cluster broadcast engines of all block processing clusters, which is implemented by using a multicast tree topology or a hierarchical forwarding mechanism, and uses a global perspective to optimize the data distribution path, so as to ensure the synchronization reliability of the cross-cluster broadcast.

[0146] Referring to Figure 12 , Figure 12 is a flowchart of data transmission during inter-cluster broadcast provided by the embodiment of the present disclosure. When the metathread of the computing unit A of the block processing cluster A is executing a task requiring cross-cluster data synchronization, the metathread initiates a broadcast request. The broadcast request can include the broadcast data to be transmitted, and the broadcast mask and related control information required for data transmission, such as data length and priority identifier. After generating the broadcast request, the computing unit A quickly reports the request to the intra-cluster broadcast engine A of the block processing cluster A through the internal preset communication interface.

[0147] When the intra-cluster broadcast engine A receives the broadcast request reported by the computing unit, it first extracts the storage address (such as the address of SRAM or DRAM) of the broadcast data and the broadcast mask from the broadcast request, and then reads the broadcast data from the local memory (such as the SRAM of the computing unit or the shared DRAM in the block processing cluster) according to the storage address. After reading is completed, the intra-cluster broadcast engine A parses the broadcast mask, determines the target computing units (such as computing unit B) in the current block processing cluster that need to receive the broadcast data by combining the identification information of the current block processing cluster with the broadcast mask. After the target computing units are determined, the intra-cluster broadcast engine A sends the broadcast data to these target computing units in parallel or serially through the intra-cluster interconnection network. In this process, after the target computing unit B receives the broadcast data, it stores the broadcast data in the local designated memory space for subsequent access and processing by the meta-thread.

[0148] After completing the intra-cluster broadcast in the current block processing cluster A, the intra-cluster broadcast engine A starts the inter-cluster broadcast process. At this time, the intra-cluster broadcast engine A uploads the broadcast data and the complete broadcast mask to the inter-cluster broadcast engine through the interface of the inter-cluster interconnection network. The inter-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engines of other block processing clusters (such as block processing cluster B) in the computing die through the inter-cluster interconnection network.

[0149] When the intra-cluster broadcast engine B of the other block processing cluster receives the broadcast data and the broadcast mask from the inter-cluster interconnection network, it immediately triggers the same intra-cluster broadcast processing procedure as the intra-cluster broadcast engine A that initiates the broadcast: parses the broadcast mask, determines the target computing units (computing units C or D) in the current cluster, and then distributes the broadcast data to these target computing units through its own intra-cluster interconnection network. In this way, all target computing units in the entire computing die (regardless of the block processing cluster where the target computing unit is located) can receive the broadcast data, thereby achieving efficient data broadcast in the global range.

[0150] Specifically, by adding the inter-cluster broadcast engine as a single entry to receive and manage all broadcast requests, conflicts caused by multiple intra-cluster broadcast engines accessing the inter-cluster interconnection network at the same time are avoided; based on the received broadcast data and broadcast mask, the inter-cluster broadcast engine uniformly performs the issuing operation to the intra-cluster broadcast engines of all block processing clusters through the inter-cluster interconnection network, so that the intra-cluster broadcast engine can focus on intra-cluster broadcast processing, and the centralized scheduling mechanism of the inter-cluster broadcast engine effectively reduces the risk of network congestion.

[0151] In an example, the inter-cluster broadcast engine can be implemented as a programmable broadcast controller integrated in the inter-cluster interconnect network switch fabric, connected with the intra-cluster broadcast engines of each block processing cluster through dedicated communication links; when the intra-cluster broadcast engine responds to the broadcast request reported by the computing unit and obtains the broadcast data and the broadcast mask, the data is uploaded to the inter-cluster broadcast engine through the reserved low-latency channel; after receiving, the inter-cluster broadcast engine encapsulates the broadcast data according to the broadcast mask, and synchronously issues the broadcast data and the broadcast mask to the intra-cluster broadcast engines of all block processing clusters through the broadcast channel of the inter-cluster interconnect network, triggering each intra-cluster broadcast engine to execute the corresponding intra-cluster broadcast processing, so that the processing load pressure of the intra-cluster broadcast engine in the inter-cluster data forwarding can be effectively relieved, the concurrent access conflict and congestion phenomenon of the inter-cluster interconnect network is reduced, and the execution efficiency of the cross-cluster global broadcast is significantly improved.

[0152] With reference to Figure 13 , Figure 13is an effect schematic diagram of inter-cluster broadcast provided by the embodiments of the present disclosure, assuming that there are 2 computing dies (such as computing die 0 and computing die 1) in the device, each computing die includes 3 block processing clusters (such as block processing cluster 0 to block processing cluster 5), and each block processing cluster also includes 4 computing units. In the computing die 0, the intra-cluster broadcast engine 0 of the block processing cluster 0 receives the broadcast request reported by the local computing unit, and needs to broadcast the broadcast data to other block processing clusters in the computing die 0, that is, the block processing cluster 1 and the block processing cluster 2. At this time, the intra-cluster broadcast engine 0 can execute the intra-cluster broadcast processing according to the broadcast request, broadcast the broadcast data to the target computing unit in the block processing cluster 0, and then upload the broadcast data and the broadcast mask to the inter-cluster broadcast engine 0 corresponding to the computing die 0. The inter-cluster broadcast engine can analyze the broadcast mask to determine which block processing cluster in the computing die is the target block processing cluster in this broadcast, and then distribute the broadcast data and the broadcast mask to the corresponding target block processing cluster, such as the block processing cluster 1 and the block processing cluster 2. The intra-cluster broadcast engine 1 corresponding to the block processing cluster 1 and the intra-cluster broadcast engine 2 corresponding to the block processing cluster 2 both receive the broadcast data and the broadcast mask distributed by the inter-cluster broadcast engine 0, and the intra-cluster broadcast engine 1 executes the intra-cluster broadcast processing in the block processing cluster 1, and the intra-cluster broadcast engine 2 executes the broadcast processing in the block processing cluster 2. Similarly, in the computing die 1, the intra-cluster broadcast engine 4 corresponding to the block processing cluster 4 receives the broadcast request reported by the local computing unit, and needs to broadcast the broadcast data to all block processing clusters in the computing die 1, that is, the block processing cluster 3 and the block processing cluster 5. At this time, the intra-cluster broadcast engine 4 can also execute the intra-cluster broadcast processing according to the corresponding broadcast request, and then upload the broadcast request to the inter-cluster broadcast engine 1 through the inter-cluster interconnection network. The inter-cluster broadcast engine 1 repeats the above processes of analyzing the broadcast mask and determining the target block processing cluster, distributes the broadcast data and the mask to the intra-cluster broadcast engines corresponding to the block processing cluster 3 and the block processing cluster 5 in the computing die 1, and executes the intra-cluster broadcast processing in the block processing cluster by each intra-cluster broadcast engine, thereby realizing the global broadcast coverage of the computing die.

[0153] In some possible implementation manners, the inter-cluster broadcast engine can also support dynamic broadcast range adjustment. For example, when the target range of the broadcast request changes in real time (such as part of the block processing cluster temporarily exits the broadcast reception due to too high load), the inter-cluster broadcast engine can update the target block processing cluster identifier in the broadcast mask according to the dynamic configuration instruction from the system scheduling module, regenerate the sub-mask to be distributed, and ensure that the broadcast data is only sent to the current valid target block processing cluster, thereby avoiding invalid data transmission and further optimizing the resource utilization rate and the broadcast response speed.

[0154] In addition, in order to cope with the scenario of large broadcast data, the inter-cluster broadcast engine can introduce a compression and decompression mechanism for broadcast data. For example, when uploading broadcast data to the inter-cluster broadcast engine, the intra-cluster broadcast engine can perform lightweight hardware compression (such as a simplified algorithm based on LZ77) on the broadcast data to reduce the amount of data transmitted; the inter-cluster broadcast engine decompresses the compressed data before delivery, and then forwards it to the target intra-cluster broadcast engine. This compression optimization effectively reduces the bandwidth occupation of the inter-cluster interconnection network without affecting the real-time performance of the broadcast, and can be applied to high-throughput parallel computing task scenarios.

[0155] In some possible implementations, as shown in Figure 3 , Figure 4 or Figure 5 , the data processor further includes a plurality of computing dies, and an inter-die interconnection network for communication between the plurality of computing dies, each computing die including a plurality of block processing clusters, at this time, if the broadcast range indicated by the broadcast request received by the intra-cluster broadcast engine is inter-die broadcast, the broadcast data and the broadcast mask can be broadcast to the intra-cluster broadcast engines in other computing dies through the inter-die interconnection network to trigger the intra-cluster broadcast engines in other computing dies to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0156] Among them, the broadcast request can dynamically indicate the broadcast range through the bit length configuration of the broadcast mask and / or the broadcast range field (a specific field in the broadcast request), to realize the control of different levels of computing units.

[0157] Specifically, the broadcast range field in the broadcast request can be preset to indicate the broadcast range. For example, the broadcast range indicates intra-cluster broadcast / local broadcast, that is, broadcast covering all computing units in the same block processing cluster, and the corresponding broadcast range field can be a first field; the broadcast range indicates inter-cluster broadcast, that is, broadcast covering computing units of other block processing clusters in the same computing die, and the corresponding broadcast range field can be a second field; the broadcast range indicates inter-die broadcast, that is, broadcast covering computing units in other computing dies, and the corresponding broadcast range field can be a third field. When it is detected that the broadcast range field in the broadcast request is the third field, that is, the broadcast range indicates inter-die broadcast covering computing units in other computing dies, the broadcast request can be broadcast to the intra-cluster broadcast engines in other computing dies through the inter-die interconnection network.

[0158] It can be understood that the intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engines in other computing dies through the inter-die interconnection network, so as to trigger the intra-cluster broadcast engines in other computing dies to perform the intra-cluster broadcast processing based on the broadcast data and the broadcast mask. Specifically, the intra-cluster broadcast engine of the initiating block processing cluster can directly encapsulate the broadcast data and the broadcast mask into a network packet, and send the network packet to the intra-cluster broadcast engines in all other computing dies through the inter-die interconnection network. For example, the inter-die interconnection network can support multicast or broadcast functions, so that the network packet can be efficiently distributed to multiple target dies. Alternatively, the intra-cluster broadcast engine of the initiating block processing cluster can send the broadcast data and the broadcast mask to a general forwarding interface of the inter-die interconnection network, and the general forwarding interface is responsible for routing the information to the intra-cluster broadcast engines in other computing dies. For example, the general forwarding interface can maintain a computing die address table, and send the broadcast data to all computing dies according to the computing die address table.

[0159] Referring to Figure 14 , Figure 14 is a flowchart of data transmission in inter-die broadcasting provided by an embodiment of the present disclosure. One device can contain one or more computing dies, that is, one device can contain one or more data processors. It is assumed that a device contains two computing dies, which are marked as computing die A and computing die B. The computing die A contains two block processing clusters, which are block processing cluster A1 and block processing cluster A2. The computing die B also contains two block processing clusters, which are block processing cluster B1 and block processing cluster B2. Each block processing cluster is internally configured with a plurality of computing units and an intra-cluster broadcast engine.

[0160] For example, when a large-scale artificial intelligence model training is performed, a meta-thread in a computing unit in the block processing cluster A1 of the computing die A needs to broadcast a model parameter update value (that is, broadcast data) obtained by the meta-thread to all related computing units in the data processor to achieve global synchronization. The meta-thread first initiates a broadcast request, which contains the model parameter update value as broadcast data and a broadcast mask indicating all target computing units. The broadcast request is received by the intra-cluster broadcast engine A1 in the block processing cluster A1.

[0161] When the intra-cluster broadcast engine A1 receives the broadcast request reported by the computing unit, it first extracts the storage address (such as the address of SRAM or DRAM) of the broadcast data and the broadcast mask from the broadcast request, and then reads the broadcast data from the local memory (such as the SRAM of the computing unit or the shared DRAM in the block processing cluster) corresponding to the meta-thread according to the storage address. After reading is completed, the intra-cluster broadcast engine A1 performs intra-cluster broadcast processing in the current block processing cluster A1. In the intra-cluster broadcast processing, the intra-cluster broadcast engine A1 parses the broadcast mask, wherein the broadcast mask is used to accurately identify the target range, so as to avoid redundant transmission caused by full-network broadcast. By combining the identification information of the current block processing cluster and the broadcast mask, the target computing unit (such as computing unit B1) that needs to receive the broadcast data in the current block processing cluster is determined. For example, in the copy mode, the broadcast mask directly indicates whether each computing unit in the current cluster is a target computing unit; in the arbitrary mode, the intra-cluster broadcast engine locates the independent mask segment corresponding to the block processing cluster in the broadcast mask, and then determines the target computing unit according to the mask segment. After the target computing unit is determined, the broadcast data is sent to these target computing units in parallel or in series through the intra-cluster interconnection network (such as a crossbar network, a ring network, or a bus network, etc.). For the intra-cluster interconnection network, if a crossbar network is used, the intra-cluster broadcast engine can simultaneously establish a connection with multiple target computing units, realizing parallel broadcast of data, thereby effectively reducing the intra-cluster broadcast delay; if a ring network is used, the data can be sent to each target computing unit in turn through the transmission of the broadcast packet on the ring. In this process, after the target computing unit receives the broadcast data, it stores the broadcast data in the local designated memory space for subsequent access and processing by the meta-thread.

[0162] After completing the intra-cluster broadcast in the current block processing cluster A1, if there are other target block processing clusters (such as block processing cluster A2) in the same computing die A, the intra-cluster broadcast engine A1 can transmit the broadcast data and the broadcast mask to the intra-cluster broadcast engine A2 of the block processing cluster A2 through the inter-die interconnection network, and the intra-cluster broadcast engine A2 performs intra-cluster broadcast processing according to the broadcast data and the broadcast mask. At the same time, the intra-cluster broadcast engine A starts the cross-die broadcast process. At this time, the intra-cluster broadcast engine A1 will broadcast the broadcast data and the complete broadcast mask through the interface of the inter-die interconnection network to the intra-cluster broadcast engines of the computing die B (for example, to the intra-cluster broadcast engine B1 of the block processing cluster B1 and the intra-cluster broadcast engine B2 of the block processing cluster B2) in parallel. Here, "parallel forwarding" means that the local intra-cluster broadcast engine processes its own intra-cluster broadcast at the same time, or after the local intra-cluster broadcast processing is completed, without waiting for the response of other block processing clusters, the broadcast data and the broadcast mask can be actively pushed out.

[0163] When the intra-cluster broadcast engines within the die B (e.g. the intra-cluster broadcast engine B1 of the block processing cluster B1 and the intra-cluster broadcast engine B2 of the block processing cluster B2) receive the broadcast data and the broadcast mask from the inter-die interconnect network, they will respond and perform the same intra-cluster broadcast processing flow as the initiating intra-cluster broadcast engine A1 within the current block processing cluster B1 based on the information: parsing the broadcast mask, determining the target computing units within the current cluster, and then distributing the broadcast data to these target computing units through the local intra-cluster interconnect network, to achieve the broadcast of the model parameter update value to all target computing units within the block processing cluster B1. Thus, through the hierarchical data transmission mechanism, the broadcast request initiated from one meta-thread first completes data distribution within the local block processing cluster, then expands to other computing dies through the inter-die interconnect network, and continues the intra-cluster broadcast processing in the block processing clusters within other computing dies, to achieve the global synchronization of the model parameter update value in all target computing units within the entire data processor, and to achieve efficient data broadcast in the global range of different levels (block processing cluster, computing die, data processor).

[0164] It is worth noting that the identification field of the broadcast range carried in the broadcast request is a control parameter for identifying the broadcast data distribution boundary. The broadcast range includes three levels: the first is the block processing cluster range, i.e. the broadcast data is only broadcast and distributed to the computing units within the current block processing cluster; the second is the computing die range, i.e. the broadcast data is broadcast and distributed to the computing units of all block processing clusters within the computing die; and the third is the device range, i.e. the broadcast data is broadcast and distributed to all computing units within all computing dies in the device. The intra-cluster broadcast corresponds to the block processing cluster range in the broadcast range, the inter-cluster broadcast corresponds to the computing die range in the broadcast range, and the die broadcast corresponds to the device range in the broadcast range. A specific bit identification bit or field value can be reserved in the broadcast request message to realize the identification field of the broadcast range, so that the intra-cluster broadcast engine can quickly identify the data distribution range after receiving the broadcast request, and trigger the corresponding broadcast operation. For example, when the intra-cluster broadcast engine receives the broadcast request, it parses the broadcast request and identifies the identification bit representing the broadcast range. If the broadcast range is inter-cluster broadcast, i.e. the computing die range, the operation of cross-cluster forwarding is performed, i.e. the local intra-cluster broadcast engine cooperates with the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network, or uploads to the inter-cluster broadcast engine to perform inter-cluster broadcast processing. If the broadcast range is die broadcast, i.e. the device range, the operation of cross-die forwarding is performed, i.e. the local intra-cluster broadcast engine, or the inter-cluster broadcast engine / die broadcast engine after uploading, forwards the broadcast data to the intra-cluster broadcast engines of other computing dies through the inter-die interconnect network.

[0165] As shown in FIG. 13, the intra-cluster broadcast engine A1 of the block processing cluster A1 receives the broadcast request from the inter-die interconnect network, and the broadcast request includes the identification field of the broadcast range. The intra-cluster broadcast engine A1 parses the broadcast request and identifies the identification bit representing the broadcast range. If the broadcast range is intra-cluster broadcast, i.e. the block processing cluster range, the intra-cluster broadcast engine A1 performs the intra-cluster broadcast operation, i.e. the intra-cluster broadcast engine A1 distributes the broadcast data to the target computing units within the block processing cluster A1 through the local intra-cluster interconnect network. If the broadcast range is inter-cluster broadcast, i.e. the computing die range, the intra-cluster broadcast engine A1 performs the inter-cluster broadcast operation, i.e. the intra-cluster broadcast engine A1 forwards the broadcast data to the intra-cluster broadcast engines of other block processing clusters within the computing die through the inter-cluster interconnect network. If the broadcast range is die broadcast, i.e. the device range, the intra-cluster broadcast engine A1 performs the die broadcast operation, i.e. the intra-cluster broadcast engine A1 forwards the broadcast data to the intra-cluster broadcast engines of other computing dies through the inter-die interconnect network. Figure 15 As shown in FIG. 13, the intra-cluster broadcast engine A1 of the block processing cluster A1 receives the broadcast request from the inter-die interconnect network, and the broadcast request includes the identification field of the broadcast range. The intra-cluster broadcast engine A1 parses the broadcast request and identifies the identification bit representing the broadcast range. If the broadcast range is intra-cluster broadcast, i.e. the block processing cluster range, the intra-cluster broadcast engine A1 performs the intra-cluster broadcast operation, i.e. the intra-cluster broadcast engine A1 distributes the broadcast data to the target computing units within the block processing cluster A1 through the local intra-cluster interconnect network. If the broadcast range is inter-cluster broadcast, i.e. the computing die range, the intra-cluster broadcast engine A1 performs the inter-cluster broadcast operation, i.e. the intra-cluster broadcast engine A1 forwards the broadcast data to the intra-cluster broadcast engines of other block processing clusters within the computing die through the inter-cluster interconnect network. If the broadcast range is die broadcast, i.e. the device range, the intra-cluster broadcast engine A1 performs the die broadcast operation, i.e. the intra-cluster broadcast engine A1 forwards the broadcast data to the intra-cluster broadcast engines of other computing dies through the inter-die interconnect network. Figure 15is a schematic diagram of the effect of the cross-computing die broadcast range provided by the embodiments of the present disclosure, assuming that the data processor includes two computing dies, each computing die includes two block processing clusters, and each block processing cluster includes 2 computing units. In the related art, the data broadcast range is limited to all computing units within the same block processing cluster, as shown by the Figure 15 shadowed area. The data broadcast range of the computing units is strictly limited within the block processing cluster to which it belongs and cannot directly reach the computing units of other block processing clusters (i.e., the Figure 15 white area shown). However, by using the data transmission method provided by the embodiments of the present disclosure, the data broadcast range of the computing units is extended to all computing units within the data processor, realizing cross-cluster and cross-computing die data broadcast, as shown by the white area in Figure 15 . The computing unit initiating the broadcast can accurately send the broadcast data to the specified computing unit of the specified target block processing cluster within the specified computing die within the data processor by configuring the broadcast mask (such as the unified broadcast mask in the copy mode or the independent mask segment in the arbitrary mode), thereby extending the data broadcast range from the same block processing cluster to the data broadcast range of all block processing clusters within the same device.

[0166] It should be noted that in the cross-computing die data broadcast scenario, that is, the broadcast request indicates that the broadcast range is inter-die broadcast, there will also be a broadcast mode corresponding to cross-block processing cluster, that is, the copy mode and the arbitrary mode.

[0167] In the copy mode, similarly, each flag bit corresponds to a computing unit within the block processing cluster. At this time, the number of flag bits in the broadcast mask is equal to the number of computing units within the block processing cluster, as shown in Figure 7 . One block processing cluster within the data processor contains 4 computing units, that is, M = 4. When the broadcast mode specified in the broadcast request is the copy mode, the broadcast mask can be set to 4 flag bits. Assuming that the broadcast mask is 1101, at this time, computing unit 0, computing unit 1, and computing unit 3 within all block processing clusters can receive the broadcast data, and only computing unit 2 does not receive the broadcast data. Moreover, the broadcast mask is applied to all block processing clusters, and the inter-die interconnection network is broadcast to the block processing clusters within each computing die by the intra-cluster broadcast engine, so that all block processing clusters perform data broadcast according to the broadcast mask. Assuming that block processing cluster 0 and block processing cluster 1 are within the same computing die 0, and block processing cluster 2 is the block processing cluster of computing die 1, as shown in Figure 7 , block processing cluster 0 in computing die 0, as the block processing cluster initiating the broadcast, uses the same broadcast mask as block processing cluster 1, which is the receiving cluster in the same computing die, while block processing cluster 2 in computing die 1, as the receiving cluster across dies, also receives the broadcast mask 1101 and performs intra-cluster broadcast according to the broadcast mask 1101.

[0168] Specifically, the arbitrary mode refers to that the target compute unit position of each block processing cluster can be independently configured, and the allocation to partial compute units is implemented on demand. In the broadcast process across compute dies, the broadcast mask contains KxNXM flag bits, K represents the number of compute dies involved in the current data processing, N represents the total number of block processing clusters in the current compute die, meanwhile, the broadcast mask is divided into KxN independent mask segments, each independent mask segment corresponds to a block processing cluster, and M represents the number of compute units in one block processing cluster. Each independent mask segment also includes M flag bits, each of which corresponds to each compute unit in the current data processor. For example, the jth independent mask segment in the broadcast mask independently determines whether the compute unit in the jth block processing cluster in the compute die receives the broadcast data. When the broadcast mode is specified as the arbitrary mode in the broadcast request, the broadcast mask is designed to contain KxMxN flag bits corresponding to the compute units one by one, which means that for each compute unit in the data processor, no matter which compute die and block processing cluster it is located in, it has a unique and independently controllable flag bit in the broadcast mask. When the metathread initiates a broadcast request and specifies the arbitrary mode, the system accurately identifies all target compute units according to the broadcast mask of KxMxN length. For example, if a flag bit is set to valid (for example, 1), the corresponding compute unit will be selected as the target compute unit; if it is set to invalid (for example, 0), the compute unit will not receive the broadcast data, realizing fine-grained data broadcast and flexibly selecting any one or any combination of compute units in the data processor as the target, thereby avoiding the limitations of the traditional broadcast mechanism when specifying the target in the global range and improving the efficiency and accuracy of data synchronization in complex distributed computing environments.

[0169] When the broadcast request specifies the broadcast mode as the arbitrary mode, the broadcast mask is organized into KxN independent mask segments, each of which corresponds to a block processing cluster. After receiving the broadcast request reported by the compute unit, the intra-cluster broadcast engine first parses the broadcast mask and the broadcast data, then determines the target compute unit based on the independent mask segment corresponding to the current block processing cluster, and transmits the broadcast data to the target compute unit through the intra-cluster interconnection network. Meanwhile, the intra-cluster broadcast engine that initiates the broadcast broadcasts the broadcast data and the complete broadcast mask to the intra-cluster broadcast engines of other compute dies through the inter-die interconnection network, triggering other clusters to perform corresponding intra-cluster broadcast processing. In the forwarding process, the KxN independent mask segments of the broadcast mask ensure that each target block processing cluster only receives the mask segment content related to it, thereby realizing accurate routing of cross-die broadcast, enabling cross-die broadcast to independently specify the target compute unit set for each block processing cluster, avoiding the limitations of the traditional single-mask structure in the broadcast scenario, and effectively improving the flexibility and execution efficiency of global data synchronization.

[0170] In one example, with reference toFigure 16 , Figure 16 is a schematic diagram of data transmission in any mode provided by the embodiments of the present disclosure. Similarly, the data processor includes 3 computing dies, K=3, and each computing die includes 3 block processing clusters, i.e., N=3, i.e., the data processor includes 9 block processing clusters, and the broadcast mask also includes 9 independent mask segments, and each block processing cluster includes 4 computing units, i.e., M=4, and each independent mask segment corresponds to a block processing cluster. When the broadcast mode specified in the broadcast request is any mode, the broadcast mask sets 36 flag bits. Assuming that the broadcast mask is 0011, 1100, 1100, 1010, 0101, 1101, 1001, 0110, 1010. As can be seen, the independent mask segment 0 corresponding to the block processing cluster 0 in the computing die 0 is 0011, the independent mask segment 1 corresponding to the block processing cluster 1 is 1100, and the independent mask segment 2 corresponding to the block processing cluster 2 is 1100, i.e., the computing unit 2 and the computing unit 3 in the block processing cluster 0 receive broadcast data, the computing unit 0 and the computing unit 1 in the block processing cluster 1 receive broadcast data, and the computing unit 0 and the computing unit 1 in the block processing cluster 2 receive broadcast data, and the remaining computing units do not receive broadcast data. The independent mask segment 3 corresponding to the block processing cluster 3 in the computing die 1 is 1010, the independent mask segment 4 corresponding to the block processing cluster 4 is 0101, and the independent mask segment 5 corresponding to the block processing cluster 5 is 1101. The independent mask segment 6 corresponding to the block processing cluster 6 in the computing die 2 is 1001, the independent mask segment 7 corresponding to the block processing cluster 7 is 0110, and the independent mask segment 8 corresponding to the block processing cluster 8 is 1010. It is worth noting that in any mode, the flag bits of some independent mask segments can be the same or different. Therefore, the intra-cluster broadcast engine transmits the broadcast data to the specified target computing unit according to the received corresponding independent mask segment content, thereby realizing accurate cross-cluster data synchronization.

[0171] Therefore, the length of the broadcast mask can be used to determine different broadcast range levels. If the length of the broadcast mask is only MxN bits, it can be determined that the broadcast range indicated by the broadcast request is inter-cluster broadcast, i.e., covering the computing units of other block processing clusters in the same computing die. If the length of the broadcast mask is KxMxN bits, it can be determined that the broadcast range indicated by the broadcast request is inter-die broadcast, i.e., covering the computing units of other computing dies.

[0172] It can be understood that, in processing large-scale artificial intelligence model parallel training, the conventional technology is difficult to directly implement global broadcast across clusters or even across dies, which seriously affects the parallel expansion capability of the artificial intelligence model. For example, in the above-mentioned model parameter update scenario, if only relying on a single level of broadcast mechanism, it will be difficult to efficiently synchronize the update value to all computing units distributed in different block processing clusters or even different computing dies. The present disclosure adopts a hierarchical broadcast mechanism. Specifically, first, the intra-cluster broadcast engine and the intra-cluster interconnection network are used to realize efficient local data distribution within the block processing cluster, and then the broadcast request and data are expanded to other computing dies through the inter-die interconnection network, and the intra-cluster broadcast processing is continued in the intra-cluster broadcast engine within the other computing dies. This hierarchical design enables data to spread from the meta-thread to all target computing units within the entire data processor. Compared with the simple point-to-point transmission or global flooding broadcast in the conventional technology, for example, in the above-mentioned model parameter update example, if point-to-point transmission is used, each computing unit needs to send or receive data individually, resulting in huge communication overhead; if global flooding is used, a large number of redundant data packets may be generated, increasing the network burden. Through the broadcast mask mechanism and the hierarchical broadcast architecture, the present disclosure can dynamically specify the target range according to the broadcast mask, and data is only distributed to the required target computing units, rather than directly broadcasted to the entire network. Combined with the low-delay high-rate transmission characteristics of the intra-cluster interconnection network and the distributed forwarding capability of the inter-die interconnection network, the global broadcast operation is decomposed into multiple parallel intra-cluster broadcast processes, and the intra-cluster broadcast engine of each block processing cluster can independently analyze the mask and perform local intra-cluster broadcast processing operation. Therefore, the communication bottleneck across dies is effectively alleviated, efficient data synchronization is achieved, and the redundant transmission problem caused by blind broadcast in the conventional technology is avoided.

[0173] In one example, with reference to Figure 17 , Figure 17is a schematic diagram of the effect of the inter-die broadcast provided by the embodiments of the present disclosure, assuming that a device contains two computing dies, each of which is configured with two block processing clusters, one computing die includes block processing cluster 0 and block processing cluster 1 respectively, and the other computing die includes block processing cluster 2 and block processing cluster 3. Each block processing cluster contains a computing unit and an intra-cluster broadcast engine, and the computing die is also configured with an inter-cluster interconnection network. When a meta-thread of a computing unit in block processing cluster 0 initiates a broadcast request to distribute broadcast data to specific computing units in block processing cluster 0, block processing cluster 2 and block processing cluster 3, the intra-cluster broadcast engine of block processing cluster 0 will respond to the broadcast request and obtain the broadcast data and the broadcast mask. First, the intra-cluster broadcast engine will broadcast the broadcast data to the target computing units in block processing cluster 0 through the intra-cluster interconnection network within block processing cluster 0. At the same time, the intra-cluster broadcast engine recognizes that the broadcast range also includes block processing clusters 2 and 3, so it can broadcast the broadcast data and the broadcast mask to the intra-cluster broadcast engine in another computing die block processing cluster 2 through the inter-die interconnection network. After receiving this information, the intra-cluster broadcast engine of block processing cluster 2 will perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask to broadcast the data to the target computing units in block processing cluster 2. At this time, the intra-cluster broadcast engine of block processing cluster 2 also recognizes that the broadcast also includes block processing cluster 3, so it can forward the broadcast data and the broadcast mask to the intra-cluster broadcast engine in block processing cluster 3 through the inter-cluster interconnection network. After receiving this information, the intra-cluster broadcast engine of block processing cluster 3 will perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask to broadcast the data to the target computing units in block processing cluster 3; or, after recognizing that the broadcast range also includes block processing clusters 2 and 3, the intra-cluster broadcast engine of block processing cluster 0 can broadcast the broadcast data and the broadcast mask to the intra-cluster broadcast engines in block processing cluster 2 and block processing cluster 3 in another computing die through the inter-die interconnection network, without the need for block processing cluster 2 to perform intra-die forwarding. In this process, the inter-cluster interconnection network can be a high-speed data bus inside the computing die, or a dedicated on-chip network channel, ensuring that data between block processing clusters can be quickly and directly transmitted. Therefore, the inter-cluster interconnection network in the computing die can provide a direct and efficient communication path between block processing clusters in the same computing die, and the intra-cluster broadcast engine can use this network to directly forward the broadcast data and the broadcast mask to the intra-cluster broadcast engine of other block processing clusters in the same computing die. In the process of cross-computing die broadcast, the additional delay and resource consumption caused by indirect communication through the inter-die interconnection network can be reduced, the data broadcast efficiency can be improved, and the parallel processing capability of the data processor can be optimized.

[0174] In some possible implementations, the computing die of the data processor can also be provided with an inter-cluster broadcast engine, in which case the intra-cluster broadcast engine, in response to a broadcast request initiated by a meta-thread in the computing unit, acquires broadcast data and a broadcast mask; the intra-cluster broadcast engine performs intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask; the intra-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine through the inter-cluster interconnect network; the inter-cluster broadcast engine forwards the broadcast data and the broadcast mask to other inter-cluster broadcast engines in other computing dies through the inter-die interconnect network, so that each inter-cluster broadcast engine performs inter-cluster broadcast processing; wherein the inter-cluster broadcast processing includes the inter-cluster broadcast engine issuing the broadcast data and the broadcast mask to the intra-cluster broadcast engine in the same computing die through the inter-cluster interconnect network, so that the intra-cluster broadcast engine performs intra-cluster broadcast processing.

[0175] The inter-cluster broadcast engine can be a dedicated hardware or software module configured inside the computing die, which can manage and coordinate the broadcast data transmission inside the computing die and between other computing dies. As an intermediate layer between the intra-cluster broadcast engine and the inter-die interconnect network, the inter-cluster broadcast engine can be responsible for receiving a broadcast request from the intra-cluster broadcast engine in the same die and forwarding the broadcast request to other computing dies through the inter-die interconnect network, or receiving a broadcast request from other computing dies and distributing it to the intra-cluster broadcast engine in the same die.

[0176] The inter-cluster broadcast engine can be a piece of dedicated logic circuit on the computing die, such as a functional unit in a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), or a specific software service running on one of the general-purpose processor cores in the computing die.

[0177] Specifically, after receiving the broadcast request, the intra-cluster broadcast engine sends the data to be broadcast and the broadcast mask indicating the target computing unit to the inter-cluster broadcast engine in the same computing die through the inter-cluster interconnect network inside the computing die, and promotes the broadcast task from the local (block processing cluster) to the die level, which is scheduled and managed by the higher-level inter-cluster broadcast engine. After receiving the broadcast data and the broadcast mask, the inter-cluster broadcast engine can send these information to the inter-cluster broadcast engine in the target computing die through the inter-die interconnect network. The inter-die interconnect network is a high-speed communication link connecting different computing dies, which is used to exchange data across dies. The broadcast request is diffused to other computing dies that need to receive data through the inter-die interconnect network, and further processed by the inter-cluster broadcast engine of each computing die. The forwarding process of the broadcast request can be based on routing table addressing, and the data packet transmission on the inter-die interconnect network can be based on point-to-point or multicast mechanism.

[0178] Specifically, the inter-cluster broadcast processing refers to a processing procedure started by each target computing die's inter-cluster broadcast engine after receiving the broadcast data and the broadcast mask, is a process of coordinating and distributing the broadcast task at the die level by the inter-cluster broadcast engine, and is implemented by executing the inter-cluster broadcast processing by the inter-cluster broadcast engine to broadcast the broadcast data to the related intra-cluster broadcast engine in the current computing die. The inter-cluster broadcast processing further includes that the inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engine in the same computing die through the inter-cluster interconnection network, that is, the inter-cluster broadcast engine distributes the received broadcast data and the broadcast mask to one or more intra-cluster broadcast engines in the current computing die through the inter-cluster interconnection network in the computing die, and that the die-level broadcast task is refined to the cluster level, and the intra-cluster broadcast engine is responsible for the data distribution in the cluster.

[0179] It can be understood that, with reference to Figure 18 , Figure 18is a flowchart of data transmission of inter-die broadcast provided by another embodiment of the disclosure. In the data processor, in order to realize efficient data transmission, especially in the scene of global broadcast across the computing die, each computing die is configured with an inter-cluster interconnection network and an inter-cluster broadcast engine in addition to a plurality of block processing clusters, an intra-cluster interconnection network, an intra-cluster broadcast engine, and a plurality of computing units. When the meta-thread in the computing unit initiates a broadcast request, the intra-cluster broadcast engine responds to the broadcast request and obtains broadcast data and a broadcast mask. Then, instead of forwarding the data to other computing dies through the inter-die interconnection network, the intra-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine in the same computing die through the inter-cluster interconnection network. At this time, the inter-cluster broadcast engine acts as a die-level broadcast coordinator, and after receiving the uploaded broadcast data and the broadcast mask, it forwards the broadcast data and the broadcast mask to the inter-cluster broadcast engine in other target computing dies through the inter-die interconnection network, that is, the complex cross-die communication is separated from the intra-cluster broadcast engine and is specially responsible for the inter-cluster broadcast engine, thereby optimizing the communication path between the dies. Each inter-cluster broadcast engine that receives the broadcast data performs inter-cluster broadcast processing, and the inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the related intra-cluster broadcast engine in the same computing die through the inter-cluster interconnection network, and these intra-cluster broadcast engines that receive the broadcast data and the broadcast mask perform corresponding intra-cluster broadcast processing based on the broadcast data and the broadcast mask, that is, determine the target computing unit in the current block processing cluster based on the broadcast mask, and broadcast the broadcast data to these target computing units through the intra-cluster interconnection network. Therefore, by introducing the inter-cluster broadcast engine, the global broadcast task is divided into three levels of block processing cluster, computing die and device. The intra-cluster broadcast engine processes the broadcast in the block processing cluster it is in, and the inter-cluster broadcast engine is responsible for the data coordination between the block processing clusters in the computing die it is in and between the current computing die and other computing dies, thereby avoiding the intra-cluster broadcast engine directly processing complex cross-die communication, reducing the redundancy of the communication path, reducing the delay of data transmission, and effectively saving communication resources.

[0180] In one example, assume a device contains four compute dies, each compute die contains two block processing clusters, and each block processing cluster contains four compute units. When a meta-thread within one of the compute units needs to broadcast data to a specific set of compute units in the entire data processor, the meta-thread initiates a broadcast request to the intra-cluster broadcast engine in the cluster where the meta-thread resides and provides the broadcast data and a broadcast mask. The intra-cluster broadcast engine receives the request and first performs an intra-cluster broadcast process to broadcast the data to the target compute units in the block processing cluster where the intra-cluster broadcast engine resides. Meanwhile, the intra-cluster broadcast engine uploads the broadcast data and the broadcast mask to an inter-cluster broadcast engine in the same compute die through an inter-cluster interconnect network, such as a specific channel of a network-on-chip (NoC). The inter-cluster broadcast engine receives the data and parses the broadcast mask to determine which other compute dies need to forward the data. For example, if the broadcast mask indicates that the target compute units are distributed in all four compute dies, the inter-cluster broadcast engine forwards the broadcast data and the broadcast mask to the inter-cluster broadcast engines in the other three compute dies through an inter-die interconnect network, such as a high-speed serial link. When the inter-cluster broadcast engines in the other compute dies receive the broadcast data and the broadcast mask, they can perform an inter-cluster broadcast process. Specifically, each receiving inter-cluster broadcast engine determines which block processing clusters in the compute die where the inter-cluster broadcast engine resides contain the target compute units according to the broadcast mask, and then the inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines in the target block processing clusters through the inter-cluster interconnect network in the compute die. The intra-cluster broadcast engines that receive the data each perform an intra-cluster broadcast process in the block processing cluster where the intra-cluster broadcast engine belongs to distribute the broadcast data to the target compute units in the cluster. Thus, by uploading the broadcast task from the intra-cluster broadcast engine to the inter-cluster broadcast engine, forwarding the broadcast task across dies by the inter-cluster broadcast engine through the inter-die interconnect network, and then distributing the broadcast task to the intra-cluster broadcast engines by the inter-cluster broadcast engine in the target compute die, the hierarchical concurrent processing of the broadcast task is achieved, the data transmission path is optimized, the unnecessary communication overhead is reduced, and the delay of global data synchronization is reduced.

[0181] In some possible implementations, in the process that the inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines in the same compute die through the inter-cluster interconnect network, the inter-cluster broadcast engine can determine target block processing clusters in the current compute die that contain the target compute units based on the broadcast mask; and the inter-cluster broadcast engine distributes the broadcast data to the intra-cluster broadcast engines in the target block processing clusters.

[0182] The broadcast mask is a bitmap or a data structure indicating target receivers of the broadcast data, and encodes which compute units or block processing clusters should receive the broadcast data. Specifically, the broadcast mask can be a binary bit string, where each bit corresponds to a specific compute unit, and a bit value of 1 indicates that the compute unit is a target receiver, and a bit value of 0 indicates that the compute unit is not a target. The inter-cluster broadcast engine determines target block processing clusters containing target compute units in the current compute die based on the broadcast mask, identifies clusters in which target compute units actually need to receive the broadcast data in the current compute die, and avoids sending data to irrelevant clusters. The inter-cluster broadcast engine can parse the broadcast mask. For example, if the broadcast mask is in the form of a bitmap, the inter-cluster broadcast engine can check bit segments corresponding to each block processing cluster in the current compute die. If there is at least one bit with a value of 1 in a bit segment, it indicates that the corresponding block processing cluster contains a target compute unit, and thus the block processing cluster is determined as a target block processing cluster. Alternatively, the inter-cluster broadcast engine can maintain a mapping table recording the range or identifier of compute units contained in each block processing cluster. When receiving the broadcast mask, the inter-cluster broadcast engine can query the mapping table according to target compute unit identifiers in the broadcast mask, thereby determining block processing clusters containing the target compute units. The inter-cluster broadcast engine delivers the broadcast data to the intra-cluster broadcast engine of the target block processing cluster, ensuring that the broadcast data is only sent to block processing clusters that have been determined to contain target compute units, thereby achieving precise data broadcasting. The inter-cluster broadcast engine can send the broadcast data and the broadcast mask directly to the intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnect network according to the identifier of the determined target block processing cluster. Specifically, the inter-cluster broadcast engine can include address information of the target cluster in the data packet header to achieve this. Alternatively, the inter-cluster broadcast engine can use the routing function of the inter-cluster interconnect network to set a specific routing path for the broadcast data packet, so that the broadcast data packet can only reach the intra-cluster broadcast engine of the target block processing cluster, but not broadcast to all clusters. This achieves fine-grained management on the basis of the inter-cluster broadcast mechanism, making data transmission more accurate and efficient.

[0183] In one example, assume that a compute die contains four block processing clusters, namely block processing cluster A, block processing cluster B, block processing cluster C and block processing cluster D. Each block processing cluster contains multiple compute units. When the inter-cluster broadcast engine receives a broadcast request forwarded by another inter-cluster broadcast engine, which contains broadcast data and a broadcast mask, the broadcast mask can be a bitmap, in which specific bit segments correspond to each block processing cluster in the compute die. The inter-cluster broadcast engine parses the broadcast mask, for example, if the bit segments of the broadcast mask indicate that there are target compute units in block processing cluster C and block processing cluster D, but not in block processing cluster A and block processing cluster B, then the inter-cluster broadcast engine will determine block processing cluster C and block processing cluster D as target block processing clusters, and the inter-cluster broadcast engine sends the broadcast data and the broadcast mask to the intra-cluster broadcast engine of block processing cluster C and the intra-cluster broadcast engine of block processing cluster D through the inter-cluster interconnect network, but not to the intra-cluster broadcast engines of block processing cluster A and block processing cluster B. After the intra-cluster broadcast engines of block processing cluster C and block processing cluster D receive the data, they perform intra-cluster broadcast processing based on the broadcast mask in their respective clusters to distribute the data to the target compute units in their clusters, avoiding sending broadcast data to block processing clusters that do not contain target compute units, and reducing unnecessary data transmission volume in the inter-cluster interconnect network.

[0184] In some possible implementations, the inter-cluster broadcast engine, in response to the broadcast data and the broadcast mask sent by another inter-cluster broadcast engine through the inter-die interconnect network, distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engine in the same compute die; the inter-cluster broadcast engine determines a plurality of target compute dies containing target compute units based on the broadcast mask; and the inter-cluster broadcast engine determines an adaptive compute die matching the compute die to which the inter-cluster broadcast engine belongs from the plurality of target compute dies based on a preset broadcast rule, and forwards the broadcast mask and the broadcast data to the inter-cluster broadcast engine in the adaptive compute die.

[0185] Wherein, in the cluster-to-cluster broadcast engine of the initiating block processing cluster, based on which computing dies need to participate in this broadcast identified by the broadcast engine, the broadcast mask usually contains detailed information about the target computing unit, such as the global serial number of the computing unit or the logical address in the entire system, and the cluster-to-cluster broadcast engine can identify all computing dies containing at least one target computing unit by analyzing the broadcast mask, thereby forming a target computing die set. Based on the preset broadcast rule, the adaptive computing die matching the current computing die is determined in the target computing die set, and the preset broadcast rule can be based on various strategies, for example, the forwarding path can be defined according to the topology of the inter-die interconnection network (such as mesh, ring, tree, etc.), and the die with the shortest physical distance or the least network hop is preferentially selected as the next broadcast target. Alternatively, the broadcast rule can also consider network load balancing, and select the die with lower network congestion level for forwarding to avoid hotspots and delays, for example, the rule can specify that the next hop die is selected according to a specific grouping order, a chain delivery order, or based on load conditions.

[0186] Specifically, when an inter-cluster broadcast engine receives broadcast data and a broadcast mask (whether it is uploaded from the local intra-cluster broadcast engine or forwarded from an inter-cluster broadcast engine on another computing die), the inter-cluster broadcast engine will respond to the broadcast request and distribute the data and mask to the relevant intra-cluster broadcast engine in the computing die to which it belongs, ensuring that the target computing unit (such as a non-initiating computing unit) in the current computing die can timely receive the broadcast data, thereby realizing local intra-cluster broadcast processing. At the same time, or thereafter, the inter-cluster broadcast engine will analyze the broadcast mask to identify all computing dies containing target computing units, forming a target computing die set. In order to avoid blindly forwarding broadcast data to all these target computing dies, the inter-cluster broadcast engine can select one or more "adaptive computing dies" as the next broadcast target from the target computing die set based on the preset broadcast rule, and then forward the broadcast data and the broadcast mask to the inter-cluster broadcast engine in the adaptive computing die, so that the broadcast data can be transmitted between computing dies along the preset optimized path, thereby reducing the load of the inter-die interconnection network.

[0187] In one example, with reference to Figure 19 , Figure 19is a schematic diagram of the effect of inter-die broadcast provided by another embodiment of the present disclosure, assuming a device contains four compute dies connected in a mesh topology through an inter-die interconnect network. Each compute die contains two block processing clusters, and each block processing cluster contains four compute units. When the intra-cluster broadcast engine 0 on the compute die 0 initiates a broadcast request and uploads to the inter-cluster broadcast engine 0, the inter-cluster broadcast engine 0 receives a copy of the broadcast data and the broadcast mask, and first distributes the data to the relevant intra-cluster broadcast engines within the compute die 0 (i.e., the intra-cluster broadcast engine 0 and the intra-cluster broadcast engine 1). Then, the inter-cluster broadcast engine parses the broadcast mask and finds that, in addition to the compute die 0, the compute die 1, the compute die 2, and the compute die 3 also contain target compute units. At this time, the preset broadcast rule can be a "unified forwarding" strategy, that is, the inter-cluster broadcast engine 0 of the compute die 0 as the only sender needs to forward the broadcast data and the broadcast mask to the inter-cluster broadcast engine 1 of the compute die 1, the inter-cluster broadcast engine 2 of the compute die 2, and the inter-cluster broadcast engine 3 of the compute die 3, as shown in the upper (a) part of Figure 19 ; or, as shown in the middle (b) part of Figure 19 , the broadcast rule can be a "nearest chain forwarding" strategy, that is, the inter-cluster broadcast engine 0 of the compute die 0 determines according to the broadcast rule that the compute die 1 is the nearest target die that has not received the broadcast data, and then forwards the broadcast data and the broadcast mask to the inter-cluster broadcast engine 1 of the compute die 1. After the inter-cluster broadcast engine 1 of the compute die 1 receives the data, it also first performs local distribution, and then determines according to the broadcast rule that the compute die 2 is the adaptive compute die and continues to forward. This process continues until all target compute dies receive the broadcast data; or, as shown in the lower (c) part of Figure 19As shown in the lower side (c) part of FIG. 1, the broadcast rule can be a "group forwarding" strategy. According to the broadcast rule, the inter-cluster broadcast engine 0 of the computing die 0 selects the computing die 1 in the same group and not having received the broadcast data as the adaptive computing die, and forwards the broadcast data and the broadcast mask to the inter-cluster broadcast engine 1 of the computing die 1. After the inter-cluster broadcast engine 1 of the computing die 1 receives the data, it also performs local distribution first. Then, the computing die 2 and the computing die 3 in the target computing dies have not received the broadcast data, and the computing die 2 and the computing die 3 belong to the same group. Therefore, the inter-cluster broadcast engine 0 of the computing die 0 selects the computing die 2 in the same group as the adaptive computing die according to the broadcast rule, and forwards the broadcast data and the broadcast mask to the inter-cluster broadcast engine 2 of the computing die 2. After the inter-cluster broadcast engine 2 of the computing die 2 receives the data, it also performs local distribution first, and then forwards the broadcast data to the computing die 4 in the same group and not having received the broadcast data as the adaptive computing die according to the broadcast rule. Therefore, when the broadcast data needs to be sent to multiple target computing dies, the broadcast path can be dynamically selected based on the preset broadcast rule. For example, the broadcast data can be transmitted according to the group, can be transmitted according to the order, or can be transmitted according to the direction, and the die with a short distance or a low load is preferentially matched to optimize the forwarding efficiency and reduce the delay and congestion risk.

[0188] In some possible implementation manners, the data processor further includes an inter-die broadcast engine. In the process of forwarding the broadcast data and the broadcast mask to the inter-cluster broadcast engine in other computing dies through the inter-die interconnection network by the inter-cluster broadcast engine, the inter-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-die broadcast engine through the inter-die interconnection network. Then, the inter-die broadcast engine downloads the broadcast data and the broadcast mask to the inter-cluster broadcast engine in other computing dies through the inter-die interconnection network.

[0189] The inter-die broadcast engine is a hardware or software module for coordinating and managing cross-die broadcast operations, which can be responsible for receiving broadcast requests from the inter-cluster broadcast engines in each computing die, and uniformly distributing the broadcast data and broadcast mask to the target computing dies. The inter-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-die broadcast engine through the inter-die interconnection network, that is, the broadcast data and the broadcast mask are aggregated from the inter-cluster broadcast engine in the computing die initiating the broadcast to the inter-die broadcast engine, so as to concentrate the broadcast information, facilitating the subsequent unified scheduling and broadcast of the inter-die broadcast engine. Specifically, the transmission can be performed through a specific message type or protocol in the inter-die interconnection network, for example, the inter-cluster broadcast engine encapsulates the broadcast data and the broadcast mask into a specific broadcast message packet, and sends it to the specified address of the inter-die broadcast engine through the inter-die interconnection network. The inter-die broadcast engine distributes the broadcast data and the broadcast mask to the inter-cluster broadcast engines in other computing dies through the inter-die interconnection network, that is, the inter-die broadcast engine distributes the concentrated broadcast information to the inter-cluster broadcast engines in other target computing dies to realize global broadcast. Specifically, the inter-die broadcast engine can send the broadcast message packet to the inter-cluster broadcast engines in the target computing dies according to the target die information parsed from the broadcast mask, or use the multicast / broadcast capability of the inter-die interconnection network for efficient distribution.

[0190] It can be understood that, with reference to Figure 20 , Figure 20 is a flow diagram of data transmission in inter-die broadcast provided by another embodiment of the present disclosure. When the meta-thread in the computing unit initiates a broadcast request, the intra-cluster broadcast engine obtains the broadcast data and the broadcast mask, and first uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine in the computing die to which it belongs through the inter-cluster interconnection network. The inter-cluster broadcast engine also first uploads the broadcast data and the broadcast mask to the inter-die broadcast engine in the data processor through the inter-die interconnection network. The inter-die broadcast engine will distribute the broadcast data and the broadcast mask to the inter-cluster broadcast engines in each target computing die (including the target computing die initiating the broadcast) based on the broadcast mask through the inter-die interconnection network. These inter-cluster broadcast engines receiving the data will further perform inter-cluster broadcast processing, that is, distribute the data and the mask to the intra-cluster broadcast engines in the computing die to which they belong through the inter-cluster interconnection network, and then complete the broadcast to the target computing unit by the intra-cluster broadcast engine. Therefore, by adding the inter-die broadcast engine as an intermediary node, the data flow across the dies is uniformly managed and optimized, centralized data broadcast is realized, conflicts between multiple inter-cluster broadcast engines are avoided, and data broadcast delay and reliability are improved.

[0191] It can be understood that, with reference to Figure 21 , Figure 21is a schematic diagram of the effect of the inter-die broadcast provided by another embodiment of the present disclosure, assuming that a device contains four compute dies, which are connected in a mesh topology through an inter-die interconnect network. Each compute die contains two block processing clusters, and each block processing cluster contains four compute units. When a meta-thread of a compute unit initiates a broadcast request, the intra-cluster broadcast engine in the cluster where the meta-thread is located uploads the received broadcast request to the inter-cluster broadcast engine through the inter-cluster interconnect network, and then uploads the broadcast request to the inter-die broadcast engine through the inter-die interconnect network by the inter-cluster broadcast engine. The inter-die broadcast engine can receive broadcast requests from the inter-cluster broadcast engines in each compute die, and uniformly distribute the broadcast requests to the inter-cluster broadcast engines in each compute die through the inter-die interconnect network, and then broadcast the broadcast requests to the intra-cluster broadcast engines of each block processing cluster by the inter-cluster broadcast engines in each compute die, so as to realize unified coordination and scheduling of die broadcast by the inter-die broadcast engine.

[0192] In some possible implementations, a broadcast mask and a data storage address are parsed from the broadcast request in response to the broadcast request initiated by the meta-thread in the compute unit; and the broadcast data is read from the local memory of the compute unit based on the data storage address. Specifically, the meta-thread can send a broadcast request to the intra-cluster broadcast engine by executing a specific instruction or calling a predefined application programming interface (API). Alternatively, the meta-thread can write relevant information of the broadcast request to a preset shared memory area, and the intra-cluster broadcast engine detects and responds to the broadcast request by polling the area or receiving an interrupt signal. The broadcast request can be a structured data packet, and the broadcast mask and the data storage address are represented by predefined fields, so that the intra-cluster broadcast engine can efficiently extract the required information by parsing the header and corresponding fields of the data packet.

[0193] The data storage address can be an identifier of a storage location of the broadcast data in the local memory, and can be implemented by a memory address pointer, which can be used as a lightweight pointer to point to the data location, so as to avoid carrying redundant data in the request. The local memory can be an internal or adjacent high-speed memory of the compute unit, and can be implemented by an SRAM or a cache. Specifically, the intra-cluster broadcast engine directly accesses the local memory of the compute unit that initiates the request through an internal memory controller, and reads the broadcast data of a specified length according to the parsed data storage address. Alternatively, the local memory of the compute unit can be mapped to the address space of the intra-cluster broadcast engine, so as to allow the intra-cluster broadcast engine to directly read the required data through a memory mapping mechanism.

[0194] It can be understood that when the meta-thread in the computing unit needs to initiate a broadcast, the meta-thread directly sends a broadcast request to the intra-cluster broadcast engine, and the intra-cluster broadcast engine receives the broadcast request and parses the broadcast mask and the data storage address from the broadcast request. The data storage address can be extracted based on the predefined field structure of the request message. Then, the intra-cluster broadcast engine directly reads the broadcast data from the local memory of the computing unit that initiates the broadcast by using the parsed data storage address. Therefore, the broadcast data acquisition process is separated into two independent links of storage address parsing and local reading, so that the request message only needs to transmit lightweight mask and address information, rather than complete data blocks, thereby significantly reducing the amount of data transmitted through the inter-cluster interconnection network, and avoiding repeated data transmission by using the low-delay characteristics of the local memory, thereby ensuring the efficiency of the broadcast operation.

[0195] In one example, when the AI accelerator performs a distributed training task, when the meta-thread in the computing unit needs to broadcast gradient data, the broadcast request contains a broadcast mask (for example, 0x11 represents all computing units in the current device) and a data storage address (for example, a starting address pointing to a gradient buffer in the local memory); After the intra-cluster broadcast engine parses the broadcast request, the intra-cluster broadcast engine can read the broadcast data of a specified length from the local cache (such as L1 or L2 cache) or local static random access memory (SRAM) of the computing unit to the internal buffer of the intra-cluster broadcast engine through the direct memory access controller, and then determine the target computing unit based on the broadcast mask and perform the intra-cluster broadcast processing. Therefore, the message size of the broadcast request can be effectively reduced, the communication burden of the inter-cluster interconnection network can be reduced, the broadcast efficiency can be improved, and the large-scale data transmission demand of parameter synchronization in the AI training scene can be met.

[0196] It can be understood that although each step in each of the above flowcharts is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified in this embodiment, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least part of the steps in the above flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0197] Referring to Figure 22 , Figure 22An optional structure diagram of the broadcast engine 2200 provided by the embodiments of the present disclosure is provided, which is applied to a data processor including a plurality of block processing clusters, and an inter-cluster interconnection network for communication transmission between the plurality of block processing clusters, the block processing cluster including the broadcast engine 2200, a plurality of computing units, and an intra-cluster interconnection network for communication transmission between the plurality of computing units, and each computing unit is configured with a meta-thread broadcast engine 2200, which includes:

[0198] The data acquisition unit 2210 is configured to acquire the broadcast data and the broadcast mask in response to the broadcast request initiated by the meta-thread in the computing unit.

[0199] The intra-cluster broadcast unit 2220 is configured to perform the intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask.

[0200] The inter-cluster broadcast unit 2230 is configured to forward the broadcast data and the broadcast mask to the intra-cluster broadcast engine of the other block processing cluster through the inter-cluster interconnection network, so as to trigger the other intra-cluster broadcast engine to perform the intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0201] In some possible implementation manners, the inter-cluster broadcast unit 2230 is further configured to determine the target computing unit in the current block processing cluster based on the broadcast mask; and broadcast the broadcast data to the target computing unit through the intra-cluster interconnection network.

[0202] In some possible implementation manners, the inter-cluster broadcast unit 2230 is further configured to determine the target block processing cluster containing the target computing unit based on the broadcast mask; and forward the broadcast data and the broadcast mask to the intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnection network.

[0203] In some possible implementation manners, the inter-cluster broadcast unit 2230 is further configured to determine the target block processing cluster containing the target computing unit based on each independent mask segment; take the independent mask segment corresponding to the target block processing cluster as a target mask; and forward the broadcast data and the target mask to the intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnection network.

[0204] In some possible implementation manners, the intra-cluster broadcast unit 2220 is further configured to perform the intra-cluster broadcast processing in the current block processing cluster based on the broadcast data and the broadcast mask in response to the broadcast data and the broadcast mask sent by the other intra-cluster broadcast engine.

[0205] In some possible implementation manners, the inter-cluster broadcast unit 2230 is further configured to broadcast the broadcast data and the broadcast mask to the intra-cluster broadcast engine in the other computing die through the inter-die interconnection network in response to the broadcast range indicated by the broadcast request being inter-die broadcast, so as to trigger the intra-cluster broadcast engine in the other computing die to perform the intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0206] In some possible implementation manners, the inter-cluster broadcast engine is arranged in the inter-cluster interconnection network or is arranged close to the inter-cluster interconnection network.

[0207] In a possible implementation manner, the intra-cluster broadcast unit 2220 is further configured to perform, in response to receiving the broadcast data and the broadcast mask broadcast by the other inter-cluster broadcast engine, intra-cluster broadcast processing in the current block based on the broadcast data and the broadcast mask.

[0208] The inter-cluster broadcast unit 2230 is further configured to broadcast the broadcast data and the broadcast mask to other broadcast engines in the same computing die, so as to trigger the broadcast engines in other computing dies to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

[0209] In a possible implementation manner, the intra-cluster broadcast unit 2220 is further configured to upload the broadcast data and the broadcast mask to the other broadcast engine.

[0210] The inter-cluster broadcast unit 2230 is further configured to broadcast the broadcast data and the broadcast mask to the broadcast engine of the other computing die, so as to enable the respective broadcast engine to perform inter-cluster broadcast processing.

[0211] The inter-cluster broadcast processing comprises that the broadcast engine distributes the broadcast data and the broadcast mask to the broadcast engine in the same computing die, so as to enable the broadcast engine to perform intra-cluster broadcast processing.

[0212] In a possible implementation manner, the inter-cluster broadcast unit 2230 is further configured to determine, based on the broadcast mask, a target block processing cluster containing a target computing unit in the current computing die, and distribute the broadcast data to the broadcast engine of the target block processing cluster.

[0213] In a possible implementation manner, the intra-cluster broadcast unit 2220 is further configured to broadcast, in response to the broadcast data and the broadcast mask broadcast by the other broadcast engine, the broadcast data and the broadcast mask to all broadcast engines in the same computing die.

[0214] The inter-cluster broadcast unit 2230 is further configured to determine, based on the broadcast mask, a plurality of target computing dies containing a target computing unit, and determine, based on a preset broadcast rule, an adaptive computing die matching the computing die to which the current computing die belongs from the plurality of target computing dies, and broadcast the broadcast mask and the broadcast data to the broadcast engine of the adaptive computing die.

[0215] In a possible implementation manner, the inter-cluster broadcast unit 2230 is further configured to upload the broadcast data and the broadcast mask to the broadcast engine through the inter-die interconnection network, or distribute the broadcast data and the broadcast mask to the broadcast engine in the other computing die through the inter-die interconnection network.

[0216] In a possible implementation, the data obtaining unit 2210 is further configured to parse, in response to a broadcast request initiated by an intra-unit thread of the computing unit, a broadcast mask and a data storage address from the broadcast request; and read the broadcast data from a local memory of the computing unit based on the data storage address.

[0217] In a possible implementation, the intra-cluster broadcasting unit 2220 is further configured to determine, by the intra-cluster broadcasting engine, target computing units in a current block processing cluster based on the broadcast mask; and broadcast the broadcast data to the target computing units.

[0218] It should be noted that the broadcast engine can be the intra-cluster broadcasting engine mentioned above, or the inter-cluster broadcasting engine, or the inter-die broadcasting engine.

[0219] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which includes a memory, a processor, a program stored in the memory and capable of running on the processor, and a data bus used for realizing connection communication between the processor and the memory. The program is run by the processor to realize the data transmission method in the first aspect.

[0220] The embodiments of the present disclosure further provide an electronic device, which includes at least one processor, and a memory connected with the at least one processor in communication. The processor and the memory are connected in communication through a data bus. The memory stores a program. The program is executed by the at least one processor to enable the at least one processor to execute instructions to realize the method in any one of the above embodiments of the present disclosure.

[0221] The following will be described in combination with Figure 23 The hardware structure of the electronic device will be described in detail. The electronic device includes a processor 2310, a memory 2320, an input / output interface 2330, a communication interface 2340, and a bus 2350.

[0222] The processor 2310 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute related programs to realize the technical solutions provided by the embodiments of the present disclosure.

[0223] The memory 2320 can be implemented in the form of a Read Only Memory (ROM), a static storage device, a dynamic storage device, or a Random Access Memory (RAM), etc. The memory 2320 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 2320 and are called and executed by the processor 2310 to perform the data transmission method of the embodiments of the present disclosure;

[0224] The input / output interface 2330 is configured to realize information input and output.

[0225] The communication interface 2340 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0226] The bus 2350 is configured to transmit information between various components (for example, the processor 2310, the memory 2320, the input / output interface 2330, and the communication interface 2340) of the device.

[0227] The processor 2310, the memory 2320, the input / output interface 2330, and the communication interface 2340 are connected to each other through the bus 2350 to realize the communication connection between the device.

[0228] The embodiments of the present disclosure further provide a computer readable storage medium, which is configured to store a computer program. The computer program is configured to execute the data transmission method of the above-mentioned various embodiments.

[0229] The embodiments of the present disclosure further provide a computer program product, which includes a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program to make the computer device execute the data transmission method.

[0230] The terms "first", "second", "third", "fourth" and the like in the description of the disclosure and the above drawings, if any, are used to distinguish similar objects, and do not necessarily have to be described in a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "contain" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in the present disclosure, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0232] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is two or more, and greater than, less than, more than, etc. are not included in the number, and above, below, etc. are included in the number.

[0233] In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0234] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0235] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0236] It should also be understood that various embodiments provided by the present disclosure can be combined in any manner to achieve different technical effects.

[0237] The above is a specific description of the embodiments of the present disclosure, but the present disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present disclosure, and these equivalent modifications or replacements are included in the scope defined by the claims of the present disclosure.

Claims

1. A data transmission method, characterized by, A method applied to a data processor, the data processor comprising a plurality of compute dies, and an inter-die interconnect network for communication between the plurality of compute dies, each of the compute dies comprising a plurality of block processing clusters, and an inter-cluster interconnect network for communication between the plurality of block processing clusters, each of the block processing clusters comprising an intra-cluster broadcast engine, a plurality of compute units, and an intra-cluster interconnect network for communication between the plurality of compute units, each of the compute units configured with a meta-thread; the method comprising: the intra-cluster broadcast engine obtaining broadcast data and a broadcast mask in response to a broadcast request initiated by a meta-thread within the compute unit; the intra-cluster broadcast engine performing an intra-cluster broadcast process within the current block processing cluster based on the broadcast data and the broadcast mask; the intra-cluster broadcast engine forwarding the broadcast data and the broadcast mask to intra-cluster broadcast engines of other block processing clusters within the same compute die through the inter-cluster interconnect network in response to the broadcast range indicated by the broadcast request being other block processing clusters within the same compute die, to trigger the other intra-cluster broadcast engines to perform an intra-cluster broadcast process based on the broadcast data and the broadcast mask; the intra-cluster broadcast engine broadcasting the broadcast data and the broadcast mask to intra-cluster broadcast engines within other compute dies through the inter-die interconnect network in response to the broadcast range indicated by the broadcast request being an inter-die broadcast, to trigger the intra-cluster broadcast engines within the other compute dies to perform an intra-cluster broadcast process based on the broadcast data and the broadcast mask.

2. The data transmission method of claim 1, wherein, the intra-cluster broadcast process comprising: the intra-cluster broadcast engine determining target compute units within the current block processing cluster based on the broadcast mask; the intra-cluster broadcast engine broadcasting the broadcast data to the target compute units through the intra-cluster interconnect network.

3. The data transmission method of claim 1, wherein, the forwarding the broadcast data and the broadcast mask to intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network comprising: the intra-cluster broadcast engine determining a target block processing cluster containing target compute units based on the broadcast mask; the intra-cluster broadcast engine forwarding the broadcast data and the broadcast mask to an intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnect network.

4. The data transmission method of claim 3, wherein, the forwarding the broadcast data and the broadcast mask to intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnect network comprising: the intra-cluster broadcast engine determining a target block processing cluster containing target compute units based on each independent mask segment; the intra-cluster broadcast engine taking an independent mask segment corresponding to the target block processing cluster as a target mask; the intra-cluster broadcast engine forwarding the broadcast data and the target mask to an intra-cluster broadcast engine of the target block processing cluster through the inter-cluster interconnect network.

5. The data transmission method of claim 1, wherein, the data transmission method further comprising: the intra-cluster broadcast engine performing an intra-cluster broadcast process within the current block processing cluster based on broadcast data and a broadcast mask sent by other intra-cluster broadcast engines in response to the broadcast data and the broadcast mask.

6. The data transmission method of claim 1, wherein, The data processor further comprises an inter-cluster broadcast engine; the intra-cluster broadcast engine forwards the broadcast data and the broadcast mask to the intra-cluster broadcast engines of other block processing clusters through the inter-cluster interconnection network, comprising: The intra-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine; The inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines of all the block processing clusters through the inter-cluster interconnection network; The intra-cluster broadcast engine performs intra-cluster broadcast processing based on the broadcast data and the broadcast mask in the current block processing cluster in response to the broadcast data and the broadcast mask distributed by the inter-cluster broadcast engine.

7. The data transmission method of claim 1, wherein, The data transmission method further comprises: The intra-cluster broadcast engine performs intra-cluster broadcast processing based on the broadcast data and the broadcast mask in the current block processing cluster in response to receiving the broadcast data and the broadcast mask broadcast by other intra-cluster broadcast engines, and broadcasts the broadcast data and the broadcast mask to other intra-cluster broadcast engines in the same computing die to trigger other intra-cluster broadcast engines in the same computing die to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

8. The data transmission method of claim 1, wherein, The computing die further comprises an inter-cluster broadcast engine; the intra-cluster broadcast engine broadcasts the broadcast data and the broadcast mask to the intra-cluster broadcast engines in other computing dies through the inter-die interconnection network, further comprising: The intra-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-cluster broadcast engine; The inter-cluster broadcast engine broadcasts the broadcast data and the broadcast mask to the inter-cluster broadcast engines of other computing dies, so that each inter-cluster broadcast engine performs inter-cluster broadcast processing; The inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines in the same computing die, so that the intra-cluster broadcast engines perform intra-cluster broadcast processing.

9. The data transmission method of claim 8, wherein, The inter-cluster broadcast engine distributes the broadcast data and the broadcast mask to the intra-cluster broadcast engines in the same computing die, comprising: The inter-cluster broadcast engine determines a target block processing cluster containing a target computing unit in the current computing die based on the broadcast mask; The inter-cluster broadcast engine distributes the broadcast data to the intra-cluster broadcast engine of the target block processing cluster.

10. The data transmission method of claim 8, wherein, The data transmission method further comprises: The inter-cluster broadcast engine broadcasts the broadcast data and the broadcast mask to all the intra-cluster broadcast engines in the same computing die in response to the broadcast data and the broadcast mask broadcast by other inter-cluster broadcast engines; The inter-cluster broadcast engine determines a plurality of target computing dies containing target computing units based on the broadcast mask, and determines an adaptive computing die matching the computing die to which the current computing die belongs from the plurality of target computing dies based on a preset broadcast rule, and broadcasts the broadcast mask and the broadcast data to the inter-cluster broadcast engine of the adaptive computing die.

11. The data transmission method of claim 8, wherein, The data processor further comprises an inter-die broadcast engine; The inter-cluster broadcast engine broadcasts the broadcast data and the broadcast mask to other inter-cluster broadcast engines within the computing die, and further includes: The inter-cluster broadcast engine uploads the broadcast data and the broadcast mask to the inter-die broadcast engine through the inter-die interconnection network; The inter-die broadcast engine distributes the broadcast data and the broadcast mask to inter-cluster broadcast engines within other computing dies through the inter-die interconnection network.

12. The data transmission method of claim 1, wherein, The number of the computing units included in each of the block processing clusters is M, and when the broadcast mode specified in the broadcast request is a copy mode, the broadcast mask includes M flag bits, each of which corresponds to one of the computing units in the block processing cluster.

13. The data transmission method of claim 1, wherein, The number of the block processing clusters included in the data processor is N, and when the broadcast mode specified in the broadcast request is an arbitrary mode, the number of the computing units included in each of the block processing clusters is M, the broadcast mask includes N independent mask segments corresponding to the block processing clusters, each of the independent mask segments includes M flag bits, and each of the independent mask segments is the same or different.

14. The data transmission method of claim 1, wherein, The number of the computing dies included in the data processor is K, the number of the block processing clusters included in each of the computing dies is N, and the number of the computing units included in each of the block processing clusters is M, and when the broadcast mode specified in the broadcast request is an arbitrary mode, the broadcast mask includes K×M×N flag bits corresponding to the computing units one by one.

15. The data transmission method of claim 1, wherein, The intra-cluster broadcast engine obtains broadcast data and a broadcast mask in response to a broadcast request initiated by the meta-thread in the computing unit, and includes: The intra-cluster broadcast engine obtains a broadcast mask and a data storage address from the broadcast request in response to a broadcast request initiated by the meta-thread in the computing unit; The intra-cluster broadcast engine reads the broadcast data from the local memory of the computing unit based on the data storage address.

16. A broadcast engine, comprising: The broadcast engine is applied to a data processor, the data processor includes a plurality of computing dies and an inter-die interconnection network for communication between the plurality of computing dies, each of the computing dies includes a plurality of block processing clusters and an inter-cluster interconnection network for communication transmission between the plurality of block processing clusters, the block processing cluster includes the broadcast engine, a plurality of computing units, and an intra-cluster interconnection network for communication transmission between the plurality of computing units, each of the computing units is configured with a meta-thread, and the broadcast engine includes: A data acquisition unit configured to obtain broadcast data and a broadcast mask in response to a broadcast request initiated by the meta-thread in the computing unit; An intra-cluster broadcast unit configured to perform an intra-cluster broadcast process in the current block processing cluster based on the broadcast data and the broadcast mask, wherein the intra-cluster broadcast process includes determining target computing units in the current block processing cluster based on the broadcast mask and broadcasting the broadcast data to the target computing units through the intra-cluster interconnection network. inter-cluster interconnect network to other intra-cluster broadcast engines in other block processing clusters within the same compute die to trigger the other intra-cluster broadcast engines to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask; the inter-cluster broadcast unit is further configured to, in response to the broadcast request indicating an inter-die broadcast, broadcast the broadcast data and the broadcast mask to intra-cluster broadcast engines in other compute dies via the inter-die interconnect network to trigger the intra-cluster broadcast engines in the other compute dies to perform intra-cluster broadcast processing based on the broadcast data and the broadcast mask.

17. An electronic device, comprising: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program, when executed by the processor, realizes the data transmission method according to any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, and the one or more programs are executable by one or more processors to realize the data transmission method according to any one of claims 1 to 15.

Images