Processing node, communication system and communication method

Abstract

The application provides a processing node, a communication system and a communication method, and relates to the technical field of computers. A topology structure formed by the processing node provides two physically independent transmission paths for data transmission between any two processing nodes, supports simultaneous parallel transmission of data in two directions, effectively breaks through the upper limit of single-link bandwidth of a shared bus structure, and improves the overall communication concurrency and data throughput capacity of the system. Meanwhile, the two links are independent of each other, full-duplex bidirectional communication can be realized, and the utilization rate of the links is greatly improved. In addition, the bidirectional double-ring topology structure has regularity and symmetry, and when the system is expanded, only the newly added nodes need to be connected to the two links, the system bandwidth is linearly expanded with the increase of the number of nodes, high performance and low hardware overhead are considered, and an efficient and scalable inter-core communication infrastructure is provided for a multi-core NPU system.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a processing node, a communication system, and a communication method. Background Technology

[0002] Against the backdrop of rapid development in artificial intelligence and deep learning, Neural Processing Units (NPUs), as dedicated AI acceleration hardware, are widely used in computationally intensive scenarios such as cloud training and edge inference. With the continuous growth in the scale of deep learning models and the increasing complexity of computational tasks, the computing power of a single NPU core is insufficient to meet practical application needs, leading to the emergence of multi-core NPU systems. In multi-core NPU systems, frequent large-scale data exchanges are required between multiple computing cores, such as gradient aggregation of model parameters, synchronization of intermediate feature maps, and parallel data communication in distributed inference. The efficiency of these communication operations directly affects the overall system's computational performance and scalability. Therefore, inter-core interconnect technology, as a key infrastructure connecting various computing cores and storage resources, largely determines the overall performance of a multi-core NPU system.

[0003] Currently, existing inter-core interconnect technologies mainly adopt a shared bus architecture, where all computing cores and storage resources are connected through a common communication channel, and a bus arbitrator coordinates the access requests of each core. This architecture is simple to implement, has low hardware overhead, and is still applicable in scenarios with a small number of cores and a light communication load.

[0004] However, as the number of cores continues to increase, the inherent defects of the shared bus architecture become increasingly prominent. First, all data transmissions must compete for the same bus resources, making bus bandwidth a bottleneck for system performance, and causing severe congestion during multi-core concurrent communication. Second, bus bandwidth cannot scale linearly with the number of cores. As the system scales up, bus contention intensifies, leading to a decrease in the system's effective bandwidth instead of an increase, thus limiting the overall computing efficiency and scalability of multi-core NPUs. Summary of the Invention

[0005] The purpose of this application is to provide a processing node, a communication system, and a communication method to improve the problems of limited bus bandwidth and restricted scalability in the prior art.

[0006] In a first aspect, embodiments of this application provide a processing node, the processing node comprising: The first transmission port is coupled to the first direction transmission link and is used to receive data from the upstream processing node or send data through the first direction transmission link. The second transmission port is coupled to the second direction transmission link and is used to receive data from the upstream processing node or send data through the second direction transmission link. In this structure, N processing nodes are connected end-to-end via the first direction transmission link and the second direction transmission link, forming a bidirectional double-ring topology. The data transmission directions of the first direction transmission link and the second direction transmission link are opposite and independent of each other, and N is greater than or equal to 2.

[0007] In the above implementation, this topology provides two physically independent transmission paths for data transmission between any two processing nodes, supporting simultaneous parallel data transmission in both directions. This effectively breaks through the single-link bandwidth limit of the shared bus structure, improving the overall communication concurrency and data throughput of the system. Simultaneously, the two links are independent of each other, enabling full-duplex bidirectional communication and significantly improving link utilization. Furthermore, this bidirectional dual-ring topology is regular and symmetrical; expansion only requires connecting new nodes to the two links, and the system bandwidth expands linearly with the number of nodes, balancing high performance with low hardware overhead. This provides an efficient and scalable inter-core communication infrastructure for multi-core NPU systems.

[0008] Secondly, embodiments of this application provide a communication system between multiple processing nodes, the communication system comprising N processing nodes.

[0009] Thirdly, embodiments of this application provide a communication method between multiple processing nodes, applied to processing nodes in a communication system between multiple processing nodes, the communication method comprising: In response to the MPI set communication command, the data to be sent in the current processing node is divided into K sub-blocks. The communication system includes N processing nodes, which are connected end to end in sequence through a first direction transmission link and a second direction transmission link to form a bidirectional double-ring topology. The data transmission directions of the first direction transmission link and the second direction transmission link are opposite and independent of each other. N is greater than or equal to 2 and K is greater than or equal to 2. According to a preset number of steps, the K sub-blocks are distributed to other processing nodes using the first direction transmission link and the second direction transmission link; In each transmission step, the current processing node sends a corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link; and the current processing node receives the sub-block sent by the adjacent processing node.

[0010] In the above implementation, this method fully utilizes the two physically independent and opposite-direction transmission links provided by the bidirectional dual-ring topology. In each transmission step, the current processing node simultaneously sends sub-blocks in both directions and receives sub-blocks from both directions, achieving bidirectional parallel transmission and reception, effectively improving communication efficiency. Furthermore, the communication latency increases only linearly with the number of nodes, avoiding the drastic performance degradation caused by bandwidth contention in multi-core scenarios with a shared bus. Secondly, by responding to MPI aggregate communication commands and distributing data according to a preset number of steps, this method ensures high determinism and predictability in data transmission between nodes, facilitating precise task scheduling and performance optimization by upper-layer software. Moreover, by dividing the data to be sent into K sub-blocks and distributing them simultaneously in both directions on the dual rings, this method fully utilizes the sending and receiving bandwidth of each node, significantly improving link resource utilization and effectively reducing the overall number of transmission steps required for multi-node aggregate communication.

[0011] Fourthly, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps of the method provided in the third aspect above are performed.

[0012] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method provided in the third aspect above.

[0013] Sixthly, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the steps of the method provided in the third aspect above.

[0014] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1This application provides a schematic diagram of the structure of a communication system between multiple processing nodes according to an embodiment of the present application. Figure 2 A detailed structural diagram of a communication system between multiple processing nodes is provided in an embodiment of this application; Figure 3 This application provides a schematic diagram of a read / write connection between multiple processing nodes in an embodiment of the present application. Figure 4 This is a schematic diagram illustrating a processing node implementing a bypass function, provided in an embodiment of this application. Figure 5 A flowchart illustrating a communication method between multiple processing nodes provided in an embodiment of this application; Figure 6 A flowchart illustrating a broadcast operation between multiple processing nodes provided in an embodiment of this application; Figure 7 A flowchart illustrating a protocol operation between multiple processing nodes provided in an embodiment of this application; Figure 8 A schematic diagram illustrating another protocol operation between multiple processing nodes provided in an embodiment of this application; Figure 9 A flowchart illustrating another full collection operation between multiple processing nodes provided in an embodiment of this application; Figure 10 A flowchart illustrating another full-swap operation between multiple processing nodes provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of an electronic device for performing a communication method between multiple processing nodes, provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0018] It should be noted that the terms "system" and "network" in the embodiments of this invention can be used interchangeably. "Multiple" refers to two or more; therefore, in the embodiments of this invention, "multiple" can also be understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / ", unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0019] It should also be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0020] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a communication system 100 between multiple processing nodes 110 provided in an embodiment of this application. The communication system 100 is used to achieve efficient data exchange between multiple processing nodes 110 in a multi-core processor. The communication system 100 adopts a bidirectional double-ring topology. Specifically, the communication system 100 includes N processing nodes 110, where N is greater than or equal to 2. The N processing nodes 110 are connected end-to-end sequentially via a first-direction transmission link 120 and a second-direction transmission link 130, forming a bidirectional double-ring topology. The transmission directions of the first-direction transmission link 120 and the second-direction transmission link 130 are opposite and independent of each other.

[0021] Each processing node 110, as a node in a bidirectional dual-ring topology, is simultaneously coupled to the first-direction transmission link 120 and the second-direction transmission link 130. It can act as a data sender injecting data into the ring network, a data receiver retrieving data from the ring network, and, when not a target node, as a data forwarding node, relaying data from upstream downstream along the current-direction transmission link. This bidirectional dual-ring structure provides two physically independent transmission paths for communication between any two processing nodes 110, supporting simultaneous data transmission in both clockwise and counterclockwise directions, thereby effectively improving link utilization and communication concurrency.

[0022] The first-direction transmission link 120 and the second-direction transmission link 130 refer to the physical data transmission paths connecting adjacent processing nodes 110. Their transmission directions are opposite and independent. For example, the first-direction transmission link 120 can be defined as clockwise (from left to right), and the second-direction transmission link 130 can be defined as counterclockwise (from right to left). The two links are completely separated electrically and logically, allowing for simultaneous bidirectional data transmission without interference. Together, the two links form a bidirectional double-ring topology.

[0023] Each processing node 110 may include a first transmission port 112 and a second transmission port 114. The first transmission port 112 is coupled to a first-direction transmission link 120 for receiving data from an upstream processing node 110 or transmitting data through the first-direction transmission link 120. The second transmission port 114 is coupled to a second-direction transmission link 130 for receiving data from an upstream processing node 110 or transmitting data through the second-direction transmission link 130.

[0024] The two transmission ports are physically independent and functionally independent, and can work simultaneously to achieve bidirectional concurrent data transmission and reception. Specifically, when a processing node 110 needs to send data to other processing nodes 110 in the ring network, the node can choose to send data via the first direction transmission link 120 or the second direction transmission link 130 according to the location of the target node, or even send different data to two different target nodes simultaneously via both direction transmission links to maximize communication efficiency. Similarly, when a processing node 110 acts as a receiver, its first transmission port 112 and second transmission port 114 can simultaneously receive data from different source nodes from the two direction transmission links. Each processing node 110 also integrates a data buffer unit (e.g., on-chip memory (OCM)) and a control logic unit (e.g., a local synchronization manager in subsequent embodiments). The data buffer unit is used to temporarily store data to be sent and received data, and the control logic unit is used to parse transmission instructions, configure transmission parameters (e.g., source address, target address, data length, etc.), and control the data flow direction.

[0025] In a specific implementation, processing node 110 may refer to the basic computing unit constituting a multi-core processor system, such as a neural network processor core, a graphics processing unit (GPU) core, a digital signal processor (DSP) core, a CPU core, or other processor cores with independent computing and storage resources. Each processing node 110 is a node in a ring network topology.

[0026] The transmission link can use the AXI Stream bus protocol to transmit data packets, and the data width can be 512 bits or other width configurations adapted to actual bandwidth requirements. The transmission link can also be called a "loop" or "circuit", in which the first direction transmission link 120 and the second direction transmission link 130 are opposite in direction, for example, the former is clockwise and the latter is counterclockwise. The two are electrically and logically independent and can transmit data simultaneously without interfering with each other.

[0027] The above architecture provides underlying hardware support for the efficient implementation of subsequent multi-step MPI (Message Passing Interface) collection communication operations (such as broadcasting, reduction, full collection, full exchange, etc.). The specific data transfer scheduling strategy and the execution flow of each communication primitive will be described in detail in subsequent embodiments.

[0028] In the above implementation, this topology provides two physically independent transmission paths for data transmission between any two processing nodes 110, supporting simultaneous parallel data transmission in both directions. This effectively breaks through the single-link bandwidth limit of the shared bus structure, improving the overall communication concurrency and data throughput of the system. Simultaneously, the two links are independent of each other, enabling full-duplex bidirectional communication and significantly improving link utilization. Furthermore, this bidirectional dual-ring topology is regular and symmetrical; expansion only requires connecting new nodes to the two links, and the system bandwidth expands linearly with the number of nodes, balancing high performance and low hardware overhead. This provides an efficient and scalable inter-core communication infrastructure for multi-core NPU systems.

[0029] Based on the above embodiments, please refer to Figure 2 Each processing node 110 further includes M virtual execution units (vNPUs), where M is greater than or equal to 2. Each virtual execution unit includes a first port and a second port. The first port is connected to the first direction transmission link 120 via the first transmission port 112, and the second port is connected to the second direction transmission link 130 via the second transmission port 114. In other words, the first ports of each of the M virtual execution units are physically connected to the same physical first transmission port 112 and share the bandwidth of the first direction transmission link 120 corresponding to that port. Similarly, the second ports of each of the M virtual execution units are connected to the same physical second transmission port 114 and share the bandwidth of the second direction transmission link 130 corresponding to that port.

[0030] A virtual execution unit (VMU) refers to multiple logically independent communication engine instances formed within a single physical processing node 110 through hardware resource replication and address space partitioning. Each VMU has its own dedicated configuration register set, state machine, task queue, and address translation logic, and is represented as an independent communication endpoint at the software level. The control flow of each VMU is independent of the others; that is, starting, pausing, or resetting any VMU will not affect the running state of other VMUs. However, at the physical level, multiple VMUs share the on-chip storage resources and external physical transmission port bandwidth of their respective processing node 110. In this embodiment, the number M of VMUs is an integer greater than or equal to 2, preferably 2, but can also be extended to 4, 8, etc.

[0031] The first port and the second port refer to the logical communication endpoints within the virtual execution unit, not physically independent hardware pins. The first port is connected to the first direction transmission link 120 via the first transmission port 112 of the processing node 110, and the second port is connected to the second direction transmission link 130 via the second transmission port 114 of the processing node 110. A virtual execution unit can logically connect to two direction transmission links simultaneously through its first port and second port, realizing bidirectional transmission and reception capabilities.

[0032] It should be noted that although the M virtual execution units share physical transmission ports and link bandwidth, they are completely independent in the control plane. That is, each virtual execution unit can be configured, started, paused, or reset independently. Its configuration parameters (such as source address, destination address, transmission length, destination node identifier, etc.) are stored in their own independent register sets, and the state machines of each virtual execution unit do not affect each other. For example, suppose that a certain processing node 110 has two virtual execution units (denoted as vNPU_0 and vNPU_1). When vNPU_0 sends a large amount of data to the adjacent processing node 110 through its first port via the first direction transmission link 120, vNPU_1 also sends data to another adjacent processing node 110 through its second port via the second direction transmission link 130. The two take turns occupying their respective physical port bandwidth through time-division multiplexing on the physical link. Macroscopically, it appears that both directions of transmission links are in a parallel and fully loaded state, while the transmission progress of the two virtual execution units does not block each other. For example, when only vNPU_0 has data transmission needs while vNPU_1 is idle, the data of vNPU_0 can be directly transmitted to the physical port, so that vNPU_0 enjoys the full bandwidth of the transmission link in that direction, thereby realizing dynamic adaptation between high throughput of a single task and fair sharing of multiple tasks.

[0033] Furthermore, the number M of virtual execution units in this embodiment can be flexibly configured according to the actual chip area and performance requirements. It is not limited to 2, but can be expanded to 4, 8, etc., only requiring adjustment of the number of channels and arbitration algorithm. When M is greater than 2, time-division multiplexing can adopt a multi-path polling method, for example, cyclically selecting among the 4 virtual execution units in a fixed order. The bandwidth share obtained by each virtual execution unit can be adjusted according to the actual application requirements by configuring the weight register. The above architecture enables a single physical processing node 110 to logically act as multiple independent communication endpoints simultaneously. This provides a flexible and efficient hardware foundation for simulating multi-node MPI communication behavior within a single chip, or for simultaneously carrying out data transfer needs for multiple different AI inference tasks.

[0034] In the above implementation process, the dual-port separation design also allows the same virtual execution unit to send different data on two transmission links in two directions through its first port and second port at the same time, thereby realizing bidirectional concurrent communication and further improving the communication efficiency of a single virtual execution unit.

[0035] Based on the above embodiments, continue to refer to Figure 2 The first transmission port 112 includes a first multiplexer port and a first multiplexer distribution port, and the second transmission port 114 includes a second multiplexer port and a second multiplexer distribution port.

[0036] This solution supports an N2N (NPU-to-NPU) dual-ring interconnect architecture for multiple virtualized NPUs (vNPUs), enabling efficient, low-latency, and high-concurrency data communication between multiple NPUs. Based on a dual-ring topology, each NPU core integrates an N2N wrapper module. Each wrapper module is interconnected with the wrapper modules of its left and right adjacent NPUs via two AXI Stream (AXIS) buses, forming two data rings in opposite directions. The preferred data bus width is 512 bits to support high-throughput transmission.

[0037] like Figure 2 Each N2N Wrapper contains: two multiplexing ports (Mux), two multiplexing delimiters (Demux), two N2N execution units (N2N EUs), and input / output data channels corresponding to the left and right loops, respectively. The two N2N EUs are dedicated to two different vNPUs (e.g., vNPU0 and vNPU1), logically isolated from each other, but sharing the physical link bandwidth between NPUs and the Fabric bus bandwidth for accessing on-chip memory. Each N2N EU internally contains two sets of RDMA (Remote Direct Memory Access) read engines (i.e., read ports) and two sets of WDMA (Write Direct Memory Access) write engines (i.e., write ports). RDMA_left and WDMA_left are responsible for communication with the NPU adjacent to the left, while RDMA_right and WDMA_right are responsible for communication with the NPU adjacent to the right. The left and right channels are independent and can operate simultaneously, enabling bidirectional concurrent transmission.

[0038] The Mux and Demux modules require no software configuration. Their core function is to implement time-division multiplexing of the AXIS data streams of the two vNPUs to the inter-core bus at the physical layer, while ensuring the control independence of each vNPU. That is, the working state of one vNPU (such as start, stop or abnormal) will not affect the data transmission of the other vNPU.

[0039] The first multiplexing port (demux) is coupled to the upstream processing node 110 of the first direction transmission link 120, and is used to receive data from the upstream processing node 110 and distribute it to the corresponding virtual execution unit in this processing node 110.

[0040] Specifically, the first multiplexing port may contain packet parsing logic and a channel mapping table. When data is transmitted from the upstream processing node 110 to this processing node 110 via the first direction transmission link 120, the first multiplexing port first parses the received data stream, extracts the identification information of the target virtual execution unit (such as the vNPU_ID field) carried in the data header, and then routes the data payload through the internal receive bus and writes it to the receive buffer of the first write port of the corresponding virtual execution unit according to the identification information. The first write port then stores the data in a specified address of the local on-chip memory. If the data header information indicates that this processing node 110 is not the target processing node 110 (for example, the data needs to continue to be forwarded along the ring network), the first multiplexing port can cooperate with the bypass mode to directly forward the data to the first multiplexing selection port, which continues to send it downstream without going through the virtual execution unit of this processing node 110.

[0041] The first multiplexing port (mux) is coupled to the downstream processing node 110 of the first direction transmission link 120, and is used to send data of at least two virtual execution units to the downstream processing node 110 through the first direction transmission link 120.

[0042] Specifically, the input of the first multiplexing port is simultaneously connected to the output data channel of the first read port of each virtual execution unit within this processing node 110, and its output is connected to the physical transmission interface of the first direction transmission link 120. Under the scheduling of the local synchronization manager or multiplexing control logic, the first multiplexing port receives a data tick from the first read port of the virtual execution unit granted link access in each clock cycle, and transmits the data tick to the downstream processing node 110 through the first direction transmission link 120.

[0043] When multiple virtual execution units (VExecution Units) simultaneously request to send data, the first multiplexing port, under the control of the local synchronization manager, polls and alternates the data streams of each VExecution Unit at a granularity of a single clock cycle, thereby merging the multiple parallel input data streams into a single composite data stream and outputting it to the first direction transmission link 120. When sending data, the first multiplexing port inserts the identifier of the source VExecution Unit into the accompanying control information of each data cycle, so that the receiving end's first multiplexing distribution port can correctly identify the data source and distribute the data to the corresponding target VExecution Unit. The first multiplexing port also supports receiving data forwarded from the first multiplexing distribution port in bypass mode and sending it directly downstream to achieve data continuity between non-target nodes.

[0044] The second multiplexing port is coupled to the upstream processing node 110 of the second direction transmission link 130, and is used to receive data from the upstream processing node 110 and distribute it to the corresponding virtual execution unit.

[0045] Similarly, the internal structure and working principle of the second multiplexing port are completely symmetrical to those of the first multiplexing port, the only difference being that the direction of the connected link is opposite.

[0046] The second multiplexing port is coupled to the downstream processing node 110 of the second direction transmission link 130, and sends the data of at least two virtual execution units to the downstream processing node 110 through the second direction transmission link 130.

[0047] Similarly, the internal structure and working principle of the second multiplexing port are completely symmetrical to those of the first multiplexing port, the only difference being that the direction of the connected link is opposite.

[0048] With the separation design of the multi-channel distribution port for receiving and the multi-channel selection port for sending and aggregation, the sending and receiving data streams of each processing node 110 are independent at the physical port level, without the need for arbitration of switching sending and receiving directions within the same port, thus supporting full-duplex simultaneous sending and receiving operations.

[0049] Based on the above embodiments, in order to achieve shared connections between virtual units, each processing node 110 also includes a local synchronization manager, which is coupled to each virtual execution unit. The local synchronization manager is used to monitor the requests of each virtual execution unit to occupy the transmission link, and when at least two virtual execution units simultaneously request to occupy the transmission link in the same direction, it arbitrates the multiple requests in a time-division multiplexing manner so that each virtual execution unit can share the physical bandwidth of the transmission link in a time-division manner.

[0050] The local synchronization manager refers to the hardware control unit located inside each processing node 110, which is coupled to each virtual execution unit within the node. It is used to monitor the requests of each virtual execution unit to occupy the transmission link, and to arbitrate in a time-division multiplexing manner when resource contention occurs, thereby coordinating the time-division access of each virtual execution unit to the shared physical link.

[0051] The main function of the local synchronization manager is to monitor the occupancy request status of each virtual execution unit for the first direction transmission link 120 and the second direction transmission link 130 in real time. When it is detected that at least two virtual execution units simultaneously request to occupy the same direction transmission link, it is determined that there is a contention conflict for the current physical link resources. At this time, the local synchronization manager starts the arbitration mechanism to sort and schedule multiple requests in a time-division multiplexing manner, so that each virtual execution unit shares the physical bandwidth of the transmission link in that direction in a time-division manner, thereby avoiding data conflicts or bus contention caused by multiple virtual execution units driving the physical link at the same time.

[0052] In one specific implementation, when each virtual execution unit needs to send data, it first sends a link occupancy request signal to the local synchronization manager. This request signal may carry a request direction identifier (indicating whether the request is for the first direction transmission link 120 or the second direction transmission link 130) and the readiness status of the data to be sent. The local synchronization manager internally maintains a current occupancy status table and a waiting queue for each direction of transmission links. When a new occupancy request is received, it first determines whether the link in the corresponding direction is currently occupied by another virtual execution unit. If it is not occupied, the link usage right is immediately granted to the virtual execution unit, the link status is marked as busy, and the data stream of the virtual execution unit is selected to the physical transmission port. If the link in the corresponding direction is occupied by another virtual execution unit, the local synchronization manager inserts the current request into the waiting queue according to a first-come, first-served or round-robin priority strategy, and after the current occupant releases the link, it selects the next waiting unit from the waiting queue in sequence to grant the link usage right.

[0053] In the aforementioned arbitration process, the granting and release of link usage rights are precisely controlled at the clock cycle granularity. Each virtual execution unit (VLU) is allocated one or more consecutive clock cycles as its exclusive physical link time slice. Within its time slice, the VLU can continuously send data to the physical link. After the time slice ends, the local synchronization manager immediately switches the link usage rights to the next VLU. When a fixed time slice rotation method is used, if a VLU has no actual data to send in its current time slice, the local synchronization manager can automatically skip that time slice and allocate bandwidth to other VLUs with data, thus achieving zero waste of bandwidth resources.

[0054] Optionally, the interleaving granularity of time-division multiplexing can be dynamically adjusted by the local synchronization manager according to the system configuration. For example, a longer time slice can be used when there are only a few virtual execution units active to obtain better packet continuity and lower switching overhead, while fine-grained polling with a granularity of a single clock cycle can be used when multiple virtual execution units are in a high-load transmission state to achieve smoother bandwidth allocation and shorter queuing delay.

[0055] For example, the local synchronization manager uses a single clock cycle as the smallest arbitration unit to poll and alternately select the data streams of at least two virtual execution units, forming a composite data stream that is output to the transmission link.

[0056] At the start of each clock cycle, the local synchronization manager checks in parallel whether each virtual execution unit has data to be sent and has initiated a request to occupy the corresponding transmission link. If at least two virtual execution units are simultaneously in a valid request state on the same transmission link, the local synchronization manager selects only one virtual execution unit's data channel in the current clock cycle according to a preset polling order, sends its data to the multiplexing module in the sending direction, and outputs it to the physical transmission port; when the next clock cycle arrives, the local synchronization manager switches to the next virtual execution unit with a request, and so on in a cyclical polling manner.

[0057] The phrase "in units of a single clock cycle" means that each selected virtual execution unit (VLU) receives only a one-clock-cycle transmission window in each arbitration round. Within this window, it can transmit one data tick. Subsequently, regardless of whether the VLU has any remaining data to transmit, its transmission right for the current cycle is revoked and transferred to the next VLU in the polling sequence in the next cycle. This fine-grained, periodic-level alternating gating mechanism divides the data stream of each VLU into independent single-cycle data blocks in time. These data blocks are arranged alternately on the physical link according to the polling order, forming a composite data stream that is sequentially continuous but has alternating data sources.

[0058] During the aforementioned alternating gating process, the local synchronization manager maintains a polling pointer to record which virtual execution unit should be prioritized for service. After each clock cycle of gating, the pointer automatically moves to the next virtual execution unit. However, if the virtual execution unit pointed to by the pointer has no valid data request in the current cycle, the local synchronization manager skips that unit and continues moving the pointer forward until it finds the first virtual execution unit with a request and selects it. This dynamic polling method ensures that bandwidth is not wasted by idle virtual execution units due to fixed allocation. When only one virtual execution unit in the system has a data transmission request, during the local synchronization manager's cycle-by-cycle polling process, that virtual execution unit is selected in every cycle, equivalent to continuous gating and pass-through, thus enjoying the full physical bandwidth of the transmission link in that direction, without introducing additional gaps or bandwidth loss due to the existence of arbitration logic.

[0059] To illustrate with a specific application scenario: Suppose that a processing node 110 deploys two virtual execution units, denoted as vNPU_0 and vNPU_1, both of which need to send a large amount of data to an external target node through the first-direction transmission link 120. The local synchronization manager detects that both virtual execution units have initiated valid link occupancy requests on the first-direction transmission link 120, and then initiates a single-cycle polling arbitration. In clock cycle T0, the local synchronization manager selects the data channel of vNPU_0, and one data clock cycle (512 bits) of vNPU_0 is sent to the composite data stream and transmitted to the first direction transmission link 120; in clock cycle T1, the selection is switched to vNPU_1, and one data clock cycle of vNPU_1 is transmitted; in clock cycle T2, the selection is switched back to vNPU_0, and its next data clock cycle is transmitted; this process is repeated until all the data to be transmitted by one of the virtual execution units is transmitted. At this time, the virtual execution unit cancels the request signal, and the local synchronization manager detects that it has no request in the polling of the next cycle, automatically skips it, and allocates all subsequent cycles to virtual execution units that still have data, so that they can complete the transmission of the remaining data with full bandwidth. For example, suppose three virtual execution units (M=3) are deployed within the same processing node 110. vNPU_0 and vNPU_2 request to occupy the first direction transmission link 120, while vNPU_1 requests to occupy the second direction transmission link 130. The local synchronization manager polls vNPU_0 and vNPU_2 on the first direction transmission link 120 at a single-cycle granularity (T0 selects vNPU_0, T1 selects vNPU_2, T2 selects vNPU_0, T3 selects vNPU_2, and so on). Simultaneously, it independently and continuously selects vNPU_1 on the second direction transmission link 130 (since there is no contention, this is equivalent to pass-through). Arbitration on the two direction transmission links is independent and parallel, without interference.

[0060] For the receiving end, in the composite data stream formed by the single-cycle granularity interleaving described above, the data clock ticks transmitted in each clock cycle originate from different source virtual execution units (VPUs). To correctly recover the original data at the receiving end, each transmitted data clock tick must carry the source VPU identifier (vNPU_ID) in the accompanying control information, or be implicitly identified by the timing correspondence between the valid signals accompanying the composite data stream and the strobing mode. Based on this identifier or the known polling mode, the receiving end allocates the data clock ticks of each clock cycle in the composite data stream to the corresponding target VPU. Since the polling alternates on a single clock cycle basis, the data from each VPU exhibits an equally spaced, interleaved distribution at the receiving end, eliminating the possibility of a VPU being blocked for an extended period due to arbitration, thus ensuring the determinism and fairness of the communication delay between the VPUs.

[0061] In this embodiment, the polling alternating gating can also be replaced by a weighted polling method. That is, the local synchronization manager can assign different gating weights to different virtual execution units according to the system configuration. For example, a high-priority virtual execution unit can be gated once every two clock cycles, while a normal-priority virtual execution unit is gated once every four clock cycles. However, the above weighted method still needs to use a single clock cycle as the basic gating granularity, and the sending window of each virtual execution unit is still a non-continuous single clock cycle.

[0062] It is worth noting that although the local synchronization manager undertakes the link arbitration function, it does not participate in the data transfer operation on the data path. That is, all data is still directly transmitted between the virtual execution unit and the physical transmission port. The local synchronization manager only indirectly controls the data flow by controlling the strobe signal of the transmission port. Therefore, the arbitration decision has a very small impact on the latency of the data path and can be controlled within a single clock cycle.

[0063] Through the centralized arbitration mechanism of the local synchronization manager, each virtual execution unit does not need to be aware of the other's data transmission details when sharing the same physical link, nor does it need to implement complex backoff and retry logic. It only needs to send a request to the local synchronization manager and wait for the grant signal to complete the reliable transmission of data, which reduces the design complexity of the virtual execution unit and ensures the orderly allocation and high utilization of physical link bandwidth in multi-concurrency scenarios.

[0064] Based on the above embodiments, continue to refer to Figure 2The first port mentioned above includes a first read port (e.g., rdma_right) and a first write port (e.g., wdma_right), and the second port includes a second read port (e.g., rdma_left) and a second write port (e.g., wdma_left). The first read port is used to read data from the local on-chip memory and send it to the first direction transmission link 120, and the first write port is used to receive data from the first direction transmission link 120 and write it to the local on-chip memory. The second read port is used to read data from the local on-chip memory and send it to the second direction transmission link 130, and the second write port is used to receive data from the second direction transmission link 130 and write it to the local on-chip memory.

[0065] The local on-chip memory (OCM) refers to the storage resources integrated within each processing node 110, used to temporarily store data to be sent and received, as well as intermediate protocol results during MPI communication. The OCM connects to the read and write ports of each virtual execution unit via the on-chip memory access bus, supporting high-bandwidth parallel access. The OCM uses a dedicated Fabric bus (Fabric OCM bus) for data access. All read and write requests from virtual execution units must access the OCM through this Fabric bus. To support multi-vNPU sharing, this embodiment merges the wdma_left channels of two vNPUs into one through arbitration, connecting it to Port0 of the Fabric bus; similarly, wdma_right, rdma_left, and rdma_right are merged through independent arbitration logic and connected to their respective ports (Port1, Port2, Port3). The arbitration strategy employs a round-robin or weighted fair algorithm to ensure fair competition for OCM bandwidth among each vNPU.

[0066] In particular, in subsequent embodiments, the Fabric OCM bus interface also incorporates a near-memory computing unit, supporting synchronous addition and reduction operations when data is written to the OCM. Specifically, when near-memory computing mode is enabled, the written data does not directly overwrite the original content at the target address. Instead, it is added to the original data read from the OCM using an adder, and the result is then written back to the same address. This function is crucial for reduction operations (such as All-Reduce) in MPI aggregate communication, significantly reducing the number of data transfers and external computational overhead.

[0067] The aforementioned read and write ports are physically and logically independent, each possessing its own independent data path, control logic, and buffer resources. This allows the same virtual execution unit to simultaneously perform data read, send, receive, and write operations in both directions without blocking each other.

[0068] The first read port is used to read data to be transmitted from the local on-chip memory (OCM) and send the read data to the first direction transmission link 120 via the first transmission port 112, thereby transmitting data to the adjacent processing node 110 or the remote processing node 110 located in that direction. This first read port can automatically initiate a read request to the local OCM according to the pre-configured source address, transmission length, and burst transmission type (such as sequential burst or step burst), and encapsulate the data returned from the OCM according to the AXI Stream protocol format before outputting it to the multiplexing module in the transmission direction, and finally sending it to the first direction transmission link 120. Correspondingly, the first write port is used to receive data sent from the upstream or remote processing node 110 from the first direction transmission link 120, and write the received data to a specified target address in the local OCM via the first transmission port 112. This first write port can parse the target address information in the received data packet and initiate a write request to the local OCM according to the address, continuously writing data to the specified storage area. When the amount of data received exceeds the bandwidth of a single write request, the first write port can automatically split the write operation and complete the storage of all data in a burst manner.

[0069] Similarly, the second read port is used to read data from the local OCM and send it to the second direction transmission link 130 via the second transmission port 114. Its internal logic and function are symmetrical with the first read port, the only difference being the direction transmission link it is coupled to. The second write port is used to receive data from the second direction transmission link 130 and write it to the local OCM. Its function is symmetrical with the first write port.

[0070] With the aforementioned port design, each virtual execution unit has the ability to simultaneously send and receive data along two transmission links in both directions, and read and write operations in each direction can be executed independently and concurrently. For example, a virtual execution unit can send data to the adjacent processing node 110 on the right via the first read port along the first direction transmission link 120, and simultaneously send data to the adjacent processing node 110 on the left via the second read port along the second direction transmission link 130, achieving bidirectional simultaneous transmission; or it can receive data from the first direction transmission link 120 and write it to the OCM via the first write port, and simultaneously read another set of data from the local OCM via the second read port and send it along the second direction transmission link 130, achieving concurrent operation of receiving in one direction and sending in the other.

[0071] like Figure 3 As shown, two sets of RDMA and two sets of WDMA serve data transmission in the left and right directions, respectively. Figure 3(A schematic diagram of the N2N EU structure within a single NPU) Taking this as an example, the `rama_right` of the left N2N EU and the `wdma_left` of the right N2N EU work together to read data from the local NPU OCM and write it to the right NPU OCM; conversely, `rama_left` and `wdma_right` work together to transmit data to the left. The two NPUs communicate, supporting send / receive semantics and switch semantics.

[0072] Each read port has built-in address generation logic, supporting sequential or step-by-step access modes. It can continuously read data blocks from a specified base address on the local OCM and encapsulate them into AXIS stream packets, adding the target NPU identifier and data length information. Each write port includes a receive buffer and address decoding logic. After receiving a data packet, it parses the target address and writes the data to the local OCM according to that address. All read and write operations support configurable start, pause, and query states, and the left and right channels can be configured independently and run simultaneously, thus supporting bidirectional full-duplex communication.

[0073] The aforementioned read / write port separation design enables the virtual execution unit to perform multi-step bidirectional transmission scheduling of MPI aggregate communication without ping-pong switching between sending and receiving operations. In each clock cycle, it can simultaneously utilize the two-way transmission links for full-duplex data transfer, thereby maximizing the use of the physical bandwidth provided by the bidirectional dual-ring topology.

[0074] Based on the above embodiments, each processing node 110 further includes a local on-chip memory, a memory access bus, and a near-memory computing unit. The first write port and the second write port access the local on-chip memory via the memory access bus, respectively. The near-memory computing unit is integrated into the write path of the memory access bus and is used to perform an addition operation between the data to be written and the original data at the target address in the local on-chip memory when enabled, and write the operation result back to the target address.

[0075] The near-memory computing unit refers to a hardware computing module integrated into the memory access bus write path, located between the memory and the communication interface. It can perform simple arithmetic operations (such as addition) on data before it is written to memory. This unit completes the calculation while the data is passing through memory, avoiding the back-and-forth movement of data between the memory and the independent computing unit, thereby reducing access latency and bus power consumption. This embodiment further introduces a near-memory computing unit to support the synchronous completion of addition and reduction operations during the data writing process to the local on-chip memory, thus providing efficient hardware acceleration for reduction-type operations (such as All-Reduce) in MPI aggregate communication.

[0076] The memory access bus refers to the internal data path connecting the write port and read port of the virtual execution unit to the local on-chip memory. This bus typically adopts a dual-channel structure with separate read and write operations. The write channel integrates a near-memory computing unit, which allows write data to be intercepted and processed by the near-memory computing unit before reaching the memory.

[0077] The virtual execution unit accesses the local OCM through a multi-channel architecture. The first and second write ports are connected to the OCM's write ports via an on-chip memory access bus (such as a Fabric bus), while the first and second read ports are connected to the OCM's read ports via the same Fabric bus or separate read buses. In practice, independent read and write buses can be configured to eliminate read-write conflicts, or arbitration logic can be used to time-division schedule concurrent access from multiple read or write ports. When multiple write ports need to write to the OCM simultaneously, the write arbitration unit grants access permissions to each port sequentially according to priority or round-robin. Similarly, when multiple read ports need to read from the OCM simultaneously, the read arbitration unit performs corresponding scheduling. Because read and write operations use independent bus channels, there are no bus conflicts between read and write operations, enabling simultaneous read and write operations.

[0078] In some implementations, the internal structure of the near-memory computing unit may include an adder, control logic, and a working mode register. The working mode register, which can be written by upper-layer software or lower-layer microcode through a configuration interface, controls the enable state of the near-memory computing unit. When the working mode register is configured to be enabled, the near-memory computing unit automatically reads the existing data from the target address during each write operation, performs an addition operation between the new data to be written and the existing data using the adder, and writes the addition result back to the target address. This process is transparent to the virtual execution unit that initiates the write request. The virtual execution unit only needs to initiate the write request and transmit the data to be written according to the normal write operation. The near-memory computing unit automatically completes the atomic operation sequence of "read-addition-write-back" on the write path, without requiring the virtual execution unit to issue additional read instructions or perform any explicit addition scheduling at the software level. When the working mode register is configured to be disabled (i.e., normal write mode), the near-memory computing unit is in pass-through mode. The data to be written is directly written to the target address of the on-chip memory without any arithmetic processing. In this case, the write operation is equivalent to a normal data overwrite write.

[0079] In terms of timing, a single near-memory addition write operation typically requires two or three clock cycles: the first cycle reads the existing data from the target address in the local on-chip memory, the second cycle performs the addition operation and writes the result back to the target address, and the third cycle submits the write completion status. Since the read-modify-write operation is executed by the near-memory computing unit in a hardware pipelined manner, it appears as an atomic write and accumulation transaction on the memory access bus. The bus protocol provides a unified external interface with other conventional write operations, distinguishing whether near-memory computing is enabled only through specific control signal bits. When multiple write ports simultaneously initiate write requests with near-memory computing enabled, the memory access bus's write arbitration logic grants access permissions to each port sequentially according to priority or a round-robin strategy. The near-memory computing unit independently completes the read-modify-write operation sequence in each authorized write transaction, and near-memory computing operations between different write ports do not interfere with each other.

[0080] The application of near-memory computing units is illustrated using the All-Reduce reduction operation in the MPI protocol as an example. Assume four processing nodes 110 (NPU0, NPU1, NPU2, NPU3) participate in the reduction. Each node's local on-chip memory stores its own data blocks D0, D1, D2, and D3, respectively. The result of adding these four data blocks needs to be synchronized to all nodes. During the All-Reduce multi-step transmission based on Send / Receive semantics, each node sends its own data block to the adjacent processing node 110 according to a predetermined schedule. While the receiving end is writing the received data into its local on-chip memory, it automatically performs addition with the existing local data by enabling the corresponding near-memory computing unit and writes it back. The entire reduction process is completed during transmission along the data write path, eliminating the need for additional operations such as reading data from on-chip memory to the computing unit, performing addition, and then writing it back to memory. This significantly reduces the number of round trips between on-chip storage and the computing unit, as well as the bus bandwidth usage.

[0081] In this embodiment, which flexibly switches between enabling and disabling the near-memory computing unit, the enable control is supported at the individual write transaction granularity. Specifically, when configuring each data transfer operation, the software can specify whether the near-memory computing unit is enabled for this write operation via a flag bit in the transfer control descriptor. For example, in the Reduce-Scatter phase of the All-Reduce operation, each node needs to enable near-memory computing to accumulate data from different source nodes when receiving local reduction data from adjacent processing nodes 110. However, in the subsequent All-Gather phase, each node needs to disable near-memory computing and only perform data overwrite writes when receiving the final reduction results from other nodes. The software only needs to set the corresponding mode flag bit when configuring each read / write transfer task. The near-memory computing unit automatically selects its operating mode based on this flag bit when each write operation starts, eliminating the need for additional mode switching configuration during the operation.

[0082] Furthermore, the addition operation supported by the near-memory computing unit can be extended to other types of arithmetic and logical operations, such as maximum value, minimum value, average value, logical AND and OR operations, which can be implemented by configuring different operation mode registers.

[0083] In the above implementation process, the near-memory computing unit can be flexibly enabled or disabled through the working mode register, which can support the accumulation requirements in reduction operations and also pass-through writing in conventional data transfer scenarios, adapting to multiple communication modes and having good versatility and flexibility.

[0084] Based on the above embodiments, processing node 110 can also implement a bypass function, allowing data to skip non-target nodes during ring network transmission and continue forwarding directly along the current direction transmission link, thereby effectively reducing communication latency and improving link utilization. For example, the local synchronization manager in processing node 110 can also be used to directly forward upstream data received via the first direction transmission link 120 and / or the second direction transmission link 130 to the downstream processing node 110 of the corresponding direction transmission link when the current processing node 110 is not designated as the target node for the current data transmission, thereby bypassing the local processing logic of the current processing node 110.

[0085] like Figure 4 As shown, when an NPU node does not need to participate in the current communication or is in an idle state, the node can directly forward data from upstream to downstream without going through the local virtual execution unit cache or processing. For example, if the vNPU of NPN2 (i.e., NPU2) is not configured with a transport task, data packets from NPN1 can be sent directly to NPN3 through a bypass path, such as... Figure 4As shown by the thick black line, this bypasses inactive nodes, reduces transmission latency, and makes the point-to-point communication latency between any two non-adjacent NPUs independent of the physical hop count, depending only on the loop bandwidth, greatly enhancing the system's flexibility and scalability.

[0086] Specifically, when a certain processing node 110 is not designated as the target node for the current data transmission, the local synchronization manager of the processing node 110 controls its first multiplexing port or second multiplexing port to directly forward the data received from the upstream via the first direction transmission link 120 or the second direction transmission link 130 to the first multiplexing port or the second multiplexing port of the corresponding direction transmission link. The selected port then continues to send the data to the downstream processing node 110, thereby bypassing the local processing logic of the current processing node 110 (including virtual execution unit, on-chip memory access, and near-memory computing unit, etc.).

[0087] The aforementioned bypass forwarding operation is completed entirely on the internal path of the physical transmission port, without needing to be cached or processed by the node's virtual execution unit. It does not consume the node's on-chip storage and computing resources, and introduces only a very short path switching delay (usually within one clock cycle), minimizing the impact of non-target nodes on data transmission.

[0088] In its implementation, when the first multiplexing port receives a data packet from the first direction transmission link 120, it first parses the header information of the data packet and extracts the target node identifier field. If the target node identifier matches the node number of the current processing node 110, the first multiplexing port performs a normal distribution operation, distributing the data to the corresponding first write port according to the target virtual execution unit identifier, and the virtual execution unit writes the data to the local on-chip memory. If the target node identifier does not match the node number of the current processing node 110, the local synchronization manager determines that the current node is not the target node and sends a bypass enable signal to the first multiplexing port. Accordingly, the first multiplexing port bypasses the received data packet along with its accompanying control information directly to the input buffer of the first multiplexing selection port. The first multiplexing selection port then continues to send the data packet along the first direction transmission link 120 to the downstream processing node 110.

[0089] Throughout the bypass process, the data undergoes no processing by the virtual execution unit and no write operations are performed on the local on-chip memory. This is equivalent to the current node merely acting as a conductor on the physical link. The bypass process of the second-direction transmission link 130 is completely symmetrical to the above process. The second multiplexing port receives upstream data from the second-direction transmission link 130 and, after determining that the current node is not the target node, directly bypasses it to the second multiplexing port to continue forwarding downstream.

[0090] Based on the aforementioned bypass function, in the bidirectional dual-ring topology provided by this solution, data transmission between any two non-adjacent processing nodes 110 has at least two physically independent transmission paths. The first path is along the first direction transmission link 120, which is bypassed and forwarded by the intermediate node to reach the target node; the second path is along the second direction transmission link 130, which is also bypassed and forwarded by the intermediate node to reach the target node.

[0091] Taking a double-ring structure consisting of four processing nodes 110 (NPU0, NPU1, NPU2, and NPU3) connected end-to-end as an example, when the source node NPU0 needs to send data to the target node NPU2, there are two optional transmission paths: Path 1 is to bypass NPU1 and reach NPU2 via transmission link 120 (clockwise) in the first direction; Path 2 is to bypass NPU3 and reach NPU2 via transmission link 130 (counterclockwise) in the second direction. Both paths are multi-hop transmissions, but non-target nodes on each hop forward data in bypass mode, without introducing store-and-forward latency. Therefore, the actual transmission latency of both paths is much lower than that of traditional store-and-forward multi-hop networks. The two paths are redundant. When a link or node on one path becomes congested, the source node can flexibly select the other path for data transmission through the local synchronization manager, thereby achieving balanced distribution of communication load and effectively avoiding a single link becoming a performance bottleneck due to traffic concentration. In the multi-step scheduling of MPI aggregate communication, directional transmission links can be dynamically allocated to different data transmission tasks according to the current load status of each node, so that the bandwidth of both directional transmission links can be fully utilized.

[0092] The bypass function described above ensures that the communication delay between any node in the ring network depends primarily on the transmission delay of the physical link between the source node and the target node, and is independent of the processing capacity or current load status of intermediate nodes, thereby guaranteeing the deterministic nature of the communication delay and its low latency characteristics.

[0093] In some implementations, the local synchronization manager maintains a bypass mode register for the current node. This register can be dynamically configured by control signals broadcast by the global synchronization manager, or it can be actively set or cleared by the local virtual execution unit based on its own communication task status. When the bypass mode register is active, all data passing through that node and destined for a non-local target is automatically bypassed and forwarded. When the bypass mode register is inactive, data passing through that node and destined for a non-local target can still be bypassed and forwarded, but it requires additional arbitration permission from the local synchronization manager to ensure that there is no bus conflict with the node's own transmission operations. During data packet transmission, the header information of each data packet also contains a hop count field. This field is decremented by one for each bypass node passed through. When the hop count reaches zero, the data packet is discarded even if the target node identifier does not match, to prevent data from infinitely looping in the ring network due to configuration errors.

[0094] Through the above mechanism, this solution achieves efficient, reliable and flexible bypass forwarding capabilities, providing key hardware support for the practical application of bidirectional dual-ring topology in large-scale multi-core processor systems.

[0095] Based on the above embodiments, the communication system 100 also includes a global synchronization manager coupled to each processing node 110, which is used to broadcast synchronization control signals to each processing node 110 to coordinate the execution order of multi-step communication operations between multiple processing nodes 110.

[0096] The global synchronization manager is a single instance in the entire system. It can be deployed inside any processing node 110 as part of its functional modules, or it can be deployed as a dedicated control unit independent of each processing node 110. Its physical location does not affect its logical functions.

[0097] In some implementations, the global synchronization manager internally maintains a global step counter and a status tracking table for each participating node. When performing complex MPI aggregate communication operations (such as Broadcast, All-Reduce, All-Gather, All-to-All, etc.), the upper-layer software or lower-layer microcode first decomposes the entire communication process into multiple sequentially executed transmission steps, and preloads the transmission configuration of each step (including source node, target node, data block identifier, transmission direction, whether to enable near-memory computation, etc.) into the local configuration registers of each processing node 110. Subsequently, the upper-layer software sends a start command to the global synchronization manager, specifying the set of processing nodes 110 involved in this communication and the total number of steps. After receiving the start command, the global synchronization manager broadcasts a synchronization control signal of "Start Step 1" to all participating nodes, and each participating node synchronously starts its pre-configured Step 1 transmission operation upon receiving the signal. Once the global synchronization manager detects that all participating nodes have completed step 1, after a preset synchronization interval (usually several clock cycles), it broadcasts a synchronization control signal to all participating nodes to "start step 2," triggering each node to synchronously execute step 2. This process continues until all steps are completed. At this point, the global synchronization manager broadcasts an "operation complete" signal to each node and reports the end of this aggregation communication operation to the upper-layer software.

[0098] Optionally, the global synchronization manager and the local synchronization managers in each processing node 110 have a clear hierarchical division of labor and a collaborative mechanism. The local synchronization manager is responsible for the micro-link time-division arbitration between multiple virtual execution units within a single processing node 110; while the global synchronization manager is responsible for the macro-level multi-step scheduling across processing nodes 110. The two types of managers operate independently and are not dependent on each other. Even if a local synchronization manager of a processing node 110 experiences a slight arbitration delay due to competition among internal virtual execution units, the global synchronization manager will only proceed to the next step after receiving the step completion signal returned by that node, thereby ensuring the correctness of global operations and data consistency. Specifically, when the global synchronization manager broadcasts the "start step n" control signal, the local synchronization managers of each participating node schedule the corresponding virtual execution units to execute the configured data transmission tasks within their local scope; after all virtual execution units of each node complete the local transmission task of step n, the local synchronization manager of that node returns a step completion acknowledgment signal (Ack) to the global synchronization manager; after collecting the acknowledgment signals from all participating nodes, the global synchronization manager determines that the current step has been completed and then broadcasts the start signal for the next step.

[0099] Regarding the coordination between the global synchronization manager and the bypass function, when a processing node 110 is neither a source node nor a destination node in a certain transmission step (i.e., a pure bypass node), the start signal broadcast by the global synchronization manager will still be sent to that node. After receiving the signal, the local synchronization manager of that node only needs to confirm that its bypass channel is unobstructed (i.e., data can be forwarded through that node without obstacles) before returning an acknowledgment signal, without waiting for the actual data transmission to complete. This design ensures that non-participating nodes do not become bottlenecks in global synchronization, while also ensuring that even if a node only undertakes bypass forwarding tasks, the global synchronization manager can correctly grasp its status.

[0100] In addition, the global synchronization manager also supports a timeout protection mechanism. When the global synchronization manager broadcasts the "start step n" signal, if it fails to receive confirmation signals from all participating nodes within the preset timeout window, it determines that an abnormality has occurred in the current step (such as a link failure or configuration error of a node). The global synchronization manager broadcasts an "operation abnormality" signal to all nodes and terminates the current communication operation. At the same time, it reports the error status to the upper-layer software, thereby avoiding the system from waiting indefinitely in a fault state.

[0101] Through the centralized coordination mechanism of the global synchronization manager, a unified communication execution rhythm can be established among multiple processing nodes 110, enabling complex multi-step MPI set communication operations to be completed in an orderly, efficient, and reliable manner.

[0102] Based on the implementation process of the above communication system, please refer to the following... Figure 5 , Figure 5 The flowchart illustrates a communication method between multiple processing nodes provided in this application embodiment. This method is applied to any one of the processing nodes in the aforementioned communication system and includes the following steps: Step S210: In response to the MPI set communication instruction, divide the data to be sent in the current processing node into K sub-blocks, where K is greater than or equal to 2.

[0103] Among them, MPI set communication commands refer to communication commands issued by upper-layer software according to the message passing interface standard, which are used to instruct processing nodes to perform specific data exchange modes, including but not limited to broadcast, reduce, all-reduce, all-gather, and all-to-all. The command usually includes parameters such as communication operation type, set of participating nodes, data address, and length.

[0104] A sub-block refers to several independent data segments obtained by dividing the data to be sent from the current processing node into equal or unequal parts according to the number of target nodes. Each sub-block is sent to the corresponding target processing node in subsequent distribution steps. The number of sub-blocks K is usually equal to the total number of processing nodes participating in the communication, but it can also be adjusted according to the actual communication mode.

[0105] Specifically, the MPI collection communication command is sent from the upper-layer software or the global synchronization manager to the control register of the current processing node through the configuration channel. This command includes information such as the communication operation type (e.g., broadcast, reduction, full collection, or full exchange), the target node set, and the local storage address and length of the data to be sent. Upon receiving the command, the local synchronization manager of the current processing node divides the data to be sent into K consecutive sub-blocks with uniform granularity, starting from the starting address, according to the number K nodes participating in the communication specified in the command. Each sub-block corresponds to a target processing node. For broadcast operations, the K sub-blocks correspond to K different target processing nodes, and the data size of each sub-block is equal (e.g., P bytes per sub-block). For All-to-All operations, each sub-block corresponds to one processing node in the target processing node set, and the data size of each sub-block can be configured to be equal or unequal depending on the actual communication mode.

[0106] After the sub-blocks are divided, the current processing node records the starting address, length, and target processing node number of each sub-block in its local transfer descriptor table for use in subsequent transfer scheduling steps. The above division operation only involves the logical partitioning of the local address space and does not involve actual data movement, thus having extremely low instruction overhead.

[0107] Step S220: Distribute K sub-blocks to other processing nodes using the first direction transmission link and the second direction transmission link according to the preset number of steps.

[0108] In each transmission step, the current processing node sends the corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link, and the current processing node receives the sub-block sent by the adjacent processing node.

[0109] The preset number of steps refers to the total number of transmission steps required to complete one MPI aggregate communication operation. This value is pre-calculated and determined by the global synchronization manager based on the number of participating nodes, the dual-ring topology, and the type of communication operation, and is broadcast to all participating nodes via a synchronization control signal before communication begins.

[0110] Specifically, the current processing node determines, based on the current step number and the local transmission descriptor table, the sub-blocks to be sent via the first direction transmission link and their target processing nodes, as well as the sub-blocks to be sent via the second direction transmission link and their target processing nodes. Subsequently, the local synchronization manager sends transmission configurations to the first and second read ports of the corresponding virtual execution units, and sends reception configurations to the corresponding first and second write ports, enabling simultaneous initiation and parallel execution of transmission and reception operations within the current step. Since the first read port and first write port of each virtual execution unit are independent, and the second read port and second write port are also independent, the same processing node can complete the transmission of one sub-block and the reception of another sub-block within the same clock cycle. That is, simultaneous transmission and reception are performed on each direction transmission link, achieving a total of four-way parallel data flow across the two direction transmission links.

[0111] In the parallel transmission and reception process described above, the local synchronization manager arbitrates and schedules the data at a single clock cycle granularity at each step to ensure that the transmitted and received data do not conflict when they share the same physical link. In multi-hop transmission scenarios, the intermediate processing node forwards the data of the non-local target directly to the downstream through the bypass function, and the current processing node does not need to be aware of the internal processing process of the intermediate processing node.

[0112] After being distributed step by step according to a preset number of steps, the K sub-blocks eventually reach their respective target processing nodes, completing the data distribution phase in the MPI set communication operation.

[0113] In the above implementation process, each current processing node uses two directional transmission links to send two different sub-blocks simultaneously, effectively utilizing the double transmission bandwidth provided by the dual-ring topology, and greatly reducing the number of transmission steps required by the traditional single-ring structure.

[0114] Based on the above embodiments, if the MPI set communication instruction is a broadcast operation instruction, then K equals N. When distributing sub-blocks, in the first step of transmission, the current processing node, as the source node, sends the first sub-block to the first adjacent processing node along the first direction transmission link, and sends the Kth sub-block to the second adjacent processing node along the second direction transmission link. In subsequent transmission steps, the current processing node continues to forward the sub-blocks received from the upstream processing node to the downstream processing node along the same transmission direction. The current processing node also receives sub-blocks forwarded from the upstream processing node in another transmission direction. After N-1 forwarding steps, the current processing node saves its original K sub-blocks and the data forwarded from each downstream processing node, thus completing the broadcast operation.

[0115] Broadcast operation refers to a one-to-many set communication operation defined in the MPI protocol. Its function is to send specified data from one processing node (source node) to all other processing nodes in the communication domain, so that each participating node eventually holds a complete copy of the data. In this scheme, the source node divides the data to be broadcast into N sub-blocks and distributes them to all nodes in N-1 steps through a double-ring topology.

[0116] Before the broadcast operation is initiated, the global synchronization manager broadcasts a synchronization control signal to all participating nodes, and each node enters the ready state after completing its local configuration. In the first transmission step, the source node acts as the starting node for data distribution and simultaneously initiates transmission operations on two directional transmission links: sending the first sub-block along the first directional transmission link to the first adjacent processing node in that direction, and simultaneously sending the Kth sub-block (i.e., the last sub-block) along the second directional transmission link to the second adjacent processing node in that direction. Taking N=4 as an example, NPU0 acts as the source node, sending the D0 sub-block to NPU1 via the first directional transmission link (clockwise) in the first step, and simultaneously sending the D3 sub-block to NPU3 via the second directional transmission link (counterclockwise). At this time, NPU1 and NPU3, as directly adjacent target processing nodes, each receive data through their corresponding directional multiplexing ports and store it in their local on-chip memory.

[0117] In subsequent transmission steps, the processing node that has received the sub-block continues to forward the sub-block to the next adjacent processing node along the same transmission direction, while the source node continues to send the undistributed sub-blocks in both directions. Specifically, for the first direction transmission link, the source node sends the second sub-block along this direction in the second step, while the first sub-block that was sent by the source node to the first adjacent processing node in the first step is forwarded by the first adjacent processing node to the next processing node along the same direction; for the second direction transmission link, the source node sends the (K-1)th sub-block along this direction in the second step, while the Kth sub-block that was sent to the second adjacent processing node in the first step is forwarded by the second adjacent processing node to the next processing node along the same direction.

[0118] The above process can be summarized as follows: In each transmission step, each processing node that has obtained a sub-block continues to forward it downstream along the original direction, while the source node continuously injects new sub-blocks in both directions, forming a pipeline effect where data advances step by step along the double rings. During this process, the current processing node (including the source node and intermediate forwarding nodes at each level) also receives the forwarded sub-blocks from the upstream processing nodes in both directions and stores them in its local on-chip memory. When an intermediate node receives a sub-block whose target is itself, the sub-block is stored at that node, and the node continues to forward the sub-block to the downstream processing node (because subsequent nodes also need the sub-block); when an intermediate node receives a sub-block whose target is another node, the node only performs bypass forwarding and does not store it. After N-1 forwarding steps, each sub-block propagates along the ring network through N-1 links, ensuring that each of the N nodes receives all N sub-blocks, and the broadcast operation is completed.

[0119] For example, such as Figure 6 As shown, the four sub-graphs in the figure represent the original state, the state after the first step, the state after the second step, and the state after the third step from left to right. D0+D1+D2+D3 represent all the data to be transmitted, which is first stored in a contiguous block of local on-chip memory of NPU0 and is now divided into 4 parts.

[0120] In a 4-node system (with NPU0 as the source node), the broadcast operation requires 3 steps.

[0121] Step 1: NPU0 sends D0 to NPU1 along the first direction transmission link, and at the same time sends D3 to NPU3 along the second direction transmission link; Step 2: NPU0 sends D1 to NPU1 along the first direction transmission link, and simultaneously sends D2 to NPU3 along the second direction transmission link. At the same time, NPU1 forwards D0 received in Step 1 to NPU2 along the first direction transmission link, and NPU3 forwards D3 received in Step 1 to NPU2 along the second direction transmission link. At this point, NPU2 obtains D0 from NPU1 via the first direction transmission link and D3 from NPU3 via the second direction transmission link. Step 3: NPU0 sends D2 to NPU1 along the first direction transmission link and sends D1 to NPU3 along the second direction transmission link; at the same time, NPU1 forwards the D1 received in step 2 to NPU2 along the first direction transmission link, and NPU3 forwards the D2 received in step 2 to NPU2 along the second direction transmission link; at the same time, NPU2 forwards the D0 received from NPU1 in step 2 to NPU3 along the first direction transmission link, and forwards the D3 received from NPU3 in step 2 to NPU1 along the second direction transmission link.

[0122] At this point, NPU1 has received D0, D1, and D2 sequentially via the first-direction transmission link, and D3 via the second-direction transmission link, thus fully holding all four sub-blocks after adding the data it forwarded. NPU2 has received D0 and D1 via the first-direction transmission link, and D3 and D2 via the second-direction transmission link (where D2 comes from the second-step data forwarded by NPU3), also fully holding all four sub-blocks. NPU3 has received D3, D2, and D1 sequentially via the second-direction transmission link, and D0 via the first-direction transmission link, similarly fully holding all four sub-blocks. NPU0, as the source node, originally held all the sub-blocks. After the above three steps of bidirectional parallel transmission, all nodes have obtained a complete copy of the original data, and the broadcast operation is complete.

[0123] In this process, two sub-blocks are transmitted simultaneously on two directional transmission links at each step. Compared with the single-ring structure, its transmission efficiency is greatly improved, and the number of steps required is reduced from 2(N-1) steps in the traditional store-and-forward method to N-1 steps.

[0124] Based on the above embodiments, the MPI set communication instruction is a reduction operation instruction. When distributing sub-blocks, in each transmission step, the current processing node sends the corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link. When the current processing node writes the received sub-block to the local on-chip memory, it starts the near-memory calculation unit. The near-memory calculation unit adds the received data with the original data in the target address of the local on-chip memory and writes the calculation result back to the target address. After a preset number of bidirectional parallel transmissions and near-memory reductions, the current processing node obtains the final reduction result of all data.

[0125] The Reduce operation, as defined in the MPI protocol, is a many-to-one set communication operation that combines data from multiple processing nodes into a single result based on specified operations (such as addition, maximum value, minimum value, etc.) and stores the result in a specified root node. When it is necessary to synchronize the reduce results of all nodes to every node, it is called an All-Reduce operation.

[0126] When the MPI set communication instruction is a reduction operation instruction, each participating processing node divides the data to be reduced stored in its local on-chip memory into several sub-blocks according to the total number of participating processing nodes. Then, in each transmission step, each processing node simultaneously sends the corresponding sub-blocks to adjacent processing nodes through the first and second direction transmission links. When writing the received sub-blocks into the local on-chip memory, the local synchronization manager enables the near-memory computing unit. The near-memory computing unit adds the received data to the existing data at the target address in the local on-chip memory and writes the result back to the target address. The entire reduction operation is scheduled in multiple steps by the global synchronization manager. After the preset number of bidirectional parallel transmissions and near-memory reductions, each participating processing node obtains the final reduction result of all data, thus achieving the dual goals of data reduction and result synchronization in the All-Reduce operation.

[0127] In each transmission step, the current processing node performs both sending and receiving operations simultaneously. In the sending direction, the current processing node, based on the current step number and the local transmission descriptor table, reads the sub-block to be sent from the local on-chip memory via the first read port, and sends it to the adjacent downstream node via the first multiplexing port along the first direction transmission link; simultaneously, it reads another sub-block from the local on-chip memory via the second read port, and sends it to another adjacent downstream node via the second multiplexing port along the second direction transmission link. In the receiving direction, the current processing node's first multiplexing port receives the sub-block from the upstream node via the first direction transmission link, and its second multiplexing port receives the sub-block from the upstream node via the second direction transmission link. The received sub-blocks are then sent to the first and second write ports of the corresponding virtual execution units, respectively. During the writing process to the local on-chip memory, the current processing node enables the near-memory computing unit according to the configuration of this step, ensuring that each write operation executes an atomic operation sequence of "reading the original data at the target address, adding it to the received data, and writing the addition result back to the target address." The above-mentioned sending and receiving operations are carried out simultaneously on the two-way transmission links. That is, each node sends data to and receives data from both directions in each step, realizing bidirectional full-duplex protocol data exchange.

[0128] For example, such as Figure 7 As shown, the three subgraphs in the diagram, from left to right, represent the initial state, the state after the first step, and the state after the second step, respectively. In the initial state, NPU0, NPU1, NPU2, and NPU3 store their respective data. NPU0: D0A, D0B; NPU1: D1A, D1B; NPU2: D2A, D2B; NPU3: D3A, D3B.

[0129] Taking a 4-node system (NPU0, NPU1, NPU2, NPU3) as an example, each node holds two data sub-blocks (part A and part B), and the entire All-Reduce operation requires two steps to complete. The goal of All-Reduce is to ensure that all NPUs ultimately obtain the same reduction result. D0A+D1A+D2A+D3A; D0B+D1B+D2B+D3B.

[0130] Step 1: Each node simultaneously performs pairwise exchanges according to the preset schedule: NPU0 and NPU1 exchange data on part A, and NPU3 and NPU2 exchange data on part A. Simultaneously, NPU0 and NPU3 exchange data on part B, and NPU1 and NPU2 exchange data on part B. Taking NPU0 as an example, it sends part A data D0A to NPU1 via the first direction transmission link, and simultaneously sends part B data D0B to NPU3 via the second direction transmission link. At the same time, NPU0's first multiplexing port receives D1A from NPU1 via the first direction transmission link, and its second multiplexing port receives D3B from NPU3 via the second direction transmission link. When writing D1A to the local on-chip memory, NPU0 enables the local memory computing unit, automatically performs the addition operation of D0A (existing) + D1A (received), and writes the partial sum of D0A + D1A back to the original address; similarly, when writing D3B, the local memory computing unit automatically performs the addition operation of D0B (existing) + D3B (received), and writes the partial sum of D0B + D3B back.

[0131] After the first step is completed, the local reduction results obtained by each NPU are as follows: NPU0: D0A+D1A, D0B+D3B; NPU1: D0A+D1A, D1B+D2B; NPU2: D2A+D3A, D1B+D2B; NPU3: D2A+D3A, D0B+D3B.

[0132] In other words, the first step completed the local summation between each pair of NPUs.

[0133] Step 2: Each node continues to exchange the local protocol results obtained in Step 1 according to the scheduling. Specifically, NPU0 and NPU3 exchange data on Part A, and NPU1 and NPU2 exchange data on Part A; simultaneously, NPU0 and NPU1 exchange data on Part B, and NPU3 and NPU2 exchange data on Part B. Taking NPU0 as an example, it sends the partial sum of Part B (D0B+D3B) to NPU1 through the first direction transmission link, and simultaneously sends the partial sum of Part A (D0A+D1A) to NPU3 through the second direction transmission link. NPU0's first multiplexing port receives the partial sum of Part B (D1B+D2B) from NPU1 from the first direction transmission link, and its second multiplexing port receives the partial sum of Part A (D2A+D3A) from NPU3 from the second direction transmission link. During the write process, NPU0 re-enables the near-memory computing unit: it adds the received (D1B+D2B) to the existing local (D0B+D3B) to obtain the final reduction result of D0B+D1B+D2B+D3B and writes it back to the B part address; it adds the received (D2A+D3A) to the existing local (D0A+D1A) to obtain the final reduction result of D0A+D1A+D2A+D3A and writes it back to the A part address.

[0134] After the second step is completed, parts A and B of NPU0's local on-chip memory store the complete reduction results from all four nodes. The other nodes also obtain the exact same final reduction results in the same second step through their own bidirectional transmit / receive and near-memory computations, thus completing the All-Reduce operation.

[0135] In other words, when the preset number of steps is 2, in the first step of transmission, each processing node exchanges sub-blocks with its adjacent processing nodes and performs local reduction to obtain the local reduction result between the two processing nodes. In the second step of transmission, each processing node continues to exchange the obtained local reduction results along the first direction transmission link and the second direction transmission link and performs global reduction to obtain the final reduction result of all processing nodes.

[0136] In the above process, the near-memory computing unit is only enabled in steps that require reduction accumulation (such as the write operations in steps one and two above), while it is disabled in pure data transfer stages (such as the final result collection step in some implementations), operating in a normal overwrite write mode. Whether near-memory computing is enabled in each step is pre-specified by the upper-layer software through flag bits when configuring the transfer descriptor, and the local synchronization manager controls the operating mode of the near-memory computing unit according to the flag bits at the start of each step.

[0137] For example, such as Figure 8As shown, the five sub-graphs in the figure, from left to right, represent the original state, the state after the first step, the state after the second step, the state after the third step, and the final state. Unlike the previous embodiment which utilized Switch semantics to achieve bidirectional data exchange and complete reduction at the same address, this embodiment uses only Send / Receive semantics (i.e., one-way sending and receiving, with different source and destination addresses) to complete the entire reduction and synchronization process, without relying on the data exchange capabilities of Switch semantics. This method also utilizes the dual-link parallel transmission capability of the bidirectional dual-ring topology and the write-time accumulation capability of the near-memory computing unit. Through a preset number of bidirectional parallel transmission steps, it gradually completes the local reduction and full synchronization of data at each node across multiple sub-block dimensions, ultimately ensuring that all participating nodes obtain the exact same global reduction result.

[0138] In the reduction operation based on Send / Receive semantics, each processing node follows the step rhythm broadcast by the global synchronization manager. In each step, according to the preset transmission schedule table, it sends a portion of the locally stored reduction data to neighboring nodes through the first and second direction transmission links, while simultaneously receiving data from neighboring nodes. Unlike Switch semantics, the data sent by the sender is not exchanged bidirectionally with the receiver at the same address. Instead, a specific sub-block of data is sent unidirectionally to the target node. During the process of writing the data into its local on-chip memory, the target node enables or disables the nearest-memory computing unit according to the configuration of the current step. This embodiment divides the entire All-Reduce process into two stages: the first stage is the Reduce-Scatter stage (reduction-scattering stage), where through multi-step bidirectional transmission, each node locally accumulates the complete reduction result of a subset of a certain sub-block from all participating nodes; the second stage is the All-Gather stage (full collection stage), where each node broadcasts its final reduction result along the double ring to all other nodes, and ultimately each node holds the complete reduction result of all sub-blocks.

[0139] The following explanation uses a 4-node system (NPU0, NPU1, NPU2, NPU3) as an example. Initially, each node holds four data sub-blocks: NPU0 holds D00, D01, D02, and D03; NPU1 holds D10, D11, D12, and D13; NPU2 holds D20, D21, D22, and D23; and NPU3 holds D30, D31, D32, and D33. The goal is to add the corresponding sub-blocks from the four nodes, resulting in four sets of global reduction results for each node: D00+D10+D20+D30, D01+D11+D21+D31, D02+D12+D22+D32, and D03+D13+D23+D33. This embodiment requires four steps: the first three are Reduce-Scatter stages, and the fourth is the All-Gather stage.

[0140] In the first step, each node sends data in one direction according to a preset schedule: NPU0 sends D00 to NPU1 via the first direction transmission link and sends D02 to NPU3 via the second direction transmission link; NPU1 sends D12 to NPU2 via the first direction transmission link and sends D11 to NPU0 via the second direction transmission link; NPU2 sends D21 to NPU3 via the first direction transmission link and sends D23 to NPU1 via the second direction transmission link; NPU3 sends D33 to NPU0 via the first direction transmission link and sends D30 to NPU2 via the second direction transmission link.

[0141] Each receiving end enables its local memory computing unit when writing data, adding the received data to the existing data at the local target address before writing it back. For example, after NPU0 receives D11 from NPU1, it adds it to the existing local D01, resulting in a local sum of D01 + D11, which is then stored. After receiving D33 from NPU3, it adds it to the existing local D03, resulting in a local sum of D03 + D33, which is then stored. After the first step is completed, each node holds a portion of the pairwise local reduction results.

[0142] In the second step, each node continues to send the local reduction results obtained in the first step in one direction according to the scheduling. NPU0 sends D01+D11 to NPU1 through the first direction transmission link and sends D03+D33 to NPU3 through the second direction transmission link; NPU1 sends D13+D23 to NPU2 through the first direction transmission link and sends D00+D10 to NPU0 through the second direction transmission link; NPU2 sends D20+D30 to NPU3 through the first direction transmission link and sends D12+D22 to NPU1 through the second direction transmission link; NPU3 sends D02+D32 to NPU0 through the first direction transmission link and sends D21+D31 to NPU2 through the second direction transmission link.

[0143] In this step, the receiving end does not enable the near-memory computing unit when writing, and only performs regular overwrite writing. The purpose is to move the local reduction result to the predetermined node to prepare for subsequent global accumulation.

[0144] In the third step, some nodes send key partial reduction results to the target node to complete the final accumulation. Specifically, NPU3 sends D21+D31 to NPU0 via the first-direction transmission link. After receiving it, NPU0 enables near-memory computation and adds it to its existing D00+D10 to obtain the complete reduction result D00+D10+D20+D30. NPU3 sends D02+D32 to NPU2 via the second-direction transmission link. After receiving it, NPU2 enables near-memory computation and adds it to its existing D12+D22 to obtain D02+D1. The complete reduction result is 2+D22+D32; NPU0 sends D02+D32 to NPU1 via the first direction transmission link. After receiving it, NPU1 enables near-memory computation and adds it to the existing D12+D22, thus obtaining D02+D12+D22+D32; NPU0 sends D00+D10 to NPU3 via the second direction transmission link. NPU3 enables near-memory computation and adds it to the existing D20+D30, thus obtaining D02+D22+D32+D32+D2 ... 00+D10+D20+D30; NPU1 sends D00+D10 to NPU2 via the first direction transmission link. NPU2 enables near-memory computation and adds it to the existing D20+D30 to obtain D00+D10+D20+D30; NPU1 sends D13+D23 to NPU0 via the second direction transmission link. NPU0 enables near-memory computation and adds it to the existing D03+D33 to obtain D03+D13+ D23+D33; NPU2 sends D13+D23 to NPU3 through the first direction transmission link. NPU3 enables near-memory computation and adds it to the existing D03+D33 to obtain D03+D13+D23+D33. NPU2 sends D21+D31 to NPU1 through the second direction transmission link. NPU1 enables near-memory computation and adds it to the existing D01+D11 to obtain D01+D11+D21+D31. After the third step, each node holds one or more complete sets of global reduction results: NPU0 holds D00+D10+D20+D30 and D03+D13+D23+D33; NPU1 holds D01+D11+D21+D31 and D02+D12+D22+D32; NPU2 also holds D00+D10+D20+D30 and D02+D12+D22+D32; and NPU3 holds D00+D10+D20+D30 and D03+D13+D23+D33. At this point, the data is scattered, requiring a fourth step for full collection.

[0145] In the fourth step (All-Gather phase), each node sends its existing global reduction results to other nodes along the bidirectional ring network. The receiving end does not enable nearby computing units during write operations, storing data only through overwrite. After the bidirectional parallel transmission in the fourth step, each node receives the missing global reduction results from its neighboring nodes. Finally, all nodes have four complete sets of global reduction results stored in their local on-chip memory: D00+D10+D20+D30, D01+D11+D21+D31, D02+D12+D22+D32, and D03+D13+D23+D33. The All-Reduce operation is then complete.

[0146] This embodiment uses Send / Receive semantics to implement All-Reduce. It only relies on the RDMA read engine and WDMA write engine of each virtual execution unit in conjunction with the near-memory computing unit to complete data reduction and result synchronization. It does not require the special switching logic of bidirectional address matching required by Switch semantics, which reduces the complexity of hardware design. This allows the system to naturally have reduction acceleration capabilities on the basis of supporting conventional Send / Receive transmission, and has better versatility and scalability.

[0147] Through the above implementation method, this scheme integrates reduction computation and data synchronization into a single multi-step transmission process. It utilizes the dual-link parallelism provided by the bidirectional dual-ring topology and the write-time accumulation capability provided by the near-memory computing unit to complete the data reduction and result synchronization of all nodes in the fewest steps.

[0148] Based on the above embodiments, the MPI collection communication instruction is a full collection operation instruction or a full exchange operation instruction. When distributing sub-blocks, in each transmission step, the current processing node sends the corresponding sub-blocks to the adjacent processing nodes through the first direction transmission link and the second direction transmission link. The current processing node writes the received sub-blocks into the local on-chip memory in an overwrite manner. After a preset number of transmission steps, the current processing node obtains all the data sub-blocks of all processing nodes.

[0149] The All-Gather operation, defined in the MPI protocol, is a many-to-many set communication operation. Its function is to collect the data held by each processing node and distribute the data from all nodes to every participating node. After the operation is complete, each node stores the complete dataset of all nodes. This operation involves only data transfer and replication, and does not involve any reduction computation.

[0150] All-to-All (ATU) is a many-to-many set communication operation defined in the MPI protocol. Its function is that each processing node divides its data into several sub-blocks according to the target nodes, sends each sub-block to its corresponding target node, and simultaneously receives its own data sub-blocks from each source node. After the operation is completed, each node receives data from all source nodes, and each data sub-block has only one final target node; there is no data replication.

[0151] Both of these operations are pure data transfer types, not involving enabling near-memory computing units. All received sub-blocks are directly written to the local on-chip memory in an overwrite manner, overwriting the original data at the target address. During execution, each participating processing node follows the steps broadcast by the global synchronization manager, simultaneously transmitting and receiving bidirectionally through the first and second direction transmission links at each step. After a preset number of forwarding steps, each participating node finally obtains all the data sub-blocks from all processing nodes.

[0152] For the full-gather operation, each participating processing node divides the data to be collected in its local on-chip memory into N sub-blocks according to the total number N of participating processing nodes. However, each node only holds its own complete copy of the data, rather than having a single source node holding all the data as in broadcasting. The core objective of the entire All-Gather operation is to collect and synchronize the data blocks held by each node to all nodes, so that each node ultimately stores the complete dataset of all nodes.

[0153] For example, such as Figure 9 As shown, the four subgraphs in the diagram represent the initial state, the state after the first step, the state after the second step, and the final state, respectively. Taking a 4-node system as an example, in the initial state, NPU0 holds D0A and D0B, NPU1 holds D1A and D1B, NPU2 holds D2A and D2B, and NPU3 holds D3A and D3B. The goal is for each node to eventually hold all data from D0A, D0B, D1A, D1B, D2A, D2B, D3A, and D3B.

[0154] The entire All-Gather operation requires three steps. In each step, each node simultaneously sends the data sub-blocks it currently holds that have not yet been sent to the target node to the adjacent node along the two-way transmission links, and at the same time receives the data sub-blocks sent by the adjacent nodes from the two-way transmission links and writes them to the corresponding address of the local on-chip memory in an overwrite manner.

[0155] In the first step, NP0 sends D0B to NPU1 via the first-direction transmission link and simultaneously sends D0A to NPU3 via the second-direction transmission link; NPU1 sends D1A to NPU0 via the first-direction transmission link and sends D1B to NPU2 via the second-direction transmission link; NPU2 sends D2A to NPU1 via the first-direction transmission link and sends D2B to NPU3 via the second-direction transmission link; NPU3 sends D3B to NPU0 via the first-direction transmission link and sends D3A to NPU2 via the second-direction transmission link. Each pair of adjacent nodes completes bidirectional data exchange, and the receiving end stores the received sub-blocks using an overwrite write method.

[0156] In the second and third steps, each node continues to forward the sub-blocks received in the first step from the remote node to the next node along the loop according to the preset schedule. At the same time, it receives new sub-blocks from the upstream. After three steps of forwarding, each sub-block propagates along the ring network through three links to reach all nodes, so that each node holds all 8 sub-blocks, and the All-Gather operation is completed.

[0157] For an all-to-all exchange operation, each participating processing node divides the data to be exchanged in its local on-chip memory into N sub-blocks according to the target node, where the target node of the j-th sub-block is the j-th processing node. The core objective of the entire all-to-all operation is to deliver each node's sub-blocks to their corresponding target nodes, so that the j-th node eventually receives the j-th sub-block from all source nodes.

[0158] For example, such as Figure 10 As shown, the four subgraphs in the diagram represent the initial state, the state after the first step, the state after the second step, and the final state, respectively. Taking a 4-node system as an example, in the initial state, NPU0 holds D00 (target NPU0), D01 (target NPU1), D02 (target NPU2), and D03 (target NPU3); NPU1 holds D10 (target NPU0), D11 (target NPU1), D12 (target NPU2), and D13 (target NPU3); and NPU2 holds D20 (target NPU0), D21 (target NPU1), and D22 (target NPU3). U2), D23 (target NPU3), NPU3 holds D30 (target NPU0), D31 (target NPU1), D32 (target NPU2), D33 (target NPU3), the goal is to make NPU0 eventually obtain D00, D10, D20, D30, NPU1 obtain D01, D11, D21, D31, NPU2 obtain D02, D12, D22, D32, and NPU3 obtain D03, D13, D23, D33.

[0159] The entire All-to-All operation requires 3 steps to complete.

[0160] In the first step, adjacent nodes perform bidirectional data exchange. For example, NPU0 sends D01 to NPU1 through the first-direction transmission link, while NPU1 sends D10 to NPU0 through the first-direction transmission link, thus realizing the first set of data exchange between adjacent nodes with a span of 1. After this round of transmission, the first set of data exchange between adjacent NPUs is completed.

[0161] In the second step, each node exchanges data with a span of 2 (i.e., exchanges data with the far-end node in the opposite direction). For example, NPU0 sends D02 to NPU1 through the first direction transmission link (the final destination of this data is NPU2, which is relayed through NPU1), while NPU0 sends D03 to NPU3 through the second direction transmission link (which is relayed to NPU2 through NPU3), thus gradually realizing multi-hop data forwarding.

[0162] In the third step, each node continues to complete the remaining data exchange with a span of 3. After three steps of bidirectional parallel transmission, each target node receives the target sub-block it needs from each source node, and the All-to-All operation is completed.

[0163] Throughout the All-to-All execution process, each node simultaneously uses two transmission links in both directions for bidirectional transmission and reception at each step. All received data is written to the target address in an overwrite manner, and the nearest memory computing unit is not enabled.

[0164] In this embodiment, the core difference between the all-gather and all-to-all operations lies in the data block arrangement and target node allocation logic: In All-Gather, each node sends its own complete data block, and each data block needs to be copied to all other nodes; the data blocks of each node are independent of each other. In All-to-All, each node pre-divides the data according to the target node, and each sub-block has only one unique target receiving node. The data does not need to be copied multiple times, but it needs to reach its respective target node through multi-hop routing in the ring network. Both operations make full use of the two independent links of the bidirectional dual-ring topology to achieve parallel transmission, completing the all-gather or all-to-all data exchange with the fewest steps.

[0165] In the above process, All-Gather and All-to-All aggregate communication are implemented based on Send / Receive semantics. Data routing and distribution are completed entirely by relying on RDMA read ports, WDMA write ports, and bidirectional inter-core interconnect links. This eliminates the need for a dedicated switch network or additional routing hardware, enabling flexible and efficient data rearrangement among multiple nodes. By fully utilizing the parallel transmission capability of the bidirectional loop, this solution can simultaneously use two transmission links in each direction for data transmission and reception, effectively improving the bandwidth utilization of physical links, reducing communication latency, and possessing good scalability, making it suitable for larger-scale multi-core processor systems.

[0166] Please refer to Figure 11 , Figure 11 This application provides a schematic diagram of an electronic device for executing a communication method between multiple processing nodes. The electronic device may include: at least one processor 310, such as a CPU; at least one communication interface 320; at least one memory 330; and at least one communication bus 340. The communication bus 340 is used to establish communication between these components. In this embodiment, the communication interface 320 is used for signaling or data communication with other node devices. The memory 330 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 330 may also be at least one storage device located remotely from the processor. The memory 330 stores computer-readable instructions, which, when executed by the processor 310, enable the electronic device to perform the aforementioned method process.

[0167] Understandable. Figure 11 The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 11 The more or fewer components shown, or having the same Figure 11 The different configurations shown. Figure 11 The components shown can be implemented using hardware, software, or a combination thereof.

[0168] This application provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it performs the method process executed by the electronic device in the above method embodiments.

[0169] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can perform the methods provided in the above-described method embodiments.

[0170] In summary, the embodiments of this application provide a processing node, a communication system, and a communication method. The topology formed by the processing nodes provides two physically independent transmission paths for data transmission between any two processing nodes, supporting simultaneous parallel data transmission in both directions. This effectively breaks through the single-link bandwidth limit of the shared bus structure, improving the overall communication concurrency and data throughput of the system. Simultaneously, the two links are independent of each other, enabling full-duplex bidirectional communication and significantly improving link utilization. Furthermore, this bidirectional dual-ring topology is regular and symmetrical; expansion only requires connecting new nodes to the two links, and the system bandwidth expands linearly with the number of nodes, balancing high performance and low hardware overhead. This provides an efficient and scalable inter-core communication infrastructure for multi-core NPU systems.

[0171] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0172] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0173] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0174] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0175] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A processing node, characterized in that, The processing nodes include: The first transmission port is coupled to the first direction transmission link and is used to receive data from the upstream processing node or send data through the first direction transmission link. The second transmission port is coupled to the second direction transmission link and is used to receive data from the upstream processing node or send data through the second direction transmission link. In this structure, N processing nodes are connected end-to-end via the first direction transmission link and the second direction transmission link, forming a bidirectional double-ring topology. The data transmission directions of the first direction transmission link and the second direction transmission link are opposite and independent of each other, and N is greater than or equal to 2.

2. The processing node according to claim 1, characterized in that, The processing node also includes: There are M virtual execution units, where M is greater than or equal to 2; Each virtual execution unit includes a first port and a second port. The first port is connected to the first direction transmission link via the first transmission port, and the second port is connected to the second direction transmission link via the second transmission port.

3. The processing node according to claim 2, characterized in that, The processing node also includes a local synchronization manager, which is coupled to each virtual execution unit. The local synchronization manager is used to monitor the requests of each virtual execution unit to occupy the transmission link, and when at least two virtual execution units simultaneously request to occupy the transmission link in the same direction, it arbitrates the multiple requests in a time-division multiplexing manner so that each virtual execution unit can share the physical bandwidth of the transmission link in a time-division manner.

4. The processing node according to claim 2, characterized in that, The processing node also includes: local on-chip memory, memory access bus, and near-memory computing unit; The first port and the second port respectively access the local on-chip memory via the memory access bus; The near-memory computing unit is integrated into the write path of the memory access bus. When enabled, it performs an addition operation between the data to be written and the existing data at the target address in the local on-chip memory, and writes the result back to the target address.

5. The processing node according to claim 1, characterized in that, The processing node also includes a local synchronization manager, which is used to directly forward upstream data received via the first direction transmission link and / or the second direction transmission link to the downstream processing node of the corresponding direction transmission link when the current processing node is not designated as the target node for the current data transmission, so as to bypass the local processing logic of the current processing node.

6. A communication system between multiple processing nodes, characterized in that, The communication system includes N processing nodes as described in any one of claims 1-5.

7. The communication system according to claim 6, characterized in that, The communication system also includes a global synchronization manager, coupled to each processing node, for broadcasting synchronization control signals to each processing node to coordinate the execution order of multi-step communication operations between multiple processing nodes.

8. A communication method between multiple processing nodes, characterized in that, A communication method for processing nodes in a multi-processing node communication system includes: In response to the MPI set communication command, the data to be sent in the current processing node is divided into K sub-blocks. The communication system includes N processing nodes, which are connected end to end in sequence through a first direction transmission link and a second direction transmission link to form a bidirectional double-ring topology. The data transmission directions of the first direction transmission link and the second direction transmission link are opposite and independent of each other. N is greater than or equal to 2 and K is greater than or equal to 2. According to a preset number of steps, the K sub-blocks are distributed to other processing nodes using the first direction transmission link and the second direction transmission link; In each transmission step, the current processing node sends a corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link; and the current processing node receives the sub-block sent by the adjacent processing node.

9. The communication method according to claim 8, characterized in that, The MPI aggregate communication command is a broadcast operation command, where K equals N. The step of distributing the K sub-blocks to other processing nodes according to a preset number of steps, using the first direction transmission link and the second direction transmission link, includes: In the first step of transmission, the current processing node, as the source node, sends the first sub-block to the first adjacent processing node along the first direction transmission link, and sends the Kth sub-block to the second adjacent processing node along the second direction transmission link. In subsequent transmission steps, the current processing node will continue to forward the sub-blocks received from the upstream processing node to the downstream processing node along the same transmission direction; and the current processing node will also receive sub-blocks forwarded from the upstream processing node in another transmission direction. After N-1 forwarding steps, the current processing node saves its original K sub-blocks and the data forwarded back from each downstream processing node, thus completing the broadcast operation.

10. The communication method according to claim 8, characterized in that, The MPI aggregate communication command is a protocol operation command. The step of distributing the K sub-blocks to other processing nodes according to a preset number of steps, using the first direction transmission link and the second direction transmission link, includes: In each transmission step, the current processing node sends the corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link; When the current processing node writes the received sub-blocks into the local on-chip memory, it starts the near-memory computing unit. The near-memory computing unit performs an addition operation between the received data and the original data in the target address of the local on-chip memory, and writes the result back to the target address. After a preset number of bidirectional parallel transmissions and near-memory reductions, the current processing node obtains the final reduction result of all data.

11. The method according to claim 8, characterized in that, The MPI collection communication command is a full collection operation command or a full exchange operation command. The step of distributing the K sub-blocks to other processing nodes according to a preset number of steps, using the first direction transmission link and the second direction transmission link, includes: In each transmission step, the current processing node sends the corresponding sub-block to the adjacent processing node through the first direction transmission link and the second direction transmission link; The current processing node writes the received sub-blocks to the local on-chip memory in an overwrite manner; After a preset number of transmission steps, the current processing node obtains all data sub-blocks from all processing nodes.

Images