Chip, set communication method, device, storage medium, and program product

Abstract

Embodiments of the present application provide a chip, a collective communication method, equipment, a storage medium and a program product. In the embodiments of the present application, a first collective communication control component is included in a PE in the chip, and a second collective communication control component is included in a router. A collective communication topology relationship is formed between a plurality of routers. The second collective communication control component can execute a corresponding sub-operation based on the upstream and downstream positions of the router in the collective communication topology relationship, and pass data upstream until a convergence terminal completes the corresponding sub-operation to obtain an execution result of a specified collective communication. In the embodiments of the present application, the collective communication task is offloaded from the PE to the router, and the router completes the sub-operation in the collective communication, thereby realizing the physical separation of calculation and communication, improving the utilization rate of the calculation resources of the PE, and passing data by using the collective communication topology relationship between the routers, thereby reducing the communication rounds, reducing the congestion of the NoC network, and further reducing the delay of the service provided by the PE.

Description

Technical Field

[0001] This application relates to the field of chip communication, and more particularly to a chip, a communication method, a device, a storage medium, and a program product. Background Technology

[0002] With the increasing integration of chips, distributed on-chip architecture has been widely used. This means that a single chip includes multiple PEs (Processing Elements), and these PEs are connected to the NoC (Network-on-Chip) through corresponding routers to achieve interconnection and communication.

[0003] In practical applications, PEs in a distributed on-chip architecture need to frequently perform aggregated communication, such as reduction operations. Taking a chip with four PEs labeled a0, a1, b0, and b1 as an example, the reduction operation process includes: each PE reads data from the other three PEs, that is: a0 reads data from a1, b0, and b1; a1 reads data from a0, b0, and b1; b0 reads data from a0, a1, and b1; and b1 reads data from a0, a1, and b0. Each PE then performs the reduction operation locally based on the read data.

[0004] During the above execution process, each PE needs to send and receive data with multiple other PEs, and each PE also needs to perform reduction operations locally. This not only consumes the PE's computing resources and reduces the utilization rate of computing resources, but also leads to a large number of concurrent point-to-point communications in the NoC network, exacerbating the congestion of the NoC network and resulting in a large delay in PE services. Summary of the Invention

[0005] This application provides a chip, a collection communication method, a device, a storage medium, and a program product to improve the utilization of PE's computing resources and reduce PE's service latency.

[0006] This application provides a chip comprising: multiple processing units, each processing unit including a first aggregated communication control component; multiple routers, each router corresponding to at least one processing unit and including a second aggregated communication control component; the multiple routers forming an aggregated communication topology, the aggregated communication topology indicating the upstream and downstream positions of the multiple routers in a specified aggregated communication process, different upstream and downstream positions corresponding to different sub-operations in the specified aggregated communication; the first aggregated communication control component being used to send an aggregated communication instruction and the original data of the processing unit participating in the aggregated communication to the second aggregated communication control component in the local router corresponding to its local processing unit; the second aggregated communication control component being used to execute the corresponding sub-operation in the specified aggregated communication according to the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instruction, and to transmit the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the convergence endpoint has completed the corresponding sub-operation, thereby obtaining the execution result of the specified aggregated communication.

[0007] This application provides a aggregated communication method applied to a chip. The chip includes multiple processing units and multiple routers. Each processing unit includes a first aggregated communication control component, and each router corresponds to at least one processing unit and includes a second aggregated communication control component. The multiple routers form an aggregated communication topology, which indicates the upstream and downstream positions of the multiple routers in a specified aggregated communication process. Different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication. The method includes: the first aggregated communication control component sending an aggregated communication instruction and the original data of the processing unit participating in the aggregated communication to the second aggregated communication control component in the local router; the second aggregated communication control component executing the corresponding sub-operation in the specified aggregated communication according to the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instruction; the second aggregated communication control component transmitting the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the convergence endpoint completes the corresponding sub-operation, thus obtaining the execution result of the specified aggregated communication.

[0008] This application provides a method for aggregated communication, applied to a first aggregated communication control component located in a local processing unit of a chip. The method includes: sending an aggregated communication instruction and raw data for the local processing unit to participate in aggregated communication to a second aggregated communication control component in a local router, so that the second aggregated communication control component executes a corresponding sub-operation in the specified aggregated communication according to the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instruction, until the execution result of the specified aggregated communication is obtained; wherein, the aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in the specified aggregated communication process, and different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

[0009] This application provides a method for aggregated communication, applied to a second aggregated communication control component located in a local router of a chip. The method includes: receiving an aggregated communication instruction sent by a first aggregated communication control component in a local processing unit and the original data of the processing unit participating in aggregated communication; executing a corresponding sub-operation in the specified aggregated communication according to the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instruction; transmitting the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint has completed the corresponding sub-operation, thereby obtaining the execution result of the specified aggregated communication; wherein, the aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in the specified aggregated communication process, and different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

[0010] This application embodiment also provides a computing device, including: a memory and a processor; the memory is used to store a computer program; the processor is coupled to the memory and is used to execute the computer program in the memory to implement the steps in the collective communication method.

[0011] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the collective communication method.

[0012] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, enable the processor to implement the steps in the collective communication method.

[0013] In this embodiment, the aggregated communication task originally executed in the processing unit can be offloaded to the router to free up computing resources in the processing unit. Specifically, a first aggregated communication control component can be built into the processing unit, and a second aggregated communication control component can be built into the router. Multiple routers can form an aggregated communication topology. The aggregated communication topology can indicate the upstream and downstream positions of each router node in a specified aggregated communication process. The second aggregated communication control component can execute corresponding sub-operations based on the upstream and downstream positions of the routers and transmit the original data or result data corresponding to the sub-operations upstream until the second aggregated communication control component at the convergence endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication. In this way, the sub-operations involved in the aggregated communication process (such as data computation) are completed by the second aggregated communication component in the router. The processing unit does not need to participate in the corresponding sub-operations, realizing the physical separation of computing and communication tasks, freeing up the computing resources of the processing unit, and improving the computing resource utilization of the PE. Furthermore, through hardware-level pipelined collaboration between routers and the aggregated communication topology, the number of communication rounds is reduced, NoC network congestion is reduced, and thus the latency of the PE providing services is reduced. Attached Figure Description

[0014] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of multi-chip integrated communication. Figure 2 A schematic diagram of internal chip communication provided for an exemplary embodiment of this application; Figure 3 A schematic diagram illustrating communication among a set of multiple processing units as provided in an embodiment of this application; Figure 4 A schematic diagram of the chip structure provided for an exemplary embodiment of this application; Figure 5 A schematic diagram of the structure of the first communication control component provided in an exemplary embodiment of this application; Figure 6 A schematic diagram of the structure of the second communication control component provided in an exemplary embodiment of this application; Figure 7 A schematic diagram illustrating the physical connection relationship between multiple routers, provided for an exemplary embodiment of this application; Figure 8 A flowchart illustrating a collection communication method provided for an exemplary embodiment of this application; Figure 9 A flowchart illustrating another collection communication method provided for an exemplary embodiment of this application; Figure 10 A flowchart illustrating yet another collection communication method provided for an exemplary embodiment of this application; Figure 11 This is a schematic diagram of the structure of a computing device provided for an exemplary embodiment of this application. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to language models or large models) comply with relevant laws and standards.

[0017] Additionally, it should be noted that when user interaction operations or triggering operations are involved in the embodiments of this application, these operations include, but are not limited to, various interaction methods such as touch operations, gesture operations, voice operations, head movement operations, and eye movement operations. Touch operations include, but are not limited to, click operations, double-click operations, long-press operations, swipe operations, pinch operations, or mouse hover operations. Swipe operations include, but are not limited to, straight-line swipes and curved-line swipes.

[0018] With the development of internet services, the scale of these services is growing rapidly. To enhance service capabilities, multiple chips can work together to provide services, forming a distributed system. These chips are interconnected via a high-speed network to collaboratively provide distributed services. The chips can be GPUs (Graphics Processing Units), CPUs (Central Processing Units), NPUs (Neural Processing Units), AI acceleration chips, etc.

[0019] In practical applications, distributed overhead (such as cross-chip data transmission latency, communication bandwidth bottlenecks, and synchronization waiting overhead between multiple chips) has become one of the bottlenecks to improving service efficiency, and this distributed overhead is mainly reflected in aggregated communication. To accelerate aggregated communication, the industry's common practice is to add computing functionality to the physical network's switch nodes, i.e., on-network computing, such as... Figure 1 As shown. Figure 1 This is a schematic diagram of multi-chip integrated communication. Figure 1 In this system, chips A1, A2, A3, and A4 can all communicate with the switch node via a high-speed interconnect network. While forwarding data, the switch node can perform aggregation operations (such as summation and finding the maximum value) on data from multiple chips and return the aggregation results to each chip, thereby reducing the amount of data transmission and communication rounds between chips.

[0020] However, the aforementioned technologies primarily address the acceleration of inter-chip aggregated communication and have not yet resolved the bottleneck issue of intra-chip aggregated communication. As chip integration increases, more and more physical processors (PEs) are integrated within a single chip, and these PEs require frequent aggregated communication to complete distributed computing tasks, such as... Figure 2 As shown. Figure 2 This is a schematic diagram of internal chip communication provided for an exemplary embodiment of this application. Figure 2 The chip contains four physical exciters (PEs), labeled a0, a1, b0, and b1. Each PE has a corresponding router and needs to access the NoC (No.C) through its respective router. Data flow between PEs must pass through their respective routers and the on-chip network to reach the destination PE. For example, if a0 sends data to b0, a0 must first send the data to its corresponding router 1. This data is then transmitted via the on-chip network to router 3, and finally forwarded by router 3 to b0, thus enabling data communication between a0 and b0.

[0021] When performing aggregated communication in the manner described above, Figure 2 Taking the reduction operation among the four PEs as an example, each PE reads data from the other three PEs, that is: a0 reads data from a1, b0 and b1, a1 reads data from a0, b0 and b1, b0 reads data from a0, a1 and b1, and b1 reads data from a0, a1 and b0; each PE performs the reduction operation locally based on the read data.

[0022] During the above execution process, each PE needs to send and receive data with multiple other PEs, and each PE also needs to perform reduction operations locally. This not only consumes the PE's computing resources and reduces the utilization rate of computing resources, but also leads to a large number of concurrent point-to-point communications in the NoC network, exacerbating the congestion of the NoC network and resulting in a large delay in PE services.

[0023] To address the aforementioned technical issues, in this embodiment, the aggregated communication tasks originally executed in the processing unit can be offloaded to the router, thereby freeing up computing resources in the processing unit. Specifically, a first aggregated communication control component can be built into the processing unit, and a second aggregated communication control component can be built into the router. Multiple routers can form an aggregated communication topology. This topology indicates the upstream and downstream positions of each router node in a specified aggregated communication process. The second aggregated communication control component can execute corresponding sub-operations based on the upstream and downstream positions of the routers and transmit the original or result data corresponding to the sub-operations upstream until the second aggregated communication control component at the convergence endpoint completes the corresponding sub-operation, obtaining the execution result of the specified aggregated communication. Through this method, the sub-operations involved in the aggregated communication process (such as data computation) are completed by the second aggregated communication component in the router, eliminating the need for the processing unit to participate in the corresponding sub-operations. This achieves physical separation of computation and communication tasks, freeing up computing resources in the processing unit and improving the utilization rate of the PE's computing resources. Furthermore, through hardware-level pipelined collaboration between routers and the aggregated communication topology, the number of communication rounds is reduced, NoC network congestion is lowered, and the latency of the PE providing services is reduced.

[0024] Furthermore, from a conceptual standpoint, the embodiments of this application employ the concept of in-network computing, offloading the collective communication operations originally performed by the processing unit (PE) to the router in the NoC network for execution, such as... Figure 3 As shown. Figure 3 This is a schematic diagram illustrating communication between a set of multiple processing units, as provided in an embodiment of this application. Please refer to [link / reference]. Figure 3 Multiple processing units (PEs) are interconnected through a NoC network with on-chip computing capabilities, forming a unified communication acceleration architecture within the chip. Taking full reduction operation as an example, in this architecture, when PEs perform unified communication, the data to be aggregated is transmitted via the on-chip network. Along the data transmission path, network nodes (such as routers) perform the aggregation operation, and the aggregation result is then returned to each PE via the on-chip network.

[0025] Since data flow and aggregation calculations in the on-chip network are all completed by the hardware pipeline, the processing unit can inject the data to be processed into the NoC network and finally receive the execution result without participating in sub-operations such as data aggregation calculations in the aggregation communication process. This effectively releases the computing resources of the PE, reduces the amount of data transmission and communication rounds between PEs, reduces the communication latency and congestion of the on-chip network, and reduces the latency of the PE in providing services.

[0026] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0027] Figure 4 A schematic diagram of the chip structure provided for an exemplary embodiment of this application. Please refer to... Figure 4 Chip 400 can include multiple processing units. Figure 4 Taking nine processing units as an example, they are labeled PE401, PE402, PE403, PE404, PE405, PE406, PE407, PE408, and PE409. In the embodiments of this application, in Figure 4 The example given has nine processing units, but there is no limit to the number of processing units in a chip.

[0028] Each processing unit includes a first set of communication control components. For example, PE401 includes a first set of communication control components 419.

[0029] Chip 400 includes multiple routers, in Figure 4 Taking nine routers as an example, they are labeled as router 410, router 411, router 412, router 413, router 414, router 415, router 416, router 417, and router 418. In this embodiment of the application, in Figure 4 The example uses nine routers, but there is no limit to the number of routers in the chip.

[0030] For any given router, there is at least one processing unit. Figure 4 In this example, each router corresponds to one processing unit. For example, PE401 corresponds to router 410, PE402 corresponds to router 411, and so on. In this embodiment of the application, in Figure 4 In the diagram, the routers corresponding to each processing unit are drawn within the boxes of each processing unit. This is to illustrate the correspondence between the processing units and the routers, and to make the diagrams more intuitive and concise. It does not imply that the physical location of the router is inside the corresponding processing unit. In this embodiment, the physical location of the router is not limited.

[0031] For any given router, the router includes a second set of communication control components. For example, router 410 includes a second set of communication control components 420.

[0032] Multiple routers form a collective communication topology. This topology is created to enable collective communication between multiple processing units. For example, a collective communication topology can be any of the following: tree, ring, or star topology.

[0033] The routers included in a collective communication topology can be all or some of the routers in the chip. In other words, one or more collective communication topologies can exist between the routers in the chip. That is, within the chip, some routers can form collective communication topology 1, and other routers can form collective communication topology 2. The router nodes in collective communication topology 1 and collective communication topology 2 can partially overlap. Figure 4 The following example illustrates the communication topology formed by all routers in the chip.

[0034] A collection communication topology can be used to indicate the upstream and downstream positions of multiple routers in a specified collection communication process. Different upstream and downstream positions correspond to different sub-operations in the specified collection communication.

[0035] like Figure 4 In this context, the communication topology is a tree structure. Routers at the bottom of the tree are leaf nodes, each with an upstream node (parent node) but no downstream node (child node). Routers in the middle layers are intermediate nodes, each with both upstream and downstream nodes. The root node is at the top of the tree, with a downstream node but no upstream node.

[0036] For example, in Figure 4 In this diagram, routers 410, 413, 416, and 412 are all leaf nodes. Routers 411, 414, 417, and 415 are all intermediate nodes, and router 418 is the root node. The upstream node of router 410 is router 411. Routers 410 and 411 can communicate to achieve data transmission, and the same applies to the other routers.

[0037] In the embodiments of this application, the specified collection communication can be any collection communication, such as an All Reduce operation, a Reduce operation, a Broadcast operation, or a Gather operation.

[0038] Chip 400 can be applied in a variety of scenarios that require collective communication, such as distributed training scenarios in the field of artificial intelligence, high-performance computing scenarios, multi-sensor fusion scenarios in autonomous driving, and multi-core parallel computing scenarios in communication baseband processing. This application embodiment does not limit the application scenarios.

[0039] In this embodiment of the application, for any first set communication control component, the processing unit to which the first set communication control component belongs can be called a local processing unit, and the router corresponding to the local processing unit can be called a local router. For example, if the first set communication control component 419 belongs to PE401, then for the first set communication control component 419, PE401 is the local processing unit, and the corresponding router 410 is the local router.

[0040] Similarly, for any second-set communication control component, the router to which the second-set communication control component belongs is called the local router, and the processing unit corresponding to the local router can be called the local processing unit. For example, if the second-set communication control component 420 belongs to router 410, then for the second-set communication control component 420, router 410 is the local router, and the corresponding PE401 is the local processing unit. That is to say, for the first-set communication control component 419 and the second-set communication control component 420, PE401 is the local processing unit, and router 410 is the local router, to distinguish it from other processing units (such as PE402, etc.) or routers (such as router 411, etc.) that do not have a belonging relationship with the first-set communication control component 419 and the second-set communication control component 420.

[0041] In this embodiment of the application, the first collective communication control component can send collective communication instructions and raw data for the processing unit to participate in collective communication to the second collective communication control component in the local router.

[0042] Collective communication commands can be used to instruct routers to perform corresponding sub-operations in collective communication. For example, a collective communication command could be the ALL Reduce command.

[0043] In one optional embodiment, the original data used by a processing unit in the aggregate communication can be the initial data to be used in the aggregate communication stored locally in the processing unit. The specific content of the original data is related to the application scenario of the specified aggregate communication. For example, in the All Reduce operation in a distributed AI training scenario, the original data can be gradient data (such as floating-point tensors) calculated locally by the processing unit; in the Gather scenario, the original data can be feature vectors, intermediate calculation results, or sensor-collected data stored locally by each processing unit. In non-computational aggregate communication scenarios (such as pure data distribution scenarios), the original data can be the data to be forwarded in the processing unit.

[0044] The specific format of the original data (such as floating-point format or integer data format) and the size of the data (such as one-dimensional vector or multi-dimensional tensor) can be flexibly configured according to the actual application requirements. This application embodiment does not impose any restrictions on this.

[0045] For example, taking the All Reduce operation in the AI ​​training scenario as an example, the first set communication control component 419 can send the All Reduce command and the gradient data of PE401 participating in the All Reduce operation to the second set communication control component 420.

[0046] In this embodiment, the second aggregated communication control component can execute the corresponding sub-operation in the specified aggregated communication based on the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instructions. It can then transmit the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication.

[0047] The sub-operation corresponding to the local router is related to its upstream and downstream position in the aggregate communication topology. Different upstream and downstream positions correspond to different sub-operations in the specified aggregate communication.

[0048] If the local router has an upstream node but no downstream node, the second set communication control component can send the raw data to the upstream second set communication control component according to the set communication command. If the local router has both upstream and downstream nodes, the second set communication control component can perform local aggregation on the received data according to the set communication command and send the resulting local data to the upstream second set communication control component. If the local router has a downstream node but no upstream node, the second set communication control component can perform global aggregation on the received data according to the set communication command to obtain the execution result.

[0049] In an optional embodiment, the upstream and downstream position of the local router in the aggregated communication topology can also indicate the node type corresponding to the local router in the aggregated communication topology.

[0050] For example, if the aggregate communication topology is tree-like, the node types can include leaf nodes, intermediate nodes, and sink nodes. If a router has upstream nodes but no downstream nodes, it is a leaf node; if a router has both upstream and downstream nodes, it is an intermediate node; if a router has downstream nodes but no upstream nodes, it can be a root node. In other words, different node types can also correspond to different sub-operations in aggregate communication.

[0051] In the embodiments of this application, regardless of whether the aggregated communication topology is tree, ring, star or other topology, each router can perform corresponding sub-operations according to the upstream and downstream positions of the local router in the topology, thereby flexibly adapting to different types of aggregated communication topologies and having good versatility and scalability.

[0052] A convergence node is a router node in a converged communication topology used to perform global aggregation operations; it is the endpoint of data aggregation. During converged communication, the raw data from each processing unit is transmitted along the converged communication topology and eventually converges at the convergence node. The convergence node then performs global aggregation operations to obtain the execution result of the specified converged communication.

[0053] For example, if the set communication topology is a tree relationship, the sink node can be the root node in the tree relationship; if the set communication topology is a star relationship, the sink node is the center node in the star relationship; if the set communication topology is a ring relationship, the sink node can be a specified node in the ring relationship.

[0054] For example, the second set communication control component 420 can determine that router 410 is a leaf node and the corresponding upstream router is router 411 based on the upstream and downstream positions of router 410 in the set communication topology. Based on the received All Reduce instruction, it can execute the sub-operation (i.e. forwarding the original data) corresponding to the leaf node in the All Reduce operation, encapsulate the received original data into a message M1, and send it to the second set communication control component in router 411 to transmit the original data upstream.

[0055] For example, the second set communication control component in router 411 can receive the All Reduce command and gradient data of PE402 participating in the All Reduce operation sent by the first set communication control component in PE402, and can also receive message M1 sent by the second set communication control component 420 in router 410. This second set communication control component can then determine that router 411 is an intermediate node and the corresponding upstream router is router 414 based on the upstream and downstream positions of the local router (router 411) in the set communication topology. Based on the received All Reduce command, it can execute the corresponding sub-operation of the intermediate node in the All Reduce operation, that is, perform local aggregation on the received data to obtain local result data. This second set communication control component can encapsulate the local result data into message M2 and send it to the second set communication control component in router 414 to transmit the local result data upstream.

[0056] For example, the second set communication control component in router 418 can receive the All Reduce command and gradient data of PE409 participating in the All Reduce operation sent by the first set communication control component in PE409. It can also receive message M3 sent by the second set communication control component in router 417 and message M4 sent by the second set communication control component in router 415. Then, the second set communication control component can determine that router 418 is the root node according to the upstream and downstream position of the local router, i.e., router 418, in the set communication topology. According to the received AllReduce command, it can execute the sub-operation corresponding to the root node in the All Reduce operation, that is, perform global aggregation on the received data (including the gradient data of PE409 participating in the All Reduce operation, the local result data carried in message M3, and the local result data carried in message M4) to obtain the execution result.

[0057] In this embodiment, by embedding a first collective communication control component in the processing unit and deploying a second collective communication control component in the router, complex sub-operations (such as reduction and broadcasting) in collective communication are offloaded from the processing unit to the router for execution. This decouples computing resources from communication tasks, freeing up the computing resources of the processing unit and allowing it to focus on core computing tasks, thus improving the utilization rate of the processing unit's computing resources. Furthermore, a collective communication topology is formed between multiple routers, allowing the routers to automatically complete collective communication sub-operations such as data aggregation and distribution based on the collective communication topology. The processing unit only needs to send the original data and receive the execution results during the specified collective communication process, greatly reducing the amount of data to be transmitted. Moreover, the collective communication topology between routers enables efficient pipelined data transmission, reducing the load and latency of the processing unit during the collective communication process, reducing the waste of computing power caused by communication congestion, reducing the latency of the processing unit providing services, and improving the overall communication throughput and computing efficiency of the chip in large-scale parallel computing scenarios.

[0058] In this embodiment, the internal structure of the first communication control component is not limited. In an optional embodiment, the first communication control component may include a request register, a command register, and a data transmission subcomponent.

[0059] Below, in conjunction with Figure 5 The first communication control component will be described. Figure 5 A schematic diagram of the structure of a first communication control component provided for an exemplary embodiment of this application. Please refer to... Figure 5Taking the first communication control component 419 as an example, the first communication control component 419 may include a request register 421, a command register 422, and a data transmission sub-component 423. The request register 421 can be used to store communication parameters written by the communication software.

[0060] Communication software refers to the driver or firmware program running in the processing unit, used to configure and trigger the router corresponding to the processing unit before the aggregate communication begins. In the embodiments of this application, the same communication software is deployed in each processing unit of the chip. Each communication software can run independently in each processing unit, writing communication parameters and aggregate communication instructions to its corresponding router, thereby enabling each processing unit to initiate the aggregate communication initialization process in parallel.

[0061] In an optional embodiment, the communication parameters may include at least one of the following: the identifier of the processing unit (rankid), the total number of processing units participating in the aggregate communication (nranks), the list of processing unit identifiers participating in the aggregate communication (PE id list), the data type, the data length, and the storage address of the original data in the local memory.

[0062] Command register 422 can be used to store aggregate communication instructions written through the communication software. Aggregate communication instructions can be used to instruct the router to perform corresponding aggregate communication operations, such as All Reduce, Reduce, Broadcast, and Gather instructions.

[0063] The data transmission subcomponent 423 can be used to read raw data from the local memory of the local processing unit in response to a set communication instruction, encapsulate the communication parameters, raw data, and set communication instruction into a local communication message, and send the local communication message to the second set communication control component. For example, the data transmission subcomponent can be a DMA (Direct Memory Access) controller.

[0064] In this embodiment of the application, the local communication message refers to the message generated by the first collective communication control component and sent to the second collective communication control component in the local router. It is used to carry the original data and control information (such as communication parameters and collective communication instructions) of the processing unit participating in collective communication, so as to trigger the collective communication processing flow on the router side.

[0065] In the embodiments of this application, the local memory can be a hardware unit inside the processing unit used for data storage. Exemplarily, the local memory can be SRAM (Static Random-Access Memory) or DRAM (Dynamic Random Access Memory) for storing data required by the processing unit during computation and communication.

[0066] In one optional embodiment, the local memory may include a send buffer and a receive buffer. The send buffer can be used to store raw data to be sent (such as gradient data, data to be broadcast, etc.); the receive buffer can be used to store received execution results, etc. In this way, the local memory physically achieves directional isolation of the data flow, ensuring that sending and receiving operations do not interfere with each other.

[0067] In an alternative embodiment, the first communication control component 419 may further include a response register 424.

[0068] The response register 424 can be used to store the execution status of a specified set of communications returned by the second communication control component.

[0069] For example, in the All Reduce operation, after the second set communication control component at the aggregation endpoint completes global aggregation and obtains the execution result, the execution result is passed down to each router level by level through the set communication topology. The second set communication control component in each router can encapsulate the received execution result into an execution result message and send it to each processing unit. The response register in each processing unit can store the execution status, and the receive buffer can store the execution result.

[0070] For example, after receiving the execution result, the second set communication control component 420 can encapsulate the execution result into an execution result message, which can be sent to the first set communication control component 419 in PE401 via the network interface of router 410. Optionally, other NoC components between PE401 and router 410 (such as the network interface NI) will parse the execution result message to obtain the execution result, write the data in the execution result into the receive buffer (recvbuff) of PE401, and write information such as the completion status or data length of the execution result into the response register 424.

[0071] The communication software running on the PE401 can detect that the response register 424 has been updated through polling or interruption. It can then read the completion status of the execution result from the response register 424 and read the execution result from the receive buffer to complete the current aggregation communication operation. For example, in the All Reduce operation, the PE401 reads the global aggregation result (such as the sum or average of the gradients of each PE) from the receive buffer.

[0072] In this embodiment, the request register and command register in the first collective communication control component provide a standard interface for the communication software to issue configuration and trigger instructions to the hardware; the response register provides a low-latency, highly deterministic execution status acquisition channel for the processing unit, so that the processing unit does not need to query the communication status through software methods such as interrupts or polling, but can obtain the execution status of the collective communication through register read operations, further reducing the software overhead of the processing unit; the data transmission sub-component realizes the automated encapsulation and transmission of raw data and communication parameters, so that the processing unit does not need to participate in the subsequent data handling and message encapsulation process, thereby realizing the effective offloading of the collective communication task from the processing unit side.

[0073] Throughout the entire aggregation communication process, the processing unit writes configuration parameters and trigger instructions to the request register and command register. After communication is complete, it reads the execution status from the response register and the execution result from the receive buffer. Data transmission, routing, aggregation operations, and other intermediate processes are all automatically handled by the hardware without the processing unit's involvement. Therefore, the processing unit's computing resources are freed up from the aggregation communication task, allowing it to focus on core computing tasks and improving the utilization rate of its computing resources.

[0074] In this embodiment, the internal structure of the second communication control component is not limited. In an optional embodiment, the second communication control component may include a communication topology subcomponent, a computation subcomponent, a buffer memory, a message generation subcomponent, and a routing subcomponent.

[0075] Below, in conjunction with Figure 6 The second communication control component will be described. Figure 6 A schematic diagram of the structure of the second communication control component provided for an exemplary embodiment of this application. Please refer to... Figure 6 Taking the second communication control component 420 as an example, the second set of communication control components 420 may include: a communication topology subcomponent 425, an operation subcomponent 426, a cache memory 427, a message generation subcomponent 428, and a routing subcomponent 429.

[0076] The communication topology sub-component 425 can be used to store communication topology relationships and the upstream and downstream locations of the local router.

[0077] The operation sub-component 426 can be used to perform local or global aggregation on the received data based on the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instructions.

[0078] For example, taking All Reduce as an example of aggregated communication, if the local router to which the second aggregated communication control component belongs is an intermediate node, the operation sub-component 426 can perform local aggregation on the received data to obtain local result data; if the local router to which the second aggregated communication control component belongs is the aggregation endpoint (such as the root node in a tree topology), the operation sub-component 426 can perform global aggregation on the received data to obtain the execution result of the specified aggregated communication.

[0079] The cache memory 427 can be used to temporarily store at least one of the following: raw data, partial result data, and execution results. For example, the cache memory 427 can be SRAM or DRAM.

[0080] The second set of communication components belongs to different local router node types, and the data content stored in the cache memory 427 is different.

[0081] For example, if the local router to which the second set communication control component belongs is a leaf node, the cache memory 427 can be used to temporarily store the original data; if the local router to which it belongs is an intermediate node, the cache memory 427 can be used to temporarily store the original data and local result data; if the local router to which it belongs is a convergence endpoint, the cache memory 427 can be used to temporarily store the execution result.

[0082] The message generation subcomponent 428 can be used to generate corresponding messages based on the upstream and downstream positions of the local router in the aggregate communication topology and the data stored in the cache memory 427.

[0083] The message generation subcomponent 428 generates different messages depending on the node type of the local router to which the second set of communication components belongs.

[0084] For example, if the local router to which the second set communication control component belongs is a leaf node, the message generation subcomponent 428 can generate a message carrying the original data based on the original data temporarily stored in the cache memory 427, and send it to the upstream node; if the local router to which it belongs is an intermediate node, the message generation subcomponent 428 can generate a message carrying the local result data based on the local result data obtained by the operation subcomponent 426, and send it to the upstream node; if the local router to which it belongs is the aggregation endpoint, the message generation subcomponent 428 can generate a message carrying the execution result based on the execution result obtained by the operation subcomponent 426, and send it to the downstream node or to the local processing unit during the broadcast phase.

[0085] The routing subcomponent 429 can be used to send the message generated by the message generation subcomponent 428 according to the upstream and downstream position of the local router in the set communication topology.

[0086] In an optional embodiment, the second set of communication control components 420 may further include: a message processing subcomponent 430 and a status management subcomponent 431.

[0087] The message processing subcomponent 430 can be used to extract data from a message.

[0088] In an optional embodiment, if the message is a local communication message, the message processing subcomponent 430 can extract the original data and communication control instructions from the local communication message; if the message is a downstream communication message, the message processing subcomponent 430 can extract the partial result data from the downstream communication message. Here, a downstream communication message refers to a message sent by a downstream router.

[0089] The state management subcomponent 431 can be used to switch the communication state of the local router in a specified set of communications when the state switching conditions are met. The communication state is either aggregated or broadcast.

[0090] The aggregation state refers to the state when a local router participates in the upstream data aggregation. In this state, leaf nodes forward the original data upstream, intermediate nodes perform local aggregation on the received data and send the local result data upstream, and the aggregation endpoint performs global aggregation on the received data to obtain the execution result.

[0091] Broadcast state refers to the state in which the local router participates in distributing the execution results downstream. In this state, the sink sends a message carrying the execution results to downstream nodes. After receiving the execution results, intermediate nodes forward them downstream and simultaneously send the execution results to their local processing units. After receiving the execution results, leaf nodes send them to their local processing units.

[0092] In an optional embodiment, the second set communication control component can also be used to send the execution result to the local processing unit; and, if there is a downstream second set communication control component, to send the execution result to the downstream second set communication control component according to the set communication topology.

[0093] For example, if the router to which the second aggregate communication control component belongs is a leaf node, then the second aggregate communication control component can send the received execution result to the local processing unit. If the router to which the second aggregate communication control component belongs is an intermediate node, then the second aggregate communication control component sends the execution result to the local processing unit, and simultaneously forwards the execution result to downstream second aggregate communication control components according to the aggregate communication topology. If the router to which the second aggregate communication control component belongs is an aggregation endpoint (such as the root node in a tree topology), then after completing global aggregation and obtaining the execution result, the second aggregate communication control component sends the execution result to the local processing unit, and also sends a message carrying the execution result to all downstream second aggregate communication control components according to the aggregate communication topology to initiate data distribution in the broadcast phase.

[0094] by Figure 4 Taking the tree topology shown as an example, after the aggregation endpoint (router 418) completes global aggregation and obtains the execution result, it can send the execution result to the local processing unit (PE409) and send the message carrying the execution result to the downstream intermediate nodes, namely routers 417 and 415. After receiving the execution result, each intermediate node can send the execution result to its local processing unit and continue to forward the execution result to the downstream leaf nodes node by node. For example, router 417 can send the execution result to PE408, and then to routers 416 and 414; and so on. After receiving the execution result, routers 410, 413, 416, and 412 can execute and send it to the corresponding local processing unit. In this way, the execution result is passed to each processing unit along the aggregate communication topology, thereby completing the distribution process of the broadcast phase.

[0095] Specifically, when the second set communication control component 420 sends the execution result to the local processing unit (PE401), the message generation subcomponent 428 can generate an execution result message carrying the execution result based on the execution result temporarily stored in the cache memory 427. The execution result message may include the execution result and destination address information (such as the address of the local processing unit PE401), and is sent to the local processing unit PE401 through the routing subcomponent 429.

[0096] In this embodiment, by setting a communication topology subcomponent, an operation subcomponent, a cache memory, a message generation subcomponent, a routing subcomponent, a message processing subcomponent, and a state management subcomponent in the second aggregated communication control component, the aggregated communication task is fully offloaded and hardware-accelerated from the processing unit to the router. This allows data aggregation operations to be completed step-by-step by the hardware pipeline along the data transmission path without the need for processing unit intervention. This effectively reduces the computing resource consumption of the processing unit, reduces the service latency of the processing unit, reduces the amount of on-chip data transmission and communication rounds, alleviates on-chip network congestion, and thus improves the overall aggregated communication efficiency and concurrent processing capability of the chip.

[0097] In one optional embodiment, the router includes a packet parsing component. The packet parsing component can parse received packets to determine the packet type, and if the packet type is a collective communication type, forward the packet to a second collective communication control component. If the packet type is a non-collective communication type (such as a regular data packet), the packet is forwarded to the router's normal routing processing path, whereby the router forwards it according to its regular routing policy.

[0098] Each router receives various messages. Each message may carry a message type identifier. The message type identifier can be indicated by specific fields or flags in the message header. For example, message types may include ordinary data messages and aggregated communication messages. Ordinary data messages refer to regular data messages that do not involve aggregated communication operations; aggregated communication messages refer to messages related to aggregated communication operations. Optionally, aggregated communication messages can be further subdivided into local communication messages, downstream communication messages, and upstream communication messages, corresponding to aggregated communication messages from different directions or sources.

[0099] In one optional embodiment, when the message type identifier in the message header is a first preset value (e.g., 0), the message can be determined to be a normal data message; when the message type identifier in the message header is a second preset value (e.g., 1), the message can be determined to be a collective communication message. After determining that the message is a collective communication message, the message parsing component can forward the message to the second collective communication control component, where the message processing subcomponent in the second collective communication control component can further extract the data content from the message.

[0100] by Figure 4Taking router 410 as an example, router 410 may include a message parsing component. The message parsing component can parse the received message and read the message type identifier in the message header. If the message type identifier included in the message is a normal message identifier (e.g., the flag bit is 0), then the message can be determined to be a normal data message. The message parsing component forwards the normal data message to the normal routing processing path of router 410, and router 410 forwards it according to the regular routing policy. If the message type identifier included in the message is a set communication identifier (e.g., the flag bit is 1), then the message can be determined to be a set communication message. In this case, the message parsing component sends the set communication message to the second set communication control component 420 in router 410, and the second set communication control component 420 performs subsequent set communication processing, such as extracting data from the message, performing aggregation operations, or forwarding.

[0101] In this embodiment, the message parsing component can quickly identify and distribute messages at the router level, enabling aggregated communication messages to be routed to the second aggregated communication control component for processing in a timely manner, while ordinary data messages are forwarded according to the conventional path. This reduces the interference of aggregated communication processing logic on ordinary data communication, realizes the separate processing of aggregated communication messages and ordinary data messages, thereby improving the router's processing efficiency and the overall performance of the chip.

[0102] Before the specified set communication process begins, an initialization process must be performed. In an optional embodiment, the first set communication control component can also be used to send topology generation instructions and initialization parameters to the second set communication control component before the specified set communication process begins.

[0103] In this embodiment, the topology generation instruction can be used to instruct the second set communication control component to initiate the generation process of the set communication topology relationship. The topology generation instruction can be a preset opcode (e.g., 0x01), or it can be an independent command word or trigger signal; this application does not impose specific limitations on it.

[0104] The initialization parameters include the identifiers (such as a list of PE IDs) of multiple processing units participating in the specified aggregate communication process. Optionally, the initialization parameters may also include at least one of the following communication parameters: the rank ID of each processing unit, the total number of processing units participating in aggregate communication (nranks).

[0105] Taking the first set communication control component 419 as an example, before the specified set communication process begins, the communication software in PE401 can write initialization parameters to the request register 421 and topology generation instructions to the command register 422. Writing the topology generation instructions to the command register 422 triggers the data transmission subcomponent 423. The data transmission subcomponent 423 can obtain the initialization parameters from the request register 421 and pass the initialization parameters and topology generation instructions to the network interface. The network interface encapsulates the initialization parameters and topology generation instructions into a message M5 according to the on-chip network protocol format, adds a message type identifier to the message M5, and sends the message M5 to the second set communication control component 420 in router 410 through the physical link.

[0106] Optionally, the Network Interface (NI) is a hardware interface module between the processing unit and the router. It can encapsulate data and commands from the PE side into messages conforming to the NoC transmission format according to the on-chip network protocol and send them to the router via the physical link. In this embodiment, the network interface can exist independently of the first set of communication control components and the second set of communication control components, serving as the communication interface between the processing unit and the router in the chip; it can also be integrated into the data transmission sub-component. This embodiment does not limit this.

[0107] The second set communication control component can also be used to respond to a topology generation command, determine multiple routers participating in the specified set communication based on multiple processing units participating in the specified set communication process, and generate set communication topology relationships based on the physical connection relationships and topology algorithms between the multiple routers.

[0108] The aggregated communication topology is identical and generated independently across multiple second aggregated communication control components. This ensures consistency in the aggregated communication topology across all routers, eliminating the need for additional synchronization negotiation, reducing synchronization overhead between routers, and improving the efficiency of aggregated communication topology generation.

[0109] The physical connections between multiple routers are embedded in the chip. This means the physical connections are determined by the chip's NoC network architecture during the chip design phase and remain fixed after chip manufacturing. In other words, the wiring topology between routers is a pre-defined, fixed on-chip network physical topology. Routers are connected via physical links to form a NoC interconnect network. After the chip powers on, each router can obtain this physical connection through hardware registers, read-only memory (ROM), or hard-coded logic, without requiring software configuration or dynamic detection.

[0110] like Figure 7 middle, Figure 7This is a schematic diagram illustrating the physical connection relationship between multiple routers, provided as an exemplary embodiment of this application. Figure 7 The chip internally uses a 2DMesh (two-dimensional mesh) topology as an example to illustrate the physical connections between multiple routers. Routers 410-418 are arranged in a mesh pattern, and each router is connected to its surrounding neighboring routers via physical links. This physical connection relationship is determined during chip design and can be read by each router after power-on.

[0111] In the embodiments of this application, Figure 7 This example uses a 2D mesh topology, but the physical connection relationship can also be any of the following: ring, star, tree, torus, or a custom topology. This application does not limit the physical topology type between routers within the chip.

[0112] The communication topology subcomponent of the second set communication control component can have a pre-defined topology algorithm. Based on the physical connection relationships and combined with the list of processing units participating in the set communication, the set communication topology can be generated based on the topology algorithm. The topology algorithm can be any of the following: spanning tree algorithm, BFS (Breadth-First Search) spanning tree algorithm, minimum spanning tree algorithm (such as Prim's algorithm), shortest path tree algorithm, or deterministic topology generation algorithm based on node identifier sorting, etc.

[0113] In this embodiment, the spanning tree algorithm is used as an example to illustrate the topology. Based on the spanning tree algorithm, a portion of the physical links between multiple routers can be selected to form an acyclic logical tree that connects all routers participating in the aggregated communication, serving as the aggregated communication topology.

[0114] Specifically, the spanning tree algorithm can construct an acyclic logical tree based on the physical connections between multiple routers, ensuring that all routers participating in the aggregate communication are connected, by selecting a subset of physical links. In the logical tree, each router node has a defined upstream and downstream position, and data is transmitted between routers along the tree structure.

[0115] For example, the spanning tree algorithm can construct the topology of a set communication network as follows: First, a router is determined from the routers participating in the set communication network as the aggregation endpoint (i.e., the root node). For example, the participating router with the smallest or largest identifier can be determined as the root node. Then, starting from the root node, the network expands layer by layer in the physical connection relationship according to a breadth-first or depth-first traversal method until all routers participating in the set communication network are included in the tree. Each router node records its corresponding parent and child node information, thus forming a tree-like relationship with clear upstream and downstream relationships.

[0116] In a tree-like structure, data can flow along the tree. Leaf nodes can pass raw data from their local processing units upstream (parent nodes); intermediate nodes receive data from their corresponding local processing units and downstream (child nodes), perform local aggregation, and then pass the resulting local data upstream; the final destination (root node) receives data from its corresponding local processing units and downstream, performs global aggregation, and obtains the execution result. Subsequently, the final destination passes the execution result downstream along the tree structure until all leaf nodes have received the execution result, thus completing the entire process of aggregate communication.

[0117] Since all routers participating in the aggregated communication use the same topology algorithm in their communication topology sub-components and are calculated based on the same physical connections and the same set of participating routers, the aggregated communication topology relationships generated independently by each router are consistent, providing a topological foundation for the subsequent aggregated communication process. This ensures topology consistency while reducing the synchronization communication overhead during the initialization phase, thereby further reducing the overall latency of the aggregated communication.

[0118] by Figure 4 Taking router 410 as an example, before the aggregated communication process begins, after receiving message M5 sent by the first aggregated communication control component 419, router 410 can parse message M5 through the message parsing component to determine the message type of message M5. Since message M5 carries an aggregated communication identifier, the message parsing component can determine that message M5 is an aggregated communication type message and send message M5 to the second aggregated communication control component 420.

[0119] The message processing subcomponent 430 in the second set communication control component 420 can extract the topology generation command and initialization parameters from the message M5, and pass them to the communication topology subcomponent 425. The communication topology subcomponent 425 can determine the multiple routers participating in this set communication based on the list of participating processing units in the initialization parameters. Then, the communication topology subcomponent 425 can, based on... Figure 7The physical connections between the routers shown are calculated independently using preset topology algorithms (such as the spanning tree algorithm) to generate the topology as follows: Figure 4 The set communication topology is shown in the figure.

[0120] After generating the aggregated communication topology, the communication topology subcomponent 425 can store the aggregated communication topology, the corresponding relationship identifier, and the upstream and downstream positions of the local router (i.e., router 410) in the aggregated communication topology. Specifically, the communication topology subcomponent 425 can store information such as the node type (e.g., leaf node, intermediate node, or aggregation endpoint) corresponding to router 410, the upstream node identifier (parent node), and the downstream node identifier (child node list).

[0121] The communication topology subcomponent 425 can pass the identifier of the set communication topology relationship (such as topology ID) to the message generation subcomponent 428.

[0122] The message generation subcomponent 428 can write the topology identifier of the aggregated communication topology relationship into the response message, and send the response message to the local processing unit (i.e., PE401) through the routing subcomponent 429. Specifically, the response message is sent to the first aggregated communication control component 419 in PE401 via the network interface of router 410. The first aggregated communication control component 419 writes the topology identifier in the response message into the response register 424. The communication software can read the response register 424 to obtain and record the identifier of the aggregated communication topology relationship.

[0123] Optionally, if the communication topology subcomponent 425 generates multiple set communication topology relationships (e.g., corresponding to different communication domains or different set communication types), the message generation subcomponent 428 can write the topology identifiers of the multiple set communication topology relationships into the corresponding response messages and send them to the local processing unit in sequence, so that the communication software can uniformly manage and maintain the multiple topology identifiers.

[0124] In this embodiment, for other routers participating in this aggregated communication within the chip, their internal second aggregated communication control component also executes the aforementioned topology generation and identifier return process, ensuring that the communication software corresponding to each processing unit participating in the aggregated communication obtains the same topology identifier for the aggregated communication topology relationship. Since each router independently generates the aggregated communication topology relationship based on the same physical connection relationship, the same list of participating processing units, and the same topology algorithm, the topology identifiers returned by each router are identical. The topology identifiers recorded by the communication software on any processing unit are consistent, enabling each processing unit to reference the same aggregated communication topology relationship through the topology identifier in subsequent aggregated communication operations, eliminating the need to repeatedly execute the topology generation process and reducing the overhead of aggregated communication.

[0125] Figure 8This is a flowchart illustrating a collection communication method provided for an exemplary embodiment of this application. Figure 8 The method shown can be applied to a chip, which includes multiple processing units and multiple routers. Each processing unit includes a first set communication control component, and each router corresponds to at least one processing unit and includes a second set communication control component. A set communication topology is formed among the multiple routers. This topology indicates the upstream and downstream positions of the multiple routers in a specified set communication process, with different upstream and downstream positions corresponding to different sub-operations within the specified set communication. See also... Figure 8 The method may include: S801, the first collective communication control component sends collective communication instructions and raw data for the processing unit to participate in collective communication to the second collective communication control component in the local router.

[0126] S802, the second collective communication control component executes the corresponding sub-operation in the specified collective communication according to the upstream and downstream position of the local router in the collective communication topology and the received collective communication instructions.

[0127] S803, the second set communication control component transmits the original data or result data corresponding to the sub-operation to the upstream second set communication control component until the second set communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified set communication.

[0128] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0129] Figure 9 A flowchart illustrating another collection communication method provided for an exemplary embodiment of this application. Please refer to... Figure 9 The method may include: S901. Send a collective communication instruction and the original data of the local processing unit participating in the collective communication to the second collective communication control component in the local router, so that the second collective communication control component can execute the corresponding sub-operation in the specified collective communication according to the upstream and downstream position of the local router in the collective communication topology and the received collective communication instruction, until the execution result of the specified collective communication is obtained. The aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in a specified aggregated communication process. Different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

[0130] In one optional embodiment, sending a collective communication instruction and the original data for collective communication involving the local processing unit to the second collective communication control component in the local router may include: reading the original data from the local memory of the local processing unit; reading communication parameters from the request register and reading the collective communication instruction from the command register; encapsulating the communication parameters, the original data, and the collective communication instruction into a local communication message and sending the local communication message to the second collective communication control component.

[0131] In an optional embodiment, the above method may further include: receiving the execution result and the execution status of the specified collection communication returned by the second collection communication control component.

[0132] In an optional embodiment, the method may further include: before the specified aggregate communication process begins, sending a topology generation command and initialization parameters to a second aggregate communication control component, so that the second aggregate communication control component responds to the topology generation command, determines multiple routers participating in the specified aggregate communication based on multiple processing units participating in the specified aggregate communication process, and generates an aggregate communication topology relationship based on the physical connection relationship and topology algorithm between the multiple routers, wherein the aggregate communication topology relationship is the same and generated independently in multiple second aggregate communication control components; wherein the initialization parameters include the identifiers of the multiple processing units participating in the specified aggregate communication process.

[0133] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0134] Figure 10 A flowchart illustrating yet another collection communication method provided for an exemplary embodiment of this application. Please refer to... Figure 10 The method may include: S1001: Receive the collection communication instruction and the original data of the processing unit participating in the collection communication sent by the first collection communication control component in the local processing unit.

[0135] S1002. Based on the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instructions, execute the corresponding sub-operation in the specified aggregated communication, and transmit the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication.

[0136] The aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in a specified aggregated communication process. Different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

[0137] In one optional embodiment, based on the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instructions, the corresponding sub-operation in the specified aggregated communication is executed, and the original data or result data corresponding to the sub-operation is transmitted to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication. This includes: if the local router has an upstream node but no downstream node, then according to the aggregated communication instructions, the original data is sent to the upstream second aggregated communication control component; if the local router has both upstream and downstream nodes, then according to the aggregated communication instructions, the received data is locally aggregated, and the obtained local result data is sent to the upstream second aggregated communication control component; if the local router has a downstream node but no upstream node, then according to the aggregated communication instructions, the received data is globally aggregated to obtain the execution result.

[0138] In an optional embodiment, the method further includes: sending the execution result and the execution status of the specified collection communication to the local processing unit; and, if there is a downstream second collection communication control component, sending the execution result to the downstream second collection communication control component according to the collection communication topology.

[0139] In an optional embodiment, the method further includes: before the specified aggregate communication process begins, receiving a topology generation instruction and initialization parameters sent by a first aggregate communication control component, wherein the initialization parameters include identifiers of multiple processing units participating in the specified aggregate communication process; in response to the topology generation instruction, determining multiple routers participating in the specified aggregate communication based on the multiple processing units participating in the specified aggregate communication process; and generating an aggregate communication topology relationship based on the physical connection relationship between the multiple routers and a topology algorithm, wherein the aggregate communication topology relationship is the same and generated independently in multiple second aggregate communication control components.

[0140] The detailed implementation methods and beneficial effects of each step in this embodiment have been described in detail in the foregoing embodiments, and will not be elaborated here.

[0141] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps S1001 to S1002 can be device A; or the execution subject of step S1001 can be device A, and the execution subject of step S1002 can be device B; and so on.

[0142] Furthermore, in some of the processes described in the above embodiments and accompanying drawings, multiple operations appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The sequence numbers are merely used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0143] Figure 11 This is a schematic diagram of the structure of a computing device provided for an exemplary embodiment of this application. Please refer to... Figure 11 The computing device 1100 may include a memory 1101 and a chip 1102.

[0144] Memory 1101 is used to store computer programs and can be configured to store various other data to support operation on the computing platform. Examples of such data include, but are not limited to: computer program instructions, training data, intermediate computation results, and collective communication configuration parameters.

[0145] In this embodiment of the application, the computer program stored in the memory 1101 can be used to trigger the initiation and configuration of the aggregate communication operation, but does not participate in the core operations such as data transfer, routing forwarding and aggregation calculation in the aggregate communication process.

[0146] Chip 1102, coupled to memory 1101, is used to execute the computer program in memory 1101 to perform aggregated communication operations. Specifically: chip 1102 can send aggregated communication instructions and raw data for processing units to participate in aggregated communication to the second aggregated communication control component in the local router through the first aggregated communication control component; chip 1102 can execute the corresponding sub-operation in the specified aggregated communication through the second aggregated communication control component according to the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instructions; chip 1102 can transmit the raw data or result data corresponding to the sub-operation to the upstream second aggregated communication control component through the second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication.

[0147] Furthermore, such as Figure 11 As shown, the computing device also includes: a communication component 1103, a display 1104, a power supply component 1105, an audio component 1106, and other components.

[0148] Figure 11The diagram only shows some components and does not mean that the computing device includes only these components. Figure 11 The components shown. Additionally... Figure 11 The components within the dashed box are optional, not mandatory, and their specific requirements depend on the form factor of the computing device. The computing device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server-side device such as a conventional server, cloud server, or server array. If the computing device in this embodiment is implemented as a terminal device such as a desktop computer, laptop computer, or smartphone, it may include... Figure 11 The components within the dashed box; if the computing device in this embodiment is implemented as a conventional server, cloud server, or server array, etc., then it may not include... Figure 11 The component within the dashed box.

[0149] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as SRAM, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0150] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0151] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0152] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0153] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0154] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a chip, enables the chip to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium. Accordingly, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a chip, the chip is able to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by a computer program or instructions. In addition, these computer programs or instructions can be applied to chips of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices, so that the chips of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices can be implemented as means to implement the corresponding functions in the above method embodiments.

[0155] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0156] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A chip, characterized in that, include: Multiple processing units, the processing units including a first set of communication control components; Multiple routers, each router corresponding to at least one processing unit and including a second aggregated communication control component; the multiple routers form an aggregated communication topology, which is used to indicate the upstream and downstream positions of the multiple routers in a specified aggregated communication process, with different upstream and downstream positions corresponding to different sub-operations in the specified aggregated communication; The first aggregate communication control component is used to send aggregate communication instructions and the original data of the processing unit participating in aggregate communication to the second aggregate communication control component in the local router corresponding to the local processing unit; The second aggregated communication control component is used to execute the corresponding sub-operation in the specified aggregated communication according to the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instruction, and to transmit the original data or result data corresponding to the sub-operation to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint has completed the corresponding sub-operation and obtained the execution result of the specified aggregated communication.

2. The chip according to claim 1, characterized in that, The second set of communication control components is specifically used for: If the local router has an upstream node but no downstream node, then according to the aggregated communication instruction, the original data is sent to the upstream second aggregated communication control component; If the local router has upstream and downstream nodes, it performs local aggregation on the received data according to the aggregated communication instruction, and sends the obtained local result data to the upstream second aggregated communication control component. If the local router has downstream nodes but no upstream nodes, then the received data is globally aggregated according to the aggregated communication instruction to obtain the execution result.

3. The chip according to claim 1, characterized in that, The second set of communication control components includes: a communication topology subcomponent, a computation subcomponent, a buffer memory, a message generation subcomponent, and a routing subcomponent, wherein, The communication topology sub-component is used to store the communication topology relationships and the upstream and downstream locations of the local router; The computational subcomponent is used to perform local or global aggregation on the received data based on the upstream and downstream position of the local router in the aggregated communication topology and the received aggregated communication instructions. The cache memory is used to temporarily store at least one of the original data, partial result data, and execution result; The message generation subcomponent is used to generate corresponding messages based on the upstream and downstream positions of the local router in the aggregated communication topology and the data stored in the cache memory; The routing sub-component is used to send the message according to the upstream and downstream position of the local router in the aggregate communication topology.

4. The chip according to claim 3, characterized in that, The second set of communication control components also includes: A message processing subcomponent is used to extract data from the message; The state management subcomponent is used to switch the communication state of the local router in the specified set of communications when the state switching conditions are met. The communication state is either an aggregation state or a broadcast state.

5. The chip according to claim 4, characterized in that, When the message processing subcomponent extracts data from the message, it is specifically used for: If the message is a local communication message, then extract the original data and communication control instructions from the local communication message; If the message is a downstream communication message, then extract the local result data from the downstream communication message.

6. The chip according to claim 4, characterized in that, The local router includes a packet parsing component; The message parsing component is used to parse the received message to determine the message type, and when the message type is a collection communication type, forward the message to the second collection communication control component.

7. The chip according to any one of claims 1-2 or 4-6, characterized in that, The first set of communication control components includes a request register, a command register, and a data transmission sub-component, wherein, The request register is used to store communication parameters written by the communication software; The command register is used to store the collective communication instructions written by the communication software; The data transmission subcomponent is configured to, in response to the aggregated communication instruction, read the original data from the local memory of the local processing unit, encapsulate the communication parameters, the original data, and the aggregated communication instruction into a local communication message, and send the local communication message to the second aggregated communication control component.

8. The chip according to any one of claims 1-2 or 4-6, characterized in that, The second set of communication control components is also used for: Send the execution result and the execution status of the specified set to the local processing unit; and, If a downstream second collective communication control component exists, the execution result is sent to the downstream second collective communication control component according to the collective communication topology. Accordingly, the first set of communication control components also includes: A response register is used to store the execution status of the specified set communication returned by the second set communication control component.

9. The chip according to any one of claims 1-2 or 4-6, characterized in that, The first set communication control component is further configured to send a topology generation instruction and initialization parameters to the second set communication control component before the specified set communication process begins, wherein the initialization parameters include identifiers of multiple processing units participating in the specified set communication process; The second set communication control component is further configured to, in response to the topology generation instruction, determine multiple routers participating in the specified set communication based on multiple processing units participating in the specified set communication process, and generate the set communication topology relationship based on the physical connection relationship and topology algorithm between the multiple routers, wherein the set communication topology relationship is the same and generated independently in multiple second set communication control components.

10. A collective communication method, characterized in that, The method is applied to a chip, which includes multiple processing units and multiple routers. Each processing unit includes a first aggregated communication control component, and each router corresponds to at least one of the processing units and includes a second aggregated communication control component. The multiple routers form an aggregated communication topology, which indicates the upstream and downstream positions of the multiple routers in a specified aggregated communication process. Different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication. The first aggregate communication control component sends aggregate communication instructions and the original data for aggregate communication to the second aggregate communication control component in the local router; The second aggregate communication control component executes the corresponding sub-operation in the specified aggregate communication based on the upstream and downstream position of the local router in the aggregate communication topology and the received aggregate communication instruction; The second set communication control component transmits the original data or result data corresponding to the sub-operation to the upstream second set communication control component until the second set communication control component at the convergence endpoint completes the corresponding sub-operation and obtains the execution result of the specified set communication.

11. A collective communication method, characterized in that, Applied to a first set of communication control components, wherein the first set of communication control components is located in a local processing unit of a chip, the method includes: Send a collective communication instruction and the original data of the local processing unit participating in the collective communication to the second collective communication control component in the local router, so that the second collective communication control component executes the corresponding sub-operation in the specified collective communication according to the upstream and downstream position of the local router in the collective communication topology and the received collective communication instruction, until the execution result of the specified collective communication is obtained; The aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in the specified aggregated communication process, and different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

12. The method according to claim 11, characterized in that, Sending a collective communication command and the raw data for collective communication involving the local processing unit to the second collective communication control component in the local router, including: The original data is read from the local memory of the local processing unit; Read the communication parameters from the request register and read the set communication instructions from the command register; The communication parameters, the raw data, and the collective communication instructions are encapsulated into a local communication message, and the local communication message is sent to the second collective communication control component.

13. The method according to claim 11 or 12, characterized in that, The method further includes: Receive the execution result and the execution status of the specified set communication returned by the second set communication control component.

14. The method according to claim 11 or 12, characterized in that, The method further includes: Before the specified set communication process begins, a topology generation instruction and initialization parameters are sent to the second set communication control component, so that the second set communication control component responds to the topology generation instruction, determines multiple routers participating in the specified set communication based on multiple processing units participating in the specified set communication process, and generates the set communication topology relationship based on the physical connection relationship and topology algorithm between the multiple routers. The set communication topology relationship is the same and generated independently in multiple second set communication control components. The initialization parameters include identifiers of multiple processing units participating in the specified set communication process.

15. A collective communication method, characterized in that, The method, applied to a second set of communication control components located in a local router of the chip, includes: Receive the collection communication instruction sent by the first collection communication control component in the local processing unit and the original data of the processing unit participating in the collection communication; Based on the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instructions, the corresponding sub-operation in the specified aggregated communication is executed, and the original data or result data corresponding to the sub-operation is transmitted to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation and obtains the execution result of the specified aggregated communication. The aggregated communication topology is used to indicate the upstream and downstream positions of multiple routers in the specified aggregated communication process, and different upstream and downstream positions correspond to different sub-operations in the specified aggregated communication.

16. The method according to claim 15, characterized in that, Based on the upstream and downstream positions of the local router in the aggregated communication topology and the received aggregated communication instructions, the corresponding sub-operation in the specified aggregated communication is executed, and the original data or result data corresponding to the sub-operation is transmitted to the upstream second aggregated communication control component until the second aggregated communication control component at the aggregation endpoint completes the corresponding sub-operation, thereby obtaining the execution result of the specified aggregated communication, including: If the local router has an upstream node but no downstream node, then according to the aggregated communication instruction, the original data is sent to the upstream second aggregated communication control component; If the local router has upstream and downstream nodes, it performs local aggregation on the received data according to the aggregated communication instruction, and sends the obtained local result data to the upstream second aggregated communication control component. If the local router has downstream nodes but no upstream nodes, then the received data is globally aggregated according to the aggregated communication instruction to obtain the execution result.

17. The method according to claim 15 or 16, characterized in that, The method further includes: Send the execution result and the execution status of the specified set to the local processing unit; and, If a downstream second set communication control component exists, the execution result is sent to the downstream second set communication control component according to the set communication topology.

18. The method according to claim 15 or 16, characterized in that, The method further includes: Before the specified set communication process begins, a topology generation instruction and initialization parameters are received from the first set communication control component, wherein the initialization parameters include identifiers of multiple processing units participating in the specified set communication process; In response to the topology generation instruction, multiple routers participating in the specified set communication process are determined based on the multiple processing units participating in the specified set communication process; Based on the physical connection relationship and topology algorithm between the multiple routers, the aggregate communication topology relationship is generated. The aggregate communication topology relationship is the same and generated independently in multiple second aggregate communication control components.

19. A computing device, characterized in that, Includes memory and the chip as described in any one of claims 1-9; The memory is used to store computer programs; The chip, coupled to the memory, is used to execute a computer program in the memory to implement the steps of the method according to any one of claims 10-18.

20. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it causes the processor to perform the steps of the method according to any one of claims 10-18.

21. A computer program product, characterized in that, Includes a computer program / instruction that, when executed by a processor, causes the processor to perform the steps of the method according to any one of claims 10-18.

Images