In-network computing method and related device

Abstract

The embodiment of the invention discloses an in-network computing method and a related device. The method comprises the following steps: judging whether a current microchip entering an input port corresponding to a microchip checking unit is a head microchip or not through the microchip checking unit; if the current flit is not the head flit, controlling the current flit according to task information carried by the current flit, and executing a calculation task corresponding to the task information carried by the current flit in one calculation unit of one in-network calculation virtual channel of the input port to obtain a first calculation result; and after authorization of the control unit is obtained, the first calculation result is forwarded to a next-hop router. According to the invention, the computing unit is embedded in the virtual channel of the traditional router, so that the virtual channel supports intra-network computing, and the computing operation of the data packet can be completed in the transmission process, thereby facilitating the reduction of communication delay and the improvement of the overall performance of the system.

Description

Technical Field

[0001] This invention relates to the field of integrated circuits and artificial intelligence, and in particular to an in-network computing method and related apparatus. Background Technology

[0002] With the widespread application of artificial intelligence in complex tasks such as image recognition, speech processing, and natural language understanding, the scale of deep neural network (DNN) models continues to expand, and their computational intensity and data dependence place higher demands on hardware acceleration platforms. To meet the deployment requirements of high throughput, low latency, and low power consumption, neural network accelerators based on network-on-chip (NoC) have become a mainstream research direction. This type of architecture achieves large-scale parallel computing by interconnecting a large number of processing elements (PEs) on the NoC, thereby effectively supporting DNN inference tasks.

[0003] However, in actual operation, especially when dealing with large models, frequent inter-layer data exchanges cause a surge in NoC traffic, which can easily lead to local or global network congestion. At this time, data packets queue up in the router's input buffer waiting to be forwarded, causing significant communication delays. This delay not only directly lengthens the end-to-end inference time, but also causes backend computing units to idle due to "data starvation," severely reducing the overall energy efficiency of the system. Summary of the Invention

[0004] This application provides an intra-network computing method and related apparatus. By embedding a computing unit in the virtual channel of a traditional router, the virtual channel supports intra-network computing, allowing data packets to complete computing operations during transmission, thereby reducing communication latency and improving the overall performance of the system.

[0005] The first aspect of this application provides an intra-network computing method applied to a virtual channel-based router. The router includes at least one chip inspection unit, at least one input port, and a control unit. Each chip inspection unit is connected to a corresponding input port and then to the control unit. Each input port includes at least one intra-network computing virtual channel, and each intra-network computing virtual channel includes at least one computing unit. The method includes:

[0006] The microchip inspection unit determines whether the current microchip entering the input port corresponding to the microchip inspection unit is the head microchip. If the current microchip is not the head microchip, then according to the task information carried by the current microchip, the current microchip is controlled to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in one of the input ports, and obtain the first calculation result; After obtaining authorization from the control unit, the first calculation result is forwarded to the next-hop router.

[0007] Optionally, the method further includes: If the current micro-piece is a header micro-piece, then determine the data packet type of the header micro-piece and determine whether the current router is the destination router corresponding to the header micro-piece; If the data packet type of the header fragment is normal data transmission type, or if the current router is not the destination router of the header fragment, then the header fragment is forwarded to the next-hop router; If the data packet type of the header micro-piece is task data calculation type, and the current router is the destination router of the header micro-piece, then a post-processing operation is performed.

[0008] Optionally, determining the data packet type of the header micro-piece includes: The data packet type of the header micro-chip is determined based on the bit status flag of the header micro-chip. The bit status flag values ​​corresponding to the normal data transmission type and the task data calculation type are different.

[0009] Optionally, the intranet computing virtual channel further includes a post-processing unit, wherein performing the post-processing operation includes: The control unit locks the target network virtual computing channel so that the target network virtual computing channel suspends participation in the forwarding of subsequent calculation results, and transmits the routing configuration information carried by the head micro-chip to the target post-processing unit. The target network virtual computing channel is the network virtual computing channel currently occupied by the head micro-chip, and the target post-processing unit is the post-processing unit included in the target network virtual computing channel. The remaining computing units in the target network's virtual computing channel execute corresponding computing tasks based on the task information carried by the body micro-piece and tail micro-piece to obtain a second computing result, and transmit the second computing result to the target post-processing unit. The head micro-piece, the body micro-piece, and the tail micro-piece together form a target data packet. The target post-processing unit processes the second calculation result to obtain the final calculation result; and according to the routing configuration information, the source router address and destination router address of the target data packet are exchanged to obtain new routing configuration information; The target post-processing unit encapsulates a new data packet based on the final calculation result and the new routing configuration information, wherein the data type of the new data packet is the ordinary data transmission type. The control unit releases the virtual channel within the target network from occupancy and forwards the new data packet to the next-hop router.

[0010] Optionally, when the intra-network calculation method is used to accelerate neural network inference, the head micro-pie also carries the bias parameters and weight parameters corresponding to the current neural network layer. The step of processing the second calculation result through the target post-processing unit to obtain the final calculation result includes: Calculate the weighted product of each of the second calculation results and the corresponding weight parameter to obtain multiple weighted products; The weighted products are summed and accumulated, and the bias parameter is added to obtain the value of the independent variable; The final calculation result is calculated based on the values ​​of the independent variables and the activation function.

[0011] Optionally, the intranet computing virtual channel further includes at least one buffer, each buffer corresponding to one computing unit, and the buffer is used to store the input data, intermediate data and computing results required by the corresponding computing unit to perform computing tasks.

[0012] Optionally, the router further includes a crossbar switch, with each of the output ports connected to the crossbar switch. The crossbar switch is also connected to the control unit, which is used to assign the crossbar switch to a target input port so that the microchip or data packet output from the target input port is forwarded to the next-hop router through the crossbar switch.

[0013] A second aspect of this application provides an intranet computing device applied to a virtual channel-based router. The router includes at least one chip inspection unit, at least one input port, and a control unit. Each chip inspection unit is connected to a corresponding input port and then to the control unit. Each input port includes at least one intranet computing virtual channel, and each intranet computing virtual channel includes at least one computing unit. The device includes: The microchip type determination unit is used to determine, through the microchip inspection unit, whether the current microchip entering the input port corresponding to the microchip inspection unit is a head microchip; The task execution unit is configured to, if the current microchip is not the head microchip, control the current microchip to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in one of the input ports, according to the task information carried by the current microchip, and obtain a first calculation result; The data forwarding unit is used to forward the first calculation result to the next-hop router after obtaining authorization from the control unit.

[0014] A third aspect of this application provides an electronic device, including: a processor and a memory; The processor is connected to a memory, wherein the memory is used to store computer programs and the processor is used to invoke the computer programs to execute the methods as described in the first aspect of the embodiments of this application.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a processor, perform the method as described in the first aspect of this application.

[0016] This invention provides an intra-network computing method applied to virtual channel routers. By introducing a chip inspection unit into the router architecture, introducing a computing unit into the intra-network computing virtual channel at the input port, and combining it with a refined chip-level scheduling and control mechanism, a deep integration of communication and computing is achieved, resulting in the following significant benefits: 1. Traditional routers typically only perform store-and-forward operations when processing non-header picts. Especially during network congestion, these picts can wait in the input buffer for extended periods, resulting in idle resources. This invention uses a pict checking unit to accurately identify the type of pict. For picts that are not headers (i.e., body or tail picts), the system utilizes the task information they carry to immediately execute computational tasks through a computation unit within the intranet computation virtual channel included in the input port. This mechanism cleverly transforms the "communication delay" originally caused by network queuing into valuable "effective computation time," thereby masking some communication overhead, significantly reducing the total latency of end-to-end inference, and improving the overall throughput performance of the system.

[0017] 2. This invention embeds the computing unit directly into the virtual channel of the router's input port, constructing a distributed integrated "computing-communication" node. This design avoids centrally configuring large-scale, high-power dedicated processing unit (PE) arrays for all computing tasks. Computing tasks are decomposed and offloaded onto the data flow path, where they are collaboratively completed by routers along the way. This not only significantly reduces dependence on central computing resources but also reduces the need for long-distance data transport on the NoC, thereby effectively saving chip area and interconnect power consumption, making it particularly suitable for resource-constrained edge devices.

[0018] 3. By performing computations in real-time along the data transmission path, this method reduces the number of times data is repeatedly moved between computing and storage units, fundamentally alleviating the energy consumption caused by the "memory wall" problem. Simultaneously, by converting network idle periods (congestion waiting periods) into computation cycles, the system completes more effective work per unit time, avoiding the energy consumption of computing units idling while waiting for data and the pure maintenance power consumption of network resources. This "turning idle time into busy time" strategy greatly improves the utilization rate of hardware resources, thereby significantly improving the energy efficiency ratio of the entire accelerator system.

[0019] 4. The forwarding of calculation results requires explicit authorization from the control unit. This mechanism ensures that intranet calculations do not interfere with normal routing decisions and network flow control logic, maintain the orderly transmission of data packets and the integrity of network protocols, thereby improving performance while ensuring the reliability and stability of the system. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram of the structure of a virtual channel-based router according to an embodiment of this application is shown; Figure 2 A flowchart illustrating an embodiment of the intra-network computing method provided in this application is shown; Figure 3 This invention illustrates a schematic diagram of a conventional virtual tunnel router pipeline provided in one embodiment of this application; Figure 4 This diagram illustrates a non-blocking scheduling scenario using the intra-network computing method provided in this application embodiment. Figure 5 This paper shows a schematic diagram of the structure of an in-network computing device according to an embodiment of the present application; Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of this application is shown. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] Please refer to Figure 1 This illustration shows a schematic diagram of a virtual channel-based router according to an embodiment of this application. The router includes at least one flit checking unit, at least one input port, a control unit, and a crossbar switch. Each flit checking unit is connected to an input port, which is connected to both the control unit and the crossbar switch. The control unit is also connected to the crossbar switch. Each input port includes multiple intra-network computing virtual channels, each including multiple buffers and a corresponding computing unit for each buffer, as well as a post-processor. Wherein: The Flit check unit determines the type of Flow Control Unit (Flit) entering the input port connected to it. A Flit, sometimes translated as a slice, is the smallest physical unit of data transmission in a NoC (No-Chip). When a large data packet needs to be transmitted on the internal network of the chip, it is divided into multiple smaller fragments, which are called Flits. Routers buffer, route, and forward data in units of Flits, rather than entire packets. This mechanism allows for the interleaving of different data packets, greatly improving link utilization and system throughput. Based on their position and function within a complete data packet, Flits are typically classified into three main types: Head Flit, Body Flit, and Tail Flit. The Head Flit is the first flit of a data packet. It carries routing information, such as the source and destination addresses, and may also include other critical control information required for routing (such as virtual tunnel requests). Body Flits are all the intermediate flits located between the Head Flit and Tail Flit. A data packet may have one or more Body Flits, or none (if the packet is small). It primarily contains the actual data to be transmitted. The Tail Flit is the last flit of the data packet and may also contain payload data.

[0024] The input port receives Flits from upstream routers or processing units. Its internal virtual computing channels not only buffer and forward data but also integrate computational functions. Each virtual channel contains computational units that can execute simple neural network operators, such as activation functions, pooling, and lightweight convolutions. These computational units are typically dedicated hardware to achieve low latency and high energy efficiency. The post-processing unit formats, packages, or merges the computation results with other Flits, ensuring the output conforms to standard protocols. The buffer stores the input data, intermediate data, and computation results required by the corresponding computational unit. Virtual channels connect to downstream devices via a credit mechanism to implement flow control and prevent congestion. Specifically, when the downstream port buffer is full, a credit signal notifies the upstream to pause transmission, preventing data loss.

[0025] A crossbar switch is responsible for establishing physical connections between input and output ports, enabling packet routing and forwarding. The control unit is used for comprehensive management and control of the entire router; it may include, for example, a routing calculator, virtual channel calculator, and switch distributor.

[0026] When a Head Flit arrives at a router, the routing calculator reads its destination address, calculates the next-hop output port, and requests the necessary Virtual Channel (VC) and Crossbar resources. Once the resources are successfully allocated, a logical path is established for subsequent body flits. Body flits do not need to perform routing calculations again; they simply follow the path established by the Head Flit, forwarding directly to the next hop through the allocated VC and Crossbar. When a Tail Flit leaves a router's output port, it triggers a signal to notify the router to release the VC buffer and other related resources previously allocated for that packet.

[0027] The switch distributor controls the crossbar switches, determining which input ports can send data to which output ports. The virtual channel distributor coordinates the initiation and resource allocation of computing tasks, assigning appropriate virtual channels to arriving Flits and managing the occupancy status of virtual channels.

[0028] Please refer to Figure 2 This illustration shows a flowchart of an intra-network computing method provided in one embodiment of this application. The method can be applied to... Figure 1The illustrated router is based on a virtual channel. This router includes at least one chip inspection unit, at least one input port, and a control unit. Each chip inspection unit is connected to a corresponding input port and then to the control unit. Each input port includes at least one intra-network computing virtual channel, and each intra-network computing virtual channel includes at least one computing unit. The method may include the following steps: Step 201: Determine whether the current microchip entering the input port corresponding to the microchip inspection unit is a head microchip through the microchip inspection unit.

[0029] In the standard NoC protocol, each chip contains a fixed header structure, with 2 bits (or more, depending on protocol complexity) specifically reserved to identify the chip type. This header is typically located at the very beginning of the chip (either the most significant bit or the least significant bit, e.g., [FLIT_WIDTH-1 : FLIT_WIDTH-2]). Its encoding can be, for example: 00 or 01: Head Flit — Indicates the start of a new data packet.

[0030] 10: Body Flit – This indicates that this is the middle part of the data packet.

[0031] 11: Tail Flit – This indicates the end of the data packet.

[0032] The specific encoding value depends on the design. The above is just an example. For other encoding methods, the specific type can also be determined by its value.

[0033] Step 202: If the current microchip is not the head microchip, then according to the task information carried by the current microchip, control the current microchip to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in the input port, and obtain the first calculation result.

[0034] Since the current micro-chip is not a head micro-chip, but a body micro-chip or a tail micro-chip, the body micro-chip or tail micro-chip can follow the routing path established by the head micro-chip to perform computing tasks. The virtual channel corresponding to the routing path is determined according to the routing information carried by the head micro-chip. The computing unit is selected and determined by the control unit according to the load balancing strategy. The first computing result can be a new micro-chip, which is obtained by processing the original micro-chip through the computing unit.

[0035] Furthermore, the method also includes: If the current micro-piece is a header micro-piece, then determine the data packet type of the header micro-piece and determine whether the current router is the destination router corresponding to the header micro-piece; If the data packet type of the header fragment is normal data transmission type, or if the current router is not the destination router of the header fragment, then the header fragment is forwarded to the next-hop router; If the data packet type of the header micro-piece is task data calculation type, and the current router is the destination router of the header micro-piece, then a post-processing operation is performed.

[0036] Ordinary data transmission type refers to traditional communication data, which only needs to be transmitted from the source to the destination without requiring special intranet computation, or the computation has already been completed in the non-header micro-chip stage. Task data computation type refers to data packets that are explicitly marked as requiring the use of intranet computing resources. These data packets are not only for transmission, but also for data processing along the path or at the destination.

[0037] Specifically, it checks whether the destination address carried in the header chip matches the ID of the current router. If so, it means that the current router is the destination; otherwise, it means that the current router is an intermediate node and needs to continue transiting.

[0038] Specifically, determining the data packet type of the header micro-piece includes: The data packet type of the header micro-chip is determined based on the bit status flag of the header micro-chip. The bit status flag values ​​corresponding to the normal data transmission type and the task data calculation type are different.

[0039] For example, a 1-bit status flag is added to the Head Flit of each data packet. A flag of "1" indicates that the data packet is a task data calculation type and requires calculation, while a flag of "0" indicates that the data packet is a normal data transmission type and only requires forwarding. Of course, the flag could also be "0" to indicate a task data calculation type and "1" to indicate a normal data transmission type, or other values ​​could be used for identification; this is not limited here.

[0040] Specifically, the intranet computing virtual channel further includes a post-processing unit, and the execution of post-processing operations includes: The control unit locks the target network virtual computing channel so that the target network virtual computing channel suspends participation in the forwarding of subsequent calculation results, and transmits the routing configuration information carried by the head micro-chip to the target post-processing unit. The target network virtual computing channel is the network virtual computing channel currently occupied by the head micro-chip, and the target post-processing unit is the post-processing unit included in the target network virtual computing channel. The remaining computing units in the target network's virtual computing channel execute corresponding computing tasks based on the task information carried by the body micro-piece and tail micro-piece to obtain a second computing result, and transmit the second computing result to the target post-processing unit. The head micro-piece, the body micro-piece, and the tail micro-piece together form a target data packet. The target post-processing unit processes the second calculation result to obtain the final calculation result; and according to the routing configuration information, the source router address and destination router address of the target data packet are exchanged to obtain new routing configuration information; The target post-processing unit encapsulates a new data packet based on the final calculation result and the new routing configuration information, wherein the data type of the new data packet is the ordinary data transmission type. The control unit releases the virtual channel within the target network from occupancy and forwards the new data packet to the next-hop router.

[0041] During post-processing, the control unit locks the target network's virtual computing channel, temporarily halting data transmission to the next hop and acting as a temporary aggregation buffer. Even if other unrelated data packets subsequently attempt to use this virtual computing channel, they will be blocked until the current task is completed. This ensures that all other fragments (body fragments, tail fragments) of the same data packet arrive and complete their respective calculations, preventing data loss or out-of-order delivery. The routing configuration information carried in the header fragment is transmitted to the target post-processing unit, enabling it to generate a valid new data packet at the end.

[0042] The remaining computing units can perform parallel calculations, executing corresponding computational tasks based on the task information carried by the body and tail micro-pieces to obtain the second calculation result, which is then sent to the target post-processing unit. The post-processing unit performs an integrity check; that is, the processing of the original data packet is only considered complete when the calculation results corresponding to all micro-pieces in the header, body, and tail have been collected by the post-processing unit. This ensures data integrity.

[0043] For example, the target post-processing unit processes the second calculation result to obtain the final calculation result. If the previous calculation was a distributed matrix multiplication, this can be by adding multiple parts together; if the previous calculation was a parallel filtering, this can be by merging the filtered image patches. The final calculation result is a complete and meaningful data block, rather than fragmented pieces.

[0044] Once the final calculation result is obtained, the task is considered complete, and the result needs to be sent back to the original sender. By exchanging addresses, the router does not need to consult complex routing tables and can directly construct the return path configuration using the existing information. Furthermore, since the calculation task is finished, this new packet no longer requires complex intra-network calculations; it is simply a regular return result packet. Routers along the route only need to forward it according to standard procedures, without needing to reactivate the calculation unit. Therefore, it needs to be set to a normal data transmission type.

[0045] After the computation task is completed, the virtual computing channel within the target network is released from its occupancy, freeing up hardware resources and allowing the virtual computing channel within the target network to receive and process new data streams again, avoiding deadlock or resource waste. The encapsulated new data packet is then sent to the output queue and forwarded to the next hop, i.e., the first station on the original path. At this point, a complete "request-intra-network computation-response" process is completed.

[0046] Specifically, when the intra-network calculation method is used to accelerate neural network inference, the head micro-pie also carries the bias parameters and weight parameters corresponding to the current neural network layer. The step of processing the second calculation result through the target post-processing unit to obtain the final calculation result includes: Calculate the weighted product of each of the second calculation results and the corresponding weight parameter to obtain multiple weighted products; The weighted products are summed and accumulated, and the bias parameter is added to obtain the value of the independent variable; The final calculation result is calculated based on the values ​​of the independent variables and the activation function.

[0047] This application provides a scenario for applying in-network computation to accelerate neural network (NN) inference. In traditional neural network accelerators, weights and biases are typically stored in off-chip DRAM or on-chip SRAM, requiring frequent access. In this application, the head slice not only handles routing but also acts as an "instruction packet + parameter packet." It directly feeds the weights and biases required by the current layer into the router, achieving "computation moving with data" or "parameters moving with requests," reducing global storage access latency, and is particularly suitable for dynamic network structures or edge computing scenarios. Different slices compute different second computation results in parallel, and then the weighted products with the corresponding weight parameters are summed and accumulated, and the bias parameters are superimposed to obtain the independent variable values, i.e., the output of the linear unit. Inputting this into a nonlinear activation function introduces nonlinear factors, enabling the neural network to fit complex functions.

[0048] Step 203: After obtaining authorization from the control unit, forward the first calculation result to the next-hop router.

[0049] In a network-on-a-chip (NoC), resources such as output port bandwidth, crossbar switches, and next-hop buffers are shared and limited. Multiple chips with input ports may simultaneously request the same output port. If they send data directly without authorization, it may cause the next-hop router's buffer to overflow, resulting in deadlock or data loss.

[0050] The specific operation may be as follows: the control unit allocates the crossbar switch path, configures the internal wiring, connects the current input port or computing unit to the target output port, and then confirms that the corresponding virtual channel of the next hop router has space to receive data. If so, it sends an allow signal to the computing unit or buffer.

[0051] Furthermore, the intranet computing virtual channel also includes at least one buffer, each buffer corresponding to one computing unit, and the buffer is used to store the input data, intermediate data and computing results required by the corresponding computing unit to perform computing tasks.

[0052] Each virtual channel is bound to a computing unit and its corresponding buffer, allowing data to be not only stored but also directly sent to the bound computing unit for processing. The buffer can store input data, such as the original micro-pie payload carrying the activation values ​​of the neural network just received from the link, or intermediate data such as temporary values ​​during the calculation process, such as partial products in the multiplication process, the current value of the accumulator, or the first calculation result generated after the computing unit completes its operation.

[0053] Furthermore, the router also includes a cross switch, with each of the output ports connected to the cross switch. The cross switch is also connected to the control unit, which is used to assign the cross switch to a target input port so that the microchips or data packets output from the target input port are forwarded to the next-hop router through the cross switch.

[0054] The crossbar switch can be an N×M matrix grid connecting all input and output ports, allowing any input port to connect to any available output port at any time. It is non-blocking or partially non-blocking, maximizing parallel throughput. The crossbar switch itself is a passive hardware connection, obeying the control unit, such as a switch distributor within the control unit. An input port requests access to output port X from the control unit. The control unit checks if output port X is available and if other input ports are also competing for X. Once authorization is granted, the control unit sends a configuration signal to the crossbar switch, closing a specific "switch point" and establishing a physical path from the target input port to the target output port.

[0055] In a traditional virtual tunnel router, a Head Flit forwarding pipeline includes: buffer write, route calculation, virtual tunnel allocation, switch allocation, and switch transmission. Although the Body Flit and Tail Flit do not perform route calculation or virtual tunnel allocation, they always enter the pipeline immediately after the Head Flit, and their cycle time in the pipeline structure is consistent with the stage progression rhythm. Therefore, although they do not perform actual route calculation or virtual tunnel allocation logic operations in their corresponding stages, they still experience the same number of pipeline stages as the Head Flit in terms of timing. Figure 3 As shown.

[0056] The packet scheduling of this invention is based on the following timing characteristics: after a Flit is written to the buffer, its data payload is actually in an "idle" state during the pipeline stage where it awaits subsequent control logic processing. This mechanism accurately captures this "idle" window and performs calculations using the aforementioned intra-network calculation method, thereby seamlessly transforming inherent communication latency into an efficient calculation process.

[0057] Under normal forwarding conditions, each pipeline stage of the router, each single computation operation, and the operation of writing the result back to the buffer are designed to consume one clock cycle. Therefore, even without network congestion, data can complete up to four computations in parallel during a single pass through the router's normal pipeline. Figure 4 As shown, when network congestion occurs, the time that data packets spend in the buffer is automatically requisitioned by the system. The embedded computing unit will continue to perform calculations until the assigned task is completed, or until the virtual channel obtains the authorization to forward to the next hop, thus perfectly masking the communication blockage and enabling more than four calculations.

[0058] As can be seen, the intra-network computing method provided by this invention, applied to virtual channel routers, introduces a chip inspection unit into the router architecture, introduces a computing unit into the intra-network computing virtual channel at the input port, and combines it with a refined chip-level scheduling and control mechanism, thereby achieving deep integration of communication and computing, and bringing the following significant beneficial effects: 1. Traditional routers typically only perform store-and-forward operations when processing non-header picts. Especially during network congestion, these picts can wait in the input buffer for extended periods, resulting in idle resources. This invention uses a pict checking unit to accurately identify the type of pict. For picts that are not headers (i.e., body or tail picts), the system utilizes the task information they carry to immediately execute computational tasks through a computation unit within the intranet computation virtual channel included in the input port. This mechanism cleverly transforms the "communication delay" originally caused by network queuing into valuable "effective computation time," thereby masking some communication overhead, significantly reducing the total latency of end-to-end inference, and improving the overall throughput performance of the system.

[0059] 2. This invention embeds the computing unit directly into the virtual channel of the router's input port, constructing a distributed integrated "computing-communication" node. This design avoids centrally configuring large-scale, high-power dedicated processing unit (PE) arrays for all computing tasks. Computing tasks are decomposed and offloaded onto the data flow path, where they are collaboratively completed by routers along the way. This not only significantly reduces dependence on central computing resources but also reduces the need for long-distance data transport on the NoC, thereby effectively saving chip area and interconnect power consumption, making it particularly suitable for resource-constrained edge devices.

[0060] 3. By performing computations in real-time along the data transmission path, this method reduces the number of times data is repeatedly moved between computing and storage units, fundamentally alleviating the energy consumption caused by the "memory wall" problem. Simultaneously, by converting network idle periods (congestion waiting periods) into computation cycles, the system completes more effective work per unit time, avoiding the energy consumption of computing units idling while waiting for data and the pure maintenance power consumption of network resources. This "turning idle time into busy time" strategy greatly improves the utilization rate of hardware resources, thereby significantly improving the energy efficiency ratio of the entire accelerator system.

[0061] 4. The forwarding of calculation results requires explicit authorization from the control unit. This mechanism ensures that intranet calculations do not interfere with normal routing decisions and network flow control logic, maintain the orderly transmission of data packets and the integrity of network protocols, thereby improving performance while ensuring the reliability and stability of the system.

[0062] Figure 5 A schematic diagram of the structure of an intranet computing device according to an embodiment of this application is shown. Applied to a virtual channel-based router, the router includes at least one chip inspection unit, at least one input port, and a control unit. Each chip inspection unit is connected to a corresponding input port and then to the control unit. Each input port includes at least one intranet computing virtual channel, and each intranet computing virtual channel includes at least one computing unit. The device includes: The microchip type determination unit 501 is used to determine whether the current microchip entering the input port corresponding to the microchip inspection unit is a head microchip through the microchip inspection unit. The task execution unit 502 is configured to, if the current microchip is not the head microchip, control the current microchip to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in one of the input ports, according to the task information carried by the current microchip, and obtain a first calculation result; The data forwarding unit 503 is used to forward the first calculation result to the next-hop router after obtaining authorization from the control unit.

[0063] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of this application is shown, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the functions of the computer system of the network computing method in any of the above embodiments.

[0064] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, causes the computer to perform the functions of the computer system of the network computing method in any of the above embodiments.

[0065] This application also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the functions of the computer system of the network computing method in any of the above embodiments.

[0066] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation methods of this application, and are not intended to limit the scope of the invention.

[0067] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not limit the implementation process of the embodiments of this application in any way.

[0068] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and the implementation methods in this application are not limited in this respect.

[0069] Unless otherwise stated, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this application includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0070] It is understood that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0071] It is understood that the memory in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Specifically, non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0072] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0073] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0074] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0076] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0077] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0078] The above are merely specific embodiments of this application, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A network-based computing method, characterized in that, An application is made to a virtual channel-based router, the router comprising at least one chip inspection unit, at least one input port, and a control unit, each chip inspection unit being connected to a corresponding input port and then to the control unit, each input port comprising at least one intra-network computing virtual channel, each intra-network computing virtual channel comprising at least one computing unit, the method comprising: The microchip inspection unit determines whether the current microchip entering the input port corresponding to the microchip inspection unit is the head microchip. If the current microchip is not the head microchip, then according to the task information carried by the current microchip, the current microchip is controlled to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in one of the input ports, and obtain the first calculation result; After obtaining authorization from the control unit, the first calculation result is forwarded to the next-hop router.

2. The method according to claim 1, characterized in that, The method further includes: If the current micro-piece is a header micro-piece, then determine the data packet type of the header micro-piece and determine whether the current router is the destination router corresponding to the header micro-piece; If the data packet type of the header fragment is normal data transmission type, or if the current router is not the destination router of the header fragment, then the header fragment is forwarded to the next-hop router; If the data packet type of the header micro-piece is task data calculation type, and the current router is the destination router of the header micro-piece, then a post-processing operation is performed.

3. The method according to claim 2, characterized in that, The determination of the data packet type of the header micro-piece includes: The data packet type of the header micro-chip is determined based on the bit status flag of the header micro-chip. The bit status flag values ​​corresponding to the normal data transmission type and the task data calculation type are different.

4. The method according to claim 2, characterized in that, The intranet computing virtual channel further includes a post-processing unit, and the post-processing operation includes: The control unit locks the target network virtual computing channel so that the target network virtual computing channel suspends participation in the forwarding of subsequent calculation results, and transmits the routing configuration information carried by the head micro-chip to the target post-processing unit. The target network virtual computing channel is the network virtual computing channel currently occupied by the head micro-chip, and the target post-processing unit is the post-processing unit included in the target network virtual computing channel. The remaining computing units in the target network's virtual computing channel execute corresponding computing tasks based on the task information carried by the body micro-piece and tail micro-piece to obtain a second computing result, and transmit the second computing result to the target post-processing unit. The head micro-piece, the body micro-piece, and the tail micro-piece together form a target data packet. The target post-processing unit processes the second calculation result to obtain the final calculation result; and according to the routing configuration information, the source router address and destination router address of the target data packet are exchanged to obtain new routing configuration information; The target post-processing unit encapsulates a new data packet based on the final calculation result and the new routing configuration information, wherein the data type of the new data packet is the ordinary data transmission type. The control unit releases the virtual channel within the target network from occupancy and forwards the new data packet to the next-hop router.

5. The method according to claim 4, characterized in that, When the in-network calculation method is used to accelerate neural network inference, the head micro-pie also carries the bias parameters and weight parameters corresponding to the current neural network layer. The process of processing the second calculation result through the target post-processing unit to obtain the final calculation result includes: Calculate the weighted product of each of the second calculation results and the corresponding weight parameter to obtain multiple weighted products; The weighted products are summed and accumulated, and the bias parameter is added to obtain the value of the independent variable; The final calculation result is calculated based on the values ​​of the independent variables and the activation function.

6. The method according to claim 5, characterized in that, The virtual computing channel within the network also includes at least one buffer, each buffer corresponding to one computing unit. The buffer is used to store the input data, intermediate data and computing results required by the corresponding computing unit to perform computing tasks.

7. The method according to claim 5, characterized in that, The router also includes a crossbar switch, each of the output ports is connected to the crossbar switch, and the crossbar switch is also connected to the control unit. The control unit is used to assign the crossbar switch to a target input port so that the chip or data packet output by the target input port is forwarded to the next-hop router through the crossbar switch.

8. A network-based computing device, characterized in that, An apparatus for use in a virtual channel-based router includes at least one chip inspection unit, at least one input port, and a control unit. Each chip inspection unit is connected to a corresponding input port and then to the control unit. Each input port includes at least one intra-network computing virtual channel, and each intra-network computing virtual channel includes at least one computing unit. The apparatus includes: The microchip type determination unit is used to determine, through the microchip inspection unit, whether the current microchip entering the input port corresponding to the microchip inspection unit is a head microchip; The task execution unit is configured to, if the current microchip is not the head microchip, control the current microchip to execute the calculation task corresponding to the task information carried by the current microchip in one of the calculation units of one of the network calculation virtual channels in one of the input ports, according to the task information carried by the current microchip, and obtain a first calculation result; The data forwarding unit is used to forward the first calculation result to the next-hop router after obtaining authorization from the control unit.

9. An electronic device, characterized in that, include: Processor and memory; The processor is connected to a memory, wherein the memory is used to store a computer program, and the processor is used to invoke the computer program to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, perform the method as described in any one of claims 1-7.

Images