Multi-core storage and calculation integrated routing system and electronic equipment

Abstract

The invention relates to the technical field of storage and calculation integration, in particular to a multi-core storage and calculation integration-oriented routing system and electronic equipment, the system comprises a plurality of storage and calculation integration cores, each storage and calculation integration core comprises a plurality of storage and calculation macro units, a plurality of direction output interfaces, an analysis network-on-chip and a router, and the storage and calculation macro units are bound with the direction output interfaces; the analysis network-on-chip analyzes the header information of the data packet to determine the target transmission direction of the data packet, and when the target transmission direction is the same as the position direction of the target core, the direction output interface outputs data to the target core; and the router is used for generating a target planning path according to the target transmission direction and the position direction of the target core when the target transmission direction is different from the position direction of the target core, and transmitting the data packet to the target core according to the target planning path. Therefore, the problems of congestion of high-parallel signal winding between the memory macro-cell and the route, blockage caused by improper matching of data transmission directions, low path planning efficiency and the like in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of on-chip network interconnection architecture technology for in-memory computing multi-core systems, and in particular to a routing system and electronic device for multi-core in-memory computing. Background Technology

[0002] To meet the computing power demands of large-scale neural networks, multi-core in-memory computing systems require interconnect architectures to achieve data communication between cores. Many related technologies employ on-chip network architectures, connecting each in-memory computing core to a dedicated router to construct a regular topology that ensures data transmission. Simultaneously, the in-memory computing cores leverage highly parallel array input / output signals to improve computational efficiency. However, the need for highly parallel signals to be centrally converged to a router for redistribution can easily lead to congested wiring between the cores and the router, introducing signal delays. Alternatively, attempts can be made to bind the array to the output direction, but the lack of efficient direction determination and path planning mechanisms can easily result in data transmission blockages, hindering overall system performance. Summary of the Invention

[0003] This application provides a routing system and electronic device for multi-core in-memory computing to solve problems such as data transmission congestion and low path planning efficiency in related technologies.

[0004] The first aspect of this application provides a routing system for multi-core in-memory computing, comprising: multiple in-memory computing cores, each core including multiple in-memory computing macrounits, multiple directional output interfaces, a parsing on-chip network, and a router; wherein the in-memory computing macrounits are bound to the directional output interfaces; the parsing on-chip network parses the header information of data packets to determine the target transmission direction of the data packets; when the target transmission direction is the same as the location direction of the target core, the directional output interfaces output data packets to the target core; and the router is used to generate a target planned path based on the target transmission direction and the location direction of the target core when the target transmission direction is different from the location direction of the target core, and transmit data packets to the target core according to the target planned path.

[0005] Based on the aforementioned technical means, in this embodiment, the in-memory computing macro unit generates a data packet (including header information) to be transmitted, parses the on-chip network to parse the data packet header, determines the target transmission direction, and judges whether the target transmission direction is consistent with the location direction of the target core. If the directions are consistent: the direction output interface directly forwards the data packet to the target core; if the directions are inconsistent: the router generates a planned path based on the "target transmission direction" and the "target core location," and then forwards the data packet to the target core. By binding the in-memory computing macro unit with the direction output interface, and combining the direction judgment of the on-chip network with the router's path planning, data transmission is split into multiple streams: when the target direction matches, transmission is directly through the direction output interface; when they do not match, the router plans the path for transmission, thus constructing an efficient multi-core in-memory computing routing mechanism. This effectively solves the problem of congestion caused by the concentrated convergence of high-parallel signals, reducing signal transmission delay; by combining direct transmission with direction matching and path planning, data transmission blocking is avoided, improving the communication efficiency and overall performance of the multi-core in-memory computing system.

[0006] Optionally, the router is used to transmit data to the edge node along the target transmission direction when the target transmission direction is opposite to the location direction of the target core. The edge node then transmits the data back to the source core. After the source core converts the data into virtual channel data, it replans the path to obtain the target planned path and transmits the data packet to the target core according to the target planned path.

[0007] Based on the aforementioned technical means, in this embodiment, when the target transmission direction is opposite to the target core location direction, the router adopts a processing logic of "edge node backhaul - source core virtual channel conversion - path replanning": first, the data packet is transmitted to the edge node along the target direction, then the edge node transmits it back to the source core. After the source core converts the data packet into virtual channel data, the router replans the target path and finally transmits the data packet to the target core. This solves the problem of chaotic and easily blocked data transmission paths when the target direction is opposite to the core location direction, avoiding detour delays or resource waste caused by reverse transmission; through the cooperation of edge node backhaul and virtual channel conversion, the rationality and stability of data transmission in reverse scenarios are ensured, further improving the routing flexibility and overall communication efficiency of multi-core in-memory computing systems.

[0008] Optionally, the router is used to transmit data to the relay node along the target transmission direction when the target transmission direction is perpendicular to the location direction of the target core. The relay node replans the path to obtain the target planned path and transmits the data packet to the target core according to the target planned path. The relay node is an in-memory computing core among multiple in-memory computing cores.

[0009] Based on the aforementioned technical means, in this embodiment, when the target transmission direction is perpendicular to the target core location direction, the router transmits the data packet along the target direction to a relay node (the relay node is any in-memory computing core in the system). The relay node then replans the path, and finally completes the transmission of the data packet to the target core according to the newly planned target path. This solves the problems of difficult path adaptation and easy detours and delays when the target direction is perpendicular to the core location direction; the relay node reuses the existing in-memory computing core in the system, eliminating the need to add additional dedicated routing nodes and reducing hardware costs; by replanning the path in real time through the relay node, the smoothness of vertical transmission is ensured, further improving the routing adaptability and communication efficiency of the multi-core in-memory computing system.

[0010] Optionally, the router includes an allocation module, a real-channel receiving module, a virtual-channel receiving module, and an arbitration module. The allocation module receives data packets and classifies them. The data categories include real-channel data, virtual-channel data, and control flow data. The real-channel receiving module receives real-channel data. The virtual-channel receiving module receives virtual-channel data. The arbitration module receives control flow data and schedules data from the real-channel receiving module and the virtual-channel receiving module based on scheduling instructions.

[0011] Based on the aforementioned technical means, in this embodiment, the router receives data packets through an allocation module and classifies them into real-channel data, virtual-channel data, and control flow data. The real-channel receiving module and virtual-channel receiving module then receive the corresponding data types respectively. The arbitration module receives control flow data and, based on scheduling instructions, schedules data from the real and virtual-channel receiving modules, achieving classified processing and orderly scheduling of multiple data types. This solves the problem in related technologies where centralized arbitration by routers is difficult to efficiently manage multiple data types. By classifying and receiving data, it avoids different types of data from competing for resources. Combined with the targeted scheduling of the arbitration module, it ensures priority processing of control flow data while preventing real and virtual-channel data from interfering with each other, improving the orderliness and timeliness of data transmission, and further optimizing the communication efficiency of multi-core in-memory computing systems.

[0012] Optionally, the real channel receiving module includes a real channel storage area and a PE module. The real channel storage area stores real channel data, and the PE module performs logical operations based on the real channel data, responds to the scheduling instructions of the arbitration module, and reads data from the real channel storage area and sends it to the arbitration module.

[0013] Based on the above technical means, the embodiments of this application ensure the independent storage of real channel data through a dedicated storage area, avoiding it from being occupied by other types of data; the PE module completes the logical operation in advance, and combined with the fast data reading in response to the scheduling command, improves the processing efficiency and scheduling response speed of real channel data, and further optimizes the transmission and utilization efficiency of real channel data in the multi-core in-memory computing system.

[0014] Optionally, the virtual channel receiving module includes virtual channel receiving sub-modules with multiple input directions. The virtual channel receiving sub-module includes an allocation unit, a virtual channel storage area, and a polling unit. The allocation unit is used to allocate virtual channels, the virtual channel storage area stores virtual channel data, and the polling unit is used to respond to the scheduling instructions of the arbitration module, read data from the virtual channel storage area, and send it to the arbitration module.

[0015] Based on the above technical means, the embodiments of this application avoid data mixing in different directions by setting sub-modules in different directions; the allocation unit ensures reasonable allocation of virtual channel resources, the storage area realizes independent data storage, and the polling unit responds efficiently to scheduling, thereby improving the receiving efficiency and scheduling order of virtual channel data, reducing resource conflicts, and further optimizing the virtual channel communication performance of the multi-core in-memory computing system.

[0016] Optionally, the virtual channel storage area includes a request virtual channel queue, a response virtual channel queue, and a data virtual channel queue. The request virtual channel queue and the response virtual channel queue transmit core configuration data packets and configuration completion feedback information, while the data virtual channel queue is used to transmit intermediate result data in neural network streaming computation.

[0017] Based on the above technical means, the embodiments of this application separate configuration-related data from intermediate calculation results through classification queues, ensuring that virtual channel data for different purposes each has a dedicated storage and transmission path, avoiding mutual resource squeezing between configuration data and calculation data, improving the targeted utilization efficiency of virtual channel resources, ensuring the smoothness of configuration process and calculation process, and further optimizing the overall operating performance of multi-core in-memory computing system.

[0018] Optionally, when the router receives a data packet, it records the direction of the corresponding data packet and the amount of data that can be sent by the virtual channel. It synchronously monitors the remaining capacity of various virtual channels input in the corresponding direction of the downstream core. When the remaining capacity of the virtual channel of the downstream core meets the capacity of the data packet, the data packet is sent. The downstream core detects that the data transmission is completed and reports back to the upstream core and releases the storage space of the corresponding virtual channel. The output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue. The polling unit uses a polling priority strategy to determine the output order of queues of the same type.

[0019] Based on the above technical means, the capacity monitoring and feedback release mechanism in this application embodiment ensures data transmission adaptability, avoids transmission failure caused by insufficient downstream space, ensures that there is space for downstream to receive data when upstream sends data, and avoids packet loss; the queue priority and polling strategy ensure that core configuration-related data is transmitted first, and the same type of queue is output in an orderly manner, improving the reliability and scheduling rationality of virtual channel transmission, optimizing the data flow efficiency of multi-core in-memory computing system, and solving the problem of blocking caused by capacity mismatch in virtual channel data transmission and the problem of disordered data output of different types of queues.

[0020] Optionally, the polling unit is also used for dynamic reconstruction of virtual channels. The dynamic reconstruction process of virtual channels includes: during the system configuration phase, the output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue; during the data transmission phase, the request virtual channel queue and the response virtual channel queue are converted into data virtual channel queues, and the sending order of queues of the same type is determined by a polling priority strategy.

[0021] Based on the above technical means, the embodiments of this application dynamically reconstruct the virtual channel in stages. In the configuration stage, core configuration data is prioritized for transmission to efficiently complete system initialization. In the data stage, the original configuration queue is reused to expand data transmission resources. Combined with the polling scheduling of queues of the same type, the overall utilization rate of virtual channel resources is improved, and the orderly scheduling of queues in the data transmission stage is ensured. This further optimizes the communication resource adaptation capability and operating efficiency of the multi-core in-memory computing system and solves the problem of idle virtual channels and low resource utilization in the data transmission stage during the configuration stage in related technologies.

[0022] A second aspect of this application provides an electronic device including the aforementioned routing system for multi-core in-memory computing.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a structural diagram of a routing system for multi-core in-memory computing provided according to an embodiment of this application; Figure 2 This is a schematic diagram of the on-chip network data path for in-memory computing mesh according to an embodiment of this application; Figure 3 This is a flowchart illustrating the deployment process of a neural network supporting the insertion of a relay layer according to an embodiment of this application. Figure 4 This is a diagram illustrating the multi-path, two-layer arbitration routing design structure according to an embodiment of this application. Figure 5 This is a data routing flowchart provided according to an embodiment of this application; Figure 6 This is a diagram illustrating the virtual channel reconstruction structure according to an embodiment of this application. Figure 7 This is a flowchart of the virtual channel reconstruction process provided according to an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] As neural networks continue to evolve, their weights grow exponentially, and computational tasks become increasingly complex. The massive amount of weight data and the huge amount of intermediate results generated during inference pose a severe challenge to the von Neumann architecture hardware. The separation of storage and computation leads to frequent data movement between storage and computation units, making the "memory wall" problem increasingly prominent. That is, the high latency and high power consumption caused by data migration have become the core bottleneck restricting system performance.

[0027] Against this backdrop, in-memory computing technology, as a groundbreaking computing paradigm, deeply integrates storage and computing units, fundamentally avoiding data transfer costs. In particular, in-memory computing architectures based on memristors, by storing multi-bit (binary digit) weights in-situ within a memristor array and loading input data as voltage onto the array rows, can simultaneously enable multiple rows for vector multiplication and perform analog domain accumulation through parallel column activation, significantly improving the core computational efficiency of neural networks. In-memory computing technology has evolved from a single array to a complete computing core: by integrating AD (Analog-to-Digital) / DA (Digital-to-Analog) converters (to achieve analog-to-digital signal conversion), input / output registers (to ensure data timing synchronization), shift accumulators (to support multi-precision operations), and special operator circuits (such as activation function accelerators), a closed-loop computing unit is formed.

[0028] Building on this, in order to meet the computing power requirements of larger-scale neural networks, the system is rapidly expanding towards multi-core architecture, and the design of the interconnect architecture has become a key support for the performance of multi-core in-memory computing systems.

[0029] In related technologies, the interconnection methods of multi-core in-memory computing systems are mainly divided into three categories: (1) Bus mode: Multiple computing cores are connected to a shared bus through an interface and compete for communication resources through a bus arbitration mechanism. The design structure is simple, but the bus bandwidth is prone to becoming a bottleneck and it is difficult to support large-scale multi-core expansion.

[0030] (2) Exchange matrix mode: All computing cores establish physical connections between each other. Theoretically, any two cores can communicate directly at the same time with low communication latency. However, as the number of cores increases, the number of physical connections increases quadratically, and the chip area and wiring complexity increase sharply. It is only suitable for small-scale multi-core systems.

[0031] (3) On-chip network mode: Each computing core is connected to a dedicated router, and the routers are interconnected through a fixed topology. This allows for flexible construction of various structures such as Mesh, Cmesh (Configurable Mesh), and Torus, making it the mainstream choice for current large-scale multi-core in-memory computing systems. Among them: Mesh topology: The cores are arranged in rows and columns in a grid. Each router connects to only four adjacent routers (top, bottom, left, and right). It has a regular structure, strong scalability, and adding new cores only requires expanding the rows and columns at the edge of the grid. It is suitable for medium-sized multi-core systems.

[0032] (4) Cmesh topology: Optimize corner connections on the basis of Mesh, reduce the number of routing hops for long-distance data transmission, and improve communication efficiency.

[0033] (5) Torus topology: Connect the first and last routers of the Mesh to form a ring, shorten the communication distance between the edge core and further reduce latency, which is suitable for ultra-large-scale multi-core systems.

[0034] The NoC (Network on Chip) based interconnect architecture, with its advantages of strong scalability, high bandwidth, and flexible routing, can effectively adapt to the development needs of multi-core in-memory computing systems and provide solid support for the efficient operation of neural networks with hundreds of billions of parameters.

[0035] However, the relevant technologies have the following three shortcomings in in-memory computing multi-core systems: (1) There is an irreconcilable contradiction between signal transmission and routing design in high-performance in-memory computing cores. To achieve efficient computation, high-performance in-memory computing cores use highly parallel array input and output signals. If a centralized router is used for data forwarding, a large number of parallel signals need to be concentrated at the router before being distributed. This will greatly increase the wiring density between the core and the router, causing serious wiring congestion problems. Wiring congestion not only greatly increases the difficulty of chip layout and routing, but may also introduce additional signal delays due to the increased wiring length, affecting signal integrity and transmission speed. While binding the array to the output direction, i.e., fixing the output signal of a specific array to a specific transmission direction, can alleviate the wiring pressure to some extent, this requires designing an extremely complex mechanism at the routing level to ensure efficient data transmission and no deadlock. In related technologies, routing algorithms have difficulty taking this into account well, which can easily lead to data transmission blockage and reduce the overall system performance.

[0036] (2) Effective management of data transmission of different types in multi-core systems is difficult. Multi-core systems contain various types of data, including virtual channel data transmitted via a loop, real channel data directly consumed by the current core, and high-priority control data. In related technologies, router designs employ centralized crossbar arbitrators to handle the arbitration of these data transmissions. However, this approach, when faced with multiple data types, not only causes routing conflicts due to a large amount of data competing for transmission resources, but also requires data from different channels to wait for arbitration, increasing data transmission latency. For example, high-priority control data may be delayed while waiting for arbitration of low-priority data, and real channel data may be squeezed out by virtual channel data, severely impacting the efficiency and timeliness of data transmission.

[0037] (3) In memristor-based in-memory computing systems, the resource configuration and switching of virtual channels are unreasonable. For memristor-based in-memory computing systems, configuring virtual channels is only needed during system startup. After startup, these configured virtual channels are idle, while data transmission often requires more virtual channel resources. However, related technologies lack suitable switching strategies, failing to promptly allocate the resources of configured virtual channels to critical data virtual channels, resulting in resource waste and affecting bandwidth utilization. Although methods have been proposed such as using dedicated threads to interrupt data transmission links, reconstructing XY routes, and adopting independent design of virtual channels, real channels, and command channels, skipping the write FIFO (First-In, First-Out) process of real channel commands, and reconfiguring configuration strategies for virtual channels to release configured virtual channels to data virtual channels after network configuration, these methods, in practical applications, either affect the continuity of data transmission due to interrupting transmission links or have shortcomings in the coordination of independent channel design and resource switching, failing to fundamentally solve the problem of unreasonable resource configuration.

[0038] The following description, with reference to the accompanying drawings, describes a routing system and electronic device for multi-core in-memory computing according to embodiments of this application. Addressing the problems mentioned in the background art, such as congestion caused by high-parallel signal routing between in-memory macrocells and routing, blocking due to improper data transmission direction matching, and low path planning efficiency, this application proposes three technical points, as follows: (1) A method is proposed to use a dedicated thread to interrupt the data transmission link and reconstruct the XY route. When a high-performance in-memory computing core experiences routing blockage or deadlock risk due to array and output direction binding, the dedicated thread can actively intervene to temporarily interrupt the current data transmission link and replan the path based on the XY routing algorithm. This application dynamically adjusts the data transmission path, which can both adapt to the hardware design of array and direction binding to reduce wire congestion and ensure efficient data transmission at the routing level without deadlock, thereby improving the routing flexibility and reliability of the system under complex topology.

[0039] (2) A design is proposed that separate virtual channels, real channels, and instruction channels, skipping the write FIFO process for real channel instructions. The three types of channels are physically isolated at the hardware level. The virtual channel handles data transmitted via a bypass, the real channel specifically processes data directly consumed by the current core, and the instruction channel is used to transmit high-priority control signals, avoiding interference between different types of data during transmission. For the real channel, since its data is directly consumed by the local core, it does not need to wait in the FIFO buffer but is directly sent to the core processing unit, significantly reducing the data dwell time in the channel and significantly improving data transmission utilization and core response speed.

[0040] (3) A reconfigurable configuration strategy is proposed for virtual channels. During the startup phase of the memristor-based in-memory computing system, the configuration virtual channel is responsible for network initialization, parameter configuration, and other tasks. When the system enters a stable operating phase, the function of the configuration virtual channel is automatically switched, and the hardware resources it occupies are released and allocated to the data virtual channel. Through this dynamic resource allocation mechanism, the idle resources of the configuration virtual channel during system operation are avoided, effectively increasing the available bandwidth of the data virtual channel, improving the overall bandwidth utilization, and better meeting the needs of large-scale parallel data transmission.

[0041] Specifically, Figure 1 This is a structural diagram of a multi-core in-memory computing routing system provided in an embodiment of this application. The multi-core in-memory computing routing system 10 includes: multiple in-memory computing cores, taking in-memory computing core 101 as an example; multiple in-memory computing macro units, taking in-memory computing macro unit 102 as an example; a parsing on-chip network 103; multiple directional output interfaces, taking directional output interface 104 as an example; a router 105; and a target core 106.

[0042] The in-memory computing core 101 includes multiple in-memory computing macro units, multiple directional output interfaces, a parsing on-chip network 103, and a router 105. The in-memory computing macro units 102 are bound to the directional output interfaces 104. The parsing on-chip network 103 parses the header information of the data packets to determine the target transmission direction of the data packets. When the target transmission direction is the same as the location direction of the target core 106, the directional output interface 104 outputs the data packets to the target core 106. The router 105 is used to generate a target planning path based on the target transmission direction and the location direction of the target core 106 when the target transmission direction is different from the location direction of the target core 106, and transmits the data packets to the target core 106 according to the target planning path.

[0043] It is understood that in this embodiment, the in-memory computing macro unit generates data packets (including header information) to be transmitted, parses the on-chip network to parse the data packet header, determines the target transmission direction, and judges whether the target transmission direction is consistent with the location direction of the target core. If the directions are consistent: the direction output interface directly forwards the data packet to the target core; if the directions are inconsistent: the router generates a planned path based on the "target transmission direction" and the "target core location", and then forwards the data packet to the target core. By binding the in-memory computing macro unit with the direction output interface, and combining the direction judgment of the on-chip network with the path planning of the router, the data transmission is split: when the target direction matches, it is directly transmitted through the direction output interface; when it does not match, the router plans the path for transmission, thus constructing an efficient multi-core in-memory computing routing mechanism. This effectively solves the problem of congestion caused by the concentrated convergence of high-parallel signals and reduces signal transmission delay; by combining direct transmission with direction matching and path planning, data transmission blocking is avoided, improving the communication efficiency and overall performance of the multi-core in-memory computing system.

[0044] It's important to note that XY routing is a routing algorithm specifically designed for 2D mesh interconnect structures. Physical routers use this algorithm to determine the direction (east / south / west / north) of data transmission, ultimately reaching the target core. In a multi-core in-memory computing system based on 2D mesh interconnects, all in-memory computing cores are arranged like a chessboard grid, with rows (X-axis) and columns (Y-axis), and each core has a unique coordinate. The core logic of XY routing is to first transmit data along the X-axis to the column containing the target core, then along the Y-axis to the row containing the target core, and vice versa.

[0045] Specifically, in a multi-core in-memory computing system based on 2DMesh interconnect, to reduce the routing complexity caused by data interaction within the core, the system adopts a hardware design where each of the four in-memory computing macrounits within the core is bound to one of the four directional (east, south, west, and north) output interfaces. Furthermore, for scenarios where the downstream of the source core's output is directly the destination core, data does not need to undergo routing forwarding; it is directly output to the destination core through the directional interfaces bound to the in-memory computing macrounits. This transmission path is branchless and directionally unique, thus naturally avoiding the risk of deadlock. In addition, during the entire data transmission process from the source core to the destination core, scenarios are also included where the source core's output direction is opposite to the destination core's direction, and scenarios where the target transmission direction is perpendicular to the destination core's direction. The system further ensures the deadlock-free characteristics of the XY routing through refined path planning, which will be further elaborated in later embodiments.

[0046] In this embodiment, the router 105 is used to transmit data to the edge node along the target transmission direction when the target transmission direction is opposite to the location direction of the target core 106. The edge node then transmits the data back to the source core. After the source core converts the data into virtual channel data, the target planned path is obtained by replanning the path. The data packet is then transmitted to the target core 106 according to the target planned path.

[0047] It is understood that in this embodiment, when the target transmission direction is opposite to the target core location direction, the router adopts the processing logic of "edge node backhaul - source core virtual channel conversion - path replanning": first, the data packet is transmitted to the edge node along the target direction, then the edge node transmits it back to the source core. After the source core converts the data packet into virtual channel data, the router replans the target path and finally transmits the data packet to the target core. This solves the problem of chaotic and easily blocked data transmission paths when the target direction is opposite to the core location direction, and avoids detour delays or resource waste caused by reverse transmission. Through the cooperation of edge node backhaul and virtual channel conversion, the rationality and stability of data transmission in reverse scenarios are ensured, further improving the routing flexibility and overall communication efficiency of multi-core in-memory computing systems.

[0048] For example, such as Figure 2 As shown in (③④), the source core needs to output data westward, but the destination core is actually located on the east side. In this case, a loopback path is constructed using the direct connection design of the input and output interfaces of the Mesh edge nodes: the data is first transmitted to the edge node in the original direction (westward), loops back to the source core through the interface direct connection mechanism, is converted into virtual channel data, and then the XY routing algorithm replans the path (first adjusting along the X-axis to the destination core column, and then transmitting along the Y-axis), and finally forwards it to the destination core. This process avoids deadlock caused by path conflicts by maintaining the consistency of the routing direction (X first, then Y).

[0049] In this embodiment of the application, the router 105 is used to transmit data to the relay node along the target transmission direction when the target transmission direction is perpendicular to the position direction of the target core 106. The relay node replans the path to obtain the target planned path and transmits the data packet to the target core 106 according to the target planned path. The relay node is an in-memory computing core among multiple in-memory computing cores.

[0050] It is understood that in this embodiment, when the target transmission direction is perpendicular to the target core location direction, the router transmits the data packet along the target direction to a relay node (the relay node is any in-memory computing core in the system). The relay node then replans the path, and finally completes the transmission of the data packet to the target core according to the newly planned target path. This solves the problems of difficult path adaptation and easy detours when the target direction is perpendicular to the core location direction; the relay node reuses the existing in-memory computing core in the system, eliminating the need to add additional dedicated routing nodes and reducing hardware costs; and the real-time path replanning by the relay node ensures the smoothness of vertical transmission, further improving the routing adaptability and communication efficiency of the multi-core in-memory computing system.

[0051] For example, such as Figure 2 As shown in (①②), the source core outputs data in the Y direction, but the destination core is located in the X direction of the source core. In this case, a relay node needs to be inserted into the transmission path to achieve deadlock-free routing by interrupting the continuity of the virtual channel: the data is first transmitted to the relay node and temporarily stored in the real channel storage area of ​​that core, breaking the continuous transmission path of the original virtual channel; the relay node then replans the path based on the XY routing algorithm (first completing the Y-direction adjustment, then starting the X-direction transmission), ensuring that the data arrives at the destination core in the order of "Y first, then X," thus avoiding loop deadlock caused by disordered YX turns. The insertion operation of the relay node is performed during the planning phase before the neural network deployment.

[0052] During deployment, the principle of "prioritizing the deployment of in-memory computing convolutional layers" is followed. When it is detected that the weights of the next layer cannot be deployed directly downstream of the upstream PE (Processing Element), and its transmission path does not satisfy the "X first then Y" or "Y first then X" order of XY routing, the system inserts a relay task at the node where the path first turns YX: configuring the downstream of the upstream PE as the relay core, and the downstream of the relay core then points to the destination core. Through this path splitting, the potential loop path of the virtual channel is physically broken, fundamentally eliminating the risk of deadlock and ensuring the stability of large-scale multi-core in-memory computing systems under high parallel transmission scenarios.

[0053] Specifically, Figure 3 This is a flowchart of a neural network deployment that supports the insertion of intermediate layers, such as... Figure 3 As shown, the steps are as follows: In step one, after the process starts, the system locates the nearest available PE core to deploy the neural network layer. The system prioritizes processing units that are "nearby in location and have available computing / storage resources" and deploys a specific layer of the neural network (such as a convolutional layer or a fully connected layer) onto that PE core, thereby reducing data transmission latency and improving deployment efficiency.

[0054] In step two, it checks whether all network layers have been deployed. After each layer is deployed, the system determines whether all layers of the entire neural network (such as the entire structure from the input layer to the output layer) have been deployed. If yes, the process ends; if no, it proceeds to the next routing decision.

[0055] In step three, it is determined whether there is a next PE that satisfies the XY routing. The system checks whether there exists a target PE that can be directly connected via XY routing rules and has available resources for deploying the next layer of the network. If yes, return to step one to continue deployment; if no, proceed to the next step.

[0056] In step four, if there is no PE that satisfies the XY route, a relay layer is inserted on the link that breaks the XY route. When the XY route cannot directly connect to the target PE, in order to ensure the continuity of data transmission and layer deployment, a "relay layer" needs to be inserted in the "link that the XY route cannot cover" so that data can be indirectly transmitted to the target PE area. After insertion, return to step one to continue deploying the next layer.

[0057] The neural network deployment process of this application achieves efficient and continuous distributed deployment of neural network layers on PE resources through the logic of "prioritizing deployment based on proximity and XY routing, and inserting a relay layer when routing is blocked".

[0058] In this embodiment, router 105 includes an allocation module, a real channel receiving module, a virtual channel receiving module, and an arbitration module. The allocation module receives data packets and classifies them. The data categories include real channel data, virtual channel data, and control flow data. The real channel receiving module receives real channel data. The virtual channel receiving module receives virtual channel data. The arbitration module receives control flow data and schedules data from the real channel receiving module and the virtual channel receiving module based on scheduling instructions.

[0059] It is understood that in this embodiment, the router receives data packets through an allocation module and classifies them into real-channel data, virtual-channel data, and control flow data. The real-channel receiving module and virtual-channel receiving module then receive the corresponding data types respectively. The arbitration module receives control flow data and, based on scheduling instructions, schedules data from the real and virtual-channel receiving modules, achieving classified processing and orderly scheduling of multiple data types. This solves the problem in related technologies where centralized arbitration by routers is difficult to efficiently manage multiple data types. By classifying and receiving data, it avoids different types of data from competing for resources. Combined with the targeted scheduling of the arbitration module, it ensures priority processing of control flow data while preventing real and virtual-channel data from interfering with each other, improving the orderliness and timeliness of data transmission, and further optimizing the communication efficiency of multi-core in-memory computing systems.

[0060] Specifically, such as Figure 4 As shown, the virtual / real channel allocation module is the allocation module; the core buffer is the storage for the real channel receiving module; the virtual channel receiving module for inputs in the east / west / south / north directions is the virtual channel receiving module; and the fixed + round-robin priority arbitrator's TDM arbitration module for sending data in each direction is the scheduling logic of the arbitration module. The overall interaction relationship between the modules is illustrated.

[0061] In this embodiment, the real channel receiving module includes a real channel storage area and a PE module. The real channel storage area stores real channel data, and the PE module performs logical operations based on the real channel data, responds to the scheduling instructions of the arbitration module, and reads data from the real channel storage area and sends it to the arbitration module.

[0062] It is understood that the embodiments of this application ensure the independent storage of real channel data through a dedicated storage area, avoiding it from being occupied by other types of data; the PE module completes the logical operation in advance, and combined with the fast data reading in response to the scheduling command, improves the processing efficiency and scheduling response speed of real channel data, and further optimizes the transmission and utilization efficiency of real channel data in the multi-core in-memory computing system.

[0063] Specifically, such as Figure 4 As shown, the core cache is the real channel storage area; logical modules such as PE or Bypass are PE modules, which reflects the flow of real channel data from storage to PE operation and then to subsequent scheduling.

[0064] In this embodiment, the virtual channel receiving module includes multiple virtual channel receiving sub-modules with input from multiple directions. Each virtual channel receiving sub-module includes an allocation unit, a virtual channel storage area, and a polling unit. The allocation unit is used to allocate virtual channels, the virtual channel storage area stores virtual channel data, and the polling unit is used to respond to the scheduling instructions of the arbitration module, read data from the virtual channel storage area, and send it to the arbitration module.

[0065] It is understood that the embodiments of this application avoid data mixing in different directions by setting sub-modules in different directions; the allocation unit ensures reasonable allocation of virtual channel resources, the storage area realizes independent data storage, and the polling unit responds efficiently to scheduling, thereby improving the receiving efficiency and scheduling order of virtual channel data, reducing resource conflicts, and further optimizing the virtual channel communication performance of the multi-core in-memory computing system.

[0066] Specifically, such as Figure 4 As shown, the virtual channel receiving module for inputs in the east / south / west / north directions is the virtual channel receiving submodule; the virtual channel allocation module is the allocation unit; multiple virtual channel FIFOs are the virtual channel storage area; and the fixed + polling priority arbitrator is the polling unit, reflecting the internal structure of the submodule.

[0067] In this embodiment, the virtual channel storage area includes a request virtual channel queue, a response virtual channel queue, and a data virtual channel queue. The request virtual channel queue and the response virtual channel queue transmit core configuration data packets and configuration completion feedback information, while the data virtual channel queue is used to transmit intermediate result data in neural network streaming computation.

[0068] It is understood that this application embodiment separates configuration-related data from intermediate calculation results through a classification queue, ensuring that virtual channel data for different purposes each has a dedicated storage and transmission path. This avoids configuration data and computation data crowding out each other's resources, improves the targeted utilization efficiency of virtual channel resources, ensures the smoothness of the configuration process and calculation process, and further optimizes the overall operating performance of the multi-core in-memory computing system. Specifically, such as Figure 4 As shown, 3 Configuration Virtual Channel FIFO and 3 Response Virtual Channel FIFO are request / response virtual channel queues; e2s / e2w / e2n Data Virtual Channel FIFO is a data virtual channel queue, reflecting the functional classification of queues.

[0069] In this embodiment, when router 105 receives a data packet, it records the direction of the corresponding data packet and the amount of data that can be sent in the virtual channel. It synchronously monitors the remaining capacity of various virtual channels input in the corresponding direction of the downstream core. When the remaining capacity of the virtual channel of the downstream core meets the capacity of the data packet, it sends the data packet. The downstream core detects that the data transmission is completed and reports back to the upstream core and releases the storage space of the corresponding virtual channel. The output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue. The polling unit uses a polling priority strategy to determine the output order of queues of the same type.

[0070] It is understood that the capacity monitoring and feedback release mechanism in this application embodiment ensures data transmission adaptability, avoids transmission failure caused by insufficient downstream space, ensures that there is space for downstream to receive data when upstream sends data, and avoids packet loss; the queue priority and polling strategy ensure that core configuration-related data is transmitted first, and the same type of queue is output in an orderly manner, improving the reliability and scheduling rationality of virtual channel transmission, optimizing the data flow efficiency of multi-core in-memory computing system, and solving the problem of blocking caused by capacity mismatch in virtual channel data transmission and the problem of disordered data output of different types of queues.

[0071] Specifically, such as Figure 4 As shown, the downstream space counter reflects the function of monitoring the remaining capacity of the downstream virtual channel; the fixed + polling priority arbitrator reflects the priority scheduling + polling strategy.

[0072] The data routing process will be illustrated below through a specific embodiment, such as... Figure 5 As shown, the steps are as follows: In step one, the system waits for the interface to receive data to be transmitted via a "borrowed route". After the process starts, the core remains in a waiting state until the interface receives the data that needs to be transmitted via the "borrowed route" method before proceeding to subsequent processing.

[0073] In step two, it is determined whether the current Core is the destination Core. After the interface receives the data, it first determines whether the current Core is the target Core of the data: if yes, it executes "write to Core internal cache", the data has reached the destination, and is directly stored in the actual cache inside the core, and proceeds to step five to continue the determination; if no, it proceeds to step three, and the data needs to be forwarded.

[0074] In step three, the data is written to the corresponding virtual channel based on the data type and next-hop direction. Since the current Core is not the destination, data needs to be forwarded through the virtual channel. At this point, the data is written to the corresponding virtual channel based on the data type (such as configuration data packets, intermediate results of neural networks, etc.) and the next-hop direction (east, south, west, north, etc.) determined by the routing algorithm (different directions / types of data correspond to different virtual channels).

[0075] In step four, the receiving module initiates a virtual channel transmission request to the sending module. The virtual channel receiving module (responsible for temporarily storing data) initiates a "transmission request" to the sending module (responsible for transmitting data downstream), informing that "there is data in the virtual channel that needs to be forwarded".

[0076] In step five, it is determined whether the downstream core is the target core. After the request is initiated, it is determined again whether the next downstream core is the target core: if yes, "send and match out-of-band information" is executed, the data is sent to the target core, and "out-of-band information" (such as control auxiliary signals) is processed, and the process ends; if no, proceed to step six, and continue to forward downstream.

[0077] In step six, determine whether the downstream Core has sufficient VC (Virtual Channel) space. The storage space of the virtual channel is limited, so it is necessary to check whether the virtual channel corresponding to the downstream Core has enough space to receive data: If "No": return to "Step 1" and continue to wait (until there is space downstream before attempting transmission). If "Yes": proceed to step seven.

[0078] In step seven, data is sent and a signal is released to the upstream feedback space. When there is sufficient VC space downstream, two things are done: data is sent to the downstream Core; and a signal is released to the upstream Core feedback space in the input direction, informing the upstream that "the previously occupied virtual channel space has been released and new data can continue to be transmitted," after which the process ends.

[0079] In this embodiment, the polling unit is also used for dynamic reconstruction of the virtual channel. The dynamic reconstruction process of the virtual channel includes: during the system configuration phase, the output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue; during the data transmission phase, the request virtual channel queue and the response virtual channel queue are converted into data virtual channel queues, and the polling priority strategy is used to determine the sending order of queues of the same type.

[0080] It is understood that the embodiments of this application dynamically reconstruct the virtual channel in stages. In the configuration stage, core configuration data is prioritized for transmission to efficiently complete system initialization. In the data stage, the original configuration queue is reused to expand data transmission resources. Combined with the polling scheduling of queues of the same type, the overall utilization rate of virtual channel resources is improved, and the orderly scheduling of queues in the data transmission stage is ensured. This further optimizes the communication resource adaptation capability and operating efficiency of the multi-core in-memory computing system and solves the problem of idle virtual channels and low resource utilization in the data transmission stage during the configuration stage in related technologies.

[0081] It should be noted that in on-chip networks, virtual channels are parallel communication paths implemented through logical isolation on physical links. Their core is to divide a single physical link into multiple independently schedulable "logical sub-links" through independent buffer queues and control signals. This application optimizes the virtual channel from static configuration to dynamic reconfiguration.

[0082] Specifically, such as Figure 6As shown, regarding queue functions and types, in the static state: virtual channel queues are divided into "request virtual channel queues," "response virtual channel queues," and "data virtual channel queues," with fixed types and functional isolation. After dynamic reconstruction: during the data transmission phase, the original "request / response virtual channel queues" can be dynamically converted into "data virtual channel queues," realizing elastic reuse of queue resources. After the configuration phase ends, the configuration-related virtual channel resources are released to supplement the bandwidth requirements for data transmission. Regarding scheduling strategies: in the static state: a fixed priority is adopted, with "request / response virtual channel queues" having a higher priority than "data virtual channel queues." After dynamic reconstruction: during the data transmission phase, all queues (including the converted original request / response queues) uniformly adopt a "round-robin priority strategy," scheduling queues of the same type according to the principle of fairness to avoid congestion caused by a single data queue occupying resources for a long time.

[0083] For example, in terms of virtual channel management, this application introduces a dynamic reconfiguration mechanism after configuration. Once the system completes the initial configuration process—that is, after the request virtual channel and response virtual channel successfully transmit all configuration data packets and feedback information—these two types of virtual channels will no longer be idle. Instead, their functions will be reconfigured through hardware logic switching—their original configuration interaction function will be released, and they will be used as data virtual channels specifically for transmitting intermediate result data in neural network streaming computation. The core value of this reconfiguration mechanism lies in achieving efficient reuse of virtual channel resources.

[0084] During the system configuration phase, request and response virtual channels are crucial for ensuring the rapid issuance and verification of configuration commands. However, once the system enters a stable operation phase, configuration requirements decrease significantly. Converting these virtual channels to data virtual channels directly increases the available bandwidth for data transmission, effectively handling the massive amounts of intermediate data generated during neural network inference and avoiding waste of hardware resources. To adapt to the dynamic reconstruction of virtual channels, the arbitrator's priority design is also adjusted accordingly. During the configuration phase, to ensure efficient system initialization, the output priorities of the request and response virtual channel FIFOs are set higher than those of the data virtual channel FIFO. For example, `fifo_w_e_req` and `fifo_s_e_req` have the same priority, both higher than `fifo_w_e_data`, thus prioritizing the scheduling of configuration-related data. Once the virtual channels complete reconstruction and enter the data transmission phase, the arbitration priority strategy changes accordingly. At this point, the original request and response virtual channels have been converted into data virtual channels. The arbitrator adopts a round-robin priority strategy for all data virtual channels. For example, the sending order of fifo_w_e_req (converted to a data virtual channel), fifo_s_e_req (converted to a data virtual channel), and fifo_n_e_req (converted to a data virtual channel) is determined by round-robin. This adjustment ensures that data in each direction can compete fairly for transmission resources, avoids long-term bandwidth occupation in one direction, and makes data transmission in the entire Mesh network more balanced and efficient.

[0085] The process of dynamic reconstruction will be illustrated below through a specific embodiment, such as... Figure 7 As shown, the steps are as follows: In step one, monitor the congestion status of the Core virtual channels. Continuously monitor the load status of the virtual channels within the core to determine if there are scenarios requiring reconfiguration, such as queue congestion or load imbalance.

[0086] In step two, register configuration is initiated to reconfigure the address space. If a reconfiguration is detected, the address space allocation rules for the virtual channel are adjusted through register configuration to avoid address conflicts.

[0087] In step three, register configuration is initiated to reconfigure the arbitrator priority. Register configuration is then used again to adjust the arbitrator's scheduling priority for virtual channel data, ensuring the data transmission order better aligns with actual needs (e.g., prioritizing the transmission of higher-priority data).

[0088] In step four, when the virtual channel tasks are cleared, the new configuration is used for route reconstruction. After all data transmission tasks in the current virtual channel are completed, the new configuration is enabled to complete the route reconstruction of the virtual channel and make the new rules effective.

[0089] The multi-path, two-layer arbitration routing design of this application will be illustrated below through a specific embodiment, such as... Figure 4As shown, unlike related technologies where all data (including data transmitted upstream and downstream and local core data) is centrally exchanged through a shared crossbar on a single core router, this application employs a differentiated data distribution architecture and a hierarchical arbitration mechanism. After data enters the core, it is first finely classified by the virtual-physical channel allocation module: First, the real channel data or instructions directly consumed by this core (such as weight parameters and operation instructions required for local calculation). Second, virtual channel data that needs to be forwarded to the downstream core (such as intermediate feature maps for cross-core transmission). Thirdly, control flow instructions that need to be forwarded (such as task configuration, interrupt signals, and other critical control information).

[0090] This classification is not a simple logical division, but rather a hierarchical arbitration achieved through independent hardware pathways: control flow data, due to its involvement in global system scheduling, is given the highest priority and does not need to go through a cache or FIFO queue. It directly reaches the sending arbitration module from the allocation module via a dedicated "fast track," ensuring zero-latency transmission of critical control commands; real channel data, as the direct input to local core operations, does not participate in cross-core forwarding arbitration, but is directly written into the core's internal cache or drives the computing unit to start operations. Its output only needs to be sent through local request arbitration, avoiding resource contention with forwarded data; virtual channel data that needs to be forwarded enters a dedicated FIFO queue built by SRAM, which is cached in an orderly manner according to type and direction, waiting for the next level of arbitration scheduling.

[0091] The core advantages of this hierarchical arbitration mechanism are as follows: On the one hand, by allocating independent transmission paths for data of different priorities, control flow commands can be transmitted "in queue" with priority, real channel data can quickly respond to local computing needs, and virtual channel data avoids disorderly conflicts through a queuing mechanism, thus achieving "priority differentiation guarantee" for data transmission; on the other hand, in related technologies, a single-level shared Crossbar needs to process arbitration requests for all types of data simultaneously, which is prone to bandwidth saturation and conflict blocking due to a surge in data volume. However, this application distributes the arbitration pressure to three levels: "direct control flow arbitration," "real channel local arbitration," and "virtual channel queue arbitration," reducing the processing load of the single-level arbitration module by more than 60%, significantly alleviating the bandwidth bottleneck of the shared Crossbar, and greatly improving the overall data throughput and response speed of the multi-core system.

[0092] Each core determines the next transmission direction of data by parsing the packet header information in the on-chip network (NoC). If the data is not destined for the current core, it is stored in the virtual channel area of ​​SRAM. The virtual channels are organized in a FIFO (First-In, First-Out) manner, containing a total of 36 FIFOs: the system has four input directions: East (E), West (W), South (S), and North (N), with 9 FIFOs corresponding to each input direction. The specific FIFO where the data is stored is determined by the receiving direction and the transmitting direction (the other three directions besides the receiving direction)—by querying the address corresponding to the receiving and transmitting directions in the configuration table, the target FIFO where the data should be stored can be determined.

[0093] These 36 FIFOs are divided into three categories, each performing a different function: Request and Response virtual channel FIFOs are used to transmit core configuration data packets and configuration completion feedback information, while Data virtual channel FIFOs are specifically used to transmit intermediate result data in neural network streaming computation. The four transmission directions are independent at the hardware level; only the nine FIFOs corresponding to the same receiving direction require transmission arbitration. In terms of priority, the output priority of Request and Response virtual channel FIFOs is higher than that of Data virtual channel FIFOs (e.g., fifo_w_e_req and fifo_s_e_req have the same priority, and both are higher than fifo_w_e_data); a round-robin priority strategy is used among virtual channel FIFOs of the same type (e.g., the transmission order of fifo_w_e_req, fifo_s_e_req, and fifo_n_e_req FIFOs is determined by a round-robin mechanism).

[0094] When receiving data, the router records the amount of data that can be sent in the corresponding direction and type of virtual channel in real time, and simultaneously monitors the remaining capacity of various virtual channels input in the corresponding direction to the downstream core. Only when the downstream core can accommodate the current data packet size will it package and send the data along with relevant control information; after the data is sent, it will report back to the upstream core and release the storage space of the corresponding virtual channel, ensuring the efficient recycling of virtual channel resources.

[0095] According to the multi-core in-memory computing routing system proposed in this application, the in-memory computing macro unit generates data packets (including header information) to be transmitted, parses the packet header by the on-chip network, determines the target transmission direction, and judges whether the target transmission direction is consistent with the location direction of the target core. If the directions are consistent: the direction output interface directly forwards the data packet to the target core; if the directions are inconsistent: the router generates a planned path based on the "target transmission direction" and the "target core location", and then forwards the data packet to the target core. By binding the in-memory computing macro unit with the direction output interface, and combining the direction judgment of the on-chip network with the path planning of the router, the data transmission is split: when the target direction matches, it is directly transmitted through the direction output interface; when it does not match, the router plans the path for transmission, thus constructing an efficient multi-core in-memory computing routing mechanism. This effectively solves the problem of congestion caused by the concentrated convergence of high-parallel signals and reduces signal transmission delay; by combining direct transmission with direction matching and path planning, data transmission blocking is avoided, improving the communication efficiency and overall performance of the multi-core in-memory computing system.

[0096] In summary, the embodiments of this application have at least the following beneficial effects: (1) A deadlock-free XY routing mechanism based on relay node insertion ensures the stability of high-parallel transmission in large-scale systems. In the 2DMesh architecture, by binding the four macrocells in the core to the four-way output interface, three types of path strategies are designed for different transmission scenarios: direct connection transmission, edge loopback path, and relay node insertion. Relay nodes are inserted at the YX bends during the deployment phase, temporarily storing data in the real channel storage area to break the continuity of the virtual channel, ensuring transmission in the XY routing order, and eliminating the risk of deadlock from a physical perspective. This mechanism is executed before the neural network is deployed, avoiding virtual channel loops by splitting the path; at the same time, for the deadlock problem caused by disordered path bends in multi-core Mesh networks, direct connection transmission can avoid branch conflicts, edge loopback can maintain the consistency of routing direction, and relay node insertion can break the loop path of virtual channels, solving the problem of "deadlock caused by disordered bends" in related technologies, enabling large-scale multi-core in-memory computing systems to maintain stable operation during high-parallel data interaction, and providing underlying support for continuous inference of ultra-large-scale neural networks.

[0097] (2) The virtual channel routing design with multi-path hierarchical arbitration improves data transmission efficiency and alleviates bandwidth bottlenecks. A virtual-real channel allocation module divides data into real channels (local consumption), virtual channels (downstream forwarding), and control flow (highest priority). Hierarchical arbitration is achieved through independent hardware paths: the control flow is directly connected to the arbitration module, the real channel directly drives computation, and the virtual channel is cached by type and direction via a FIFO queue in SRAM. A priority and round-robin arbitration strategy is used between FIFOs in the same receiving direction. The router monitors downstream capacity in real time and dynamically releases resources, alleviating the bandwidth pressure of shared crossbars in related technologies. Simultaneously, the control flow is transmitted with zero latency via a "fast track" to avoid scheduling delays, real channel data directly drives local computation to reduce cross-module interaction, and virtual channel data is cached in an ordered manner via FIFOs and dynamically sent in conjunction with a downstream capacity awareness mechanism. Compared to the mixed-flow contention of single crossbars in related technologies, this design significantly reduces the probability of data collisions, improves bandwidth utilization, and the shortcut path between the real channel and control flow also shortens local response time, significantly improving the overall data throughput of the system.

[0098] (3) Configure a dynamic reconfiguration mechanism for virtual channels to optimize resource utilization and adapt to the needs of multiple system stages. After the system is configured, the request and response virtual channels are reconfigured into data virtual channels through hardware logic to transmit intermediate results of the neural network and realize resource reuse; the arbitrator priority is adjusted synchronously with the reconfiguration: during the configuration stage, the request / response channels are given priority to accelerate system initialization, and during the operation stage, all data channels adopt round-robin arbitration to balance the load. This design improves bandwidth utilization through function switching and ensures that there is no deadlock in the routing after reconfiguration in conjunction with the Mesh path design; at the same time, it solves the problem of "idle configuration channels wasting resources" in related technologies, so that virtual channel resources can be dynamically adapted in different stages of the system (initialization / operation) - the configuration stage accelerates system startup, and the operation stage expands data bandwidth, which not only improves hardware resource utilization, but also avoids one-way congestion through load balancing, making data transmission more efficient and balanced.

[0099] This application also provides an electronic device, such as Figure 8 As shown, this includes the aforementioned routing system for multi-core in-memory computing.

[0100] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0102] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0103] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0104] Those skilled in the art will understand that all or part of the steps of the methods implementing the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0105] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A routing system for multi-core in-memory computing, characterized in that, include: Multiple in-memory computing cores, each core including multiple in-memory computing macrounits, multiple directional output interfaces, on-chip network parsing, and routers, wherein... The stored-computing macro unit is bound to the direction output interface; The parsing on-chip network parses the header information of the data packet to determine the target transmission direction of the data packet. When the target transmission direction is the same as the location direction of the target core, the direction output interface outputs the data packet to the target core. The router is used to generate a target planning path based on the target transmission direction and the target core's location direction when the target transmission direction is different from the target core's location direction, and then transmit the data packet to the target core according to the target planning path.

2. The routing system for multi-core in-memory computing as described in claim 1, characterized in that, The router is used to transmit data along the target transmission direction to the edge node when the target transmission direction is opposite to the location direction of the target core, and then transmit the data back to the source core from the edge node. After the source core converts the data into virtual channel data, the target planned path is obtained by replanning the path, and the data packet is transmitted to the target core according to the target planned path.

3. The routing system for multi-core in-memory computing as described in claim 1, characterized in that, The router is used to transmit data along the target transmission direction to a relay node when the target transmission direction is perpendicular to the location direction of the target core. The relay node replans the path to obtain the target planned path and transmits the data packet to the target core according to the target planned path. The relay node is an in-memory computing core among the plurality of in-memory computing cores.

4. The routing system for multi-core in-memory computing as described in claim 1, characterized in that, The router includes an allocation module, a real-channel receiving module, a virtual-channel receiving module, and an arbitration module, wherein... The allocation module receives data packets and classifies them. The data packets include categories such as real channel data, virtual channel data, and control flow data. The real channel receiving module receives the real channel data; The virtual channel receiving module receives the virtual channel data; The arbitration module receives the control flow data and schedules data from the real channel receiving module and the virtual channel receiving module based on scheduling instructions.

5. The routing system for multi-core in-memory computing as described in claim 4, characterized in that, The real channel receiving module includes a real channel storage area and a PE module. The real channel storage area stores the real channel data, and the PE module performs logical operations based on the real channel data, responds to the scheduling instructions of the arbitration module, and reads data from the real channel storage area and sends it to the arbitration module.

6. The routing system for multi-core in-memory computing as described in claim 4, characterized in that, The virtual channel receiving module includes multiple virtual channel receiving sub-modules for input from multiple directions. Each virtual channel receiving sub-module includes an allocation unit, a virtual channel storage area, and a polling unit. The allocation unit is used to allocate virtual channels, the virtual channel storage area stores the virtual channel data, and the polling unit is used to respond to the scheduling instructions of the arbitration module, read data from the virtual channel storage area, and send it to the arbitration module.

7. The routing system for multi-core in-memory computing as described in claim 6, characterized in that, The virtual channel storage area includes a request virtual channel queue, a response virtual channel queue, and a data virtual channel queue. The request virtual channel queue and the response virtual channel queue transmit core configuration data packets and configuration completion feedback information. The data virtual channel queue is used to transmit intermediate result data in neural network streaming computation.

8. The routing system for multi-core in-memory computing according to claim 7, characterized in that, When the router receives a data packet, it records the direction of the data packet and the amount of data that can be sent by the virtual channel. It also monitors the remaining capacity of various virtual channels input by the downstream core in the corresponding direction. When the remaining capacity of the virtual channel of the downstream core meets the capacity of the data packet, the router sends the data packet. When the downstream core detects that the data has been sent, it reports back to the upstream core and releases the storage space of the corresponding virtual channel. The output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue. The polling unit uses a polling priority strategy to determine the output order of queues of the same type.

9. The routing system for multi-core in-memory computing according to claim 6, characterized in that, The polling unit is also used for dynamic reconstruction of the virtual channel, and the process of dynamic reconstruction of the virtual channel includes: During the system configuration phase, the output priority of the request virtual channel queue and the response virtual channel queue is higher than the output priority of the data virtual channel queue. During the data transmission phase, the request virtual channel queue and the response virtual channel queue are converted into the data virtual channel queue, and a round-robin priority strategy is used to determine the sending order of queues of the same type.

10. An electronic device, characterized in that, The system includes the multi-core in-memory computing routing system described in any one of claims 1-9.

Images