Routing system, neural network system, routing method, chip and electronic equipment
By employing multiple interfaces of the routing device in the CNN system for data broadcasting and multicasting, the problem of low data transmission efficiency is solved, achieving low-cost and high-efficiency data transmission, which is suitable for resource-constrained devices.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING ESWIN COMPUTING TECH CO LTD
- Filing Date
- 2026-01-06
- Publication Date
- 2026-05-01
AI Technical Summary
In CNN systems, data transmission efficiency is low and hardware implementation is not optimal, especially in resource-constrained devices. How to achieve efficient data transmission is an urgent problem to be solved.
A routing system is provided, including multiple routing devices, each of which is connected to nodes in multiple directions. The system enables unicast, multicast, and broadcast of data through source interfaces, destination interfaces, multiple source interfaces, and multiple destination interfaces. It adopts a minimalist routing hardware design and software rule constraints to avoid broadcast storms and deadlock risks.
It achieves low-cost and efficient data transmission, avoids the need for additional site setup and address-based routing, improves data transmission efficiency, and is suitable for resource-constrained devices.
Smart Images
Figure CN121967288A_ABST
Abstract
Description
Routing systems, neural network systems, routing methods, chips and electronic devices Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a routing system, neural network system, routing method, chip, and electronic device. Background Technology
[0002] With the rapid development of AI (Artificial Intelligence) technology, unprecedented demands have been placed on hardware computing power. Furthermore, the exponential growth in model complexity and the diversification of application scenarios have become the core driving forces behind the surge in AI computing power demand. CNN (Convolutional Neural Networks), as the core architecture in deep learning for processing grid-like data (such as images, videos, and audio), has achieved breakthroughs in fields such as image recognition and object detection through its characteristics such as local connectivity, weight sharing, and spatial downsampling.
[0003] In CNN systems, how to transmit data is a problem that urgently needs to be solved. Summary of the Invention
[0004] This application provides a routing system, a neural network system, a routing method, a chip, and an electronic device, which can be used to improve data transmission efficiency. The technical solution is as follows.
[0005] On one hand, a routing system is provided, comprising multiple routing devices, each connected to at least one adjacent node in multiple directions. Each routing device includes a source interface, a destination interface, multiple source interfaces, and multiple destination interfaces, wherein the multiple source interfaces correspond to the multiple directions, and the multiple destination interfaces correspond to the multiple directions. The source interface is configured to receive the first data and send the first data to the destination interface when connected to a data source device and the data source device sends first data, and also send the first data to the destination interface when the destination interface is enabled. The source interface is configured to receive the second data and send the second data to the destination interface when connected to a first routing device and the first routing device sends second data, and also send the second data to the destination interface when the destination interface is enabled. The destination interface is configured to receive at least one of the first data or the second data and send at least one of the first data or the second data to a second routing device when enabled. The destination interface is configured to receive at least one of the first data or the second data and send at least one of the first data or the second data to a connected computing unit.
[0006] In one possible implementation, any source interface of each routing device conforms to the following routing rule: data received from a node in a first direction among the plurality of directions needs to be sent to a destination interface in a second direction different from the first direction.
[0007] In one possible implementation, the plurality of routing devices are arranged in a two-dimensional grid topology with i rows and j columns, where i and j are positive integers, and at least one of i and j is greater than or equal to 2. Each row of routing devices in the i-row and j-column routing devices is connected, and one column of routing devices in the j-column routing devices is connected.
[0008] In one possible implementation, the plurality of routing devices are arranged in a two-dimensional grid topology with i rows and j columns, where i and j are positive integers, and at least one of i and j is greater than or equal to 2. The routing devices in the i-row of the j-column routing devices are connected, and the routing devices in each column of the j-column routing devices are connected.
[0009] In one possible implementation, only one of the plurality of routing devices has its source interface connected to the data source device; or, N of the plurality of routing devices have their source interfaces connected to the data source device, where N is greater than 1 and less than the total number of routing devices in the routing system; or, the source interface of each of the plurality of routing devices is connected to the data source device.
[0010] In a second aspect, a neural network system is provided, the neural network system comprising a data source device, a computing unit, and a routing system as described in any of the first aspects, the routing system being connected to the data source device and the computing unit respectively; the data source device is used to send data to the routing system; the routing system is used to receive the data sent by the data source device and send the data to the computing unit; the computing unit is used to receive the data sent by the routing system and perform calculations based on the data.
[0011] In one possible implementation, the number of computing units is the same as the number of routing devices in the routing system, with each routing device connected to one computing unit.
[0012] In one possible implementation, there is one data source device, and one routing device in the routing system is connected to the data source device; or, the number of data source devices is greater than one and less than the total number of routing devices in the routing system, and each of the routing devices in a subset of the routing system is connected to one data source device, the number of the subset of routing devices being the same as the number of data source devices; or, the number of data source devices is the same as the total number of routing devices in the routing system, and each routing device in the routing system is connected to one data source device.
[0013] Thirdly, a routing method is provided, which is applied to a routing device in any of the routing systems described in the first aspect. The routing device is connected to at least one adjacent node in multiple directions. The routing device includes a source interface, a destination interface, multiple source interfaces, and multiple destination interfaces. The multiple source interfaces correspond to the multiple directions, and the multiple destination interfaces correspond to the multiple directions. The method includes: when the source interface is connected to a data source device and the data source device sends first data, receiving the first data through the source interface, sending the first data to the destination interface, and sending the first data to the destination interface when the destination interface is enabled; when the source interface is connected to a first routing device and the first routing device sends second data, receiving the second data through the source interface, sending the second data to the destination interface, and sending the second data to the destination interface when the destination interface is enabled; when the destination interface is enabled, sending at least one of the first data or the second data to a second routing device through the destination interface; receiving at least one of the first data or the second data through the destination interface, and sending at least one of the first data or the second data to a connected computing unit.
[0014] In one possible implementation, any source interface conforms to the following routing rule: data received from a node in a first direction among the plurality of directions must be sent to a destination interface in a second direction different from the first direction.
[0015] A chip is also provided, the chip comprising any of the routing systems described above.
[0016] An electronic device is also provided, which includes the aforementioned chip.
[0017] The technical solution provided in this application has at least the following beneficial effects: The technical solution provided in this application enables the guidance of data flow direction through the destination interface, and enables multiple interfaces of the routing device to directly broadcast and multicast data without the need to set up additional sites or perform routing based on addresses. Therefore, the cost of routing is low and the data transmission efficiency is high. Attached Figure Description
[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] Figure 1 is a schematic diagram of the interface structure of a routing device provided in an embodiment of this application; Figure 2 is a schematic diagram of the structure of a neural network system provided in an embodiment of this application; Figure 3 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 4 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 5 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 6 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 7 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 8 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 9 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 10 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 11 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 12 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 13 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 14 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 15 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 16 is a schematic diagram of the structure of another neural network system provided in an embodiment of this application; Figure 17 is a flowchart of a routing method provided in an embodiment of this application. Detailed Implementation
[0020] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0021] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0022] With the development of AI technology, the demand for hardware computing power is constantly increasing. The exponential growth of model complexity and the diversified expansion of application scenarios are the core driving forces behind the surge in AI computing power demand. The types and functions of neural network models are also increasing, with CNN being one of them. CNNs have achieved breakthroughs in fields such as image recognition and object detection through characteristics such as local connectivity, weight sharing, and spatial downsampling. CNNs are used to process grid-like data such as images, videos, and audio. The need for image and weight reuse in CNNs is a key aspect of the core design of CNN systems.
[0023] For example, in CNNs, convolutional kernels slide across feature maps, repeatedly using the same set of weights to extract similar features at different locations; through multi-level feature collaborative extraction, the original image is processed across multiple convolutional layers. Therefore, the data path design of dedicated neural network accelerators needs to support flexible unicast, multicast, and broadcast transmission of images and weights. This reuse mechanism in CNNs plays a crucial role in reducing computational complexity, minimizing memory usage, and improving model generalization ability.
[0024] Furthermore, the widespread application of AI in resource-constrained devices such as smartphones and tablets has placed increasingly higher demands on the resources of neural network accelerators. These accelerators must ensure high computing power while minimizing resource consumption, placing higher requirements on the design of data transmission paths. The lowest-level implementation logic of convolutional neural networks is matrix multiplication, and the data path for matrix multiplication is mostly implemented using a mesh architecture. Most mesh designs rely on the coordinates of routers for routing. This routing requires mapping addresses to the coordinates of the routers, and the router hardware implementation needs to implement the control logic for routing rules. While some dedicated neural network accelerators have optimized the mesh implementation, the data transmission mode is still controlled by the router, indicating that the hardware implementation is not optimal.
[0025] To address this, this application provides a routing system for neural network systems. Through a simplified routing hardware design and constraint definition of software usage rules, it enables flexible routing of image data, weight data, and multiply-accumulate data between various PEs (Processing Element) (also known as computational units or operational units) within the neural network, enabling unicast, multicast, and broadcast routing. The routing system provided in this application includes multiple routing devices, each connected to at least one adjacent node in multiple directions. Each routing device includes a source interface, a destination interface, multiple source interfaces, and multiple destination interfaces. The multiple source interfaces correspond to multiple directions, and the multiple destination interfaces also correspond to multiple directions.
[0026] For example, the routing device provided in this application includes multiple interfaces, including but not limited to source interfaces and destination interfaces, as shown in Table 1 below. The source interfaces include a source interface (global mem) and multiple source interfaces, such as source interfaces in the four directions: north (N), south (S), east (E), and west (W). Correspondingly, the destination interfaces include a destination interface (mounted with a local PE) and four destination interfaces, such as destination interfaces in the four directions: north, south, east, and west.
[0027] Among them, the source and destination interfaces in the four directions of east, west, south, and north are used for data transmission between routers, globalmem is the source of image and weight data, and local pe is the computing unit mounted on each router.
[0028] Table 1
[0029] It should be noted that the multiple source interfaces and multiple destination interfaces described above are only illustrated using the four cardinal directions (north, south, east, and west) as examples, and are not intended to limit the directions. They can also be described using the four directions (up, down, left, right) or other directions, and the number of directions can be two, four, or other quantities. This application embodiment does not limit the names or number of directions. Furthermore, since each routing device is connected to at least one adjacent node in multiple directions, each of the multiple source interfaces and multiple destination interfaces corresponds to multiple directions.
[0030] The source interface is configured to receive first data and send first data to the destination interface when connected to a data source device and the data source device sends first data, and also send first data to the destination interface when the destination interface is enabled; the source interface is configured to receive second data and send second data to the destination interface when connected to a first routing device and the first routing device sends second data, and also send second data to the destination interface when the destination interface is enabled; the destination interface is configured to receive at least one of the first data or the second data when enabled, and send at least one of the first data or the second data to a second routing device; the destination interface is configured to receive at least one of the first data or the second data, and send at least one of the first data or the second data to the connected computing unit.
[0031] The routing system provided in this application guides the direction of data flow through the destination interface enablement, enabling direct broadcasting and multicasting of data based on multiple interfaces of the routing device without the need for additional site settings or address-based routing. Therefore, the cost of routing is low and the data transmission efficiency is high.
[0032] For example, in the embodiments of this application, the routing device includes, but is not limited to, routing rule constraints with software and hardware, in order to avoid broadcast storms that lead to bandwidth waste during broadcasting, to prevent the same data from being repeatedly sent to the same destination from different directions, and also to avoid the risk of deadlock.
[0033] The hardware routing rules include, but are not limited to, any source interface of each routing device conforming to the following routing rule: data received from a node in a first direction among multiple directions must be sent to a destination interface in a second direction different from the first direction. In other words, data received from any source interface is not sent through a destination interface in the same direction. For example, data received from a north-bound source interface (src N) is not sent from a north-bound destination interface (dest N); data received from a south-bound source interface (src S) is not sent from a south-bound destination interface (dest S); data received from a west-bound source interface (src W) is not sent from a west-bound destination interface (dest W); and data received from an east-bound source interface (src E) is not sent from an east-bound destination interface (dest E).
[0034] Furthermore, multiple routing devices are arranged in a two-dimensional grid topology with rows of i and columns of j, where i and j are positive integers, and at least one of i and j is greater than or equal to 2. The routing rules of the routing devices' software include, but are not limited to: In broadcast mode, the correct configuration of the on / off state of each routing device is crucial to prevent broadcast storms. The configuration principle is: the broadcast range is defined by a cluster or group of i*j processing units (pe clusters): either all processing units in row i are horizontally connected, and only one column of processing units in column j is vertically connected; or all processing units in column j are vertically connected, and only one row of processing units in row i is horizontally connected. That is, each row of routing devices in row i and column j is connected, and one column of routing devices in column j is connected. Alternatively, one row of routing devices in row i of the routing devices in row i and column j is connected, and each column of routing devices in column j is connected.
[0035] This application does not limit the method of row or column connectivity between routing devices. Connectivity is achieved by enabling corresponding interfaces; that is, enabling the interface on the i-th row path and the interface on one of the column paths, or simply enabling the interface on one row path and the interface on the J-th column path. This avoids broadcast storms that would occur if all interfaces on all rows and columns were enabled. A broadcast storm refers to the same data being sent from the source interfaces of different routing devices.
[0036] For example, in this embodiment of the application, only one of the multiple routing devices has its source interface connected to the data source device. In this case, the multiple routing devices have a single data source, enabling unisource multicast or broadcast.
[0037] Alternatively, in this embodiment, N routing devices have their source interfaces connected to corresponding data source devices, where N is greater than 1 and less than the total number of routing devices in the routing system. In this case, multiple routing devices have multiple data sources, but not every routing device's source interface is connected to a data source device, thus achieving multi-source multicast or broadcast.
[0038] Alternatively, in this embodiment, the source interface of each of the multiple routing devices is connected to a corresponding data source device. In this case, each routing device is connected to a corresponding data source device, and the data from each data source device is multicast and broadcast in the routing device at row i and column j.
[0039] This application provides a neural network system, which includes a data source device, a computing unit, and a routing system. The routing system is connected to both the data source device and the computing unit. The data source device is used to send data to the routing system. The routing system is used to receive data sent by the data source device and send data to the computing unit. The computing unit is used to receive data sent by the routing system and perform calculations based on the received data.
[0040] In one possible implementation, the number of processing units is the same as the number of routing devices in the routing system, with each routing device connected to one processing unit.
[0041] For example, there may be one data source device, and one routing device in the routing system may be connected to the data source device; or, the number of data source devices may be greater than one and less than the total number of routing devices in the routing system, with each of the routing devices in a subset of the routing system connected to one data source device, and the number of the subset of routing devices being the same as the number of data source devices; or, the number of data source devices may be the same as the total number of routing devices in the routing system, with each routing device in the routing system connected to one data source device.
[0042] For ease of understanding, the interfaces shown in Table 1 are represented using the interfaces of the routing device shown in Figure 1. The left side of Figure 1 shows the various interfaces of the routing device, and the right side is a simplified schematic diagram of the interfaces on the left. In the simplified schematic diagram shown on the right side of Figure 1, only the switches of the destination interfaces are shown as examples for the four cardinal directions (north, south, east, and west). Interfaces marked in black indicate that the interface is open, and GLB in black represents the initial source of the data source. Furthermore, since the destination interface is connected to the PE without switch control, the destination interface is always open; that is, the destination interface in black in the figure indicates that the destination interface is open.
[0043] Next, referring to Figure 2, the neural network system includes a first data source device 110, a first routing device 111, a second routing device 122, a first processing unit 112 connected to the first routing device 111, and a second processing unit 122 connected to the second routing device 121, and will be described in different cases.
[0044] In the neural network system shown in Figure 2, the first data source device 110 is used to send first data to the source interface of the first routing device 111; the first routing device 111 is used to receive the first data sent by the first data source device 110 through the source interface of the first routing device 111, send the first data to the first processing unit 112 through the destination interface of the first routing device 111, and send the first data to the source interface of the second routing device 121 through the first destination interface of the first routing device 111 when the first destination interface of the first routing device 111 is open; the second routing device 121 is used to receive the first data sent by the first routing device 111 through the source interface of the second routing device 121, and send the first data to the second processing unit 122 through the destination interface of the second routing device 121; the first processing unit 112 is used to receive the first data sent by the first routing device 111 and perform calculations based on the first data; the second processing unit 122 is used to receive the first data sent by the second routing device 121 and perform calculations based on the first data.
[0045] The neural network system shown in Figure 2 can be represented in the form shown in Figure 1, as shown in Figure 3. In Figure 3, the first routing device 111 is connected to the second communication device 112 (shown as a solid line in Figure 3 for illustrative purposes only, and is not limited to wired or wireless connections). Furthermore, the first routing device 111 is also connected to the first data source device 110. In Figure 3, 110 is marked in black, indicating that the source interface connecting the first routing device 111 and the first data source device 110 is in the open state.
[0046] It should be noted that the second data source device 120 is also shown in Figure 3, but it is not marked in black, indicating that the source interface connecting the second routing device 121 and the second data source device 120 is not transmitting data. Furthermore, Figure 3 is illustrated only with the second routing device 121 located vertically to the first routing device 111 as an example. In practical applications, the second routing device 121 can also be located horizontally to the first routing device 111; this application does not impose any limitations on this.
[0047] In one possible implementation, referring to Figure 4, the neural network system further includes a second data source device 120; the second data source device 120 is used to send second data to the source interface of the second routing device 121; the second routing device 121 is used to receive first data sent by the first routing device 111 through the source interface of the second routing device 121, receive second data sent by the second data source device 120 through the source interface of the second routing device 121, and send the first data and the second data to the second processing unit 122 through the destination interface of the second routing device 121; the second processing unit 122 is used to receive the first data and the second data, and perform calculations based on the first data and the second data.
[0048] The neural network system shown in Figure 4 can be represented in the form shown in Figure 1, as shown in Figure 5. In Figure 5, the first routing device 111 is connected to the second communication device 112 (shown as a solid line in Figure 5, for illustrative purposes only, and not limited to wired or wireless connections). The first routing device 111 is also connected to the first data source device 110. In Figure 5, 110 is marked in black, indicating that the source interface connecting the first routing device 111 and the first data source device 110 is not transmitting data. Figure 5 shows the second data source device 120, marked in black, indicating that the source interface connecting the second routing device 121 and the second data source device 120 is transmitting data.
[0049] It should be noted that Figure 5 is only illustrated with the example of the second routing device 121 being located in the vertical direction of the first routing device 111. In practical applications, the second routing device 121 can also be located in the horizontal direction of the first routing device 111. This application does not limit this.
[0050] In the neural network system shown in Figure 1 or Figure 4, referring to Figure 6, the neural network system further includes at least one third routing device 131 and a third processing unit 132 connected to the third routing device 131; the second routing device 121 is further configured to send first data to the source interface of the third routing device 131 through the first destination interface of the second routing device 121; the third routing device 131 is configured to receive the first data sent by the second routing device 121 through the source interface of the third routing device 131, and send the first data to the third processing unit 132 through the destination interface of the third routing device 131; the third processing unit 132 is configured to receive the first data sent by the third routing device 131 and perform calculations based on the first data.
[0051] In addition to sending the first data to the third routing device 131, the second routing device 121 may also send only the second data. In one possible implementation, the neural network system further includes at least one third routing device 131 and a third processing unit 132 connected to the third routing device 131. The second routing device 121 is also used to send the second data to the source interface of the third routing device 131 through the first destination interface of the second routing device 121 when the first destination interface of the second routing device 121 is open. The third routing device 131 is used to receive the second data sent by the second routing device 121 through the source interface of the third routing device 131 and send the second data to the third processing unit 132 through the destination interface of the third routing device 131. The third processing unit 132 is used to receive the second data sent by the third routing device 131 and perform calculations based on the second data.
[0052] The neural network system shown in Figure 6 can be represented in the form shown in Figure 1, as shown in Figure 7. In Figure 7, the first routing device 111 is connected to the second communication device 112 (shown as a solid line in Figure 7, for illustrative purposes only, and is not limited to wired or wireless connections). The first routing device 111 is also connected to the first data source device 110. In Figure 7, 110 is marked in black, indicating that the source interface connecting the first routing device 111 and the first data source device 110 is for data transmission. Figure 7 shows the second data source device 120, also marked in black, indicating that the source interface connecting the second routing device 121 and the second data source device 120 is for data transmission.
[0053] As shown in Figure 7, the second routing device 121 is also connected to the third routing device 131 (shown as a solid line in Figure 7 for illustrative purposes only, and is not limited to wired or wireless connections). As shown in Figure 7, the third routing device 131 is also connected to the third data source device 130. The 130 shown in Figure 7 is not marked in black, indicating that the source interface connecting the third routing device 131 and the third data source device 130 is not transmitting data.
[0054] It should be noted that Figure 7 is only illustrated with the example of the second routing device 121 being located in the vertical direction of the first routing device 111 and the third routing device 131 being located in the vertical direction of the second routing device 121. In practical applications, the second routing device 121 may also be located in the horizontal direction of the first routing device 111 and the third routing device 131 may be located in the horizontal direction of the second routing device 121. This application does not limit this.
[0055] In one possible implementation, referring to Figure 8, the neural network system further includes at least one third data source device 130; the third data source device 130 is used to send third data to the source interface of the corresponding third routing device 131; the third routing device 131 is used to receive first data sent by the second routing device 121 through the source interface of the third routing device 131, receive third data sent by the third data source device 130 through the source interface of the third routing device 131, and send the first data and the third data to the third processing unit 132 through the destination interface of the third routing device 131; the third processing unit 132 is used to receive the first data and the third data sent by the third routing device 131, and perform calculations based on the first data and the third data.
[0056] Optionally, referring to Figure 8, in one possible implementation, the neural network system further includes at least one third data source device 130; the third data source device 130 is used to send third data to the source interface of the corresponding third routing device 131; the third routing device 131 is used to receive the second data sent by the second routing device 121 through the source interface of the third routing device 131, receive the third data sent by the third data source device 130 through the source interface of the third routing device 131, and send the second data and the third data to the third processing unit 132 through the destination interface of the third routing device 131; the third processing unit 132 is used to receive the second data and the third data sent by the third routing device 131, and perform calculations based on the second data and the third data.
[0057] In one possible implementation, the neural network system further includes at least one third routing device 131 and a third processing unit 132 connected to the third routing device 131; the second routing device 121 is further configured to send first data and second data to the source interface of the third routing device 131 through the first destination interface of the second routing device 121; the third routing device 131 is configured to receive the first data and second data sent by the second routing device 121 through the source interface of the third routing device 131, and send the first data and second data to the third processing unit 132 through the destination interface of the third routing device 131; the third processing unit 132 is configured to receive the first data and second data sent by the third routing device 131, and perform calculations based on the first data and second data.
[0058] In one possible implementation, the neural network system further includes at least one third data source device 130; the third data source device 130 is used to send third data to the source interface of the corresponding third routing device 131; the second routing device 121 is further used to send first data and second data to the source interface of the third routing device 131 through the first destination interface of the second routing device 121; the third routing device 131 is used to receive the first data and second data sent by the second routing device 121 through the source interface of the third routing device 131, receive the third data sent by the third data source device 130 through the source interface of the third routing device 131, and send the first data, the second data and the third data to the third processing unit 132 through the destination interface of the third routing device 131; the third processing unit 132 is used to receive the first data, the second data and the third data sent by the third routing device 131, and perform calculations based on the first data, the second data and the third data.
[0059] The neural network system shown in Figure 8 can be represented in the form shown in Figure 1, as shown in Figure 9. In Figure 9, the first routing device 111 is connected to the second communication device 112 (shown as a solid line in Figure 9, for illustrative purposes only, and not limited to wired or wireless connections). The first routing device 111 is also connected to the first data source device 110. In Figure 9, 110 is marked in black, representing the source-end interface data transmission between the first routing device 111 and the first data source device 110. Figure 9 shows the second data source device 120, also marked in black, representing the source-end interface data transmission between the second routing device 121 and the second data source device 120.
[0060] As shown in Figure 9, the second routing device 121 is also connected to the third routing device 131 (shown as a solid line in Figure 9 for illustrative purposes only, and is not limited to wired or wireless connections). As shown in Figure 9, the third routing device 131 is also connected to the third data source device 130. The 130 shown in Figure 9 is marked in black, representing the source interface for data transmission between the third routing device 131 and the third data source device 130.
[0061] It should be noted that Figure 9 is only illustrated with the example of the second routing device 121 being located in the vertical direction of the first routing device 111 and the third routing device 131 being located in the vertical direction of the second routing device 121. In practical applications, the second routing device 121 may also be located in the horizontal direction of the first routing device 111 and the third routing device 131 may be located in the horizontal direction of the second routing device 121. This application does not limit this.
[0062] In one possible implementation, the neural network system includes multiple third routing devices 131 and third operation units 132, as shown in Figure 10. Figure 10 illustrates two as an example, but there can be more than two, which will not be elaborated further.
[0063] In one possible implementation, the neural network system includes multiple third routing devices 131, third operation units 132, and third data source devices, as shown in Figure 11. Figure 11 illustrates two as an example, but there can be more than two, which will not be elaborated further.
[0064] In one possible implementation, there are multiple first data source devices 110, multiple first routing devices 111, and multiple second routing devices 121, and each of the multiple first data source devices 110, multiple first routing devices 111, and multiple second routing devices 121 corresponds one-to-one. As shown in Figure 12, only three are shown as an example, but this is not intended to be a limitation, and other numbers are also possible.
[0065] In one possible implementation, referring to Figure 13, the neural network system further includes a fourth routing device 141 and a fourth processing unit 142, and optionally, a fourth data source device 140. The fourth routing device 141 is connected to the third routing device 131, and the third routing device 131 is further configured to send at least one of first data, second data, or third data to the fourth routing device 141. The fourth routing device 141 is configured to receive at least one of the first data, second data, or third data, and send at least one of the first data, second data, or third data to the fourth processing unit 142. The fourth processing unit 142 performs calculations based on at least one of the first data, second data, or third data.
[0066] It should be noted that the number of the fourth routing device 141, the fourth processing unit 142, and the fourth data source device 140 can be multiple, as shown in the different cases in Figures 14 to 16. The principle is similar and will not be described in detail again. The communication between the routing devices should comply with the routing rules of the software and hardware mentioned above. In addition, the first data, the second data, and the third data include, but are not limited to, image data, weight data, and multiply-accumulate data, etc. The embodiments of this application do not limit these and can be determined based on the application scenario.
[0067] In summary, the neural network system provided in this application has lower costs and higher data transmission efficiency because it can directly broadcast and multicast data through multiple interfaces of the routing device without the need for additional site setup or address-based routing. Furthermore, through a simplified routing hardware design and constraint definitions of software usage rules, flexible routing of unicast, multicast, and broadcast data of image data, weight data, and multiply-accumulate data between various computational units (PEs) in the neural network system is achieved.
[0068] Referring to Figure 17, this application embodiment also provides a routing method. The method is applied to a routing device in the above-mentioned routing system. The routing device is connected to at least one adjacent node in multiple directions. The routing device includes a source interface, a destination interface, and multiple source interfaces and multiple destination interfaces. The multiple source interfaces correspond to multiple directions, and the multiple destination interfaces correspond to multiple directions. The method includes: in step 1701, when the source interface is connected to a data source device and the data source device sends first data, receiving the first data through the source interface, sending the first data to the destination interface, and sending the first data to the destination interface when the destination interface is open.
[0069] In step 1702, if the source interface is connected to the first routing device and the first routing device is sending second data, the second data is received through the source interface, and the second data is sent to the destination interface. If the destination interface is enabled, the second data is sent to the destination interface. In step 1703, if the destination interface is enabled, at least one of the first data or the second data is sent to the second routing device through the destination interface.
[0070] In step 1704, at least one of the first data or the second data is received through the destination interface, and at least one of the first data or the second data is sent to the connected computing unit.
[0071] In one possible implementation, data received by any source interface will not be sent to the same destination interface in the same direction. That is, any source interface conforms to the following routing rule: data received from a node in the first direction among multiple directions must be sent to a destination interface in a second direction different from the first direction.
[0072] The implementation process of each step of the method described in Figure 17 above can be referred to the relevant descriptions of the neural network system and routing system mentioned above, and will not be repeated here.
[0073] It should be noted that those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0074] When implemented using software, it can be implemented entirely or partially in the form of a computer program product. This computer program product includes one or more computer program instructions. Furthermore, all data and signals involved in this application are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the first, second, and third data involved in this application were all obtained under fully authorized conditions.
[0075] This application embodiment also provides a routing device, which includes a source interface, a destination interface, and multiple source and destination interfaces in various directions; the source interface is used to receive first data and send first data to the destination interface when connected to a first data source device and the first data source device sends first data, and also to send first data to the destination interface when the destination interface is enabled; the source interface is used to receive second data and send second data to the destination interface when connected to a first routing device and the first routing device sends second data, and also to send second data to the destination interface when the destination interface is enabled; the destination interface is used to receive at least one of the first data or the second data when enabled, and to send at least one of the first data or the second data to a second routing device; the destination interface is used to receive at least one of the first data or the second data, and to send at least one of the first data or the second data to a connection processing unit.
[0076] In one possible implementation, data received from either source interface will not be sent to the destination interface in the same direction.
[0077] In one possible implementation, the source interface is used to receive first data sent by the data source device and to send the first data to the corresponding multiple destination interfaces and destination interface.
[0078] In one possible implementation, the source interface is used to receive the second data transmitted by the first routing device and send the second data to the corresponding multiple destination interfaces and destination end interfaces.
[0079] This application also provides a chip that includes any of the routing devices or routing systems described above.
[0080] This application also provides an electronic device that includes any of the chips described above.
[0081] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.
Claims
1. A routing system, characterized in that, The routing system includes multiple routing devices, each of which is connected to at least one adjacent node in multiple directions. Each routing device includes a source interface, a destination interface, multiple source interfaces, and multiple destination interfaces. The multiple source interfaces correspond to the multiple directions, and the multiple destination interfaces correspond to the multiple directions. The source interface is used to receive the first data and send the first data to the destination interface when the data source device is connected and the data source device sends the first data, and also to send the first data to the destination interface when the destination interface is enabled. The source interface is used to receive the second data and send the second data to the destination interface when connected to the first routing device and the first routing device sends the second data, and also to send the second data to the destination interface when the destination interface is enabled. The destination interface is configured to receive at least one of the first data or the second data when it is enabled, and to send at least one of the first data or the second data to the second routing device. The destination interface is used to receive at least one of the first data or the second data, and to send at least one of the first data or the second data to the connected computing unit.
2. The routing system according to claim 1, characterized in that, Each source interface of the routing device conforms to the following routing rule: data received from a node in the first direction among the plurality of directions must be sent to a destination interface in a second direction that is different from the first direction.
3. The routing system according to claim 1, characterized in that, The plurality of routing devices are arranged in a two-dimensional grid topology in rows i and columns j, where i and j are positive integers, and at least one of i and j is greater than or equal to 2. Each row of routing devices in the i-row-j-column routing devices is connected, and one column of routing devices in the j-column routing devices is connected.
4. The routing system according to claim 1, characterized in that, The plurality of routing devices are arranged in a two-dimensional grid topology in rows i and columns j, where i and j are positive integers, and at least one of i and j is greater than or equal to 2. The routing devices in the i-row routing devices in the j-column routing devices are connected, and the routing devices in each column of the j-column routing devices are connected.
5. The routing system according to any one of claims 1-4, characterized in that, Of the plurality of routing devices, only one routing device has its source interface connected to the data source device; or, N routing devices have their source interfaces connected to the data source device, where N is greater than 1 and less than the total number of routing devices in the routing system. Alternatively, the source interface of each of the plurality of routing devices is connected to the data source device.
6. A neural network system, characterized in that, The neural network system includes a data source device, a computing unit, and a routing system as described in any one of claims 1-5, wherein the routing system is connected to the data source device and the computing unit respectively; the data source device is used to send data to the routing system. The routing system is used to receive data sent by the data source device and send the data to the computing unit; The computing unit is used to receive data sent by the routing system and perform calculations based on the data.
7. The neural network system according to claim 6, characterized in that, The number of processing units is the same as the number of routing devices in the routing system, and each routing device is connected to one processing unit.
8. The neural network system according to claim 6 or 7, characterized in that, There is one data source device, and one routing device in the routing system is connected to the data source device; Alternatively, the number of data source devices is greater than one and less than the total number of routing devices in the routing system, with each routing device in a subset of the routing devices connected to one data source device, and the number of the subset of routing devices being the same as the number of data source devices; or, the number of data source devices is the same as the total number of routing devices in the routing system, with each routing device in the routing system connected to one data source device.
9. A routing method, characterized in that, The routing method is applied to a routing device in any one of the routing systems described in claims 1-5. The routing device is connected to at least one adjacent node in multiple directions. The routing device includes a source interface, a destination interface, multiple source interfaces, and multiple destination interfaces. The multiple source interfaces correspond to the multiple directions, and the multiple destination interfaces correspond to the multiple directions. The method includes: when the source interface is connected to a data source device and the data source device sends first data, receiving the first data through the source interface, sending the first data to the destination interface, and sending the first data to the destination interface when the destination interface is open; when the source interface is connected to a first routing device and the first routing device sends second data, receiving the second data through the source interface, sending the second data to the destination interface, and sending the second data to the destination interface when the destination interface is open; when the destination interface is open, sending at least one of the first data or the second data to a second routing device through the destination interface; receiving at least one of the first data or the second data through the destination interface, and sending at least one of the first data or the second data to a connected computing unit.
10. The method according to claim 9, characterized in that, Any source interface conforms to the following routing rule: data received from a node in the first direction among the plurality of directions must be sent to a destination interface in a second direction that is different from the first direction.
11. A chip, characterized in that, The chip includes the routing system described in any one of claims 1-5.
12. An electronic device, characterized in that, The electronic device includes the chip of claim 11.