A method of dead die bypass routing and a processing system with dead die bypass routing

TWI945263BActive Publication Date: 2026-11-11TESLA INC
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
TW111131019
Authority / Receiving Office
TW · TW
Patent Type
Patents
Current Assignee / Owner
Priority Date
2022-07-01
Filing Date
2022-08-17
Publication Date
2026-11-11
Estimated Expiration
2042-08-16

AI Technical Summary

Technical Problem

Existing signal routing systems in electronic circuits face challenges when dealing with defective or damaged dies in die arrays, leading to deadlocks and inefficiencies in data transmission, particularly in high-performance computing applications like neural network training and machine learning.

Method used

Implementing decentralized routing methods that allow packets to make multiple turns and utilize virtual channels to bypass dead dies, with routing rules that prevent loops and ensure efficient data transmission across dies, using routing tables and dedicated channels to manage traffic flow.

Benefits of technology

Enhances data transmission efficiency by allowing packets to bypass dead dies effectively, maintaining full or near-full bandwidth and preventing deadlocks, suitable for high-performance computing applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure TWG2TB001920008_001
    Figure TWG2TB001920008_001
  • Figure TWG2TB001920008_002
    Figure TWG2TB001920008_002
  • Figure TWG2TB001920008_003
    Figure TWG2TB001920008_003
Patent Text Reader

Abstract

Systems and methods for routing packets around failed dies in an array are disclosed. In some embodiments, the method for routing packets in a die array including failed dies may include routing packets from a first die to a second die via a first channel of a plurality of first channels, and routing packets from a second die to a third die via a second channel of a plurality of second channels, based on one or more routing rules. One or more routing rules may allow the first channels to route packets to a subset of the plurality of second channels including the second channels. The second channels may be orthogonal to the first channels, such that routing packets from the first die to the third die includes a turning. The method can route packets around failed dies.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure generally relates to signal routing in electronic circuits. Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 260437 entitled “Bypass Routing”, filed August 19, 2021, and U.S. Provisional Application No. 63 / 367568 entitled “Bypass Routing”, filed July 1, 2022, the disclosure of which is incorporated herein by reference in its entirety and for all purposes. Prior Technology

[0003] Some processing systems include die arrays. The dies in an array can include computing circuitry. The dies in an array can communicate with each other. One or more dies in the array may be defective or damaged, while most dies in the array are fully functional. In such a processing system, data can be routed around one or more inoperable dies. Summary of the Invention

[0004] Each innovation described in the claims has several aspects, none of which is solely responsible for its desired properties. Without limiting the scope of the claims, some of the prominent features of this disclosure will now be briefly described.

[0005] In some aspects, the techniques described herein relate to a method for dead die bypass routing, the method comprising: routing packets from a source die to an intermediate die via a first route, the first route comprising redirecting packets from a first channel of a plurality of first channels to a second channel of a plurality of second channels based on one or more routing rules, the one or more routing rules allowing the first channel to route packets to a subset of a plurality of second channels including the second channel, and the first channel being orthogonal to the second channel; and routing packets from the intermediate die to a destination die via a second route, the second route comprising redirecting packets from the second channel to a third channel orthogonal to the second channel, wherein the system on the wafer includes a die array including a source die, an intermediate die, a destination die and at least one dead die, and wherein the first route and the second route bypass the at least one dead die.

[0006] In some respects, the techniques described herein relate to a method in which the at least one dead die comprises two dead dies, and the first route and the second route bypass the two dead dies.

[0007] In some aspects, the techniques described herein relate to a dead-die bypass routing method, comprising: routing packets from a first die to a second die via a first channel of a plurality of first channels, wherein the first die, second die, third die, and dead die are included in an array; and routing packets from the second die to a third die via a second channel of a plurality of second channels based on one or more routing rules, said one or more routing rules allowing the first channel to route packets to a subset of a plurality of second channels including the second channel, and the second channels being orthogonal to the first channel such that routing packets from the first die to the third die includes a turn, wherein the method routes packets around the dead die.

[0008] In some respects, the techniques described herein relate to a method in which routing packets from a first die to a second die, and routing packets from a second die to a third die, comprises: querying a routing table at least in part based on the address of the third die, wherein the routing table conforms to one or more routing rules.

[0009] In some respects, the techniques described herein relate to a method in which one or more routing rules prevent packets from being routed in a loop.

[0010] In some respects, the techniques described herein relate to a method in which each of a plurality of first channels and each of a plurality of second channels is assigned a priority, and wherein said one or more routing rules do not allow routing from a lower priority channel of the plurality of first channels to a higher priority channel of the plurality of second channels.

[0011] In some respects, the techniques described herein relate to a method in which the routing from a first die to a third die is configured to terminate at the lowest priority first channel among a plurality of first channels or the lowest priority second channel among a plurality of second channels.

[0012] In some respects, the techniques described herein relate to a method in which one or more routing rules allow packets to be routed to an escape channel configured to allow packets to move to a channel with higher priority.

[0013] In some respects, the techniques described herein relate to a method in which routing from a first die to a third die includes a second turn from a second channel to a third channel, wherein the third channel has a lower priority than the second channel, and wherein the second channel has a lower priority than the first channel.

[0014] In some respects, the techniques described herein relate to a method in which the routing table includes a default route, which is used if no defined route exists from the first die to the second die.

[0015] In some respects, the techniques described herein relate to a method that revolves around at least two dead core routing packets.

[0016] In some respects, the techniques described herein relate to a method that includes grouping routes with multiple turns.

[0017] In some respects, the techniques described herein relate to a method in which an on-chip system includes an array.

[0018] In some respects, the techniques described herein involve a method that further includes routing packets from a third die to a die outside the array.

[0019] In some aspects, the technology described herein relates to a processing system with dead-die bypass routing, the processing system comprising: a die array including a first die, a second die, a third die, and a dead die; wherein the processing system is configured to route packets from the first die to the third die via the second die based on one or more routing rules implemented by circuitry of the processing system, thereby bypassing the dead die, the packets being routed via at least a first channel and a second channel, and the one or more routing rules allowing the first channel of a plurality of first channels to route packets to a subset of a plurality of second channels orthogonal to the plurality of first channels.

[0020] In some respects, the technology described herein relates to a processing system configured to route packets from a first die to a third die via multiple turns.

[0021] In some respects, the technology described herein relates to a processing system in which the die array includes a second die, and the processing system is configured to bypass a second dead die when routing packets from a first die to a third die.

[0022] In some respects, the techniques described herein relate to a processing system in which one or more routing rules prevent packets from being routed in loops.

[0023] In some respects, the techniques described herein relate to a processing system, wherein the processing system includes a routing table that stores information associated with one or more routing rules.

[0024] In some respects, the techniques described herein relate to a processing system in which each of a plurality of first channels and each of a plurality of second channels is assigned a priority, and wherein said one or more routing rules do not allow routing from a lower priority channel of the plurality of first channels to a higher priority channel of the plurality of second channels.

[0025] In some respects, the techniques described herein relate to a processing system configured to generate neural network training data. Simple Explanation of the Diagram

[0026] This disclosure has been described herein with reference to accompanying drawings of certain embodiments, which are intended to illustrate and not limit the scope of the disclosure. It should be understood that the drawings are incorporated in and form a part of this specification and are for illustrative purposes only and may not be drawn to scale.

[0027] [Figure 1] is a diagram depicting an example of distributed routing.

[0028] [Figure 2] is a diagram of an example of multi-turn routing according to some embodiments.

[0029] [Figures 3A-3B] depict example network routes according to some embodiments.

[0030] [Figure 4] illustrates a multi-die configuration with multiple dead dies according to some embodiments.

[0031] [Figure 5] illustrates an example of a routing layout according to some embodiments.

[0032] [Figures 6A, 6B, 6C, 7A, 7B] and [Figure 8] illustrate example routes with varying complexity according to some embodiments. Implementation

[0033] The following description of certain embodiments presents various descriptions of particular embodiments. However, the innovations described herein can be embodied in a variety of different ways, for example, as defined and covered by the claims. In this description, reference is made to the accompanying drawings, wherein similar reference numerals may indicate the same or functionally similar elements. It will be understood that the elements illustrated in the figures are not necessarily drawn to scale. Furthermore, it will be understood that some embodiments may include more elements than illustrated in the figures and / or a subset of the elements illustrated in the figures. Additionally, some embodiments may incorporate any suitable combination of features from two or more figures.

[0034] When assembling multiple dies onto a multi-chip module (MCM) or larger substrate, some dies may be damaged during the assembly process. One or more aspects of this application relate to a mechanism that allows routing around inoperable dies.

[0035] Packets can include control information and payload information. Control information can include, for example, packet source, packet destination, packet type, packet priority, etc. Packets can include, for example, data, memory read requests, semaphore requests, barrier requests, etc., or any suitable combination thereof. When routing packets on a mesh or ring network, it is desirable to avoid deadlocks that can occur when multiple packets in a packet set each expect resources that other packets in the set are using. For example, if a device has queues A, B, and C, and packets a0, a1, ..., b0, b1, ..., c0, c1, ... filling queues A, B, and C, a deadlock can occur if a0 in queue A wants to move to queue B, b0 in queue B wants to move to queue C, and c0 in queue C wants to move to queue A, but each queue is full and waiting for packets to leave before accepting new packets. One way to overcome this deadlock is to avoid all circular loops, for example, by adding rules that disallow certain packets from moving. In the example above, the deadlock can be broken if the path from queue C to queue A is removed.

[0036] One example scheme allows packets to travel horizontally first and then vertically. By disallowing turns from vertical to horizontal channels, the system avoids the possibility of packets in the source channel getting stuck while waiting for the destination channel to drain, while that channel is waiting for the same packets to drain from the source channel. This scheme might only allow packets to make one turn from source to destination, and it will work if the network is intact. However, if there are missing or inoperable dies, there may be gaps in the network that cause packets to make multiple turns to reach their destination. This scheme will not allow those packets to make turns.

[0037] On a grid spanning multiple dies, there can be many channels, rather than just a single horizontal channel and a single vertical channel. By subdividing these channel sets into smaller sets, and by allowing turns from one set to another, the system can allow packets to make several turns from source to destination while still retaining the property of not allowing circular or toroidal paths for packets.

[0038] For example, suppose that for all horizontal channels H and all vertical channels V, a packet can travel from any H channel to any V channel, making one turn from source to destination. If the system divides the horizontal channels into two groups, HA and HB, and similarly divides the vertical channels into two groups, VA and VB, then the system can allow packets to turn from HA to VA, from VA to HB, and from HB to VB. Therefore, packets can make three turns from source to destination, which may be sufficient for routing around a single dead core. A complicating situation exists if the packet source is not present in the HA row or logically adjacent to the HA row (e.g., not near the HA row), or if the destination is not near the VB column. Therefore, in addition to changing the labeling of the H and V network segments, the system can add dedicated channels to capture packets from each source to each HA row, and from each V column to each destination.

[0039] According to various aspects of this application, one or more advantages over conventional implementations / methods can be achieved. According to this application, such a method is not necessarily required for all implementations. Additional benefits or efficiencies can also be achieved according to aspects of this disclosure. Such improvements include, but are not limited to: 1) A way to label the rows / columns of a network as belonging to certain subgroups; 2) Routes packets from any source to one or more dedicated channels for a specific tagged row / column subgroup; 3) Routing packets from a specifically tagged row / column subgroup to one or more dedicated channels to any destination; or 4) A routing table, which can be programmed to ensure that packets can be redirected from a set of rows / columns of a specific tag to one or more subsets of the set of rows / columns of the tag.

[0040] The embodiments described herein can be used and / or specifically configured for high-performance computing and / or compute-intensive applications. For example, the embodiments described herein can be configured for neural network training and / or processing, machine learning, artificial intelligence, etc. Some embodiments described herein can be used for neural network training to generate data for use by autonomous driving systems of vehicles (e.g., automobiles).

[0041] Packets can be routed between different dies in various ways. For example, Figure 1 depicts a distributed routing pattern. In this pattern, a packet can be routed from die 101 to die 103, passing through die 102, where only a single turn is permitted. According to the distributed routing pattern in Figure 1, a node from a single column on die 101 is routed to a single column on die 102, and then to a single column on die 103. Traffic from die 101 to die 102 can utilize full input / output (IO) bandwidth. However, traffic from die 102 to die 103 can only utilize a portion of the bandwidth, effectively limiting the bandwidth on the second part of the packet routing from die 101 to die 103.

[0042] Figure 2 illustrates an alternative routing pattern allowing multiple turns according to an embodiment. In the multi-turn route depicted in Figure 2, physical channels are grouped into virtual channels. Rows and columns can each be grouped into virtual channels. When a turn occurs, rules exist for routing traffic between the virtual channels. A turn can include a route from one channel to an orthogonal channel, such as a route from (a) row to column or (b) column to row. In the illustrated route, traffic can only be turned from a lower-numbered (e.g., higher-priority) virtual channel to a higher-numbered (e.g., lower-priority) virtual channel. On the second die 202, each row can turn traffic into a different column. Therefore, traffic can be distributed across the input / output interfaces between the second die 202 and the third die 203. This configuration allows full or near-full bandwidth to be used for two segments of the path from the first die 201 to the third die 203. On the third die 203, each column can be turned to the corresponding destination row.

[0043] Alternatively or additionally, other rules for routing traffic can be implemented to utilize the full bandwidth or near-full bandwidth of a path from one die to another, which includes at least one turn. Such rules may include routing each row (or column) to a corresponding column (or row) or a subset of columns (or rows) when traffic is turned. In some embodiments, the available bandwidth for routing may depend on the number of turns routing from the first die to the second die, which may be affected by, for example, the number and arrangement of non-functional (dead) dies in the die array.

[0044] In assemblies with multiple dies, such as systems on a wafer assembly (in some embodiments, which may include integrated fan-out wafers or devices fabricated according to other wafer-level packaging or fan-out (InFO) wafer-level packaging technologies), multi-chip modules, etc., one or more dies may be inoperable (dead) due to manufacturing defects or damage during the assembly process. For example, a dead die itself may have internal defects (particles, impurities, cracks, broken connections, bridging connections, etc., or any combination thereof), defects may exist in the physical circuitry that enables communication with the die, and so on. In some embodiments, a dead die may be suitable for some functions but not for others, for example because only a portion of the die is damaged. This can pose serious problems for routing signals between dies. For example, routing around non-functional (dead) dies may be desired.

[0045] Besides the issues of non-functional (dead) dies, combining a large number of dies into multi-die assemblies can present additional challenges. For example, when routing packets on mesh or ring networks, it is generally desirable to avoid deadlocks that can occur when routing tables or algorithms attempt to route different packets using the same resources. In some cases, this can lead to deadlock situations where it is impossible to allocate resources as requested. As an example, consider two packets p and q and two queues A and B. If packet p is in queue A and packet q is in queue B, it may be impossible to swap the packets so that p is in queue B and q is in queue A.

[0046] Such problems can be eliminated or at least mitigated by imposing various routing rules on moving packets. For example, some packet movements may not be permitted. However, it is generally expected that routing rules still enable routing around dead cores. For example, imposing rules such as allowing only a single turn from a horizontal channel to a vertical channel, or vice versa, prevents loops but does not allow routing around dead cores. In some embodiments, if the routing network spans multiple cores, there may be multiple vertical and horizontal channels. In some cases, channels can be divided into smaller sets (e.g., virtual channels), and rules can allow turns from one set to another. Thus, packets can be allowed to make multiple turns while avoiding loops and other routing problems.

[0047] As an example, consider a network with two horizontal channels HA and HB, and two vertical channels VA and VB. The system can allow turns from HA to VA, VA to HB, and HB to VB. By allowing multiple turns, the system can allow routing around a single dead core. However, complexity may arise when the packet source is not close to the HA row or the packet destination is not close to the VB column. Therefore, the system can add additional channels to route packets from each source to each HA row and from each VB column to the packet's destination. Those skilled in the art will appreciate that such a system can be scaled to any suitable number of columns and rows, and that the source and destination can originate from either the horizontal or vertical channels.

[0048] In some embodiments, the PC0 and PC1 networks may be interleaved physical mesh networks. The PC0 network may prefer horizontal as its first direction of travel, while the PC1 network may prefer vertical as its first direction of travel. The networks may otherwise be identical or substantially identical or similar. In some embodiments, the PC0 and PC1 networks may be subdivided into one or more virtual channels. For example, the PC0 and PC1 networks may be subdivided into request Hi, request Lo, and data virtual channels. Memory read requests may be sent along the request Lo virtual channel, while data responses may be sent on the data virtual channel. The request Hi virtual channel may be used for synchronization and timing, such as for semaphore requests and barrier requests. In some embodiments, when the request Hi virtual channel is full, semaphore and barrier requests may instead be routed on the request Lo channel. In some embodiments, virtual channels may be dynamically reallocated based on the need to route various types of information and requests.

[0049] In some cases, virtual networks can be used to implement routing around dead cores. For example, a system can be configured to route packets within a virtual network until they reach the edge or turn of the core, and then potentially travel in either the PC0 or PC1 channel. The virtual network may use the physical PC0 channel for horizontal travel and the physical PC1 channel for vertical travel, although other routing configurations are possible. For example, the physical PC0 channel may not be limited to horizontal travel, and / or the physical PC1 channel may not be limited to vertical travel.

[0050] In some embodiments, various rules can be used to route packets within and across a die. For example, within a die, the system may initially force routing to either PC0 or PC1, or the system may be configured to prefer a specific first network while allowing for different routing in some cases (e.g., if the PC0 channel is busy, the system may instead route along PC1 first). In some instances, PC0 may have a higher priority than PC1. In some other instances, PC1 may have a higher priority than PC0. In some cases, the initial network may be selected based on the packet's destination address. In some embodiments, there may be erroneous routes that route to an error queue and optionally cause interruptions.

[0051] In some embodiments, different channels, sub-channels, columns, rows, and / or channel groups can have one or more routing rules. Each can have different rules, which can avoid loops, dead ends, etc., when routing packets. In some embodiments, a single routing table can be used to implement one or more routing rules, while in other embodiments, multiple routing tables can be used, for example, each column or each row can have its own routing table. The routing table can store information associated with one or more routing rules. The routing table can conform to one or more routing rules. Packets can be routed based on queries to the routing table. For example, such queries can be based on the destination die or the address of intermediate dies in the route from the source die to the destination die.

[0052] In some embodiments, a multi-chip assembly can be subdivided into one or more bays. For packets leaving the die, the system can be configured to determine whether the packet's destination is within or outside the bay. For example, a bay may include a 2 m × 2 n die grid. If the destination is outside the bay, the system may have routing tables or other routing configuration information that can be used to route packets to network devices used to connect the bays to each other.

[0053] If the destination of a packet is on an interval, the system can determine the route by looking up a table entry for the destination die address. In some embodiments, this table may include a k×k (e.g., 8×8) die address array to map dies. The interval portion covered by the array can be determined by defining minimum and maximum row and column addresses. In addition to the die destination, the routing table may include edge entries and corner entries. Row edges may include routes for k m×1 areas of address mapping, while each column edge may include routing entries for k 1×n areas of address mapping. Furthermore, the routing table may include corner entries. In some cases, corner entries may be used to provide routing information for dies located in an m×n die address grid, where each row entry and each column entry overlaps.

[0054] In some cases, the routing table may not contain complete routing information. When no matching table entry exists, the system can be configured with a default route for routing packets.

[0055] In some cases, a system can be configured with one or more secondary routes, and packet routing can be determined by specifying secondary routes instead of reading routes directly from the table. For example, a routing table can contain error routes, interval exit routes, default routes (e.g., forced or preferred first-pass networks as discussed above), routes to the core edge on the PC0 / PC1 network, routes to the core edge on the escape network (E network, discussed more fully below), routes to node rows and columns, then to the core edge on the PC0 / PC1 network, routes to node rows and columns on the E network and then to the core edge, and secondary routes. Secondary routes can include any other type of route or additional route type. For example, a route might be on the PC0 / PC1 or E network, to the row and column specified in the field, and then to the edge.

[0056] In some configurations, multiple nodes can be connected to each other. Therefore, the system may include a routing table describing how to handle arriving packets. If an arriving packet is addressed to a location on a node, it can be routed to its destination using local routing. The packet can be routed directly to its destination, or it can be routed through rows and columns specified by the node address and then redirected to its destination. In some embodiments, a turn can be configured such that the packet remains in either the PC0 or PC1 network, with PC0 traversing to PC1, or PC1 traversing to PC0. In some cases, turns may not be allowed, and therefore any attempt at a turn may result in an error.

[0057] In some cases, packets may travel across cores on their way to their final destination. The system can be configured with routing tables that allow various types of routes. For example, packets may be routed directly across cores without changing networks (i.e., staying in PC0 or PC1), or they may be routed to node rows and columns and then redirected to the edge. In the case of detours, the system may include global bits that define whether the packet will stay in the PC0 network or the PC1 network, or whether transmission from one network to another is permitted. In some embodiments, routes may be determined based on hashes, and again, transmission between the PC0 and PC1 networks may be allowed or disallowed. Furthermore, packets may be routed along secondary routes (e.g., to specific rows or columns and then redirected to the same or different networks) or routed to an error queue. Routing constraints and routing around dead cores

[0058] As discussed above, deadlock situations can occur when routing packets. Routing algorithms can use fixed routes with a finite set of available turns. For example, consider a system with queues 0, 1, and 2. If all queues are full, and the system wants to move the contents of queue 0 to queue 1, the contents of queue 1 to queue 2, and the contents of queue 2 to queue 0, the system will deadlock. Therefore, a rule could allow packets to be routed only to queues with higher numbers. Thus, for example, queue 0 can be moved to queue 1, and queue 1 can be moved to queue 2, but queue 2 cannot be moved to queue 0. Another example rule is to allow packets to be routed only to queues with lower numbers. A rule could make a queue route packets only to a subset of available queues.

[0059] In a 2D mesh (e.g., a network with PC0 and PC1 as described above), packets can traverse sequential queues when they pass directly through the set of nodes. In some embodiments, queues can be labeled individually. However, in some cases, such as when wrap-around is not possible, queues can be collectively labeled as channels.

[0060] As an illustrative example, as depicted in Figure 3A, all PC0 network horizontal queues can be numbered 0, and all PC0 network vertical queues can be numbered 1. Routing rules can specify that packets can be routed to queues with higher numbers (also referred to here as lower priority queues), but not to queues with lower numbers. Therefore, packets can be routed from any PC0 horizontal channel to any PC0 vertical channel, but not from a PC0 vertical channel to a PC0 horizontal channel. Similarly, PC1 vertical channels can be numbered 2, and PC1 horizontal channels can be numbered 3. Therefore, packets can be routed from the PC1 vertical channel to the PC1 horizontal channel. According to the routing rules, packets cannot be routed from the PC1 horizontal channel to the PC1 vertical channel.

[0061] Additional complexities can arise when routing out of an interval (e.g., to a die in another system, to a field-programmable gate array (FPGA), etc.) or when routing around a dead die. For example, when routing out of an interval, the system lacks control over the switching topology that connects the intervals to each other. Preferably, loops through the same network switch can be avoided. Therefore, in some embodiments, traffic traveling to an external device (e.g., an FPGA) can be kept on one set of channels, while traffic traveling from an external device can be kept on another set of channels. In some embodiments, the channel used to send traffic to an external device can have a lower channel number than the channel used to receive traffic from an external device.

[0062] For example, Figure 3B depicts an example of routing to an out-of-interval destination according to some embodiments. According to Figure 3B, the system can route packets to out-of-interval devices using channels 0 and 1. The system can then route packets to the destination node using channels 2 and 3. In this configuration, the switch that delivers traffic to the out-of-interval device is considered intermediate channel 1.5. As depicted in Figure 3B, channels 1 and 2 can only support vertical traffic. Therefore, this configuration will only support out-of-interval devices at the top or bottom edge of the interval. It will be appreciated that other configurations are possible that enable routing to out-of-interval destinations at the side edges of the interval. Additional channels can be used to enable routing to or from out-of-interval devices at all four edges of the interval.

[0063] Dead dies present a similar problem. Dead dies create situations where routing from one die to another involves multiple turns. Figure 4 illustrates a die array with two dead dies. In Figure 4, moving packets from source die S to destination die D involves routing around the two dead dies (shown in black). Unless multiple turns are allowed, it is impossible to reach destination die D from source die S without traversing the dead dies. The four-channel configuration depicted in Figure 3A allows for more than one turn, but problems and failures can still occur when there are multiple dead dies to route around.

[0064] While providing additional channels can mitigate potential problems, doing so can be costly because additional channels involve additional hardware and may consume additional area. In some embodiments, different node rows / columns can be treated as independent channels, thereby increasing the number of available channels. Although this may reduce the maximum bandwidth of each channel, the impact can be mitigated because the bandwidth at the die edge may limit the total bandwidth.

[0065] A grid or stripe pattern can be used to define additional channels. For example, nodes in every nth row or column can be grouped into a set of virtual channels. As just one example, channel 0 (the horizontal route of the PC0 network) above can be divided into 4 channels, and each 4th row can be placed into each channel. Thus, H-h0 would consist of rows 0, 4, 8, 12, etc., PC0-h1 would consist of rows 1, 5, 9, 13, etc., PC0-h2 would consist of rows 2, 6, 10, 14, etc., and PC0-h3 would consist of rows 3, 7, 11, 15, etc. Vertical channels can be partitioned similarly, such that, for example, PC1-v0 includes columns 0, 4, 8, 12, etc., and so on.

[0066] Figure 5 illustrates an example of routing along multiple channels according to some embodiments. As shown in Figure 5, packets may take multiple turns to reach the destination node from the source node. If a channel (e.g., PC0-h2, PC1-v3, etc.) terminates at the edge of an interval, the route may terminate at an FPGA or at a different interval connected via a Time Triggered Protocol (TTP) or a switch.

[0067] While this method can alleviate some problems, the number of virtual and horizontal channels that can be used to move packets from source to destination is limited. However, a potential issue is that when multiple dead cells exist, allowing routing to include many turns, routing may start from a lower-numbered channel because rules may not permit turning from a higher-numbered channel to a lower-numbered channel. However, in some cases, the source node may not exist in the lower-numbered row or column. Therefore, the system can be configured to route packets to a lower-numbered row or column before leaving the die.

[0068] Escaped virtual channels (E-channels) can be used to route packets to lower-numbered channels. Escaped channels may not leave the die, and therefore, while they may increase the complexity of management tokens and tracking channels, they may not contribute to the number of channels supported by the serializer / deserializer (SerDes) controller. Advantageously, although the E network shares physical infrastructure with the PC0 and PC1 networks, E-channels can help maintain the separation of local die traffic from global traffic, avoiding the problem of local routing being affected by global traffic, which can cause congestion at die boundaries.

[0069] When a packet is routed out of the die, it can be routed in the outgoing channel until it reaches the grout at the edge of the die. For example, if a packet in PC1-v6 is to be routed north along PC1-v2, the system can be configured to route north in E-v0 until it reaches the grout, and the packet can be marked as if it came from PC1-v2. Therefore, when the packet leaves the die, it will be correctly routed as a PC1-v2 packet. Alternatively, in some cases, the packet may be routed along E-h0 and then turn onto PC1-v2. Those skilled in the art will recognize that these are merely examples, and in some embodiments, the system may route packets in a different manner consistent with this disclosure.

[0070] In some cases, packets can terminate at the FPGA. For example, if the upper portion of the routing interval terminates at the FPGA, the system can be configured to route packets arriving at the grout to the correct SerDes channel, ensuring the packets reach the correct target FPGA. In some embodiments, a single die can be connected to more than one FPGA. For example, along the north edge, each die can be connected to two FPGAs. In such a configuration, the two FPGAs will have the same column address but can be assigned to different rows. Other configurations are possible, such as multiple FPGAs sharing a single column and / or a single row.

[0071] Sometimes, intra-die routing may compete with inter-die routing. In some embodiments, to alleviate intra-die packet congestion, certain channels (e.g., high-numbered channels) can be reserved for use only by the destination die. Therefore, only local packets will operate in those channels, thus avoiding congestion caused by packets routed across die boundaries.

[0072] Figures 6A-6C depict example die arrays with multiple dead dies. These figures illustrate various scenarios of routing around dead dies. In the examples of Figures 6A-6C, routing around dead dies can be completed in only two turns. Because fewer turns are achieved, this routing can be accomplished with high bandwidth. As more dead dies are introduced, routing can become more complex, for example, as depicted in Figures 7A-7B. Therefore, in the dead dies array of Figures 7A-7B, routing around dead dies can reduce the available bandwidth. Depending on the number and distribution of dies, routing around dead dies may involve many turns, and the routing may have relatively low bandwidth, for example, in the die array with dead dies depicted in Figure 8. In Figures 6A-8, dead dies are shown in black. Grouting topology

[0073] Grouting nodes are connected to the node array, the SerDes controller, and in some cases, to adjacent (e.g., left and right) grouting nodes. In some embodiments, packets can be routed from the node array to the SerDes controller, or from the SerDes controller to the node array, but can enter and leave the grouting on the node array or on the SerDes controller. In addition to routing packets, grouting provides packet classification functionality. In some embodiments, grouting can be divided into multiple channels, such as three channels traveling from left to right and three channels traveling from right to left, for both node-to-array transport and array-to-node transport. Each grouting channel can include multiple virtual channels. For example, each grouting channel can include PC0 and PC1 virtual channels, which themselves can be divided into request Hi, request Lo, and data channels. Furthermore, in some embodiments, there may be channels that are directly routed to the SerDes controller of the local node.

[0074] When packets arrive at the grout, for example on the array side, rules can determine which virtual channels the packets can be routed to. Furthermore, packets can undergo classification functions, such as managing packet classification for routing to different FPGAs that may be connected after different SerDes controllers. In some embodiments, for a given physical or virtual channel, there may be exactly one channel, and packets can be explicitly classified to their destination channel. However, in other cases, the grout can use a threshold mechanism to propagate traffic across channels enabled for a given packet.

[0075] In some embodiments, the desired SerDes channel may be busy, and packets may be routed to different channels in the same row or column, such as adjacent channels. For example, a row or column may be divided into multiple channels, such as 2 channels, 3 channels, 4 channels, 5 channels, 6 channels, 7 channels, 8 channels, and so on. In some embodiments, for example, if one channel is not working, packets may be routed to different channels.

[0076] As packets are routed from one grouting node to another, their movement can be tracked, and at each node, packets can be allowed to continue straight, turn, etc. In some embodiments, if a packet can continue straight and turn towards the SerDes controller, it can do so based on the determination of whether there is space in the channel exit buffer. For example, in some cases, the channel may be busy and / or there may be hardware defects preventing the channel from operating. In some embodiments, grouting can control which SerDes channels can receive packets from which channels / packet types per channel and / or per packet type. Additional Examples

[0077] The foregoing disclosure is not intended to limit this disclosure to the precise form or particular field of use disclosed. Accordingly, it is contemplated that various alternative embodiments and / or modifications to this disclosure, whether expressly described or implied herein, are possible. Having thus described embodiments of this disclosure, those skilled in the art will recognize that changes in form and detail may be made without departing from the scope of this disclosure. Therefore, this disclosure is limited only to the scope of the claims.

[0078] In the foregoing specification, this disclosure has been described with reference to specific embodiments. However, as those skilled in the art will appreciate, various embodiments disclosed herein may be modified or implemented in various other ways without departing from the spirit and scope of this disclosure. Therefore, this description is intended to be illustrative and for the purpose of teaching those skilled in the art how to manufacture and use various embodiments of the disclosed IC assemblies. It should be understood that the forms of disclosure shown and described herein are to be considered representative embodiments. Equivalent elements, materials, processes, or steps may be substituted for those representatively illustrated and described herein. Furthermore, certain features of this disclosure may be utilized independently of the use of other features, all of which will be apparent to those skilled in the art who benefit from the description of this disclosure. Expressions such as “comprising,” “including,” “incorporated,” “consisting of,” “having,” and “is” used to describe and claim the contents of this disclosure are intended to be interpreted in a non-exclusive manner, allowing for the presence of items, components, or elements not explicitly described. References to the singular should also be interpreted as relating to the plural.

[0079] Furthermore, the various embodiments disclosed herein should be considered illustrative and explanatory, and in no way should they be construed as limiting the scope of this disclosure. All connecting references (e.g., attachment, adhesion, coupling, connection, etc.) are provided only to aid the reader in understanding this disclosure and are unlikely to impose limitations, particularly regarding the location, orientation, or use of the systems and / or methods disclosed herein. Therefore, connecting references should be interpreted broadly, if any. Moreover, such connecting references do not necessarily imply that two elements are directly connected to each other.

[0080] Furthermore, all numerical terms, such as but not limited to “first,” “second,” “third,” “primary,” “secondary,” “primary,” or any other common and / or numerical terms, should be regarded only as identifiers to help the reader understand the various elements, embodiments, variations, and / or modifications of this disclosure, and may not impose any limitations, particularly regarding the order or preference of any element, embodiment, variation, and / or modification relative to or over another element, embodiment, variation, and / or modification.

[0081] It will also be appreciated that one or more elements depicted in the figures / graphs may also be implemented in a more discrete or integrated manner, or even removed or rendered inoperable in some cases, which may be useful depending on the specific application. Furthermore, unless specifically specified otherwise, any signal shadows in the figures / graphs should be considered exemplary only, and not limiting.

[0082] 101: Core 102: Core 103: Core 201: Core 202: Core 203: Core

Claims

1. A method for bypassing failed grains, the method comprising: The system on the wafer includes routing packets from a source die to an intermediate die via a first route, the first route including redirecting packets from a first channel of a plurality of first channels to a second channel of a plurality of second channels based on one or more routing rules, the one or more routing rules allowing the first channel to route packets to a subset of a plurality of second channels including the second channel, and the first channel being orthogonal to the second channel; and routing packets from an intermediate die to a destination die via a second route, the second route including redirecting packets from the second channel to a third channel orthogonal to the second channel, wherein the system on the wafer includes a die array including a source die, an intermediate die, a destination die and at least one failed die, and wherein the first route and the second route bypass the at least one failed die route.

2. The method of claim 1, wherein the at least one failed die comprises two failed dies, and the first route and the second route bypass the two failed die routes.

3. A method for bypassing failed grains, the method comprising: The method routes packets from a first die to a second die via a first channel of a plurality of first channels, wherein the first die, second die, third die, and failed die are included in an array; and routes packets from a second die to a third die via a second channel of a plurality of second channels based on one or more routing rules, wherein the one or more routing rules allow the first channel to route packets to a subset of a plurality of second channels including the second channel, and the second channels are orthogonal to the first channels, such that routing packets from the first die to the third die includes a turn, wherein the method routes packets around the failed die.

4. The method of claim 3, wherein routing packets from the first die to the second die and routing packets from the second die to the third die includes: The routing table is queried at least in part based on the address of the third die, wherein the routing table conforms to one or more routing rules.

5. As in request item 4, wherein one or more routing rules prevent packets from being routed in a loop.

6. The method of request item 5, wherein each of the plurality of first channels and each of the plurality of second channels is assigned a priority, and wherein the one or more routing rules do not allow routing from a lower priority channel of the plurality of first channels to a higher priority channel of the plurality of second channels.

7. The method of request item 6, wherein the route from the first die to the third die is configured to terminate at the lowest priority first channel among a plurality of first channels or the lowest priority second channel among a plurality of second channels.

8. The method of request item 6, wherein one or more routing rules allow packets to be routed to an escape channel, the escape channel being configured to allow packets to move to a channel with higher priority.

9. The method of claim 6, wherein the routing from the first die to the third die includes a second turn from the second channel to the third channel, wherein the third channel has a lower priority than the second channel, and wherein the second channel has a lower priority than the first channel.

10. The method of request item 4, wherein the routing table includes a preset route, which is used if no defined route exists from the first die to the second die.

11. The method of request item 3, wherein the method revolves around at least two failed die routing packets.

12. The method of request item 3, wherein the method includes using a multi-redirection route packet.

13. The method of claim 3, wherein the system on the wafer includes an array.

14. The method of claim 3 further includes routing the packet from the third die to a die outside the array.

15. A processing system with a failed grain bypass route, the processing system comprising: A grain array comprising a first grain, a second grain, a third grain, and a failed grain; The processing system is configured to route packets from a first die to a third die via a second die based on one or more routing rules implemented by the circuitry of the processing system, thereby bypassing a faulty die. The packets are routed via at least a first channel and a second channel, and the one or more routing rules allow a first channel of a plurality of first channels to route packets to a subset of a plurality of second channels orthogonal to the plurality of first channels.

16. The processing system of claim 15, wherein the processing system is configured to route packets from a first die to a third die via multiple sub-routes.

17. The processing system of claim 15, wherein the die array includes a second die, and the processing system is configured to bypass a second failed die when routing packets from a first die to a third die.

18. The processing system of request item 15, wherein the one or more routing rules prevent packets from being routed in a loop.

19. The processing system of claim 15, wherein the processing system includes a routing table storing information associated with one or more routing rules.

20. The processing system of request item 19, wherein each of a plurality of first channels and each of a plurality of second channels is assigned a priority, and wherein said one or more routing rules do not allow routing from a lower priority channel of the plurality of first channels to a higher priority channel of the plurality of second channels.

21. The processing system of claim 15, wherein the processing system is configured to generate neural network training data.

Citation Information

Patent Citations

  • Three dimensional memory structure

    US20130265067A1

  • Fault-tolerant power-driven synthesis

    US20160132769A1

  • Yield tolerance in a neurosynaptic system

    US20160323137A1

  • Multiple dies hardware processors and methods

    US20180101502A1

  • Automated design closure with abutted hierarchy

    US20210165856A1