Techniques for robust communication

Abstract

Device comprising: a data path for storing data to be sent via a connection (350) to a receiving node (480); an output stage (335, 355, 580) which is connected between the data path and the link (350), wherein the output stage (335, 355, 580) has double sampling mechanisms to retain a copy of data sent via the link (350) to the receiving node (480); Error detection circuits (100, 200) connected to the output stage (335, 355, 580) to detect transient timing errors in the data path or in the output stage (335, 355, 580), wherein the error detection circuits (100, 200) cause the output stage (335, 355, 580) to resend the copy of the data sent over the connection (350) in response to the detection of an error.

Description

Technical field

[0001] Embodiments of the invention relate to techniques for communication between connected nodes. In particular, embodiments of the invention relate to robust communication techniques that can be used to achieve reliable, efficient message communication. background

[0002] Many interconnect architectures, for example in a network-on-a-chip (NoC) or system-on-a-chip (SoC), rely on routers to manage message traffic between nodes (e.g., processor cores, memory). These routers consume power during operation and can contribute significantly to the overall power consumption of an SoC. One strategy for reducing power consumption is to lower the operating voltages. However, at low voltages, routers are susceptible to dynamic changes such as voltage dips or aging effects, which can potentially lead to timing errors in the router. Operational robustness is typically ensured by using a static voltage protection band, which is selected at the time of design. Using a static voltage protection band requires a higher operating voltage and increases power consumption.

[0003] US 5,959,995 A discloses a multiprocessor system comprising a multitude of nodes and an intermediate link containing routers. Each node contains a reliable packet exchanger and a fast frame exchanger. The reliable packet exchanger delivers packets to the fast frame exchanger, which adds routing information to the packet to form a frame. The route to each node is predetermined. The frame is then forwarded to the routers, which remove the route from the routing information. If the frame is lost during forwarding, the router discards it. When the packet is received at a destination node, the reliable packet exchanger at that node sends an acknowledgment to the source node if the packet has passed an error detection test. The reliable packet exchanger at the source node retransmits the packet if it does not receive an acknowledgment within a specified time.The Fast Frame Mover randomly selects the route from a multitude of predefined routes to the target node according to a probability distribution.

[0004] US 2003 / 0110422 A1 discloses a system and a method for error detection and correction during the transmission of multiple flits between sending and receiving agents connected in a network or computer interconnection environment. The method comprises embedding a sequence identifier in each flit prior to transmission, sending each flit to a connected receiving agent, examining the sequence identifiers of each received flit, and requesting the sending agent to retransmit a flit if the sequence identifier for that flit is found to be incorrect. In a preferred embodiment of this document, the sequence identifier is embedded in the control part of the flit and includes a sequence number that is incremented or otherwise modified in a predictable manner so that the order of the received flits can be predicted.If the sequence number for a flit is different than expected, the receiver requests that it be resent.

[0005] US 2010 / 0046364 A1 discloses a system comprising a first device and a second device connected via a transmission path. The first device includes a transmitting unit that sends a request as at least one packet and retransmits at least one request packet if the time interval between the request and a response is not less than a predetermined time interval; and a receiving unit that receives the response as at least one response packet and discards any error packet and any redundant packet from the received at least one response packet.The second device comprises: a receiving unit that receives the at least one request packet and discards each error packet from the at least one received request packet; and a transmitting unit that determines a response type, makes the at least one response packet selectively redundant for each response of a certain type, and transmits the redundant response packets.

[0006] US 2011 / 0219280 A1 discloses a system and method for enabling global collective high-speed, low-latency communication between interconnected processing nodes. The global collective network optimally facilitates the execution of collective reduction operations during parallel algorithm operations performed in a computer architecture with a multitude of interconnected processing nodes. Router devices are included that connect the network nodes via links to facilitate the execution of low-latency global processing operations on the nodes of the virtual network and the class structures. The global collective network can be configured to provide global locking and interrupting functions in an asynchronous or synchronous manner.When implemented in a massively parallel supercomputing structure, the global collective network can be partitioned physically and logically according to the requirements of a processing algorithm.

[0007] The object of the invention is to provide energy-efficient, robust, and reliable message transmission between nodes. This object is achieved by the independent claims. Advantageous embodiments are described in the dependent claims. Brief description of the drawings

[0008] Embodiments of the invention are illustrated in the figures of the accompanying drawings by means of examples and are not limited to the use of the same reference numbers, which refer to similar elements. Fig. Figure 1 is a block diagram of an embodiment of a fault detection circuit that can be used to support robust communication techniques. Fig. Figure 2 is a block diagram of an embodiment of a double-sampling fault detection circuit that can be used to support robust communication techniques without incurring significant additional design effort. Fig. Figure 3 is a block diagram of a router with robustness to dynamic changes, designed to send data to a router / traffic generator (TG). Fig. Figure 4 is a block diagram of a receiving router (or other device) that uses the bit error signal to validate received data. Fig. Figure 5 is a block diagram of an embodiment of a source node with reset mechanisms to support timing robustness. Fig. Figure 6 is a block diagram of an embodiment of an electronic system. Description of the embodiments

[0009] Various router designs exist that can be used in different configurations, some of which can provide fault detection and correction. In one embodiment, input-buffered wormhole-switched router architectures (suitable, for example, for a NoC) can be designed to operate as: 1) a single-cycle latency router, 2) a two-cycle latency router with no robustness, or 3) a two-cycle latency router with robustness to dynamic changes. These operating modes are described in more detail below.

[0010] In one embodiment, dynamic changes manifested by timing errors in the router are detected in robust two-cycle mode using an error detection sequential mechanism (EDS), which can be a hardware circuit. In another embodiment, the robust router architecture incorporates EDS in a processing stage that can protect timing paths within the router. Correction of messages subject to timing errors in the router can be achieved by using packet retransmission techniques.

[0011] In one embodiment, the EDS approach provides native protection against events caused by soft errors (SERs) through double sampling. Additionally, the disclosed approach can protect against SER-induced combinatorial delay pushouts and sequential state loss.

[0012] In one embodiment, a router (or component) includes a robustness-enhanced output stage that operates to protect timing paths within the router. For example, a router can employ EDS mechanisms to protect all timing paths within the router originating from an output queue (e.g., a FIFO). In another embodiment, packet (message or flit) errors resulting from timing errors in the router (or another component) are corrected using packet replay techniques.

[0013] Since the timing error is detected at the output stage after a message has been transmitted to the receiving node (e.g., router, traffic generator), an error signal (e.g., a bit, flag) is sent with the message to indicate whether a timing error has occurred. Because the error signal can be metastable, it is buffered at the input stage of the receiving node before further processing. The error signal acts as an invalidation signal, causing the receiving node to discard (or otherwise not process or use) the corresponding message.

[0014] The sending node (for example, the router) where the timing error occurred retransmits the failed message by resetting its state by the necessary number of clock cycles (for example, up to a test point). In one embodiment, two clock cycles are sufficient. In alternative embodiments, a greater reset distance can be supported. In one embodiment, this can be achieved by isolating the control and data paths of the sending node and retaining copies of critical data and control state elements (for example, via a flip-flop or latch) in the sending node.

[0015] In one embodiment, an output queue (for example, a FIFO or another type of output queue) has sufficient unused memory during normal operation that messages from previous cycles are still present and not overwritten before being used for repetition, if necessary. In another embodiment, the FIFO depth can be increased to accommodate additional message space for repetition. For example, in one embodiment, a timing error is detected, and the message is retransmitted within two clock cycles. The sending node is reset to a previous state, and the erroneous message is sent again to the receiving node.

[0016] In one embodiment, the use of positive-phase latches in the EDS mechanism can make the output stage susceptible to hold-time errors. This can be avoided by selectively buffering minimum-delay timing paths to EDS-enhanced stages and routing a pulsed clock to the output stage, as described in more detail below. In one embodiment, the output stage receives a clock signal with a configurable pulse width, and other stages receive a clock signal with a 50% duty cycle. In another embodiment, a truncated standard cell library with enlarged sequential and combinational logic states can be used to reduce the impact of in-chip and intra-chip fluctuations at very low operating voltages (for example, near a threshold voltage).In addition to retrying, current source / destination ports can be used as indicators for other ports and, through error logging logic, to trigger DVFS programs.

[0017] Fig. Figure 1 is a block diagram of an embodiment of a fault detection circuit that can be used to support robust communication techniques. The circuit of the Fig. 1 can be used to generate an error signal that can be used to retransmit messages (packets, flits) in which a timing error has occurred. In one embodiment, the error detection circuit 100 is arranged in a final stage of a transmitting node on the path to a connection to a receiving node.

[0018] The fault detection circuit 100 receives a data bit from a stage in the transmitting node. The data bit is fed to the high-phase latch 110 and the flip-flop 120. The high-phase latch 110 temporarily stores the data bit when a clock signal (see Fig. 3), which is fed to the output stage, has a high level, and flip-flop 120 stores the data bit on the transition from the low to the high level of a pulsed clock signal (see Fig. 3).

[0019] If a timing error occurs, the output signals of high-phase latch 110 and flip-flop 120 will be different, causing gate 130 to set the bit error signal. If no timing error occurs, the output signals of high-phase latch 110 and flip-flop 120 will be the same, and gate 130 will not set a bit error signal.

[0020] Fig. Figure 2 is a block diagram of an embodiment of a double-sampling fault detection circuit that can be used to support robust communication techniques. The circuit of the Fig. 2 can be used to generate an error signal that can be used to retransmit messages (packets, flits) in which a timing error has occurred. In one embodiment, the error detection circuit 200 is located in a final stage of a transmitting node on the path to a connection to a receiving node and provides double sampling, which allows packets and other information (for example, status information) to be retained for two clock cycles in order to recover from a timing error. In alternative embodiments, more than two samples can be retained; for example, triple-sampling error detection can be provided.

[0021] In one embodiment, a fault detection circuit can operate in one of the following three modes: 1) Mode 0, in which both the master latch 230 and the slave latch 250 are open; 2) Mode 1, in which the master latch 230 and the slave latch 250 operate as a single flip-flop; and 3) Mode 2, in which the master latch 230 is open and double sampling is performed by the slave latch 250 and the flip-flop 210. The flip-flop 210, the master latch 230, and the slave latch 250 are each controlled by a mode signal and a clock signal, which are operated by the logic gates 215, 220, and 240, respectively.

[0022] In mode 0, the error detection circuit 200 is effectively transparent to the surrounding circuitry. In mode 1, the error detection circuit 200 operates as a flip-flop for the data bit. In mode 2, the error detection circuit 200 operates as a positive-phase latch with double sampling and a flip-flop for timing error detection.

[0023] The data bit to be sent is the input signal of flip-flop 210 and master latch 230. The output signal of flip-flop 210 provides an input for logic gate 275, which generates the bit error signal. The output signal of master latch 230 is the input signal of slave latch 250. The output signal of slave latch 250 provides a second input for logic gate 275 and is also the data bit to be sent.

[0024] If a timing error occurs, the output signals of flip-flop 210 and slave latch 250 will differ, causing gate 275 to set the bit error signal. If no timing error occurs, the signals from flip-flop 210 and slave latch 250 will be the same, and gate 275 will not set a bit error signal.

[0025] In one embodiment, an error detection circuit is provided for each data bit to be transmitted. The bit error signals corresponding to the multiple data bits can be combined to determine whether an error exists for any of the data bits in the message to be transmitted. In another embodiment, the message is retransmitted as described here if an error occurs for any data bit.

[0026] Fig. Figure 3 is a block diagram of a router robust to dynamic changes, designed to send data to a router / traffic generator (TG). The example of the Fig. 3 can consist of two routers within a network-on-a-chip (NoC) architecture; however, the robustness concepts are also applicable to other configurations.

[0027] A sender router 310 stores data that is to be sent to the receiving router 360 via connection 350, which can be of any connection type. The sender router 310 receives data from a data source (in Fig. (3 not shown) and temporarily stores the data in output queue 320. Output queue 320 can be, for example, a first-in / first-out queue (FIFO queue) or any type of structure for collecting data to be sent.

[0028] Data from the output queue 320 is transferred to the crossbar (X-bar) 330 via the intermediate stage 325. In one embodiment, the output queue 320 operates with a negative-phase latch, and the intermediate stage 325 operates with a positive-phase latch; however, other latch configurations are also supported. In one embodiment, the output queue 320 and the intermediate stage 325 operate using a clock signal with a 50% duty cycle; however, other duty cycles can also be used.

[0029] The crossbar 330 routes data from input ports to output ports to accomplish the routing functionality of the transmitter router 310. The crossbar 330 can be of any type of crossbar mechanism. The crossbar 330 routes data to be transmitted via the link 350 to an output stage 355. In one embodiment, the output stage 335 operates with a pulsed clock signal having a duty cycle of less than 50%, or less than the duty cycle of the clock signal used by the output queue 320 and the intermediate stage 325.

[0030] The output stage 335 includes fault detection circuits, for example those in the Fig. 1 or Fig. The error detection circuits shown in Figure 2 are described. The output stage 335 causes the data from the output queue 320, along with the bit error signal, to be sent via the connections 350 to the receiving router 360. The input queue 370 in the receiving router 360 receives the data and the bit error signal. In one embodiment, the input queue 370 stores the data and the bit error signal, which can be used as described below.

[0031] Fig. Figure 4 is a block diagram of a receiving router (or similar device) that uses the bit error signal to validate the received signal. The receiving node 480 can be, for example, a router, a memory or processing core, or any other type of logic core, such as an intellectual property core, or other processing circuitry. The message 405 is sent to the receiving node 480 along with one or more error signals 410, which, as described above, can be, for example, one or more bits long.

[0032] The receiving node 480 receives messages 405 and corresponding error signals 410 and stores them in an input buffer 420. The input buffer 420 stores multiple messages together with the corresponding error signals. In one embodiment, the input buffer acts as a first-in / first-out (FIFO) queue; however, other structures are also supported. A read pointer controller 430 controls the position of a read pointer for the input buffer 420, which is used to read messages from the input buffer 420 at the correct time.

[0033] In one embodiment, the input stage 440 reads a message and a corresponding error signal from the input buffer 420, as specified by the read pointer. The input stage 440 uses the error signal from the source node to validate the output signal in the receiving node 480. The output data can be validated in the same way as the data in the source node. In one embodiment, the read pointer is advanced only when an error-free message is read from the input buffer 420. This allows the message to be repeated until an error-free transmission occurs without the need to reset the state of the receiving node 480.

[0034] Fig. Figure 5 is a block diagram of an embodiment of a source node with reset mechanisms to support timing robustness. The error signal can be used to control a reset within the source node to resend the erroneous message to the receiving node. As explained above, the output stage generates an error signal associated with the transmitted data, and one or more stages of the transmitting node can store transmitted data, for example, in a FIFO queue.

[0035] An output queue 510 receives data from any data source (in Fig. (5 not shown) and temporarily stores the data. The output queue 510 can be, for example, a FIFO queue or any type of structure for collecting data to be sent. Data from the output queue 510 is sent to the intermediate stage 520. In one embodiment, the output queue 510 operates with a negative-phase latch and the intermediate stage 520 operates with a positive-phase latch; however, other latch configurations can also be supported. In one embodiment, the output queue 510 and the intermediate stage 520 operate using a clock signal that has a duty cycle of less than 50%; however, other duty cycles can also be used.

[0036] A read pointer controller 512 manages a read pointer that is used to read data from the output queue 510. The read pointer controller 512 changes the position of the read pointer while data is being read from the output queue 510. The read pointer controller 512 has associated state information (current read pointer state 514) that is used to manage the read pointer. In one embodiment, information about previous pointer positions (previous read pointer state 516) is also stored for reset purposes. In the example embodiment, where two messages are retained for robustness, the two previous sets of state information can be retained. In alternative embodiments, more or less state information can be retained.

[0037] Several multiplexers 560 can be used to route data from the intermediate stage 520 and / or data from other sources to the output stage 580. Associated with the multiplexers 560 are intermediaries 562, which control the operation of the multiplexers 560 to provide the desired switching functionality. The intermediaries 562 have associated status information (current intermediary states 564) that is used to manage the operation of the intermediary 562. In one embodiment, status information from previous intermediary states (previous intermediary states 566) is also stored for reset purposes. In the example embodiment, where two messages are retained for robustness, the two previous sets of status information can be retained. In alternative embodiments, more or less status information can be retained.

[0038] As described above, the output stage 580 generates an error signal. This error signal can be buffered by the error latch 590, for example, a flip-flop or other device. The error signal can then be supplied to cause the read pointer controller 512 and the mediators 562 to use previous status information to revert to a previous state and retransmit a message that was sent with an error. This technique allows data to be retransmitted without proceeding with the transmission of subsequent data until error-free transmission occurs, thus eliminating any sequencing problems that might otherwise arise. Furthermore, recovery can occur in 1-3 cycles, providing an efficient and nearly transparent error recovery mechanism.

[0039] Fig. Figure 6 is a block diagram of an embodiment of an electronic system. The diagram shown in Fig. The illustrated electronic system is intended to represent a range of electronic systems (either wired or wireless), such as desktop computer systems, laptop computer systems, mobile phones, personal digital assistants (PDAs), including cellular-enabled PDAs, and set-top boxes. Alternative electronic systems may include more, fewer, and / or different components.

[0040] An electronic system 600 includes a bus 605 or other communication devices for communicating information, and a processor 610 that can process information and is connected to the bus 605. Although the electronic system 600 is illustrated with a single processor, it can include multiple processors and / or coprocessors. The electronic system 600 can also include random-access memory (RAM) or other dynamic storage device 620 (referred to as main memory) that is connected to the bus 605 and can store information and instructions that can be executed by the processor 610. The main memory 620 can also be used to store temporary variables or other intermediate information while instructions are being executed by the processor 610.

[0041] The electronic system 600 may also include a read-only memory (ROM) and / or another static storage device 630 connected to the bus 605, which stores further static information and instructions for the processor 610. A data storage device 640 may be connected to the bus 605 to store information and instructions. The data storage device 640, such as a magnetic or optical data carrier and a corresponding drive, may be connected to the electronic system 600.

[0042] The electronic system 600 can also be connected via bus 605 to a display device 650, such as a cathode ray tube (CRT) or liquid crystal display (LCD), to show information to a user. An alphanumeric input device 660, including alphanumeric and other keys, can be connected to bus 605 to communicate information and command selections to the processor 610. Another type of user input device is the cursor control 670, such as a mouse, trackball, or cursor direction keys, to communicate directional information and command selections to the processor 610 and to control cursor movements on the display 650.

[0043] The electronic system 600 may further include one or more network interface(s) 680 to provide access to a network such as a local area network. The network interface(s) 680 may, for example, include a wireless network interface with an antenna 685, which may represent one or more antennas. The network interface(s) 680 may also, for example, include a wired network interface to communicate with remote devices via a network cable 687, which may be, for example, an Ethernet cable, a coaxial cable, a fiber optic cable, a serial cable, or a parallel cable.

[0044] In one embodiment, the network interface(s) 680 can provide access to a local network, for example by complying with the IEEE 802.11b and / or IEEE 802.11g standards, and / or the wireless network interface can provide access to a personal network, for example by complying with the Bluetooth standards. Other wireless network interfaces and / or protocols may also be supported.

[0045] IEEE 802.11b conforms to IEEE Standard 802.11b-1999, entitled "Local and Metropolitan Area Networks, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications: Higher-Speed ​​Physical Layer Extension in the 2.4 GHz Band," adopted on September 16, 1999, and related documents. IEEE 802.11g conforms to IEEE Standard 802.11g-2003, entitled "Local and Metropolitan Area Networks, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, Amendment 4: Further Higher Rate Extension in the 2.4 GHz Band," adopted on June 27, 2003, and related documents. Bluetooth protocols are described in "Specification of the Bluetooth System: Core, Version 1.1," published on February 22, 2001, by the Bluetooth Special Interest Group, Inc. Associated and earlier or later versions of the Bluetooth standard can also be supported.

[0046] In addition to or instead of communication using wireless LAN standards, the network interface(s) 680 can provide wireless communication using, for example, time division multiplex (TDMA) protocols, GSM protocols, code division multiplex (CDMA) protocols and / or any other type of wireless communication protocol.

Claims

[1] Device comprising: a data path for storing data to be sent via a connection (350) to a receiving node (480); an output stage (335, 355, 580) which is connected between the data path and the link (350), wherein the output stage (335, 355, 580) has double sampling mechanisms to retain a copy of data sent via the link (350) to the receiving node (480); Error detection circuits (100, 200) connected to the output stage (335, 355, 580) to detect transient timing errors in the data path or in the output stage (335, 355, 580), wherein the error detection circuits (100, 200) cause the output stage (335, 355, 580) to resend the copy of the data sent over the connection (350) in response to the detection of an error. [2] Device according to claim 1, wherein the data path, output stage (335, 355, 580) and fault detection circuits (100, 200) are part of a router in a network-on-a-chip (NoC) architecture. [3] Device according to claim 1, wherein the data path comprises at least one first-in / first-out (FIFO) queue connected to a crossbar circuit, the crossbar circuit being connected to the output stage (335, 355, 580). [4] Device according to claim 1, wherein the data path comprises at least one crossbar circuit connected to a first-in / first-out (FIFO) queue, the FIFO queue being connected to the output stage (335, 355, 580). [5] Device according to claim 3, wherein the fault detection circuits (100, 200) detect transient timing errors in the FIFO queue and the crossbar circuit. [6] Device according to claim 3, wherein a read pointer in the FIFO queue is reset in response to the detection of the transient timing error by the fault detection circuits (100, 200). [7] Device according to claim 3, wherein, in response to the detection of the transient timing error by the fault detection circuits (100, 200), states of one or more intermediaries in the data path are reset. [8] Device according to claim 1, wherein the output stage (335, 355, 580) together with the data sent via the connection (350) sends an error indicator to invalidate the outgoing erroneous messages. [9] Device according to claim 8, wherein the receiving node (480) does not process the data transmitted via the connection (350) in response to the reception of the error indicator. [10] Device according to claim 1, wherein the receiving node (480) comprises a router. [11] Procedure, encompassing: Generating an error signal corresponding to a data signal, wherein the error signal is to be sent with the data signal, wherein the error signal is generated by error detection circuits (100, 200) connected to an output stage (335, 355, 580) to detect transient timing errors; Causing the output stage (335, 355, 580) to send a copy of the data transmitted over the connection (350) in response to the detection of an error; Double scanning to keep a copy of data sent over connection (350). [12] Method according to claim 11, wherein the data path, output stage (335, 355, 580) and fault detection circuits (100, 200) are part of a router in a network-on-a-chip (NoC) architecture. [13] Method according to claim 11, wherein the data path comprises at least one first-in / first-out (FIFO) queue connected to a crossbar circuit, the crossbar circuit being connected to the output stage (335, 355, 580). [14] Method according to claim 11, wherein the data path comprises at least one crossbar circuit connected to a first-in / first-out (FIFO) queue, the FIFO queue being connected to the output stage (335, 355, 580). [15] Method according to claim 13, wherein the fault detection circuits (100, 200) detect transient timing errors in the FIFO queue and the crossbar circuit. [16] Method according to claim 13, wherein a read pointer in the FIFO queue is reset in response to the detection of the transient timing error by the fault detection circuits (100, 200). [17] Method according to claim 13, wherein, in response to the detection of the transient timing error by the fault detection circuits (100, 200), states of one or more intermediaries in the data path are reset. [18] Method according to claim 11, wherein the output stage (335, 355, 580) together with the data sent via the connection (350) sends an error indicator to invalidate the outgoing erroneous messages. [19] Method according to claim 18, wherein the receiving node (480) does not process the data transmitted via the connection (350) in response to the reception of the error indicator. [20] Method according to claim 11, wherein the receiving node (480) comprises a router. [21] Device comprising: multiple intellectual property (IP) cores; and a connection coupling field connected to the multiple IP cores, wherein the connection coupling field contains routing logic to route flits between at least the IP cores, wherein the routing logic includes repetition logic to to check one or more flits and one or more control state elements for a number of previous cycles prior to a current cycle, to reset a state associated with the guiding logic in response to a fault associated with a flit of one or more flits, based on checking one or more flits and one or more control state elements for a number of previous cycles prior to a current cycle; and to repeat the Flit in response to the state being reset. [22] Device according to claim 21, wherein the multiple IP cores and the interconnection network are contained in an integrated circuit with a system-on-a-chip (SOC) and wherein the SOC is connected to a controller to receive inputs from a touch-sensitive display device. [23] Device according to claim 21, wherein the number of preceding cycles comprises two. [24] Device according to claim 21, wherein the number of preceding cycles comprises three.

Images