Fast recovery of dual-core lockstep

Abstract

An exemplary fault-tolerant computing system includes a secondary processor configured to execute with a primary processor from a common program store in a delay lock-step manner, a comparator in a stored data and write-back path to detect a fault based on comparing a primary processor state and a secondary processor state, and allowing a write-back path delay that aborts execution when a fault is detected prior to write-back of invalid data. Auxiliary processor execution and main processor storage data and write back may be delayed by a predetermined number of cycles, allowing fault detection prior to writing invalid data. The storage data and write-back path may include a triple module redundancy configured to delay a level of most data only through the storage data and write-back path. Some embodiments may forward data from the storage data path delay stage to a write-back stage or memory if the load data address matches the address of data in the storage data path delay stage.

Description

Cross Reference to Related Applications

[0001] None. TECHNICAL FIELD

[0002] The present disclosure relates generally to fault-tolerant computing. BACKGROUND

[0003] Fault-tolerant computing is processing implemented by a computer in the presence of a fault. The fault can be caused by a hardware error. For example, a faulty hardware component, such as a defective memory device, can cause a processor to read invalid data. This invalid data read can cause the system state to be corrupted. For example, the processor that read the invalid data can then generate invalid data that, when written to memory, can corrupt the system. In some cases, environmental factors, such as cosmic rays, can corrupt data and cause processing faults. Some computing systems operate in a mission-critical role. When a fault is detected, it can be necessary to stop a computing system operating in a safety-critical application to prevent catastrophic results. Resuming safe and correct system operation after a fault is detected can be time-consuming or difficult. SUMMARY

[0004] An example fault-tolerant computing system includes a secondary processor configured to execute in a delayed lockstep manner with a primary processor from a common program storage, a comparator in a store data and writeback path to detect a fault based on comparing a primary processor state and a secondary processor state, and a writeback path delay to allow execution to be aborted when a fault is detected prior to writeback of invalid data. The secondary processor execution and the primary processor store data and writeback can be delayed for a predetermined number of cycles, allowing for fault detection prior to writing invalid data. The store data and writeback path can include triple modular redundancy configured to pass majority data through a store data and writeback path delay stage. Some embodiments can forward data from a store data path delay stage to a writeback stage or memory if a load data address matches an address of data in the store data path delay stage.

[0005] Example implementations in accordance with the present disclosure can include a primary processor and a secondary processor. The primary processor and the secondary processor can be pipelined processors. The primary processor pipeline and the secondary processor pipeline can include a respective plurality of sequential pipeline stages. The primary processor pipeline and the secondary processor pipeline can include, in order, a fetch stage, a decode stage, an execute stage, a load / store stage, and a writeback stage. The primary processor pipeline load / store stage and the secondary processor pipeline load / store stage can be operably coupled to a respective data memory. The primary processor pipeline writeback stage and the secondary processor pipeline writeback stage can be operably coupled to a respective writeback cache. The primary processor pipeline store data path and the secondary processor pipeline store data path include a delay module connected between the execute stage and the load / store stage of the respective pipeline. The primary processor pipeline store data path delay can be a three-stage delay. The secondary processor pipeline store data path delay can be a one-stage delay, allowing the secondary processor to execute from a common program memory area with the primary processor, two cycles delayed relative to the primary processor. A store data comparator receives delayed input signals and data from the primary processor pipeline and the secondary processor pipeline. The store data comparator is configured to output a result comparing the delayed input signals and data from the primary processor and the secondary processor. The output of the store data path comparator indicates whether the primary processor state and the secondary processor state are consistent. When the primary processor state and the secondary processor state are not consistent, the output of the store data path comparator indicates that a fault exists. The store data path comparator output is connected to a control gate configured to manage memory write operations through the load / store stage, and to block memory writes when a fault exists. Delayed lockstep execution between the processor and the store data path delay allows system state corruption to be prevented based on blocking memory write operations while invalid data is still in the store data path delay stage before the invalid data is written to memory.

[0006] In another example implementation according to the present disclosure, the main processor pipeline writeback path and the secondary processor pipeline writeback path include a delay module connected between a load / store stage and a writeback stage of the respective pipeline. The main processor pipeline writeback path delay can be a three-stage delay. The secondary processor pipeline writeback path delay can be a one-stage delay, allowing the secondary processor to execute from a common program memory area with the main processor, delayed by two cycles relative to the main processor. A writeback comparator receives delayed input signals and data from the main processor pipeline and the secondary processor pipeline. The writeback comparator is configured to output a result of comparing the delayed input signals and data from the main processor and the secondary processor. The output of the writeback path comparator indicates whether the main processor state and the secondary processor state are consistent. When the main processor state and the secondary processor state are not consistent, the output of the writeback path comparator indicates that there is a fault. The writeback path comparator output is connected to a control gate configured to manage writeback operations through the writeback stage and to block writebacks when there is a fault. The delay lockstep execution between the processors, the store data path delay, and the writeback delay allow prevention of system state corruption based on blocking memory writes and writeback operations while invalid data is still in the store data path and writeback path delay stages before the invalid data is written to memory or the writeback cache.

[0007] In another example implementation according to the present disclosure, the delay module configured in the main processor pipeline writeback data path can include a plurality of buffer stages configured with triple module redundancy designed to pass majority data to the next stage only.

[0008] In another example implementation according to the present disclosure, the main processor pipeline store data path and the secondary processor pipeline store data path can be configured to forward data from the store data path delay stage to the writeback stage or memory if the load data address matches the address of the data in the store data path delay stage.

[0009] Various embodiments can achieve one or more technical effects. For example, some embodiments can improve fault detection reliability. This facilitation can be a result of fault detection implemented in the store data stage and the writeback stage, increasing the likelihood of fault detection with multiple comparisons. In some embodiments, fault recovery can be faster based on resuming execution from a known clean point that the program has likely passed through recently. This faster fault recovery can be a result of preventing system state corruption before invalid data is written to memory, a register file, or a writeback cache, allowing recovery of execution at the most recent clean point determined based on known good data in the delay stage. Embodiments in accordance with the present disclosure can ensure data coherency between two processor cores before writeback to a register file or data storage to memory, allowing reduced recovery time when a fault is detected. In an illustrative example, when a mismatch between a primary core pipeline and a secondary core pipeline is detected prior to writeback of a store data comparator or writeback as disclosed herein, fast recovery from the fault can be performed as follows: the pipelines are flushed, and the pipelines begin fetching instructions from the instruction memory at the point where the mismatch occurred. Since the pipelines are flushed prior to writeback, the detected invalid data is not written to the register file, protecting the register file from corruption. As a result of preventing register file corruption, the instructions required for fast recovery can be in the cache with very high probability, and the cycle loss to recover normal execution using fault-tolerant fast recovery techniques in accordance with the present disclosure can be less than 10 cycles.

[0010] In an aspect, an apparatus can include a primary processor including a primary processor pipeline, wherein the primary processor pipeline includes an execution stage and a load / store stage; a secondary processor including a secondary processor pipeline, wherein the secondary processor pipeline includes an execution stage and a load / store stage; a store data comparator having a first input operably coupled with the primary processor pipeline, a second input operably coupled with the secondary processor pipeline, and an output, wherein the store data comparator is configured to indicate a result of a comparison of a delayed primary processor pipeline signal and a delayed secondary processor pipeline signal at the output; a primary processor pipeline store data control gate configured to output the delayed primary processor pipeline signal to the primary processor pipeline load / store stage only when the output of the store data comparator indicates a coherence between the delayed primary processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator; and a secondary processor pipeline store data control gate configured to output the delayed secondary processor pipeline signal to the secondary processor pipeline load / store stage only when the output of the store data comparator indicates a coherence between the delayed primary processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator.

[0011] The main processor pipeline execution stage may have an output terminal that is operatively connected to the first input terminal of the main processor storage data control gate via a main processor pipeline storage data path delay module.

[0012] The main processor pipeline storage data path delay module can be a three-level buffer delay module.

[0013] The main processor pipeline storage data path delay module can be a three-level triple-module redundant buffer delay module, where each level can be configured to pass only the majority of data.

[0014] The main processor pipeline may also include a write-back stage, wherein the main processor pipeline can be configured to forward data from the main processor pipeline storage data path delay module to the write-back stage if the address of the loaded data matches the address of the data in the main processor pipeline storage data path delay module.

[0015] The auxiliary processor pipeline execution stage may have an output terminal that is operatively connected to the first input terminal of the auxiliary processor storage data control gate via an auxiliary processor pipeline storage data path delay module.

[0016] The auxiliary processor pipeline storage data path delay module can be a first-level buffer delay module.

[0017] The delay of the main processor pipeline signal compared by the storage data comparator can be connected to the first input of the storage data comparator through the main processor storage data comparison delay module, which is operatively coupled to the main processor pipeline execution stage.

[0018] The main processor's data comparison delay module can be a three-level buffer delay module.

[0019] On the other hand, an apparatus may include: a main processor including a main processor pipeline, wherein the main processor pipeline includes an extract stage, a decode stage, an execution stage, a load / store stage, and a write-back stage operably connected in sequence; an auxiliary processor including an auxiliary processor pipeline, wherein the auxiliary processor pipeline includes an extract stage, a decode stage, an execution stage, a load / store stage, and a write-back stage operably connected in sequence; and a storage data comparator having a first input, a second input, and an output, the first input being operably coupled to the output of the main processor pipeline execution stage to receive a delayed main processor pipeline signal via a main processor storage data comparison delay module including a three-stage buffer delay, the second input being operably coupled to the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor storage data comparison delay module including a one-stage buffer delay, wherein the storage data comparator is configured at its output to indicate the comparison of the delayed main processor pipeline signal with the delayed auxiliary processor pipeline signal. The result of comparing line signals; a main processor pipeline storage data control gate having a first input, a second input, and an output, wherein the first input is operatively coupled to the output of the main processor pipeline execution stage to receive a delayed main processor pipeline signal through a main processor storage data path delay module including a three-stage buffer delay, wherein the second input is operatively coupled to the output of a storage data comparator, and the main processor pipeline storage data control gate is configured to output the delayed main processor pipeline storage data path signal to the main processor pipeline load / store stage only if the output of the storage data comparator indicates consistency between the delayed main processor pipeline signal compared by the storage data comparator and the delayed auxiliary processor pipeline signal, and wherein the main processor pipeline is configured to forward data from the main processor pipeline storage data path delay module to the write-back stage or memory if the load data address matches the address of data in the main processor pipeline storage data path delay module;An auxiliary processor pipeline storage data control gate has a first input, a second input, and an output. The first input is operatively coupled to the output of the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor storage data path delay module including a first-level buffer delay. The second input is operatively coupled to the output of a storage data comparator. The auxiliary processor pipeline storage data control gate is configured to output the delayed auxiliary processor pipeline storage data path signal to the auxiliary processor pipeline load / store stage only if the output of the storage data comparator indicates a match between the delayed main processor pipeline signal compared by the storage data comparator and the delayed auxiliary processor pipeline signal. The auxiliary processor pipeline is configured to forward data from the auxiliary processor pipeline storage data path delay module to a write-back stage or memory if the load data address matches the address of data in the auxiliary processor pipeline storage data path delay module. A write-back comparator has an output operatively coupled to the output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal via a third-level buffer delay. The write-back comparison delay module receives a first input to a delayed main processor pipeline signal, a second input operatively coupled to the auxiliary processor pipeline load / memory stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor write-back comparison delay module including a first-level buffer delay, and an output, wherein the write-back comparator is configured to indicate at the output the result of comparing the delayed main processor pipeline signal with the delayed auxiliary processor pipeline signal; a main processor pipeline write-back control gate has a first input, a second input, and an output, wherein the first... An input is operatively coupled to the output of the main processor pipeline load / memory stage to receive a delayed main processor pipeline signal via a main processor write-back path delay module including a three-stage buffer delay, and wherein a second input is operatively coupled to the output of a write-back comparator, and a main processor pipeline write-back control gate is configured to output the delayed main processor pipeline write-back signal to the main processor pipeline write-back stage only if the output of the write-back comparator indicates consistency between the delayed main processor pipeline signal compared by the write-back comparator and the delayed auxiliary processor pipeline signal;And an auxiliary processor pipeline write-back control gate having a first input, a second input, and an output, wherein the first input is operatively coupled to the output of the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor write-back path delay module including a first-level buffer delay, and wherein the second input is operatively coupled to the output of a write-back comparator, and the auxiliary processor pipeline write-back control gate is configured to output the delayed auxiliary processor pipeline write-back signal to the auxiliary processor pipeline write-back stage only if the output of the write-back comparator indicates consistency between the delayed main processor pipeline signal compared by the write-back comparator and the delayed auxiliary processor pipeline signal.

[0020] The main processor pipeline extraction stage and the auxiliary processor pipeline extraction stage can be operatively coupled to a program memory area shared by the main processor and the auxiliary processor.

[0021] The auxiliary processor pipeline can be executed in a delayed, lockstep manner with the main processor pipeline, wherein the execution of the auxiliary processor pipeline is delayed by at least two cycles relative to the execution of the main processor pipeline.

[0022] The delayed main processor pipeline signals and delayed auxiliary processor pipeline signals compared by the storage data comparator may also include handshake signals and data between the data cache and the core, or handshake signals between the data memory and the core.

[0023] The delayed main processor pipeline signals and delayed auxiliary processor pipeline signals compared by the write-back comparator may also include register files, data caches, data memory, and handshake signals and data between cores.

[0024] The main processor pipeline storage data path delay module may also include a three-level triple-module redundant buffer delay module, wherein each level can be configured to transmit only the majority of data.

[0025] The main processor pipeline write-back path delay module may also include a three-level triple-module redundant buffer delay module, wherein each level can be configured to transmit only the majority of data.

[0026] On the other hand, an apparatus may include: a main processor including a main processor pipeline, wherein the main processor pipeline includes an extract stage, a decode stage, an execution stage, a load / store stage, and a write-back stage operably connected in sequence; an auxiliary processor including an auxiliary processor pipeline, wherein the auxiliary processor pipeline includes an extract stage, a decode stage, an execution stage, a load / store stage, and a write-back stage operably connected in sequence, wherein the main processor pipeline extract stage and the auxiliary processor pipeline extract stage are operably coupled to a program memory area shared by the main processor and the auxiliary processor; and a storage data comparator having a first input, a second input, and At the output, a first input is operatively coupled to the output of the main processor pipeline execution stage to receive a delayed main processor pipeline signal via a main processor store-data comparison delay module including a three-stage buffer delay; a second input is operatively coupled to the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor store-data comparison delay module including a one-stage buffer delay, wherein the store-data comparator is configured to indicate at its output the result of comparing the delayed main processor pipeline signal with the delayed auxiliary processor pipeline signal, wherein the delay compared by the store-data comparator... The main processor pipeline signals and delayed auxiliary processor pipeline signals also include handshake signals and data between the data cache and the core, and handshake signals between the data memory and the core; the main processor pipeline storage data control gate has a first input, a second input, and an output, wherein the first input is operatively coupled to the output of the main processor pipeline execution stage to receive delayed main processor pipeline signals through a main processor pipeline storage data path delay module including a three-stage triple-module redundant buffer delay, wherein each stage is configured to pass only a majority of data, and wherein the second input is coupled to a storage data comparator. The output is operatively coupled, and the main processor pipeline storage data control gate is configured to output the delayed main processor pipeline storage data path signal to the main processor pipeline load / store stage only if the output of the storage data comparator indicates the consistency between the delayed main processor pipeline signal compared by the storage data comparator and the delayed auxiliary processor pipeline signal, and wherein the main processor pipeline is configured to forward data from the main processor pipeline storage data path delay module to the write-back stage if the load data address matches the address of data in the buffer stage of the main processor pipeline storage data path delay module;An auxiliary processor pipeline storage data control gate has a first input, a second input, and an output. The first input is operatively coupled to the output of the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor storage data path delay module including a first-level buffer delay. The second input is operatively coupled to the output of a storage data comparator. The auxiliary processor pipeline storage data control gate is configured to only release the delay signal if the output of the storage data comparator indicates consistency between the delayed main processor pipeline signal compared by the storage data comparator and the delayed auxiliary processor pipeline signal. A delayed auxiliary processor pipeline storage data path signal is output to the auxiliary processor pipeline load / store stage, wherein the auxiliary processor pipeline is configured to forward data from the auxiliary processor pipeline storage data path delay module to the write-back stage if the load data address matches the address of data in the auxiliary processor pipeline storage data path delay module; a write-back comparator has a first input operatively coupled to the output of the main processor pipeline load / store stage to receive delayed main processor pipeline signals via a main processor write-back comparison delay module including a three-stage buffer delay, and is operatively coupled to the auxiliary processor pipeline load / store stage. The system includes a second input terminal and an output terminal for receiving delayed auxiliary processor pipeline signals via an auxiliary processor write-back comparison delay module that includes a first-level buffer delay. The write-back comparator is configured to indicate at its output the result of comparing the delayed main processor pipeline signal with the delayed auxiliary processor pipeline signal. The delayed main processor pipeline signal and delayed auxiliary processor pipeline signal compared by the write-back comparator also include handshake signals and data between register files, data caches, data memory, and cores. A main processor pipeline write-back control gate has a first input terminal, a second input terminal, and an output terminal, wherein the first... The input is operatively coupled to the output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal via a main processor write-back path delay module including a three-stage triple-module redundant buffer delay, wherein each stage is configured to pass only the majority of data, and wherein a second input is operatively coupled to the output of a write-back comparator, and the main processor pipeline write-back control gate is configured to output the delayed main processor pipeline write-back signal to the main processor pipeline write-back stage only if the output of the write-back comparator indicates consistency between the delayed main processor pipeline signal compared by the write-back comparator and the delayed auxiliary processor pipeline signal;And an auxiliary processor pipeline write-back control gate having a first input, a second input, and an output, wherein the first input is operatively coupled to the output of the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal via an auxiliary processor write-back path delay module including a first-level buffer delay, and wherein the second input is operatively coupled to the output of a write-back comparator, and the auxiliary processor pipeline write-back control gate is configured to output the delayed auxiliary processor pipeline write-back signal to the auxiliary processor pipeline write-back stage only if the output of the write-back comparator indicates consistency between the delayed main processor pipeline signal compared by the write-back comparator and the delayed auxiliary processor pipeline signal.

[0027] The program storage area may also include processor-executable instructions configured to resume execution from a known clean program point in response to a fault indication determined based on an inconsistency between main processor pipeline signals and auxiliary processor pipeline signals.

[0028] One or more of the main processor or auxiliary processors may also include a superscalar processor.

[0029] The device may also include a C200 processor.

[0030] Details of each aspect are set forth in the accompanying drawings and the description below. Other features and advantages will become apparent from the description, the drawings, and the claims. Brief description of the attached diagram

[0031] Figure 1 A block diagram of an exemplary fault-tolerant computing system is depicted, which has delayed lockstep execution, storage data path latency, and write-back path latency between a main processor and an auxiliary processor, and is configured to prevent system corruption when a fault is detected based on a comparison of the main processor state and the auxiliary processor state.

[0032] Figure 2 A block diagram depicts an exemplary design for comparing the states of the main processor and the auxiliary processor.

[0033] Figure 3 A block diagram of an exemplary processor pipeline is depicted.

[0034] Figure 4 A block diagram depicts a portion of a processor pipeline configured with exemplary storage data path and write-back path latency modules.

[0035] Figure 5 A block diagram depicts an exemplary write-back path delay module design.

[0036] Figure 6A block diagram depicts an exemplary storage data path delay module design.

[0037] Figure 7 An exemplary fault-tolerant computing system according to this disclosure is described, which has delayed lockstep execution, storage data path latency, and write-back path latency between a main processor and an auxiliary processor, and is configured to prevent system corruption when a fault is detected based on a comparison of the main processor state and the auxiliary processor state.

[0038] Similar reference symbols in the various figures indicate similar elements. Detailed description

[0039] To aid understanding, this document is organized as follows. First, a layered overall description, different from that in the accompanying figures, is given. Second, references... Figure 1 This paper briefly introduces a fault-tolerant computing system design for rapid recovery using latency-based lockstep execution between the main processor and auxiliary processor, storage data path latency, and write-back path latency. It is configured to prevent system corruption when a fault is detected based on a comparison of the main processor's state and the auxiliary processor's state. Third, refer to... Figures 2-4 The discussion then shifts to demonstrating exemplary implementations of a fast-recovery fault-tolerant computing design. Specifically, fault detection and fast recovery applications using comparisons of main and auxiliary processor states, delayed storage data paths, and delayed write-back paths are presented. Then, refer to... Figures 5-6 Exemplary write-back path latency and storage data path latency designs are disclosed. Finally, refer to... Figure 7 An exemplary implementation of a fault-tolerant computing system is disclosed to demonstrate improvements in fault-tolerant computing techniques.

[0040] In this disclosure, various features may be described as optional, for example, by using the verb "may"; or by using any of the following phrases: "in some implementations," "in some designs," "in various implementations," "in various designs," "in an illustrative example," or "for example." For brevity and readability, this disclosure does not explicitly describe every permutation that can be obtained by selecting from the set of optional features. However, this disclosure should be interpreted as explicitly disclosing all such permutations. For example, a system described as having three optional features can be implemented in seven different ways: utilizing only one of the three possible features, utilizing any two of the three possible features, or utilizing all three of the three possible features. In particular, it should be noted that individual implementation features, even those disclosed only in combination with other implementation features, can be combined in any configuration, except for those configurations that are obviously meaningless to those skilled in the art.

[0041] This disclosure teaches a fault-tolerant computing system. The fault-tolerant computing system may include at least one processor. The at least one processor may include a pipelined processor. The at least one pipelined processor may include multiple pipeline stages. Each of the multiple pipeline stages can be configured to perform an operation. Each of the multiple pipeline stages can be configured to have at least one input. Each of the multiple pipeline stages can be configured to have at least one output. Each of the multiple pipeline stages can be configured to perform an operation in one clock cycle, wherein the result of the operation is available at the output. One or more of the multiple pipeline stages may be buffer stages configured to pass inputs to outputs in one clock cycle. The at least one pipelined processor may include multiple pipelines. The at least one pipeline may include an fetch stage. The at least one pipeline may include a decode stage. The at least one pipeline may include an execution (EX) / address generation (AG) stage. The at least one pipeline may include a stored data stage. The stored data stage may be a memory access stage. The at least one pipeline may include a write-back stage. The write-back stage may be operatively coupled to a write-back cache. The at least one processor may be a superscalar processor. A superscalar processor may include multiple processor cores. Each processor core may be implemented using a superscalar architecture. A superscalar processor may be a RISCY processor as specified in https: / / riscv.org / technical / specifications / . A superscalar processor may be a C200 processor or a later model, as described in https: / / www.intel.com / content / dam / www / public / us / en / documents / specification-updates / 6-and-c200-chipset-specification-update.pdf.

[0042] At least one processor may include a main processor. At least one processor may include an auxiliary processor. A fault-tolerant computing system may include a main processor and an auxiliary processor. At least one processor may be implemented as at least one processor core. The main processor may be implemented as a main core. The auxiliary processor may be implemented as an auxiliary core. A fault-tolerant computing system may be implemented as a dual-core computing system. At least one processor may be operatively coupled to memory. The memory may be an instruction memory. The instruction memory may be a program memory. The memory may be a data memory. The memory may include both instruction memory and data memory. At least one processor may be configured to execute processor-executable program instructions from the instruction memory. At least one processor may be configured to perform operations on data in the data memory. At least one processor core may be operatively coupled to at least one cache. At least one cache may be a write-back cache. At least one processor may be configured with local data memory. At least one processor may be configured with a data cache. At least one processor may be configured with both local data memory and a data cache. Some fault-tolerant computing designs according to this disclosure may be implemented using processors without caches. The implementation of this disclosure using a processor without a cache can be configured to apply the fault detection and fast recovery techniques disclosed herein to a processor having only data memory and no cache. The memory may include processor-executable program instructions and data for jointly programming and configuring at least one processor to perform operations. Operations performed by at least one processor may include resuming execution from a known clean program point. At least one processor may resume execution from a known clean point in response to a fault indication.

[0043] The main processor and auxiliary processor can be configured to execute program instructions from the same memory shared by both. The auxiliary processor can be configured to execute program instructions from memory in a delayed, lockstep manner with the main processor. One or more processors can be configured with one or more delay modules. One or more delay modules can include one or more buffer stages. One or more delay modules can include a series of buffer stages, each configured to pass input to output in one clock cycle. A buffer stage can delay the input-to-output transition by one clock cycle.

[0044] Fault-tolerant computing systems may include a memory data comparator. The memory data comparator may have a first input connected to the main processor pipeline via a delay module to receive delayed main processor pipeline signals, a second input connected to the auxiliary processor pipeline via a delay module to receive delayed auxiliary processor pipeline signals, and an output. The delay module configured to delay the main processor pipeline signals input to the memory data comparator may be a three-stage delay buffer connected to the main core execution / address generation stage. The delay module configured to delay the auxiliary processor pipeline signals input to the memory data comparator may be a single-stage delay buffer connected to the auxiliary core execution / address generation stage.

[0045] The main processor pipeline's storage data path can be configured with delay modules connecting the main pipeline execution / address generation stage to the main pipeline storage data / memory access stage. The delay modules configured in the main processor pipeline's storage data path can be three-level delay buffers. The delay modules configured in the main processor pipeline's storage data path can include multiple buffer stages configured with triple module redundancy. If the load data address matches the address of data in the storage data path delay stage, the main processor pipeline's storage data path can be configured to forward the data from the storage data path delay stage to the write-back stage or memory. The auxiliary processor pipeline's storage data path can be configured with delay modules connecting the auxiliary pipeline execution / address generation stage to the auxiliary pipeline storage data / memory access stage. If the load data address matches the address of data in the storage data path delay stage, the auxiliary processor pipeline's storage data path can be configured to forward the data from the storage data path delay stage to the write-back stage or memory. The delay modules configured in the auxiliary processor pipeline's storage data path can be single-level delay buffers.

[0046] A memory data comparator can be configured to instruct on its output to compare the results of main processor pipeline signals and auxiliary processor pipeline signals from the corresponding memory data path. If the output of the memory data comparator indicates an inconsistency between the main processor pipeline signals and auxiliary processor pipeline signals from the corresponding memory data path, a fault can be indicated. The main processor pipeline signals and auxiliary processor pipeline signals compared by the memory data comparator can include pipeline internal signals or boundary signals. The processor pipeline signals compared by the memory data comparator can include handshake signals and data between the data cache and the core, and handshake signals between the data memory and the core. The handshake signals and data between the data cache and the core compared by the memory data comparator can include a request signal, a data valid signal, data, a data address, and a cache hit. The handshake signals between the data memory and the core compared by the memory data comparator can include a request signal, data valid, data, and a data address.

[0047] The output of the storage data path comparator can be connected to control gates in the main processor pipeline's storage data path and in the auxiliary processor pipeline's storage data path. The control gates in the main processor pipeline's storage data path can be configured to manage main processor pipeline write operations to the main processor pipeline's storage data / memory access stage, as determined by the output indication of the storage data path comparator. The output of the storage data path comparator managing the main processor pipeline write operations can indicate a fault or no fault. The control gates in the main processor pipeline's storage data path can be configured to allow the main processor pipeline to write data to the main pipeline's storage data / memory access stage only if the output indication of the storage data path comparator is no fault. The control gates in the auxiliary processor pipeline's storage data path can be configured to manage auxiliary processor pipeline write operations to the auxiliary processor pipeline's storage data / memory access stage, as determined by the output indication of the storage data path comparator. The output of the storage data path comparator managing the auxiliary processor pipeline write operations can indicate a fault or no fault. The control gates in the auxiliary processor pipeline's storage data path can be configured to allow the auxiliary processor pipeline to write data to the auxiliary pipeline storage data / memory access level only if the output indication of the storage data path comparator is fault-free.

[0048] Fault-tolerant computing systems may include write-back comparators. A write-back comparator may have a first input connected to the main processor pipeline via a delay module to receive delayed main processor pipeline signals, a second input connected to the auxiliary processor pipeline via a delay module to receive delayed auxiliary processor pipeline signals, and an output. The delay module configured to delay the main processor pipeline signals input to the write-back comparator may be a three-stage delay buffer connected to the main core's data / memory access stage. The delay module configured to delay the auxiliary processor pipeline signals input to the write-back comparator may be a single-stage delay buffer connected to the auxiliary core's data / memory access stage.

[0049] The main processor pipeline write-back data path can be configured with delay modules connecting the main pipeline storage data / memory access stage to the main processor pipeline write-back stage. The delay modules configured in the main processor pipeline write-back data path can be three-stage delay buffers. The auxiliary processor pipeline write-back data path can be configured with delay modules connecting the auxiliary pipeline storage data / memory access stage to the auxiliary processor pipeline write-back stage. The delay modules configured in the main processor pipeline write-back data path can include one or more buffer stages configured with triple module redundancy. The delay modules configured in the auxiliary processor pipeline write-back data path can be single-stage delay buffers.

[0050] A write-back comparator can be configured to instruct on its output to compare the results of primary processor pipeline signals and secondary processor pipeline signals from the corresponding write-back data path. A fault can be indicated if the write-back comparator's output indicates an inconsistency between the primary and secondary processor pipeline signals from the corresponding write-back data path. The processor pipeline signals compared by the write-back comparator can include pipeline internal signals or boundary signals. The processor pipeline signals compared by the write-back comparator can include handshake signals and data between the register file, data cache, data memory, and core. The handshake signals and data between the register file, data cache, data memory, and core compared by the write-back comparator can include data valid, register file ID, data, control register ID, data for the control register, and valid for the control register. The register file can have one or more write ports. The number of data handshake signals and data inputs between the register file, data cache, data memory, and core compared by the write-back comparator can be extended in multiples of the corresponding number of register file write ports. When the register file has two write ports, the data handshake signals and data between the register file, data cache, data memory, and core compared by the write-back comparator may include two data valid signals, two register file IDs, two data inputs, a control register ID, data for the control register, and data for the control register valid. In some implementations, the register file may include two write ports, which can be used when the ALU path produces double-precision data, or when there is an ALU operation and a load data operation; however, in this case, the cache and data memory writes perform one data operation per cycle.

[0051] The output of the write-back comparator can be connected to control gates in the main processor pipeline write-back path and in the auxiliary processor pipeline write-back path. The control gates in the main processor pipeline write-back path can be configured to manage main processor pipeline write-back operations to the main processor pipeline cache based on the output indication of the write-back path comparator. The output of the write-back path comparator managing the main processor pipeline write-back operation can indicate a fault or no fault. The control gates in the main processor pipeline write-back path can be configured to allow the main processor pipeline to write data back to the main pipeline cache only if the output of the write-back path comparator indicates no fault. The control gates in the auxiliary processor pipeline write-back path can be configured to manage auxiliary processor pipeline write-back operations to the auxiliary processor pipeline cache based on the output indication of the write-back path comparator. The output of the write-back path comparator managing the auxiliary processor pipeline write-back operation can indicate a fault or no fault. The control gates in the write-back path of the auxiliary processor pipeline can be configured to allow the auxiliary processor pipeline to write data back to the auxiliary pipeline cache only if the output indication of the write-back path comparator is fault-free.

[0052] Figure 1 A block diagram of an exemplary fault-tolerant computing system is depicted, which features delayed lockstep execution between a main processor and an auxiliary processor, storage data path latency, and write-back path latency, configured to prevent system corruption when a fault is detected based on a comparison of the main processor state and the auxiliary processor state. Figure 1In this system 100, a main core pipeline 105 and a secondary core pipeline 110 are included. In the depicted embodiment, the main core pipeline 105 includes a main core fetch stage 115A, a main core decode stage 120A, a main core execution (EX) / address generation (AG) stage 125A, a main core load / store stage 130A, and a main core write-back stage 135A, operably connected in sequence. In the depicted embodiment, the secondary core pipeline 110 includes a secondary core fetch stage 115B, a secondary core decode stage 120B, a secondary core execution (EX) / address generation (AG) stage 125B, a secondary core load / store stage 130B, and a secondary core write-back stage 135B, operably connected in sequence. In the depicted embodiment, the main core pipeline 105 and the secondary core pipeline 110 execute from a common program memory in a delayed lockstep manner. In the illustrative example, when data requested by the main core pipeline 105 becomes available, the same data will be provided to the auxiliary core pipeline 110 after a two-cycle delay. System 100 can be configured such that the main core pipeline 105 and the auxiliary core pipeline 110 have separate caches that remain synchronized with a lockstep delay between the main core pipeline 105 and the auxiliary core pipeline 110. For example, system 100 can be configured such that all input / output (I / O) and interrupt signals pass through a single point accessed only by the main core pipeline 105. In the illustrative example, when there is I / O, exception, or interrupt activity in the main core pipeline 105, the auxiliary core pipeline 110 will experience the same I / O, exception, or interrupt activity after the delay between the processor pipelines. The delay length between the processor pipelines is a design choice and can be any number of cycles.

[0053] In the depicted embodiment, the storage data comparator 140 receives signals from the main core pipeline 105 delayed by the main core storage data comparison delay module 145A, and signals from the auxiliary core pipeline 110 delayed by the auxiliary core storage data comparison delay module 145B. The storage data comparator 140 compares the states of the main core pipeline 105 and the auxiliary core pipeline 110 by comparing the delayed signals of the main core pipeline 105 and the delayed signals of the auxiliary core pipeline 110. The storage data operations in the main core pipeline 105 and the auxiliary core pipeline 110 are controlled based on the result of comparing the delayed signals of the main core pipeline 105 and the delayed signals of the auxiliary core pipeline 110. In the depicted embodiment, the storage data path of the main core pipeline 105 is delayed by the storage data path delay module 150A. In the depicted embodiment, the storage data path of the auxiliary core pipeline 110 is delayed by the auxiliary core storage data path delay module 150B.

[0054] In the depicted embodiment, write-back comparator 155 receives signals from the main core pipeline 105 delayed by the main core write-back comparison delay module 160A, and signals from the auxiliary core pipeline 110 delayed by the auxiliary core write-back comparison delay module 160B. Write-back comparator 155 compares the states of the main core pipeline 105 and the auxiliary core pipeline 110 by comparing the delayed signals of the main core pipeline 105 and the delayed signals of the auxiliary core pipeline 110. The write-back operation in the main core pipeline 105 and the auxiliary core pipeline 110 is controlled by the result of comparing the delayed signals of the main core pipeline 105 and the delayed signals of the auxiliary core pipeline 110. In the depicted embodiment, the write-back path of the main core pipeline 105 is delayed by the main core write-back path delay module 165A. In the depicted example, the write-back path of the auxiliary core pipeline 110 is delayed by the auxiliary core write-back path delay module 165B. When a fault is identified based on an inconsistency between the states of the main core pipeline 105 and the auxiliary core pipeline 110, if invalid data is present in the storage data path or write-back delay module, data storage or write-back operations can be prevented, thereby preventing system state corruption and maintaining the integrity of system data in register files, caches, or memory. As a result of preserving valid system data, system 100 can be restarted more quickly from the nearest known clean point.

[0055] In the depicted implementation, the main core fetch stage 115A and the auxiliary core fetch stage 115B fetch instructions from the instruction memory shared by the main core pipeline 105 and the auxiliary core pipeline 110. The main core decode stage 120A and the auxiliary core decode stage 120B perform instruction decoding, including parsing operand register sources, determining the operation type, and identifying the operation destination. The main core execution (EX) / address generation (AG) stage 125A and the auxiliary core execution (EX) / address generation (AG) stage 125B perform operations based on operands and operand types from the respective decode stages. If the operation type is identified as a load / store instruction, the main core execution (EX) / address generation (AG) stage 125A and the auxiliary core execution (EX) / address generation (AG) stage 125B also calculate memory addresses. The main core load / store stage 130A and the auxiliary core load / store stage 130B access data memory. The main core write-back stage 135A and the auxiliary core write-back stage 135B store loaded data from memory or store data from the pipeline to a register file.

[0056] Figure 2 A block diagram depicts an exemplary design for comparing the states of the main processor and the auxiliary processor. Figure 2In this configuration, the main core pipeline 105 and the auxiliary core pipeline 110 are configured to execute from a common program memory area at intervals of several cycles. The latency between the main core pipeline 105 and the auxiliary core pipeline 110 can be managed by the latency configured in the corresponding memory data paths and write-back paths of the main core pipeline 105 and the auxiliary core pipeline 110. In this illustrative example, the execution latency between processor pipelines can be set during system initialization using techniques known in the field of processor system design, such as holding a processor pipeline reset for a predetermined number of cycles.

[0057] In the depicted embodiment, the main core storage data comparison delay module 145A includes a first stage 145A1, a second stage 145A2, and a third stage 145A3 of the main core storage data comparison delay module, operably connected in sequence. In the depicted embodiment, the main core storage data comparison delay module 145A receives a main core storage data comparison signal 205A. In the depicted embodiment, the auxiliary core storage data comparison delay module 145B receives an auxiliary core storage data comparison signal 205B. The main core storage data comparison signal 205A and the auxiliary core storage data comparison signal 205B may include pipeline internal key signals or key boundary signals, such as handshake signals and data between the data cache and the core, and handshake signals between the data memory and the core. The handshake signals and data between the data cache and the core may include a request signal, a data valid signal, data, a data address, and a cache hit result. The handshake signals between the data memory and the core compared by the storage data comparator may include, for example, a request signal, data valid, data, or a data address. The main core storage data comparison delay module 145A outputs a main core storage data comparison signal 210A delayed by three cycles relative to the input delay. The auxiliary core storage data comparison delay module 145B outputs a delayed auxiliary core storage data comparison signal 210B delayed by one cycle relative to the input delay. The storage data comparator 140 compares the delayed main core storage data comparison signal 210A and the delayed auxiliary core storage data comparison signal 210B. The storage data comparator 140 is configured to indicate the obtained storage data comparison decision 215 on its output.

[0058] In the depicted embodiment, the main core write-back comparison delay module 160A includes a first-stage main core write-back comparison delay module 160A1, a second-stage main core write-back comparison delay module 160A2, and a third-stage main core write-back comparison delay module 160A3, which are operably connected in sequence. In the depicted embodiment, the main core write-back comparison delay module 160A receives a main core write-back comparison signal 220A. In the depicted embodiment, the auxiliary core write-back comparison delay module 160B receives an auxiliary core write-back comparison signal 220B. The main core write-back comparison signal 220A and the auxiliary core write-back comparison signal 220B may include pipeline internal key signals or key boundary signals, such as handshake signals and data between the register file, data cache, data memory, and the core. The handshake signals and data between the register file, data cache, data memory, and the core may include data valid, register file ID, data, control register ID, data for the control register, and data valid for the control register. The main core write-back comparison delay module 160A outputs a delayed main core write-back comparison signal 225A, which is delayed by three cycles relative to the input. The auxiliary core write-back comparison delay module 160B outputs a delayed auxiliary core write-back comparison signal 225B, which is delayed by one cycle relative to the input. The write-back comparator 155 compares the delayed main core write-back comparison signal 225A and the delayed auxiliary core write-back comparison signal 225B. The write-back comparator 155 is configured to indicate the obtained write-back comparison decision 230 on its output.

[0059] Figure 3 A block diagram depicting an exemplary processor pipeline is provided. Figure 3In the depicted embodiment, core pipeline 105 detail 300 illustrates an exemplary core pipeline 105 of an exemplary superscalar processor according to various embodiments of the present disclosure. In the depicted embodiment, core pipeline 105 is configured with an ALU pipeline, a load / store pipeline, and a floating-point pipeline to execute instructions, and can fetch up to two 32-bit instructions per cycle. Instructions can be stored in an instruction buffer in one or more fetch blocks. In the depicted embodiment, core fetch stage 115A fetches instructions from core instruction memory (IMEM) 305A based on addresses determined by core instruction memory lookahead (ITA) 310A. In the depicted embodiment, core decode stage 120A includes core decode 120A first stage 315A and core decode 120A second stage 320A. In the depicted embodiment, core decode 120A first stage 315A decodes instructions from the instruction fetch buffer and determines which execution pipeline can be used. In the depicted implementation, the main core decodes the second stage 320A of the 120A to obtain the necessary operands for the issued instruction. In the illustrative example, if instruction execution requires unavailable pipeline resources or operands that are not available from the main core register file 360A or the pipeline, the pipeline must wait for the pipeline resources to become available or for the operand conditions to be met before executing the instruction, resulting in a pipeline stall.

[0060] In the depicted implementation, the main core execution (EX) / address generation (AG) stage 125A includes a main core execution / address generation stage 125A execution first stage 325A and a main core execution / address generation stage 125A execution second stage 330A, which are configured in an arithmetic logic unit (ALU) pipeline to perform integer arithmetic operations, such as multiplication, addition, and subtraction, as well as logical and shift operations.

[0061] In the depicted implementation, the core execution (EX) / address generation (AG) stage 125A includes a first floating-point execution stage 335A, a second floating-point execution stage 340A, and a third floating-point execution stage 345A, configured to perform floating-point operations within a floating-point (FP) pipeline. In the illustrative example, the floating-point operations may require three loops to be executed by the FP pipeline.

[0062] In the depicted implementation, the main core execution (EX) / address generation (AG) stage 125A includes a main core execution / address generation stage 125A address generation stage 350A and a main core execution / address generation stage 125A data memory table look-ahead (TA) stage 355A, which is configured in the load / store pipeline to access data memory using the main core load / store stage 130A. The data memory may be the main core pipeline 105 and the auxiliary core pipeline 110 (at least in...).Figure 1 The main core execution / address generation level 125A and address generation level 350A perform address calculations, and the main core execution / address generation level 125A and data memory table look-ahead (TA) level 355A perform address translation from the virtual address calculated by the main core execution / address generation level 125A and address generation level 350A to the physical address. The virtual address is determined according to the memory management unit (MMU) configured in the processor. The virtual address is translated into a physical address by the main core execution / address generation level 125A and data memory table look-ahead (TA) level 355A. The physical address target can be in the data cache or non-cacheable memory. If the operation is a load operation (e.g., a read operation from memory), data is obtained from the data memory using the main core load / store level 130A, and the data can be saved to the main core register file 360A using the main core write-back level 135A. If the operation is a store instruction and the data is in memory, the data is stored in the data memory using the main core load / store level 130A. If the operation is a storage instruction for a cacheable region, the data is stored in the data cache using the master core write-back level 135A.

[0063] Figure 3 The depicted main core pipeline 105 illustrates a pipeline of a superscalar processor that includes dual-processor pipeline cores. Figure 3 Each of the components 305A, 310A, 315A, 320A, 325A, 330A, 335A, 340A, 345A, 350A, 355A, and 360A in the shown core pipeline 105 has at least one of the following: Figure 7 The corresponding auxiliary core pipelines shown are 110 elements: 305B, 310B, 315B, 320B, 325B, 330B, 335B, 340B, 345B, 350B, 355B, and 360B. An exemplary superscalar processor can be a dual-ordered release and ordered completion superscalar processor. The superscalar processor can be a RISC-based multithreaded processor, such as, for example, the C200 processor.

[0064] Figure 4 A block diagram depicts a portion of a processor pipeline configured with exemplary storage data path and write-back path latency modules. Figure 4 In the main core pipeline 105, data is stored and written back. Detail 400 depicts the process... Figure 1This depicts a portion of the main core pipeline 105. In the depicted embodiment, the main core memory data path delay module 150A includes a first stage 150A1, a second stage 150A2, and a third stage 150A3 of the main core memory data path delay module, operably connected in sequence. In the depicted embodiment, the main core memory data path delay module 150A receives a main core memory data comparison signal 205A and outputs a main core memory data comparison signal 205A delayed by three cycles to a main core memory data control gate 405A. The main core memory data control gate 405A receives a memory data comparator 140 (at least in...) Figure 2 (Depicted in the middle) Receive storage data comparison decision 215. The main core storage data control gate 405A is configured to output the delayed main processor pipeline storage data path signal to the main core load / storage level 130A only when the storage data comparison decision 215 indicates consistency between the main processor pipeline state and the auxiliary processor pipeline state.

[0065] In the depicted embodiment, the master core write-back path delay module 165A includes a first stage 165A1, a second stage 165A2, and a third stage 165A3 of the master core write-back path delay module, which are operably connected in sequence. In the depicted embodiment, the master core write-back path delay module 165A receives the master core write-back comparison signal 220A and outputs the master core write-back comparison signal 220A, delayed by three cycles, to the master core write-back control gate 410A. The master core write-back control gate 410A receives the write-back comparator 155 (at least in...) Figure 2 (Depicted in the middle) Receive writeback comparison decision 230. The master core writeback control gate 410A is configured to output the delayed master core writeback comparison signal 220A to the master core writeback stage 135A only when the writeback comparison decision 230 indicates consistency between the master processor pipeline state and the auxiliary processor pipeline state.

[0066] The main core pipeline 105 is configured such that if a cache data address or memory data address matches a data address in the delay stage of the main core storage data path delay module 150A, the loaded data is forwarded to the main core write-back stage 135A via the main core load data read path 415A and the main core load data majority multiplexer 420A. The main core pipeline 105 is also configured such that if a loaded data address matches a data address in the delay stage of the main core storage data path delay module 150A, the stored data is forwarded to memory or the main core write-back stage 135A via the main core load forwarding path 430A. The main core storage data path delay module 150A can be configured with triple module redundancy across multiple buffer stages, designed to only pass a majority of data from each triple-redundant buffer stage. The main core write-back path delay module 165A can be configured with triple module redundancy across multiple buffer stages, designed to only pass a majority of data from each triple-redundant buffer stage.

[0067] In the depicted implementation, the additional master core dummy pipeline stage 425A is configured with a three-stage delay in the write-back path to delay data updates to external memory or the write-back cache until the memory data comparator and write-back comparator confirm that the master core state and the auxiliary core state are consistent. When the memory data comparator and write-back comparator confirm that the master core state and the auxiliary core state are consistent, the update is executed. If the memory data comparator and write-back comparator do not confirm that the master core state and the auxiliary core state are consistent, a fault is detected. If a fault is detected, the update is canceled along with all instructions in the pipeline, and an exception is triggered to the processor. This exception will be a precise exception, requiring a period of time that can be accurately predicted. In this case, incomplete instructions can be canceled, and the program counter (PC) of the last executed instruction can be saved at the exception handling route. Since no fraudulent data is updated in the processor, the system will immediately restart after the trap software logs the fault point and PC. In some implementations, the fault point and PC can be logged to report the fault, and the fault log can be used to determine whether the fault is permanent based on identifying whether the same fault point with the same address in the PC causes the trap again.

[0068] Figure 4 The depiction of the main core pipeline 105 shows a portion of the pipeline of a superscalar processor that includes dual-processor pipeline cores. Figure 4 Each of the elements 405A, 410A, 415A, 420A, 425A, and 430A in the portion of the main core pipeline 105 depicted has at least one component... Figure 7 The corresponding auxiliary nuclear pipeline elements 110, 405B, 410B, 415B, 420B, 425B, and 430B, are depicted.

[0069] Figure 5 A block diagram depicts an exemplary write-back path delay module design. Figure 5 In the figure 500, the master core write-back path delay module 165A is depicted as an exemplary master core write-back path delay module 165A in a triple module redundancy implementation, which is designed to pass only the majority of data through each stage. The depicted master core write-back path delay module 165A includes a master core write-back path delay module first stage 165A1, a master core write-back path delay module second stage 165A2, and a master core write-back path delay module third stage 165A3, which are operatively connected in sequence. The depicted master core write-back path delay module first stage 165A1 includes delay module first stage triple module redundancy module first 505A, delay module first stage triple module redundancy module second 505B, and delay module first stage triple module redundancy module third 505C, which are cross-connected to delay module first stage triple module redundancy comparator first 510A, delay module first stage triple module redundancy comparator second 510B, and delay module first stage triple module redundancy comparator third 510C. The depicted master core write-back path delay module second stage 165A2 includes delay module second stage triple redundancy module one 505D, delay module second stage triple redundancy module two 505E, and delay module second stage triple redundancy module three 505F, which are cross-connected to delay module second stage triple redundancy comparator one 510D, delay module second stage triple redundancy comparator two 510E, and delay module second stage triple redundancy comparator three 510F. The depicted master core write-back path delay module third stage 165A3 includes delay module third stage triple redundancy module one 505G, delay module third stage triple redundancy module two 505H, and delay module third stage triple redundancy module three 505I, which are cross-connected to delay module third stage triple redundancy comparator three 510G. In the depicted embodiment, data is copied to three buffers in each stage, and these three values ​​are compared to the next stage. Only the majority of data determined by the comparator will be passed to the next level to improve data security by preventing invalid data from being written to memory or cache in the event of a failure in the storage data path delay module or write-back path delay module.

[0070] Figure 6 A block diagram depicts an exemplary storage data path latency module design. Figure 6In the figure 600, a detailed diagram of the main core storage data path delay module 150A depicts an exemplary main core storage data path delay module 150A in a triple module redundancy implementation. This triple module redundancy implementation is designed to pass most data only through each stage, and if the address of loaded data matches the address of data in the storage data path delay stage, the data is forwarded from the storage data path delay stage to the write-back stage or memory. The depicted main core storage data path delay module 150A includes a first stage 150A1, a second stage 150A2, and a third stage 150A3 of main core storage data path delay modules operably connected in sequence. In the depicted embodiment, the first stage 150A1, the second stage 150A2, and the third stage 150A3 of main core storage data path delay modules include reference... Figure 5 The publicly disclosed triple module redundancy feature also includes data forwarding features. Figure 6 In this configuration, the main core storage data path delay module 150A includes a main core storage data path delay module triple redundancy majority multiplexer 605, which is configured to pass matching majority data in the delay stages via a main core storage data path delay module triple redundancy first-stage comparator 610A, a main core storage data path delay module triple redundancy second-stage comparator 610B, and a main core storage data path delay module triple redundancy third-stage comparator 610C. In the illustrative example, because the stored data is delayed to memory or a data cache as a result of the disclosed delay-locked-step fault-tolerant system design, the data in memory or the data cache can be older data, and the data in the buffer stage of the main core storage data path delay module 150A can be newer data. In this case, if the load data address matches the stored data in these stages, the data can be forwarded to the write-back stage, thereby allowing faster recovery from the nearest clean point by preserving the integrity of the data in the delay stages.

[0071] In an example illustrating the technical effects that may result from using a data forwarding implementation according to this disclosure, consider the following execution scenario.

[0072] Store R3, R4, 100 / / Stores data from R3 into memory location R4+100. / / In this example, assume R4+100 equals R5+200 Load R6, R5, 200 / / Load data from memory in R5+200 into R6 If a load instruction must wait for the STORE operation to finish before executing the LOAD operation, then we don't need data forwarding. In this case, we encounter a pipeline pause. However, in a pipeline mode with data forwarding, the pipeline does not need to pause because the LOAD instruction does not need to wait for the STORE operation to finish.

[0073] Figure 7 An exemplary fault-tolerant computing system according to this disclosure is described, which has delayed lockstep execution between a main processor and an auxiliary processor, storage data path latency, and write-back path latency, and is configured to prevent system corruption when a fault is detected based on a comparison of the main processor state and the auxiliary processor state. Figure 7 In the description, the system 100 includes incorporated references. Figures 1-6 The main core pipeline 105 and the auxiliary core pipeline 110 are presented as features.

[0074] Although various features have been described with reference to the accompanying drawings, other features are also possible. For example, although this disclosure has presented an implementation based on a dual ordered release and ordered completion processor example, implementations according to this disclosure can also be configured in any processor system having a single ordered release and ordered completion processor architecture.

[0075] In the illustrative example, fault detection and recovery have been described with reference to a delayed-locked-step design based on a main core with three additional latency stages and an auxiliary core with a similar structure but one additional latency stage, such that the auxiliary core executes two steps after the main core. Therefore, the acknowledgment signal from the storage data comparator or write-back comparator is advanced two cycles in the auxiliary core. Embodiments according to this disclosure can be configured such that the main processor executes the same program as the auxiliary processor, wherein the main processor always executes two cycles before the auxiliary processor. This is why the main processor is configured with three additional stages while the auxiliary processor is configured with one additional stage. In the illustrative example of an embodiment of the fast recovery fault-tolerant feature disclosed herein, if the number of cycle gaps between the main processor and the auxiliary processor increases, the number of additional stages will increase accordingly.

[0076] In the foregoing overview, this detailed description, the appended claims, and the accompanying drawings, specific features of various embodiments have been referenced. It should be understood that the disclosure of specific features of various embodiments in this specification is to be interpreted as including all possible combinations of such specific features. For example, where a specific feature is disclosed in the context of a particular aspect or embodiment or a particular claim, that feature may also be used to the extent possible in combination with other specific aspects and embodiments and / or in the context of other specific aspects and embodiments, and is generally used in embodiments.

[0077] While several embodiments have been disclosed, other embodiments will become apparent to those skilled in the art from this detailed description. Numerous modifications can be made to the disclosed embodiments in a variety of obvious aspects without departing from the spirit and scope of the disclosed embodiments. Accordingly, the drawings and descriptions are to be considered illustrative in nature rather than restrictive.

[0078] It should be noted that the features shown in the accompanying drawings are not necessarily drawn to scale, and as those skilled in the art will recognize, features of one embodiment can be used in conjunction with those of other embodiments, even if not explicitly stated herein. Descriptions of well-known components and processing techniques may be omitted so as not to unnecessarily obscure the implementation features.

[0079] This article describes how coupled or connected elements can have effective relationships that can be achieved through direct connection or indirectly with one or more other intermediate elements.

[0080] In this disclosure, the term "any" can be understood to specify any number of corresponding elements, that is, to specify one, at least one, at least two, each, or all of the corresponding elements. Similarly, the term "any" can be understood to specify any set of corresponding elements, that is, to specify one or more sets of corresponding elements, including sets of one, at least one, at least two, each, or all of the corresponding elements. The various sets do not need to include the same number of elements.

[0081] While various embodiments have been disclosed and described in detail herein, it will be apparent to those skilled in the art that various changes can be made to the disclosed configurations, operations, and forms without departing from their spirit and scope. The use of singular and plural forms is for illustrative purposes only and should not be construed as restrictive.

[0082] An abstract is provided to comply with 37 CFR §1.72(b) so that the reader can quickly determine the nature of the technical disclosure and understand, at the time of submission, that it is not intended to interpret or limit the scope or meaning of the claims.

[0083] In this disclosure, all descriptions using the word "comprising" may be replaced with "consistently consisting of" or "comprises of". In this disclosure, any method or apparatus implementation may lack one or more process steps or components. In this disclosure, implementations employing negative limitations are explicitly disclosed and are considered part of this disclosure.

[0084] Certain terms and their derivatives may be used in this disclosure for convenience of reference only and not for limitation. For example, unless otherwise stated, words such as “up,” “down,” “left,” and “right” will refer to directions referenced in the accompanying drawings. Similarly, words such as “inward” and “outward” will refer to directions toward and away from the geometric center of the equipment or area and its designated portion, respectively. Unless otherwise stated, singular tense references include plural, and vice versa.

[0085] As used herein, the term "comprising" and its grammatical equivalents mean that other components, ingredients, steps, etc. are optionally present. For example, an implementation that "comprising" (or "comprising" components A, B, and C) may consist of components A, B, and C (i.e., containing only components A, B, and C), or may contain not only components A, B, and C, but also one or more other components.

[0086] When this document refers to a method that includes two or more defined steps, the defined steps may be performed in any order or simultaneously (except where the context precludes this possibility), and the method may include one or more other steps that are performed before any defined step, between two defined steps, or after all defined steps (except where the context precludes this possibility).

[0087] The term "at least" followed by a number is used in this document to indicate the beginning of a range that begins with that number (which, depending on the defined variable, may be a range with an upper limit or no upper limit). For example, "at least 1" means 1 or greater than 1. The term "at most" followed by a number (which may be a range with 1 or 0 as its lower limit, or a range without a lower limit, depending on the defined variable). For example, "at most 4" means 4 or fewer, while "at most 40%" means 40% or fewer. In this specification, when a range is given as "(first number) to (second number)" or "(first number) - (second number)", it means that its limit is the range of the second number. For example, 25-100mm refers to a range with a lower limit of 25mm and an upper limit of 100mm.

[0088] Any element not expressly referred to as an “apparatus” or “step” for performing a particular function in the claims herein shall not be construed as an “apparatus” or “step” as defined in 35 USC § 112(f). Specifically, any use of “…step” in the claims herein is not intended to reference the provisions of 35 USC § 112(f). Elements described in the apparatus plus function format are intended to be interpreted in accordance with 35 USC § 112(f).

[0089] The use of the term "first" in the claims regarding a feature or element does not necessarily imply the presence of a second or additional such feature or element.

[0090] The phrases “connected to,” “coupled to,” and “communicate with” refer to any form of interaction between two or more entities, including mechanical, electrical, magnetic, electromagnetic, fluid, and thermal interactions. Two components can be functionally coupled to each other even if they are not in direct contact. The terms “adjacent” or “mechanically joined” refer to items that are in direct physical contact with each other, although these items may not necessarily be attached together.

[0091] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as superior to other implementations. Although various aspects of this disclosure are presented with reference to the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated otherwise.

[0092] Throughout this specification, references to "implementation" or "this implementation" mean that a particular feature, structure, or characteristic described in connection with that implementation is included in at least one implementation. Therefore, all phrases or variations thereof referenced throughout this specification do not necessarily refer to the same implementation.

[0093] Similarly, it should be recognized that in the foregoing description, for the purposes of this disclosure, various features are sometimes grouped together in a single embodiment, drawing, or description thereof. However, the approach of this disclosure should not be construed as reflecting an intention that any claim in this application or any application claiming priority to this application requires more features than those expressly recited in that claim. Rather, as reflected in the appended claims, the inventive aspect may lie in a combination of fewer features than all the features of any single foregoing disclosed embodiment. Therefore, the claims following this detailed description are expressly incorporated herein by reference, with each claim existing independently as a separate embodiment. This disclosure is intended to be construed as including all permutations of the independent claims and their dependent claims.

[0094] Throughout this disclosure and elsewhere, block diagrams or flowcharts may depict methods, apparatus (i.e., systems) and computer program products. Each element of a block diagram or flowchart, and each corresponding combination of elements in a block diagram or flowchart, illustrates the function of a method, apparatus, and computer program product. Any and all such function (“depicted function”) may be implemented by computer program instructions; by a dedicated, hardware-based computer system; by a combination of dedicated hardware and computer instructions; by a combination of general-purpose hardware and computer instructions; and so on—any and all of which may generally be referred to herein as a “circuit,” a “module,” or a “system.”

[0095] While the foregoing figures and descriptions may illustrate functional aspects of the disclosed system, no specific arrangement of software used to implement these functional aspects should be inferred from these descriptions unless explicitly stated in the context or otherwise clear.

[0096] Each element in the flowchart illustration may depict a step or group of steps of a computer-implemented method. Furthermore, each step may contain one or more sub-steps. For illustrative purposes, these steps (as well as any and all other steps identified and described above) are presented sequentially. It should be understood that implementations may include alternative orders of steps suitable for a particular application of the technology disclosed herein. All such variations and modifications are intended to fall within the scope of this disclosure. The depiction and description of steps in any particular order is not intended to exclude implementations of steps in a different order unless required by the particular application, explicitly stated, or clearly apparent from the context.

[0097] Traditionally, computer programs consist of a series of computational instructions or program instructions. It should be understood that programmable devices (i.e., computing devices) can receive such computer programs and, by processing their computational instructions, produce further technical effects.

[0098] Programmable devices may include one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, programmable devices, programmable gate arrays, programmable array logic, memory devices, application-specific integrated circuits, etc., which may be suitably adopted or configured to process computer program instructions, execute computer logic, store computer data, etc. Throughout this disclosure and elsewhere, a computer may include any and all suitable combinations of at least one general-purpose computer, special-purpose computer, programmable data processing device, processor, processor architecture, etc.

[0099] It should be understood that a computer may include computer-readable storage media, and such media may be internal or external, removable and replaceable or fixed. It will also be understood that a computer may include a basic input / output system (BIOS), firmware, operating system, database, etc., which may include the software and hardware described herein, or interface with or support the software and hardware described herein.

[0100] Implementations of the systems described herein are not limited to applications involving conventional computer programs or programmable devices running them. It is conceivable, for example, that embodiments of this disclosure as claimed herein may include optical computers, quantum computers, analog computers, etc.

[0101] Unless explicitly stated in the context or otherwise clear, the verbs “execute” and “process” are used interchangeably to indicate execution, processing, interpretation, compilation, assembly, linking, loading, and any and all combinations thereof. Therefore, an implementation that executes or processes computer program instructions, computer executable code, etc., can properly act on the instructions or code in any and all the ways just described.

[0102] The functions and operations presented herein are not inherently related to any particular computer or other device. Various general-purpose systems may also be used with programs based on the teachings herein, or it may prove convenient to construct more specialized devices to perform the required method steps. The desired architectures of various such systems, as well as equivalent variations, will be apparent to those skilled in the art. Furthermore, no specific programming language is referenced in describing implementations of this disclosure. It should be understood that various programming languages ​​may be used to implement the teachings as described herein, and any references to particular languages ​​are provided for the purpose of disclosing implementations and best implementation patterns of this disclosure. Implementations of this disclosure are well-suited for various computer or network systems on multiple topologies. In this field, the configuration and management of large systems involve storage devices and computers communicatively coupled to different computers and storage devices via networks such as the Internet.

[0103] The corresponding reference numerals and descriptions of the components shown in the attached figures are summarized below.

[0104] 100 System 105 main core pipeline 110 Auxiliary Nuclear Production Line 115A main core extraction stage 115B auxiliary nuclear extraction grade 120A Main Core Decoding Level 120B Auxiliary Core Decoding Level 125A main core execution (EX) / address generation (AG) level 125B Auxiliary Core Execution (EX) / Address Generation (AG) Level 130A main core loading / storage level 130B Auxiliary Core Loading / Storage Level 135A main core write-back level 135B auxiliary core write-back level 140 Storage Data Comparator 145A main core storage data comparison delay module 145A1 main core storage data comparison delay module first stage 145A2 main core storage data comparison delay module level 2 145A3 main core storage data comparison delay module level 3 145B Auxiliary Core Storage Data Comparison Delay Module 150A Main Core Storage Data Path Delay Module 150A1 main core storage data path delay module level 1 150A2 main core storage data path delay module level 2 150A3 main core storage data path delay module level 3 150B Auxiliary Core Storage Data Path Delay Module 155 write-back comparator 160A Main Core Write-back Comparison Delay Module 160A1 main core write-back comparison delay module level 1 160A2 main core write-back comparison delay module level 2 160A3 main core write-back comparison delay module level 3 160B Auxiliary Core Write-back Comparison Delay Module 165A main core write-back path delay module 165A1 main core write-back path delay module level 1 165A2 main core write-back path delay module level 2 165A3 main core write-back path delay module level 3 165B Auxiliary Core Write-back Path Delay Module 205A main core storage data comparison signal 205B auxiliary core stores data comparison signals 210A Delay Main Core Storage Data Comparison Signal 210B delay auxiliary core stores data comparison signal 215 Data Comparison and Judgment 220A main core write-back comparison signal 220B Auxiliary Core Write-back Comparison Signal 225A Delayed Master Core Write-Back Comparison Signal 225B delayed auxiliary core write-back compare signal 230 write-back comparison judgment 300 main core pipeline 105 detailed diagram 305A Main Core Instruction Memory (IMEM) 305B Auxiliary Core Instruction Memory (IMEM) 310A Main Core Instruction Memory Table Lookahead (ITA) 310B Auxiliary Core Instruction Memory Table Outlook (ITA) 315A main core decodes 120A first stage 315B auxiliary core decodes 120B first stage 320A main core decodes 120A secondary stage 320B auxiliary core decodes 120B second stage 325A main core execution / address generation level 125A execution level 1 325B auxiliary core execution / address generation level 125B execution level 1 330A main core execution / address generation level 125A execution level 2 330B auxiliary core execution / address generation level 125B execution level 2 335A main core execution / address generation level; 125A floating-point execution level 1 335B auxiliary core execution / address generation level 125B floating-point execution level 1 340A main core execution / address generation level, 125A floating-point execution level 2 340B auxiliary core execution / address generation level 125B floating-point execution level 2 345A main core execution / address generation level; 125A floating-point execution level 3 345B auxiliary core execution / address generation level 125B floating-point execution level 3 350A main core execution / address generation level 125A address generation 350B auxiliary core execution / address generation level 125B address generation 355A main core execution / address generation stage, 125A data memory table preview (TA) 355B auxiliary core execution / address generation stage, 125B data memory table lookup (TA) 360A Main Core Register File 360B Auxiliary Core Register File 400 main core pipeline 105 storage data and write-back details 405A main core memory data control gate 405B Auxiliary Core Storage Data Control Gate 410A Main Core Write-back Control Gate 410B Auxiliary Core Writeback Control Gate 415A main core loading data read path 415B Auxiliary Loading Data Reading Path 420A main core loads data multiplexer 420B auxiliary core loads data multiplexer 425A main core virtual pipeline stage 425B auxiliary core is a dummy pipeline stage 430A main core loading and forwarding 430B auxiliary core loading and forwarding Detailed diagram of 500 main core write-back path delay module 165A 505A Delay Module Level 1 Tri-Mode Redundancy Module 505B Delay Module Level 1 Tri-Mode Redundancy Module 2 505C Delay Module Level 1 Tri-Mode Redundancy Module 505D Delay Module Level 2 Tri-Mode Redundancy Module 1 505E Delay Module Level 2 Tri-Mode Redundancy Module 2 505F Delay Module Level 2 Tri-modular Redundancy Module 505G latency module, third-level tri-mode redundancy module 1 505H Delay Module Level 3 Tri-modular Redundancy Module 2 505I delay module, third-level triple-modular redundancy module 510A Delay Module First Stage Triple Modular Redundancy Comparator 510B Delay Module First Stage Triple Modular Redundancy Comparator 510C Delay Module First Stage Triple Modular Redundancy Comparator 510D Delay Module Second Stage Tri-Mode Redundancy Comparator 510E Delay Module Second Stage Tri-Mode Redundancy Comparator 510F Delay Module Second Stage Tri-Mode Redundancy Comparator 510G delay module third-stage tri-mode redundancy comparator Detailed diagram of 600 main core storage data path delay module 150A 605 main core storage data path delay module triple-redundant majority multiplexer 610A Main Core Storage Data Path Delay Module Triple Modular Redundancy First-Stage Comparator 610B Main Core Storage Data Path Delay Module Triple Modular Redundancy Second-Stage Comparator The 610C main core storage data path delay module features a triple-redundant third-stage comparator.

[0105] Many embodiments have been described. However, it should be understood that various modifications can be made. For example, the steps of the disclosed technology may be performed in a different order, the components of the disclosed system may be combined in different ways, or these components may be supplemented with other components. Therefore, other embodiments are contemplated within the scope of the appended claims.

Claims

1. An apparatus comprising: a main processor including a main processor pipeline, wherein the main processor pipeline includes an execution stage and a load / store stage; a secondary processor including a secondary processor pipeline, wherein the secondary processor pipeline includes an execution stage and a load / store stage; a store data comparator having a first input operably coupled to the main processor pipeline, a second input operably coupled to the secondary processor pipeline, and an output, wherein the store data comparator is configured to indicate at the output a result of a comparison of a delayed main processor pipeline signal and a delayed secondary processor pipeline signal; a main processor pipeline store data control gate configured to output a delayed main processor pipeline signal to the main processor pipeline load / store stage only if the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator; and a secondary processor pipeline store data control gate configured to output a delayed secondary processor pipeline signal to the secondary processor pipeline load / store stage only if the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator.

2. The apparatus of claim 1, wherein, the main processor pipeline execution stage has an output operably connected to a first input of the main processor store data control gate through a main processor pipeline store data path delay module.

3. The apparatus of claim 2, wherein, the main processor pipeline store data path delay module is a three-stage buffer delay module.

4. The apparatus of claim 2, wherein, the main processor pipeline store data path delay module is a three-stage triple modular redundancy buffer delay module, wherein each stage is configured to pass only a majority of data.

5. The apparatus of claim 2, wherein the main processor pipeline further includes a writeback stage, and wherein the main processor pipeline is configured to forward data from the main processor pipeline store data path delay module to the writeback stage if a load data address matches an address of data in the main processor pipeline store data path delay module.

6. The apparatus of claim 1, wherein, the secondary processor pipeline execution stage has an output operably connected to a first input of the secondary processor store data control gate through a secondary processor pipeline store data path delay module.

7. The apparatus of claim 6, wherein, the secondary processor pipeline store data path delay module is a one-stage buffer delay module.

8. The apparatus of claim 1, wherein, the delayed main processor pipeline signal compared by the store data comparator is connected to the store data comparator first input through a main processor store data comparison delay module operably coupled to the main processor pipeline execution stage.

9. The apparatus of claim 8, wherein, the main processor store data comparison delay module is a three-stage buffer delay module.

10. An apparatus comprising: a main processor including a main processor pipeline, wherein the main processor pipeline includes, in order, an extract stage, a decode stage, an execute stage, a load / store stage, and a writeback stage; a secondary processor including a secondary processor pipeline, wherein the secondary processor pipeline includes, in order, an extract stage, a decode stage, an execute stage, a load / store stage, and a writeback stage; a store data comparator having a first input, a second input, and an output, the first input operably coupled with an output of the main processor pipeline execute stage to receive a delayed main processor pipeline signal through a main processor store data compare delay module including a three-stage buffer delay, the second input operably coupled with the secondary processor pipeline execute stage to receive a delayed secondary processor pipeline signal through a secondary processor store data compare delay module including a one-stage buffer delay, wherein the store data comparator is configured to indicate at the output a result of comparing the delayed main processor pipeline signal with the delayed secondary processor pipeline signal; a main processor pipeline store data control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the main processor pipeline execute stage to receive a delayed main processor pipeline signal through a main processor store data path delay module including a three-stage buffer delay, wherein the second input is operably coupled with the store data comparator output, and the main processor pipeline store data control gate is configured to output the delayed main processor pipeline store data path signal to the main processor pipeline load / store stage only if the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator, and wherein the main processor pipeline is configured to forward data from the main processor pipeline store data path delay module to the writeback stage or memory if a load data address matches an address of data in the main processor pipeline store data path delay module; an auxiliary processor pipeline store data control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor store data path delay module including a one-stage buffer delay, wherein the second input is operably coupled with the store data comparator output, and the auxiliary processor pipeline store data control gate is configured to output the delayed auxiliary processor pipeline store data path signal to the auxiliary processor pipeline load / store stage only if the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the store data comparator, and wherein the auxiliary processor pipeline is configured to forward data from the auxiliary processor pipeline store data path delay module to the writeback stage or memory if a load data address matches an address of data in the auxiliary processor pipeline store data path delay module; a writeback comparator having a first input operably coupled with an output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal through a main processor writeback comparison delay module including a three-stage buffer delay, a second input operably coupled with the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor writeback comparison delay module including a one-stage buffer delay, and an output, and wherein the writeback comparator is configured to indicate a result of comparing the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal at the output; a main processor pipeline writeback control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal through a main processor writeback path delay module including a three-stage buffer delay, and wherein the second input is operably coupled with the writeback comparator output, and the main processor pipeline writeback control gate is configured to output the delayed main processor pipeline writeback signal to the main processor pipeline writeback stage only if the output of the writeback comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator; and and an auxiliary processor pipeline writeback control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor writeback path delay module including a one-stage buffer delay, and wherein the second input is operably coupled with the writeback comparator output and the auxiliary processor pipeline writeback control gate is configured to output the delayed auxiliary processor pipeline writeback signal to the auxiliary processor pipeline writeback stage only when the output of the writeback comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator.

11. The apparatus of claim 10, wherein, the main processor pipeline fetch stage and the auxiliary processor pipeline fetch stage are operably coupled to a program store common to the main processor and the auxiliary processor.

12. The apparatus of claim 10, wherein, the auxiliary processor pipeline is executed in a delayed lockstep manner with the main processor pipeline, wherein the auxiliary processor pipeline is executed at least two cycles delayed relative to the main processor pipeline execution.

13. The apparatus of claim 10, wherein, the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the storage data comparator further include handshake signals and data between a data cache and a core, or between a data memory and the core.

14. The apparatus of claim 10, wherein, the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator further include handshake signals and data between a register file, a data cache, a data memory, and a core.

15. The apparatus of claim 10, wherein, the main processor pipeline storage data path delay module further includes a three-stage triple modular redundancy buffer delay module, wherein each stage is configured to pass only a majority of data.

16. The apparatus of claim 10, wherein, the main processor pipeline writeback path delay module further includes a three-stage triple modular redundancy buffer delay module, wherein each stage is configured to pass only a majority of data.

17. An apparatus comprising: a main processor including a main processor pipeline, wherein the main processor pipeline includes, in order, a fetch stage, a decode stage, an execute stage, a load / store stage, and a writeback stage operably connected; an auxiliary processor including an auxiliary processor pipeline, wherein the auxiliary processor pipeline includes, in order, a fetch stage, a decode stage, an execute stage, a load / store stage, and a writeback stage operably connected, wherein the main processor pipeline fetch stage and the auxiliary processor pipeline fetch stage are operably coupled to a program store common to the main processor and the auxiliary processor; a store data comparator having a first input, a second input, and an output, the first input operably coupled with an output of the main processor pipeline execution stage to receive a delayed main processor pipeline signal through a main processor store data compare delay module including a three stage buffer delay, the second input operably coupled with the secondary processor pipeline execution stage to receive a delayed secondary processor pipeline signal through a secondary processor store data compare delay module including a one stage buffer delay, wherein the store data comparator is configured to indicate at the output a result of comparing the delayed main processor pipeline signal with the delayed secondary processor pipeline signal, wherein the delayed main processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator further include handshake signals and data between a data cache and a core, and handshake signals between a data memory and the core; a main processor pipeline store data control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the main processor pipeline execution stage to receive a delayed main processor pipeline signal through a main processor pipeline store data path delay module including a three stage triple module redundancy buffer delay, wherein each stage is configured to pass only a majority of data, wherein the second input is operably coupled with the store data comparator output, and the main processor pipeline store data control gate is configured to output the delayed main processor pipeline store data path signal to the main processor pipeline load / store stage only when the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed secondary processor pipeline signal compared by the store data comparator, and wherein the main processor pipeline is configured to forward data from the main processor pipeline store data path delay module to the write back stage if a load data address matches an address of data in the main processor pipeline store data path delay module buffer stage; an auxiliary processor pipeline store data control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the auxiliary processor pipeline execution stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor store data path delay module including a one-stage buffer delay, wherein the second input is operably coupled with the store data comparator output, and the auxiliary processor pipeline store data control gate is configured to output the delayed auxiliary processor pipeline store data path signal to the auxiliary processor pipeline load / store stage only when the output of the store data comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the store data comparator, and wherein the auxiliary processor pipeline is configured to forward data from the auxiliary processor pipeline store data path delay module to the writeback stage if a load data address matches an address of data in the auxiliary processor pipeline store data path delay module; a writeback comparator having a first input operably coupled with an output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal through a main processor writeback comparison delay module including a three-stage buffer delay, a second input operably coupled with the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor writeback comparison delay module including a one-stage buffer delay, and an output, and wherein the writeback comparator is configured to indicate a result of comparing the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal at the output, wherein the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator further include handshake signals and data between a register file, a data cache, a data memory, and a core; a main processor pipeline writeback control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal through a main processor writeback path delay module including a three-stage triple module redundancy buffer delay, wherein each stage is configured to pass majority data only, and wherein the second input is operably coupled with the writeback comparator output, and the main processor pipeline writeback control gate is configured to output the delayed main processor pipeline writeback signal to the main processor pipeline writeback stage only when the output of the writeback comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator; and a main processor pipeline writeback control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the main processor pipeline load / store stage to receive a delayed main processor pipeline signal through a main processor writeback path delay module including a three-stage triple module redundancy buffer delay, wherein each stage is configured to pass majority data only, and wherein the second input is operably coupled with the writeback comparator output, and the main processor pipeline writeback control gate is configured to output the delayed main processor pipeline writeback signal to the main processor pipeline writeback stage only when the output of the writeback comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator; and An auxiliary processor pipeline writeback control gate having a first input, a second input, and an output, wherein the first input is operably coupled with an output of the auxiliary processor pipeline load / store stage to receive a delayed auxiliary processor pipeline signal through an auxiliary processor writeback path delay module including a one-stage buffer delay, and wherein the second input is operably coupled with the writeback comparator output, and the auxiliary processor pipeline writeback control gate is configured to output the delayed auxiliary processor pipeline writeback signal to the auxiliary processor pipeline writeback stage only when the output of the writeback comparator indicates a match between the delayed main processor pipeline signal and the delayed auxiliary processor pipeline signal compared by the writeback comparator.

18. The apparatus of claim 17, wherein, The program storage area also includes processor-executable instructions configured to cause the apparatus to resume execution from a known clean program point in response to a fault indication determined from an inconsistency between a main processor pipeline signal and an auxiliary processor pipeline signal.

19. The apparatus of claim 17, wherein, One or more of the main processor or the auxiliary processor also includes a superscalar processor.

20. The apparatus of claim 17, wherein, The apparatus also includes a C200 processor.

Images