Hardware-based fault scanner for detecting faults in homogeneous processing units

Abstract

Apparatuses, systems, and techniques for detecting faults in processing pipelines are described. An accelerator circuit includes a fixed function circuit that performs operations corresponding to a neural network layer. The fixed function circuit includes a set of homogeneous processing units and a fault scanner circuit. The fault scanner circuit includes an additional homogeneous processing unit to sequentially scan each processing unit of the set to find functional faults.

Description

TECHNICAL FIELD

[0001] At least one embodiment relates to processing resources for performing and facilitating artificial intelligence. For example, at least one embodiment relates to a hardware-based fault scanner for detecting faults in a processing pipeline. BACKGROUND

[0002] In many cases, the majority of the computational work for deep learning inference is based on mathematical operations, which can generally be divided into four parts: convolution, activation, pooling, and normalization. These operations share some common characteristics that make them particularly suitable for hardware implementation: their memory access patterns are predictable, and they are easily parallelizable, for example, in a parallel processing pipeline. Functional safety refers to the failure to cause unreasonable risk due to faulty behavior of a processing unit at runtime. Functional safety aims to reduce the risk of harm from an unacceptable level to a tolerable level. A previous approach uses error-correcting coding (ECC) to detect faults in memory. This approach is limited to detecting errors in data, but cannot distinguish between permanent errors and temporary errors in underlying logic. Another approach to detecting faults is to use a soft diagnostic library to periodically execute a predetermined workload and then compare the results to a known reference, also referred to as a golden reference. This approach only compares interface data, and internal data can be more precise than interface data. This approach also impacts performance because underlying resources are occupied while executing the predetermined workload. Another approach to detecting faults uses algorithm-based error detection techniques. Algorithm-based error detection techniques require high-precision mathematics, which is expensive in terms of area on silicon. Some comparisons can not have enough bits, so a certain error threshold is allowed during the comparison process. The error threshold can result in false positives. Due to unavoidable rounding errors in different computation sequences and the error threshold used during the comparison process, some faults require thousands of cycles to detect, and some faults cannot be detected at all. BRIEF DESCRIPTION OF DRAWINGS

[0003] Figure 1 is a block diagram of an accelerator core with a hardware fault scanner for detecting faults occurring in homogenous processing units of a fixed function circuit, in accordance with at least some embodiments;

[0004] Figure 2 is a block diagram of a processing pipeline with a group of homogenous processing units and a hardware fault scanner operating in a time-multiplexed manner, in accordance with at least some embodiments;

[0005] Figure 3 is a block diagram of a convolution pipeline with a group of homogenous convolution multiply-accumulate (CMAC) units and a hardware fault scanner, in accordance with at least some embodiments;

[0006] Figure 4is a flowchart of a method for detecting faults in a homogeneous processing unit of a processing pipeline, according to at least some embodiments;

[0007] Figure 5 is a block diagram of a deep learning accelerator (DLA) system, according to at least some embodiments; and

[0008] Figure 6 is a block diagram of a DLA system, according to at least some embodiments. DETAILED DESCRIPTION

[0009] As noted above, deep learning inference is based on operations that are well suited for hardware implementation. Deep learning accelerator (DLA) circuits, such as the NVIDIA® Deep Learning Accelerator (NVDLA), can address the computational demands of inference by providing building blocks that accelerate core deep learning operations. Deep learning accelerators can be used to accelerate different neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), fully connected neural networks, etc. These neural networks can have very different structures, can not follow any pre-defined network structure, and new neural networks are being introduced on a regular basis.

[0010] Currently, DLA circuits use fixed function engines (also referred to herein as fixed function units or fixed function circuits) for different types of layers in these neural networks, e.g., fixed function engines for convolution, activation functions, pooling, batch normalization, etc. It should be noted that a neural network from an algorithmic perspective can be specified with a set of layers (referred to herein as “original layers”), e.g., bias and batch normalization. These original layers can be compiled or converted into another set of layers (referred to herein as “hardware layers”). Each hardware layer is used as a basic element for scheduling execution on the accelerator circuit. The mapping between original layers and hardware layers can be m:n, where m is the number of original layers and n is the number of hardware layers. For example, the original layers bias, batch normalization, and local response normalization (LRN) (e.g., rectified linear unit (ReLU)) in a neural network can be compiled into one hardware layer. In this case, m:n is 3:1. Each hardware layer can be represented with one basic hardware instruction for one of the fixed function engines to perform the operation, and each layer communicates with another layer through a memory interface. For example, a first layer can be executed by a first fixed function engine that receives an input tensor, performs operations on the input tensor to generate an output tensor, and stores the output tensor in a system memory, e.g., a dynamic random access memory (DRAM) coupled to the accelerator. A second layer can be executed by a second fixed function engine that receives the output tensor from the first layer as a second input tensor from the memory, performs operations on the second input tensor to generate a second output tensor, and stores the second output tensor in the DRAM. Each communication incurs a tensor read operation and a tensor write operation in the memory interface.

[0011] As described above, previous methods for detecting faults in DLA circuits are insufficient because they only detect faults in memory, only compare interface data, cannot distinguish between permanent faults and temporary faults, impact the performance of the DLA circuit, and are costly to implement.

[0012] Aspects and embodiments of the present disclosure address these and other challenges by providing, for example, a hardware fault scanner to detect faults in a processing pipeline. Aspects and embodiments of the present disclosure can provide a hardware fault scanner (also referred to herein as a fault scanner circuit or fault scanner unit) that is an additional processing unit that scans for functional faults in a time-multiplexed manner. In a given cycle, the processing unit and the hardware fault scanner can have the same implementation and the same input data, allowing for a bit-by-bit comparison of the outputs of the processing unit and the hardware fault scanner. The hardware fault scanner can be used for both fixed function circuitry of a DLA circuit and parallel processing units in a GPU. Because the fault scanning is performed concurrently with the operation of the processing unit, aspects and embodiments of the present disclosure do not suffer from performance loss. Aspects and embodiments of the present disclosure can provide a bit-by-bit comparison, resulting in fewer false positives than algorithm-based error detection techniques. Unlike algorithm-based error detection techniques that require higher precision and more area, aspects and embodiments of the present disclosure can be implemented in less area and can use the same precision and implementation as the processing unit.

[0013] Figure 1 is a block diagram of an accelerator core 100 having a hardware fault scanner 102 for detecting faults in fixed function circuitry 104-114, according to at least some embodiments. The accelerator core 100 (also referred to herein as a DLA core or accelerator circuit) includes the hardware fault scanner 102 and various fixed function circuitry, such as a convolution engine 104 (also referred to herein as a convolution core), an activation engine 106 (also referred to herein as a single data processor (SDP)) that is a single point lookup engine for activation functions, a pooling engine 108 (also referred to herein as a planar data processor (PDP)) that is a planar average engine for pooling, a local response normalization (LRN) engine 110 (also referred to herein as a cross-channel data processor (CDP)) that is a specialized unit that applies the LRN function operating on the channel dimension instead of the spatial dimension, a data reshaping engine 112 (also referred to herein as RUBIK) that performs data format conversions (e.g., split or slice, merge, shrink, reshape transfer), and a bridging direct memory access (DMA) engine 114 that can move data between system DRAM and a dedicated memory interface. Additional details of the hardware fault scanner 102 are described below. In other embodiments, the accelerator core 100 can include more or fewer fixed function circuitry than Figure 1More or fewer engines as described herein. Each of these engines can be separate and independently configurable. For example, a system that does not require pooling operations can completely remove the planar averaging engine. For example, a system that requires additional convolution performance can scale up the performance of the convolution core without modifying other units in accelerator core 100.

[0014] like Figure 1 As shown, the accelerator core 100 has multiple connections to the rest of the DLA system, including a configuration interface block 116, which includes a configuration space bus (CSB) interface and an interrupt interface. The configuration interface block 116 may be a control channel interface that implements a register file (e.g., a configuration register) and an interrupt interface (labeled CSB / interrupt interface 118). In at least one embodiment, the CSB interface is a synchronous, low-bandwidth, low-power, 32-bit control bus designed to be controlled by the CPU (…). Figure 1 (Not shown) is used to access the configuration register in configuration interface block 116. The interrupt interface can be a 1-bit level driven interrupt. An interrupt line can be asserted when a task completes or an error occurs. Accelerator core 100 may also include a memory interface block 120 that interfaces with memory using one or more bus interfaces. In at least one embodiment, memory interface block 120 has a connection to system memory (…). Figure 1 Main memory interface 122 (not shown). System memory may include DRAM. Main memory interface 122 may be shared with the CPU and input / output (I / O) peripherals. In at least one embodiment, main memory interface 122 is a data backbone (DBB) interface connecting accelerator core 100 and other memory subsystems. The DBB interface is a configurable data bus that can specify different address sizes, different data sizes, and issue requests of different sizes. In at least one embodiment, the DBB interface uses an interface protocol, such as AXI (Advanced Extensible Interface) or other similar protocols. In at least one embodiment, memory interface block 120 has a second memory interface 124 that allows connection to higher bandwidth memory dedicated to accelerator core 100 or computer vision subsystems. For example, the second memory interface 124 may be used with on-chip SRAM to provide higher throughput and lower access latency.

[0015] Memory interface block 120 is coupled to each fixed-function engine 104-114. A convolution buffer 126 may be used between memory interface block 120 and convolution engine 104 to avoid repeated access to system memory. Convolution buffer 126 may be internal RAM reserved for weight and input feature / pixel storage.

[0016] During operation of the accelerator core 100, the process flow begins with a hardware layer configuration and activation command sent from a management processor (microcontroller or CPU) coupled to the accelerator core 100. Multiple hardware layers can be sent to different engines and activated simultaneously if data dependencies do not preclude this (i.e., if there is no input from another layer that depends on the output of the previous layer). In at least one embodiment, each engine can have a double buffer for its configuration registers that allows the configuration of a second layer to begin processing as the active layer completes. Once the hardware engine completes its active task, the configuration interface block 116 can interrupt the management processor to report completion, and the management processor will begin the process again. This command execution interrupt flow repeats until inference on the entire network is complete.

[0017] Returning to Figure 1 , the one or more fixed function circuits 104-114 include homogeneous processing units and one or more hardware fault scanners 102. The homogeneous processing units can be parallel processing units used in the respective fixed function circuits 104-114. As shown in Figure 1 , in at least one embodiment, the convolution core 104 includes a set of homogeneous processing units 103 (e.g., multiply-accumulate (MAC) units) and a hardware fault scanner 102 that is an additional homogeneous processing unit (e.g., a MAC unit) similar to the set of homogeneous processing units 103. In at least one embodiment, the activation engine 106 includes a set of homogeneous processing units 105 (e.g., homogeneous look-up table (LUT) logic units) and a hardware fault scanner 102 that is an additional homogeneous processing unit (e.g., a LUT logic unit) similar to the set of homogeneous processing units 105. In at least one embodiment, the LRN engine 110 includes a set of homogeneous processing units 107 (e.g., homogeneous look-up table (LUT) logic units) and a hardware fault scanner 102 that is an additional homogeneous processing unit (e.g., a LUT logic unit) similar to the set of homogeneous processing units 107. In other embodiments, the hardware fault scanner 102 can be used in conjunction with any other type of parallel processing unit in the accelerator core 100 to detect functional faults in those parallel processing units. In at least one embodiment, the hardware fault scanner 102 can be used to detect functional faults in other circuits, such as parallel processing circuits of a GPU. In at least one embodiment, the parallel processing circuits can be a collection of circuits organized in a set of pipelines. The hardware fault scanner 102 can be an additional pipeline similar to the set of pipelines.

[0018] In at least one embodiment, accelerator core 100 includes fixed function circuitry that performs operations corresponding to a neural network layer. Fixed function circuitry includes a set of homogeneous processing units and a hardware fault scanner 102. Hardware fault scanner 102 includes an additional homogeneous processing unit to scan each processing unit in the set for functional faults in a time-multiplexed manner (e.g., in a sequence). In at least one embodiment, hardware fault scanner 102 selectively receives inputs and outputs for each processing unit of a set of homogeneous processing units (e.g., 103, 105, 107) in a sequence. This sequence can be used to scan each processing unit in the set of homogeneous processing units. After scanning each processing unit, the sequence can be repeated. In some cases, the sequence can be repeated based on a number of cycles of an operation (e.g., a strip operation). If an entire strip operation has C cycles, where C is a positive integer, the sequence can be repeated based on C times the number of processing units in the set. Or, the sequence can be repeated based on C times half the number of processing units. Hardware fault scanner 102 (as an additional homogeneous processing unit) generates a second output based on inputs for each processing unit of the set of homogeneous processing units. Hardware fault scanner 102 detects a fault in response to a mismatch between an output for a respective one of the set of homogeneous processing units and the second output generated based on inputs for the respective one of the set of homogeneous processing units. Additional details of hardware fault scanner 102 that selects inputs and outputs in a time-selective manner are described below with reference to Figure 2 FIG. 2A is a block diagram of a processing pipeline 200 having a set of homogeneous processing units 204 and a hardware fault scanner 102 that operates in a time-multiplexed manner, in accordance with at least some embodiments.

[0019] Figure 2 FIG. 2A is a block diagram of a processing pipeline 200 having a set of homogeneous processing units 204 and a hardware fault scanner 102 that operates in a time-multiplexed manner, in accordance with at least some embodiments. Figure 2 In at least one embodiment, hardware fault scanner 102 includes an additional homogeneous processing unit 204(n+1) (labeled “Scanner PU”) and a comparator 206 coupled to additional homogeneous processing unit 204(n+1). Hardware fault scanner 102 also includes a selection circuit (not shown in FIG. 2A) that selects inputs and outputs for homogeneous processing units 204(1)-(n) in a time-multiplexed manner over n cycles. As shown in FIG. 2B, selection circuit selectively receives different inputs and outputs for each homogeneous processing unit 204(1)-(n) in each of the n cycles. Figure 2

[0020] ​In a first cycle 208, the first processing unit 204(1) receives a first input 210. The first input 210 can include feature data and weight data in the context of a neural network. The first input 210 can include other types of data in other processing contexts. For example, the first input 210 can include pixel data for an image processing application. The first processing unit 204(1) processes the first input 210 and generates a first output 212. The selection circuit causes an additional homogeneous processing unit 204(n+1) to receive the first input 210, and the additional homogeneous processing unit 204(n+1) generates a second output 214. The comparator 206 receives the first output 212 from the first processing unit 204(1) and the second output 214 from the additional homogeneous processing unit 204(n+1). The comparator 206 compares the first output 212 and the second output 214 to detect a fault in the first processing unit 204(1). The comparator 206 detects a fault in the first processing unit 204(1) in response to a mismatch between the first output 212 and the second output 214. In at least one embodiment, a mismatch occurs when a first value calculated by the first processing unit 204(1) is different from a second value calculated by the additional homogeneous processing unit 204(n+1). In at least one embodiment, the first value and the second value are integer values. In at least one embodiment, the first value and the second value are floating point values. Since the processing units are homogeneous and calculate the same function, the results should be identical unless a fault occurs. The comparator 206 can output a fault signal 216 in response to the fault detection. In at least one embodiment, the fault signal 216 is output only when a mismatch is detected. In another embodiment, the comparator 206 can output a pass signal, a fail signal, or a single signal that indicates a pass with a first level of the single signal and a fail with a second level of the single signal.

[0021] In a second cycle 218, the second processing unit 204(2) receives a second input 220. The second input 220 can include feature data and weight data. In at least one embodiment, the feature data of the second input 220 is different from the feature data in the first input 210, and the weight data is the same. Alternatively, the feature data can be the same and the weight data can be different for the first input 210 and the second input 220. As described above, the second input 220 can include other types of data in other contexts. The second processing unit 204(2) processes the second input 220 and generates a third output 222. The selection circuitry causes the second input 220 to be received by an additional homogeneous processing unit 204(n+1), and the additional homogeneous processing unit 204(n+1) generates a fourth output 224. The comparator 206 receives the third output 222 from the second processing unit 204(2) and the fourth output 224 from the additional homogeneous processing unit 204(n+1). The comparator 206 compares the third output 222 and the fourth output 224 to detect a fault in the second processing unit 204(2). The comparator 206 detects a fault in the second processing unit 204(2) in response to a mismatch between the third output 222 and the fourth output 224. The comparator 206 can output a fault signal 226 in response to the fault detection. In at least one embodiment, the fault signal 226 is output only when a mismatch is detected. In another embodiment, the comparator 206 can output a pass signal, a fail signal, or a single signal indicating a pass with a first level of the single signal and a fail with a second level of the single signal. The processing described above with respect to the first and second processing units 204(1)-(2) can be repeated until an nth cycle 228.

[0022] In the nth cycle 228, the nth processing unit 204(n) receives an nth input 230. The nth input 230 can include feature data and weight data. Alternatively, the nth input 230 can be other types of data. The nth processing unit 204(n) processes the nth input 230 and generates a fifth output 232. The selection circuitry causes the nth input 230 to be received by an additional homogeneous processing unit 204(n+1) and the additional homogeneous processing unit 204(n+1) generates a sixth output 234. The comparator 206 receives the fifth output 232 from the second processing unit 204(2) and the sixth output 234 from the additional homogeneous processing unit 204(n+1). The comparator 206 compares the fifth output 232 and the sixth output 234 to detect a fault in the nth processing unit 204(n). The comparator 206 detects a fault in the nth processing unit 204(n) in response to a mismatch between the fifth output 232 and the sixth output 234. The comparator 206 can output a fault signal 236 in response to the fault detection. In at least one embodiment, the fault signal 236 is output only when a mismatch is detected. In another embodiment, the comparator 206 can output a pass signal, a fail signal, or a single signal that indicates a pass with a first level of the single signal and a fail with a second level of the single signal.

[0023] In at least one embodiment, the functionality of the selection circuitry can be integrated into a sequence controller that is responsible for providing inputs to each processing unit 204. In at least one embodiment, the selection circuitry is part of a convolution sequence controller (CSC) that is responsible for loading input data (e.g., feature data weight data) from a convolution buffer 126 and sending it to a convolution core 104, as described below with respect to Figure 3 In more detail.

[0024] Figure 3 is a block diagram of a convolution pipeline of a convolution core 300 having a set of homogeneous convolution multiply-accumulate (CMAC) units 304 and a hardware fault scanner 102, in accordance with at least some embodiments. The convolution pipeline of the convolution core 300 includes a plurality of stages, including a convolution DMA stage (CDMA) Figure 3 (not shown in FIG. 1), a convolution buffer stage (CBUF) (e.g., labeled 126), a CSC stage 310, a CMAC stage 320 having a set of CMAC units, and a convolution accumulator stage 330 (CACC).

[0025] The CDMA is a pipeline stage that fetches data from memory for convolution operations and stores it in the convolution buffer 126 in an order required by the convolution core 300.

[0026] CBUF 126 is a stage in the convolution pipeline and contains a specified amount of memory (e.g., SRAM) that buffers input pixel data, input feature data, compressed / uncompressed weight data, etc. from CDMA. The CSCs in CSC stage 310 can read input data from convolution buffer 126. Convolution buffer 126 can include multiple banks that can operate as a logical circular buffer, where new input (input data, weight data) has an incremental entry address, and if the address reaches a maximum address, it wraps to zero and starts incrementing again.

[0027] The CSCs in CSC stage 310 are responsible for loading input data 301 (e.g., feature / pixel data) and weight data 303 from CBUF 126 and sending it to a particular CMAC unit 304 in CMAC stage 320. The CSCs can generate sequences to control the convolution operation, as well as fault detection described in more detail below. For example, the CSCs can generate a pair of sequences packets, including a weight load packet and a data load packet, where each packet can represent a slice operation. In at least one embodiment, a slice operation includes n cycles, where n is a programmable parameter. For each cycle, the CSCs can fetch feature data from CBUF and broadcast it to all MAC units. For each cycle, the CSCs can fetch weight data and send it to a designated MAC unit. Each MAC unit has two weight data buffers: one for active (i.e., remains constant for the entire slice operation) and one for shadow (can be updated if the CSC sends data to the particular MAC unit). The switch between the shadow buffer and the active weight buffer can occur abruptly at the end of a slice operation. The CSCs can execute a feature / pixel load sequence to load input data 301 (feature / pixel data) from CBUF and send it to the appropriate MAC unit 304. The CSCs can execute a weight load sequence to load weights from CBUF and send it to the appropriate MAC unit 304.

[0028] The CMAC stage 320 is a stage of a convolution pipeline for convolution operations. The CMAC stage 320 receives input data 301 and weight data 303 from the CSC and performs multiplication and addition and outputs output 305 representing the result of a convolution accumulator CACC 330. In at least one embodiment, when operating in Winograd mode, the CMAC stage 320 also performs a post-add on the output to convert the result back to a standard activation format. The CMAC stage 320 includes K homogenous MAC units 304. Each MAC unit 304 contains multiple multipliers (e.g., 64 16-bit multipliers for int16 / fp16 operations) and multiple adders (e.g., 72 adders for int16 / fp16, for Winograd post-add). For certain formats (e.g., int8 format), each multiplier and adder can be split into two compute units, where the throughput is doubled, and the output 305 is referred to as a partial sum. In at least one embodiment, the pipeline depth is seven cycles. Alternatively, other pipeline depths can be used.

[0029] The CACC stage 330 is a stage of the convolution pipeline after the CMAC 320. The CACC stage 330 is used to accumulate partial sums from the CMAC 320 and round / saturate the result before sending to another fixed function circuit, such as an activation engine (e.g., SDP). For example, for INT16, the final result accumulated in the CACC 320 is 48 bits, for INT8, the final result accumulated is 34 bits, and the bit width between the CACC 320 and the SDP is 32. For another example, for INT8 and INT16, the result accumulated in the CACC 320 is 32 bits. For precision INT8 and INT16, a rounding and saturation operation can be performed before sending the result to the SDP. The rounding precision can be configured through a field in a register. The CACC stage 330 can include a buffer that can smooth the peak throughput of the convolution pipeline.

[0030] Returning to the CMAC 320 in the convolution pipeline, the hardware fault scanner 102 can have K MAC units 304(1)-(K) (e.g., 32 MAC units) and an additional MAC unit 304(K+1). The convolution operation is controlled according to the sequence generated by the CSC, the input data 301 (feature / pixel data) is broadcast to all MAC units at each cycle, and the weight data 303 remains constant throughout the stripe operation (e.g., up to 64 cycles). The stripe operation is available as a time-multiplexing granularity for the hardware fault scanner 102. For example, since the active weight data remains constant during the stripe operation, to scan all K MAC units, the hardware fault scanner 102 needs the following granularity: K*stripe_lenght / num_scanners. The minimum cycle required for the stripe operation depends on how fast all MAC units can load the “new” weight data, so: min_stripe_length = K*ATOMIC_C / CSC_WEIGHT_BANDWIDTH. In at least one embodiment, for example in INT8, the parameters are: min_stripe_length = 64*64 / 64 = 64, and the corresponding scan interval is: 64*64 / 1 = 4096 cycles. Thus, to shorten the scan interval, K can be reduced, ATOMIC_C can be reduced, the CSC weight can be increased, or the number of scanners can be increased. This can be a tradeoff between the required safety level and the area cost.

[0031] For example, to iterate through all K MAC units 304(1)-(K) for fault scanning, there needs to be a number of cycles C in the stripe operation time K, the number of MAC units divided by the number of scanners num_scanners (e.g., fault scanner granularity = (K*C) / num_scanners). For a stripe operation of 64 cycles and 32 MAC units, the fault scan requires 2048 cycles. For a stripe operation of 64 cycles and 16 MAC units, the fault scan requires 1024 cycles. In at least one embodiment, the weight data remains constant over the entire stripe operation, and to scan all K MAC units, the sequence is repeated based on the number of cycles of the entire stripe operation multiplied by K MAC units 304(1)-(K). In at least one embodiment, the weight data remains constant over the entire stripe operation, and the sequence is repeated based on the number of cycles of the entire stripe operation multiplied by half of the K number of MAC units 304(1)-(K) divided by the number of scanners num_scanners.

[0032] In at least one embodiment, the additional MAC unit 304(K+1) receives input data 301 and weight data 303 for each cycle for one of the MAC units 304. The multiplexer 308 receives outputs from the K MAC units 304(1)-(K) and selects the output 305 corresponding to the MAC unit 304 for which it received input data 301 and weight data 303. The comparator 306 receives the output 307 from the additional MAC unit 304(K+1) and the output 305 and compares the output 307 and the output 307. The comparator 306 generates a fault signal 316 (e.g., pass / fail) in response to a mismatch between the output 305 and the output 307. In at least one embodiment, the additional MAC unit 304(K+1) of the hardware fault scanner 102 can have the same precision as the MAC units 304 because the comparator 306 can perform a bit-by-bit comparison of the output 305 and the output 307. In at least one embodiment, a register can store a value that can be injected into the additional MAC unit 304(K+1) of the hardware fault scanner 102 to test the hardware fault scanner 102. Because the value is injected into the additional MAC unit 304(K+1) of the hardware fault scanner 102 and not the selected MAC unit 304, the comparator 306 generates a fault signal 316 indicating a fault. This process can be performed to confirm that the additional MAC unit 304(K+1) of the hardware fault scanner 102 is functioning properly.

[0033] In at least one embodiment, one or more registers can be used to configure the hardware fault scanner 102. In at least one embodiment, a first register stores a first value (e.g., scanner enable) that enables or disables the hardware fault scanner 102. In at least one embodiment, a second register stores a second value (e.g., scanner start) that indicates which of the K MAC units 304(1)-(K) should be the selected first MAC unit. In at least one embodiment, a third register stores a third value (e.g., fault injection enable) that indicates whether to enable fault injection into the hardware fault scanner 102 to diagnose the hardware fault scanner 102.

[0034] As described above, the hardware fault scanner 102 includes additional MAC units 304(K+1) that are homogeneous to the K MAC units 304(1)-(K) of the convolution kernel 300. In other embodiments, the hardware fault scanner 102 includes processing units that are homogeneous to a set of parallel units in a GPU, CPU, processing device, microcontroller, controller, etc. In other embodiments, the hardware fault scanner 102 includes a set of processing units that are homogeneous to a respective set of processing units. For example, an integrated circuit can include a first set of processing units and a second set of processing units. The fault scanner 102 includes a third set of processing units that are homogeneous to the first and second sets of processing units so that the hardware fault scanner 102 can detect faults in the first or second set of processing units using a single processing unit in a similar manner as described above. In at least one embodiment, the first set of processing units can be organized in a first pipeline and the second set of processing units can be organized in a second pipeline. The fault scanner 102 includes a third pipeline that is homogeneous to the first and second pipelines.

[0035] As described herein, the hardware fault scanner 102 can detect temporary faults and permanent faults. It should be noted that to detect all temporary faults, the number of scanners would be the same as the number of processing units, which is more like a lockstep mechanism. In at least one embodiment, the average fault diagnosis rate per layer of observable faults detected by the hardware fault scanner 102 increases. The fault scanner 102 does not require a pre-determined workload to detect faults and the hardware fault scanner 102 has the same precision as the processing units being scanned. The fault scanner 102 does not impact the performance of the convolution kernel 300 because the hardware fault scanner 102 operates concurrently during the operation of the convolution kernel 300.

[0036] Figure 4 is a flowchart of a method 400 for detecting faults in homogeneous processing units of a processing pipeline, according to at least some embodiments. The method 400 can be performed by processing logic that can comprise hardware, software, firmware, or any combination thereof. In at least one embodiment, the method 400 is performed by the hardware fault scanner 102 of Figure 1 of the accelerator core 100. In at least one embodiment, the method 400 is performed by the hardware fault scanner 102 of Figure 1 of the accelerator core 100. In at least one embodiment, the method 400 is performed by the hardware fault scanner 102 of Figures 1-3 of the accelerator core 100. In at least one embodiment, the method 400 is performed by the hardware fault scanner 102 of

[0037] Returning to Figure 4Method 400 begins with processing logic selectively receiving, by time multiplexing, an input to and an output from each of a set of homogeneous processing units (block 402). The processing logic generates a second output based on each input of the set using an additional homogeneous processing unit (block 404). The processing logic detects a fault responsive to a mismatch between the output of a respective one of the set of homogeneous processing units and the second output generated based on the input to the respective one of the set of homogeneous processing units (block 406); method 400 ends.

[0038] In at least one embodiment, the input comprises feature data and weight data for a convolution operation. In at least one embodiment, the weight data remains constant throughout the strip operation. In at least one embodiment, the sequence is repeated based on a number of cycles of the strip operation multiplied by a number of homogeneous processing units being scanned. In at least one embodiment, the sequence is repeated based on a number of cycles of the strip operation multiplied by half of the number of homogeneous processing units being scanned. Alternatively, the input comprises other types of data processed by the groups of processing units being scanned.

[0039] In at least one embodiment, the processing logic receives a first input to a first processing unit and a first output from the first processing unit. The first output represents a computation by the first processing unit. The processing logic generates a second output using the same first input to an additional processing unit and compares the first output and the second output to determine whether a match exists. The processing logic generates and outputs a fault detection signal responsive to a mismatch of the first output and the second output. In at least one embodiment, the processing logic performs a bit-by-bit comparison of the first output and the second output.

[0040] In at least one embodiment, the processing logic scans a first processing unit in a first cycle and a second processing unit in a subsequent cycle. In the subsequent cycle, the processing logic receives a second input to the second processing unit and a third output from the second processing unit. The third output represents a computation by the second processing unit. The processing logic generates a fourth output using the same second input to an additional processing unit and compares the third output and the fourth output to determine whether a match exists. The processing logic generates and outputs a fault detection signal responsive to a mismatch of the third output and the fourth output. In at least one embodiment, the processing logic performs a bit-by-bit comparison of the third and fourth outputs. In at least one embodiment, the process is repeated for n cycles, where n is the number of processing units in the set being scanned by the hardware fault scanner 102.

[0041] In at least one embodiment, processing logic scans MAC units using additional MAC units as processing units. In another embodiment, processing logic scans LUT logic units using additional LUT logic units. In at least one embodiment, when scanning a group of multiple processing units as a single unit, processing logic scans the group of processing units using an additional group of processing units that are homogeneous with the single unit. In at least one embodiment, when the single unit is a processing pipeline, processing logic scans a group of homogeneous processing pipelines using an additional homogeneous processing pipeline. In at least one embodiment, processing logic resides in an accelerator circuit. In at least one embodiment, processing logic resides in a GPU.

[0042] Figure 5 is a block diagram of a DLA system 500 according to at least some embodiments. DLA system 500 is considered a headless system, where per-unit management of DLA subsystem 502 occurs on a host system processor CPU 504. DLA subsystem 502 includes an interrupt interface 506, a configuration space bus (CSB) interface 508, a primary data bus interface 510 (DBBIF), a secondary data bus interface 512, and the hardware fault scanner 102 described above with respect to Figures 1-4 CPU 504 and DLA subsystem 502 are coupled to system memory 514 (e.g., DRAM). DLA subsystem 502 is coupled to system memory 514 via primary data bus 510. DLA subsystem 502 can be coupled to auxiliary memory, such as SRAM (not shown in Figure 5 It should be noted that DLA system 500 can not include the optional auxiliary memory interface 512, as system memory 514 can consume less power than SRAM when overall system performance is less than a priority. DLA system 500 can more efficiently use system memory 514 as a compute cache.

[0043] Figure 5DLA system 500 represents a more cost-sensitive system than a DLA system with a dedicated controller or coprocessor for per-cell management of DLA subsystem 502. DLA system 500 can be considered a small system model. The small system model can be used for cost-sensitive Internet of Things (IoT) class devices, Artificial Intelligence (AI), and automation-oriented systems that have well-defined tasks where cost, area, and power are the main drivers. Savings in cost, area, and power can be achieved through the configurable resources of DLA subsystem 502. Neural network models can be pre-compiled and optimized for performance, thus reducing the load complexity of larger models. In turn, the reduction in load complexity enables scaled-down DLA implementations where models consume less storage space, and the system software takes less time to load and process. In at least one embodiment, DLA system 500 can execute one task at a time. Alternatively, DLA system 500 can execute multiple tasks at a time. For DLA system 500, context switching of DLA 500 does not burden CPU 504 with maintaining a large number of interrupts from DLA subsystem 502. This eliminates the need for an additional microcontroller, and CPU 504 performs memory allocation and other DLA subsystem management operations. As described herein, DLA subsystem 502 includes hardware fault scanner 102 for detecting faults in fixed function engines as well as other operations of CPU 504.

[0044] Figure 6 is a block diagram of a DLA system 600 according to at least some embodiments. DLA system 600 is considered a headless system where the host system processor CPU 602 delegates high-interrupt frequency tasks to a companion microcontroller 604 coupled to DLA subsystem 502. DLA system 600 is similar to DLA system 500 as indicated by like reference numbers, except that DLA system 600 includes companion microcontroller 604. DLA system 600 can be considered a larger system that features the addition of a dedicated control coprocessor and high-bandwidth SRAM to support DLA subsystem 600. The larger system model can be used in IoT devices that can run multiple tasks simultaneously.

[0045] In some cases, when higher performance and versatility are needed, a DLA system 700 is used Figure 6Large DLA Models. Performance-oriented IoT systems can infer on many different network topologies; therefore, they maintain a high degree of flexibility. In addition, these systems can perform many tasks at once, rather than serializing inference operations, so inference operations do not necessarily consume too much processing power on the CPU 504. To meet these needs, the DLA subsystem 502 includes an auxiliary memory interface 512 coupled to a dedicated high-bandwidth SRAM 612. The SRAM 612 is usable by the DLA subsystem 502 as a cache. The SRAM 612 is also usable by other high-performance computer vision-related components on the system to further reduce access to main system memory 514 (e.g., DRAM). The DLA subsystem 502 enables an interface with a microcontroller 604 (or a dedicated control coprocessor) to limit the interrupt load on the CPU 504. In at least one embodiment, the microcontroller 604 can be a PicoRV32 processor based on RISC-V, an ARM Cortex-M or Cortex-R processor, or other microcontroller design. Using a dedicated coprocessor (microcontroller 604), the host processor (CPU 504) can handle some of the tasks related to managing the DLA subsystem 500. For example, the microcontroller 604 can be responsible for scheduling and fine-grained programming of the DLA hardware, the CPU 504 can still handle some coarse-grained scheduling of the DLA hardware. In addition, the host can continue to be responsible for coarse-grained scheduling on the NVDLA hardware, input-output memory management (IOMMU) mapping of DLA memory accesses as needed, memory allocation of input data, and fixed weight arrays on the DLA subsystem 500, as well as synchronization between other system components and tasks running on the DLA subsystem 502.

[0046] In at least one embodiment, the DLA subsystem 502 is programmable into multiple modes of operation, such as independent mode, fused mode, etc. Each functional block can be configured in independent mode to determine when and what it executes, each block performing its assigned task (similar to an independent layer in a deep learning framework). Independent operation can start and end with the assigned block performing memory-to-memory operations from either main system memory or dedicated SRAM memory. In fused mode, some blocks can be assembled into pipelines. Pipelines can improve performance by bypassing round trips through memory, rather than having blocks communicate with each other through small first-in-first-out (FIFO) queues. For example, a convolution kernel can pass data to a single data point processor, which can pass data to a plane data processor and a cross-channel data processor.

[0047] The technology disclosed herein can be incorporated into any processor that can be used to process neural networks, such as a central processing unit (CPU), a GPU, an intelligent processing unit (IPU), a neural processing unit (NPU), a tensor processing unit (TPU), a neural network processor (NNP), a data processing unit (DPU), a vision processing unit (VPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and the like. Such a processor can be integrated in a personal computer (e.g., a laptop), a data center, an Internet of Things (IoT) device, a handheld device (e.g., a smartphone), a vehicle, a robot, a voice-controlled device, or any other device that performs inference, training, or any other processing of a neural network. Such a processor can be used in a virtualized system, such that an operating system executing in a virtual machine on the system can utilize the processor.

[0048] For example, a processor incorporating the technology disclosed herein can be used to process one or more neural networks in a machine to recognize, classify, manipulate, process, operate on, modify, or navigate a physical object in the real world. For example, such a processor can be used in an autonomous vehicle (e.g., a car, a motorcycle, a helicopter, a drone, an airplane, a boat, a submarine, a delivery robot, etc.) to move the vehicle in the real world. Further, such a processor can be used in a robot in a factory to select a component and assemble the component into an assembly.

[0049] For example, a processor incorporating the technology disclosed herein can be used to process one or more neural networks to recognize one or more features in an image, or to alter, generate, or compress an image. For example, such a processor can be used to enhance an image rendered using rasterization, ray tracing (e.g., using NVIDIA RTX), and / or other rendering techniques. In another example, such a processor can be used to reduce the amount of image data transmitted over a network (e.g., the Internet, a mobile telecommunication network, a WIFI network, and any other wired or wireless network system) from a rendering device to a display device. Such a transmission can be used to stream image data from a server or data center in the cloud to a user device (e.g., a personal computer, a video game console, a smartphone, other mobile device, etc.) to enhance a service that streams images, such as NVIDIA GeForce Now (GFN), Google Stadia, and the like.

[0050] For example, a processor incorporating the technology disclosed herein can be used to process one or more neural networks for any other type of application that can utilize a neural network. For example, such an application can involve translating languages, recognizing and negating sounds in audio, detecting anomalies or defects in the production of goods and services, monitoring living and non-living things, medical diagnosis, making decisions, and the like.

[0051] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrated embodiments are shown in the drawings and described above in detail. It should be understood, however, that there is no intent to limit the application to the particular forms disclosed but to the contrary this application is to cover all modifications, alternatives, and equivalents falling within the spirit and scope of the application as defined by the appended claims.

[0052] In the context of describing the disclosed embodiments (especially in the context of the following claims), the use of the term "a" and "an" and "the" and similar referents should be interpreted to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms "comprising," "having," "including," and "containing" are to be construed as open-ended terms (i.e., meaning "including, but not limited to,") "connected" is to be construed as partly or fully containing, connecting, or coupling together, even if used in the context of physical or mechanical connection, unless otherwise specifically stated herein. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated in the specification as if it were individually recited herein. The use of the term "set" (e.g., "a set of items") or "subset," unless otherwise noted or contradicted by context, should be construed as a non-empty set containing one or more members. Additionally, the term "subset" of a corresponding set does not necessarily denote a proper subset; that is, the subset and the corresponding set can be equal.

[0053] Unless specifically stated or otherwise clear from context, connective language, such as "at least one of A, B, and C," or "at least one of A, B, and C," is used to present one or more options for what can be included in a set of items, terms, etc. For example, in an illustrative example with a set of three members, the connective phrase "at least one of a, B, and C" and "at least one of a, B, and C" refers to any one of the following sets: {a}, {B}, {C}, {a, B}, {a, C}, {B, C}, {a, B, C}. Thus, this connective language typically does not mean that at least one of A, at least one of B, and at least one of C must be present. Further, unless otherwise noted or clear from context, the term "plurality" denotes a plural state (e.g., "a plurality of items" denotes multiple items). In at least one embodiment, the number of items in a plurality is at least two, but can be more when explicitly or implicitly indicated by the context

[0054] The operations of a process described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. In at least one embodiment, processes such as those described herein (or variations and / or combinations thereof) are performed under the control of one or more computer systems configured with executable instructions (e.g., computer-executable instructions, one or more computer programs or one or more applications) to cause the computer systems to carry out operations to implement the processes described herein. In at least one embodiment, the computer systems are configured with the executable instructions to cause the computer systems to perform the operations to implement processes described herein collectively on one or more processors of the computer systems by hardware or combinations thereof. In at least one embodiment, the computer-executable instructions are stored on a computer-readable storage medium of the computer system. In at least one embodiment, the computer system is configured to read the computer-executable instructions from the computer-readable storage medium and execute the computer-executable instructions to cause the computer system to carry out the operations described herein. In at least one embodiment, the computer system is configured to read the computer-executable instructions from the computer-readable storage medium and execute the computer-executable instructions to cause the computer system to carry out the operations described herein.

[0055] Accordingly, in at least one embodiment, computer systems are configured to implement one or more services that individually or collectively perform operations of processes described herein, and these computer systems are configured with suitable hardware and / or software to perform the operations. Moreover, a computer system that implements at least one embodiment of the present disclosure is a single device, and in another embodiment, is a distributed computer system including multiple devices that operate differently such that the distributed computer system performs operations described herein, and a single device does not perform all of the operations.

[0056] The use of any and all examples, or exemplary language (e.g., "for instance") provided herein is intended merely to better illuminate embodiments of the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.

[0057] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein, irrespective of any disclosure restriction.

[0058] In the description and claims, the terms "coupled" and "connected," along with derivatives thereof, can be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, "connected" or "coupled" is used to indicate that two or more elements are in direct or indirect physical or electrical contact with each other. "Coupled" can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.

[0059] Unless specifically stated otherwise, it can be appreciated that throughout the specification terms such as "processing," "computing," "calculating," "determining," or the like, refer to the action and / or processes of a computer or computing system, or similar electronic device, that manipulates and / or transforms data represented as physical quantities (e.g., electronic quantities) within the computer's registers and / or memories into other data similarly represented as physical quantities within the computer's memories, registers or other such information storage, transmission or display devices.

[0060] In a similar manner, the term "processor" can refer to any device or portion of a device that processes electronic data, from registers and / or memory to transform that electronic data into other electronic data that can be stored in registers and / or memory. As a non-limiting example, a "processor" can be a CPU or GPU. A "computing platform" can include one or more processors. As used herein, a "software" process can include, for example, software and / or hardware entities that perform work over time, such as tasks, threads, and intelligent agents. Also, each process can refer to multiple processes for sequentially or concurrently executing instructions, continuously or intermittently. In at least one embodiment, the terms "system" and "method" are used interchangeably herein as long as the system can embody one or more methods and the method can be viewed as a system.

[0061] In this document, reference can be made to obtaining, acquiring, receiving, or inputting analog or digital data into a subsystem, computer system, or computer-implemented machine. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog and digital data can be accomplished in a variety of ways, such as by receiving the data as a parameter to a function call or to a call to an application programming interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting the data via a serial or parallel interface. In at least one embodiment, the process of obtaining, acquiring, receiving, or inputting analog or digital data can be accomplished by transmitting the data from a providing entity to an acquiring entity via a computer network. In at least one embodiment, reference can also be made to providing, outputting, transmitting, sending, or presenting analog or digital data. In various examples, the process of providing, outputting, transmitting, sending, or presenting analog or digital data can be accomplished by transmitting the data as an input or output parameter to a function call, a parameter to an application programming interface, or an interprocess communication mechanism.

[0062] Although the description herein sets forth example embodiments of the technology, other architectures can be used to implement the described functionality, and are intended to be within the scope of the present disclosure. Moreover, although specific allocations have been defined in the description, alternative embodiments can allocate and partition various functions and responsibilities differently, and are intended to be within the scope of the present disclosure.

[0063] Moreover, although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claims.

Claims

1. An integrated circuit, comprising: The accelerator core consists of multiple homogeneous processing units used to perform neural network operations; as well as The fault scanning circuit of the accelerator core includes additional homogeneous processing units for testing the plurality of homogeneous processing units while they are performing the operation of the neural network, wherein the fault scanning circuit is used for: Selectively receive the input and output of each of the plurality of homogeneous processing units in the sequence; Using the additional homogeneous processing unit, a second output is generated based on the input of each of the plurality of homogeneous processing units; A fault is detected in response to a mismatch between the output of a corresponding one of the plurality of homogeneous processing units and a second output generated based on the input of the corresponding one of the plurality of homogeneous processing units; as well as After testing all of the plurality of homogeneous processing units, the sequence is repeated at least once.

2. The integrated circuit according to claim 1, wherein the fault scanning circuit further comprises: A comparator coupled to the additional homogeneous processing unit; as well as Selection circuitry is configured to enable the additional homogeneous processing unit to selectively receive the input and the output of each of the plurality of homogeneous processing units in the sequence.

3. The integrated circuit of claim 1, wherein the homogeneous processing unit is a homogeneous multiplication-accumulation MAC unit of the convolution kernel, wherein the additional homogeneous processing unit is an additional homogeneous MAC unit of the convolution kernel, and wherein the input of each of the plurality of homogeneous processing units includes feature data and weight data.

4. The integrated circuit of claim 3, wherein the weight data remains constant throughout the strip operation, and wherein the sequence is repeated based on the number of cycles of the entire strip operation multiplied by the number of the plurality of homogeneous processing units.

5. The integrated circuit of claim 3, wherein the weight data remains constant throughout the strip operation, and wherein the sequence is repeated based on half the number of cycles of the entire strip operation multiplied by the number of the plurality of homogeneous processing units.

6. The integrated circuit of claim 1, wherein the homogeneous processing unit is a homogeneous lookup table (LUT) logic unit of a single data point processor (SDP) or a cross-channel data processor (CDP), wherein the additional homogeneous processing unit is an additional homogeneous LUT logic unit of the SDP or the CDP.

7. The integrated circuit of claim 1, wherein the homogeneous processing unit is a homogeneous processing pipeline, and wherein, The additional homogeneous processing unit is an additional homogeneous processing pipeline.

8. The integrated circuit of claim 1, wherein the integrated circuit is an accelerator circuit.

9. The integrated circuit of claim 1, wherein the integrated circuit is a graphics processing unit (GPU).

10. An accelerator circuit, comprising: Memory interface block; as well as A fixed-function circuit, coupled to the memory interface block, for performing operations corresponding to the neural network layer, wherein the fixed-function circuit includes: Multiple homogeneous processing units; and A fault scanning circuit, including additional homogeneous processing units and a selection circuit, wherein the fault scanning circuit is used to scan for functional faults in each of the plurality of homogeneous processing units in the sequence, wherein, in the first loop of the sequence: The selection circuit enables the fault scanning circuit to receive the first input and first output of the first processing unit of the plurality of homogeneous processing units when the plurality of homogeneous processing units perform the operation corresponding to the neural network layer; The additional homogeneous processing unit generates a second output based on the first input; and The fault scanning circuit detects a first functional fault in response to a mismatch between the first output and the second output; The sequence is repeated at least once after all of the plurality of homogeneous processing units have been tested.

11. The accelerator circuit according to claim 10, wherein, In order to perform a functional fault scan for each of the plurality of homogeneous processing units in the sequence, the fault scanning circuit is used to: The additional homogeneous processing unit selectively receives the input and output of each of the plurality of homogeneous processing units in the sequence; Using the additional homogeneous processing unit, a third output is generated based on the input of each of the plurality of homogeneous processing units; as well as A fault is detected in response to a mismatch between the output of a corresponding one of the plurality of homogeneous processing units and a third output generated based on the input of a corresponding one of the plurality of homogeneous processing units.

12. The accelerator circuit of claim 10, wherein the fault scanning circuit further comprises: A comparator coupled to the additional homogeneous processing unit, the comparator being used to compare the first output and the second output to detect a mismatch; as well as The selection circuit is configured to enable the additional homogeneous processing unit to selectively receive the input and the output of each of the plurality of homogeneous processing units in the sequence.

13. The accelerator circuit of claim 10, wherein the homogeneous processing unit is a homogeneous multiplication-accumulation MAC unit of the convolution kernel, wherein the additional homogeneous processing unit is an additional homogeneous MAC unit of the convolution kernel, and wherein the input of each of the plurality of homogeneous processing units includes feature data and weight data.

14. The accelerator circuit of claim 13, wherein the weight data remains constant throughout the strip operation, and wherein the sequence is repeated based on the number of cycles of the entire strip operation multiplied by the number of the plurality of homogeneous processing units.

15. The accelerator circuit of claim 13, wherein the weight data remains constant throughout the strip operation, and wherein the sequence is repeated based on the number of cycles of the entire strip operation multiplied by half the number of the plurality of homogeneous processing units.

16. The accelerator circuit of claim 10, wherein the homogeneous processing unit is a homogeneous lookup table (LUT) logic unit of a single data point processor (SDP) or a cross-channel data processor (CDP), wherein the additional homogeneous processing unit is an additional homogeneous LUT logic unit of the SDP or the CDP.

17. A method comprising: The fault scanning circuit of the accelerator circuit selectively receives the inputs and outputs of each of the plurality of homogeneous processing units of the accelerator circuit in the sequence to test the plurality of homogeneous processing units while they are performing neural network operations. The fault scanning circuit generates a second output based on the input of each of the plurality of homogeneous processing units using an additional homogeneous processing unit of the accelerator circuit; as well as A fault is detected by the fault scanning circuit in response to a mismatch between the output of a corresponding one of the plurality of homogeneous processing units and a second output generated based on the input of a corresponding one of the plurality of homogeneous processing units; and The fault scanning circuit repeats the sequence at least once after testing all of the plurality of homogeneous processing units.

18. The method according to claim 17, wherein, The operation is a convolution operation, and the input of each of the plurality of homogeneous processing units includes feature data and weight data for the convolution operation.

19. The method according to claim 18, wherein, The weight data remains constant throughout the entire strip operation, wherein the sequence is repeated based on the number of cycles of the entire strip operation multiplied by the number of the plurality of homogeneous processing units.

20. The method according to claim 18, wherein, The weight data remains unchanged throughout the entire strip operation of the convolution operation, wherein the sequence is repeated based on the number of cycles of the entire strip operation multiplied by half of the plurality of homogeneous processing units.

Images