An NPU array architecture with online fault detection and PE remapping capabilities and its implementation method.

Abstract

This invention discloses an NPU array architecture and its implementation method with online fault detection and PE remapping capabilities. The architecture includes a main controller, an on-chip configuration and data bus, and functional areas. These functional areas include an online fault detection section, a health status management section, a PE remapping section, a reconfigurable computing array section, a fault recovery section, and a degraded operation section. The technical solution disclosed in this application improves the long-term operating capability of NPUs in high-reliability scenarios such as industrial, automotive, aerospace, and energy IoT applications. This solution reduces hardware overhead and power consumption through sampling detection and backup PE remapping, making it more suitable for edge-side NPU chips.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an NPU array architecture and its implementation method that has online fault detection and PE remapping capabilities. Background Technology

[0002] NPU (Neural Processing Unit) computing arrays typically consist of a large number of PEs (Processing Units), each of which performs multiplication, accumulation, local registers, and partial result output. For neural network operators such as convolution, matrix multiplication, fully connected layers, and attention calculations, the NPU requires a large number of PEs to work continuously in parallel. Therefore, the PE array is the most densely packed and critical computing resource in the NPU.

[0003] As process dimensions shrink and application scenarios become more complex, NPU arrays face increasing reliability risks. For example, in industrial edge devices, NPUs may be exposed to high temperatures, strong electromagnetic interference, and power fluctuations for extended periods; in automotive or aerospace applications, NPUs may be affected by single-event upsets, radiative disturbances, or aging effects; and in energy IoT edge devices, NPUs may require long-term online operation with low power consumption, and voltage drops or clock drift may cause computational errors.

[0004] Neural network inference typically has a certain degree of fault tolerance, but inference errors in critical scenarios can have serious consequences. For example, errors in the output of a fault diagnosis model may lead to false alarms or missed alarms; errors in the output of an object detection model may affect safety judgments; and errors in the output of a measurement and identification model may affect business results.

[0005] High-reliability NPUs not only need to pass testing during the manufacturing stage, but also need to possess online detection and self-recovery capabilities during operation. Current technologies address the lack of online detection, fault location, fault isolation, and automatic remapping capabilities in high-reliability applications such as long-term operation, complex temperature and voltage fluctuations, industrial fields, automotive, aerospace, and energy IoT, when NPUs experience transient, intermittent, or permanent failures in the internal power supply (PE) array. The main solutions fall into the following categories:

[0006] (1) NPU array screening scheme based on factory testing.

[0007] This approach tests the PE array during the chip production testing phase to screen out defective chips. Its advantage is a mature testing process; its disadvantage is that it can only detect manufacturing defects and cannot handle aging failures, transient failures, and intermittent failures that occur during chip operation.

[0008] (2) Storage protection scheme based on parity check or ECC.

[0009] This approach is typically used to protect weight caches, input caches, or on-chip SRAM, and can detect or correct stored data errors. Its advantage is its effectiveness against storage errors; its disadvantage is that it cannot directly determine whether a calculation error has occurred within the PE multiply-accumulate logic itself.

[0010] (3) Error detection scheme based on repeated calculation or dual-mode redundancy.

[0011] This scheme detects errors by having two computing units perform the same task and comparing the results. Its advantages are strong detection capabilities; its disadvantages are high hardware overhead, low utilization of computing resources, and the inability to continue operation even if errors are detected if a remapping mechanism is lacking.

[0012] (4) Array repair scheme based on spare rows or spare columns.

[0013] This scheme reserves spare rows and columns in memory or a regular array, and replaces them when a fault is detected. Its advantage is that it is suitable for regular structures; its disadvantage is that the NPU's PE array not only has computational logic, but also complex connections such as data input, partial sum accumulation, and output routing, and simple replacement of spare rows and columns is difficult to meet the mapping requirements of various operators.

[0014] (5) Degradation scheme based on software shielding of fault units.

[0015] This approach uses software to record faulty areas and avoids faulty units in subsequent task scheduling. Its advantages include flexible implementation; its disadvantages include slow fault detection, high switching latency, and the need for deep software intervention, making it difficult to meet the requirements of real-time inference and low-power scenarios.

[0016] Since existing NPUs mainly rely on factory testing, they can only guarantee that the chip meets functional requirements at the time of manufacture. After the chip has been running for a period of time, if the PE fails due to aging, voltage disturbances, abnormal temperature, or soft errors, traditional solutions cannot detect it in time, causing erroneous results to continue to propagate to subsequent network layers.

[0017] Because existing NPUs lack PE-level health status recording and fault location mechanisms, even if the system detects an anomaly in the final inference result, it is difficult to determine whether the fault originates from the input data, weight data, cache, interconnect, or the multiply-accumulate logic of a specific PE. Therefore, the system cannot mask or replace specific faulty PEs.

[0018] Because existing NPUs lack a modifiable mapping table from logical PEs to physical PEs, computational tasks are typically mapped to a fixed row, column, or cluster of PEs in the array. When a PE permanently fails, computational data may still be sent to that PE, causing subsequent computations to continue to fail. Even if idle or spare PEs exist in the array, it is difficult to automatically take over the tasks on that PE.

[0019] Since repetitive computation and dual-mode redundancy schemes require additional computing resources, performing complete repetitive computation on all computational tasks would significantly reduce throughput and increase power consumption, making them unsuitable for edge-side low-power NPUs. Relying solely on periodic software testing results in long detection cycles, slow response times, and an inability to handle online faults promptly.

[0020] In summary, existing solutions lack a complete hardware architecture capable of performing online detection, accurately locating faulty physical exciters (PEs), updating PE health status, automatically modifying the mapping relationship between logical PEs and physical PEs during NPU operation, and taking over faulty computing tasks through backup or healthy PEs.

[0021] Therefore, there is an urgent need for a highly reliable NPU array architecture with online fault detection and PE remapping capabilities to solve the following technical problems:

[0022] (1) Solve the problem that existing NPUs cannot detect PE calculation errors in a timely manner during operation. Traditional NPUs are mostly tested during the factory testing phase. If a PE is erroneous due to aging, voltage disturbance, abnormal temperature or radiation reversal during operation, the system has difficulty detecting it in a timely manner.

[0023] (2) Solve the problem of coarse granularity in existing NPU fault location. Existing solutions can often only detect abnormal model output, but cannot pinpoint which row, column or PE is faulty, resulting in the inability to perform precise masking and replacement.

[0024] (3) Solve the problem of a single PE failure causing the entire NPU array to fail. In traditional NPU arrays, if a PE experiences a permanent failure and there is no backup PE or routing bypass mechanism, the entire computing array may become unusable.

[0025] (4) Solve the problem of incompatibility between fault detection and normal computation during NPU online inference. If the entire array is paused for self-testing, it will affect the real-time inference task; if no detection is performed at all, long-term reliability cannot be guaranteed.

[0026] (5) Solve the problem of the lack of a dynamic mapping mechanism from logical PE to physical PE in the existing NPU. Even if spare computing resources are reserved in the chip, if the task allocation and data routing cannot be modified in the hardware, the spare resources will be difficult to take over the computing tasks of the faulty PE.

[0027] (6) Address the issue of the lack of a degradation operation mechanism for the NPU in high-reliability scenarios. When the number of failures exceeds the number of backup PEs, the system should be able to reduce parallelism and continue operating instead of failing directly. Summary of the Invention

[0028] The purpose of this application is to provide a highly reliable NPU array architecture with online fault detection and PE remapping capabilities, enabling the NPU to detect PE faults, record PE health status, isolate faulty PEs, remap the computing tasks of faulty PEs to standby PEs or healthy PEs, and continue to execute inference tasks in a way that reduces throughput when necessary.

[0029] One aspect of this application discloses an NPU array architecture with online fault detection and PE remapping capabilities, including a main controller, an on-chip configuration and data bus, and a functional area. The functional area includes an online fault detection section, a health status management section, a PE remapping section, a reconfigurable computing array section, a fault recovery section, and a degraded operation section.

[0030] The main controller sends inference tasks, model parameters, detection frequency, error threshold, remapping strategy and degraded operation strategy to the functional area through the on-chip configuration and data bus. The main controller receives the fault status and operation status returned by the functional area through the on-chip configuration and data bus.

[0031] The online fault detection section is used to detect PE calculation errors during NPU operation. The online fault detection section includes an idle cycle self-test unit, a sentinel test vector generator, a sampling repeated calculation unit, and a result verification and comparison unit. The online fault detection section is directly connected to the reconfigurable computing array section.

[0032] The health status management section is used to store the health status and fault history of each PE. The health status management section includes a PE health status table, a fault counter, an error threshold register, and a fault classification unit.

[0033] The PE remapping part is used to modify the computing task allocation relationship after a faulty PE is detected. The PE remapping part includes a logical PE to physical PE mapping table, a PE remapping controller, a spare PE pool, and a mapping update state machine.

[0034] The reconfigurable computing array section is used to ensure that input data, weight data and accumulation results can be transmitted according to the new mapping path after the mapping relationship changes. The reconfigurable computing array section includes a normal PE array, a backup PE, a reconfigurable data routing network, a weight distribution network and a partial reduction network.

[0035] The fault recovery section is used to process the current task after fault detection. The fault recovery section includes a security checkpoint manager and a compute block re-execution controller.

[0036] The degradation operation section is used to reduce the array parallelism when the spare PE is insufficient, and the degradation operation section includes a degradation operation controller;

[0037] The fault recovery section works in conjunction with the online fault detection section, the health status management section, and the PE remapping section to process the current task. If the fault is determined to be a transient fault, the fault recovery section re-executes the affected computational block from the nearest network layer boundary, operator boundary, or tile boundary through the security checkpoint manager and the computational block re-execution controller. If the fault is determined to be a permanent fault, the fault recovery section waits for the PE remapping and route reconstruction to be completed before re-executing the affected computational block to prevent erroneous results from continuing to propagate to subsequent network layers.

[0038] According to some embodiments, the online fault detection part collects PE output results, local accumulation results or self-test results during the execution of the inference task; the online fault detection part sends fault events to the health status management part, and the fault events include abnormal PE number, error type, error occurrence time, corresponding calculation block and / or tile number.

[0039] During NPU operation, the online fault detection section uses any one or any combination of the following four detection methods to detect PE calculation errors:

[0040] Sentinel Test Vector Detection: The steps of the sentinel test vector detection include: the sentinel test vector generator generating a set of input data and weight data as sentinel test vectors, wherein the correct output results of the input data and the weight data are known; the system sends the sentinel test vectors to a designated PE or PE cluster during NPU idle periods, task switching intervals, or low-load periods to obtain corresponding output results, and compares the output results with the correct output results; if the comparison results are inconsistent, the corresponding PE is marked as a suspected fault;

[0041] Sampling repeated calculation: The sampling repeated calculation steps include the system extracting a portion of the calculation blocks from the normal inference task, having the target PE execute it once, and having the backup PE or an adjacent healthy PE execute the same calculation block at the same time; the result verification and comparison unit compares the two output results, and if the difference between the two output results exceeds the allowable threshold, the target PE is recorded as abnormal;

[0042] Local result verification: The steps of local result verification include the system generating a checksum, parity check or CRC check value for the local output result; if the check value of a certain PE output does not match the prediction range, or the check value of a certain PE output is inconsistent with the relationship of adjacent calculation results, an anomaly record is triggered.

[0043] Idle Cycle Self-Test: The steps of the idle cycle self-test include performing a short test on certain PEs when they are temporarily not used in the current layer or the current tile. The short test includes checking whether the multiplier, adder, accumulator, register and input selector are normal.

[0044] According to some embodiments, the health status management section performs statistics, retesting, and classification of fault events; the health status management section determines whether the anomaly is a transient fault, intermittent fault, or permanent fault based on the error threshold, fault count, retesting results, and PE historical status, and writes the determination result into the PE health status table; the PE health status table simultaneously provides status information to the PE remapping section, the fault recovery section, and the degraded operation section, enabling the PE remapping section, the fault recovery section, and the degraded operation section to perform subsequent processing based on the same PE health information;

[0045] The PE health status table records the health status of each PE; each PE corresponds to one table entry, which includes several fields, including PE location, current status, error count, most recent error time, retest result, whether it is allowed to participate in the calculation, whether it has been remapped, and the corresponding spare PE identifier; the PE health status table is stored in on-chip SRAM, register array, or dedicated status memory; the main controller reads the PE health status table for system diagnosis and maintenance; the PE remapping controller directly reads the PE health status table to generate a remapping scheme.

[0046] According to some embodiments, the current state includes:

[0047] Health status, which indicates that the PE can participate in the calculation normally;

[0048] Suspected fault status, which indicates that the PE has encountered an error, but it has not yet been confirmed whether it is a permanent fault;

[0049] Transient fault state, which indicates that the PE has one or a few errors, and returns to normal after retesting;

[0050] Intermittent fault state, which indicates that the PE fails multiple times at different times, but not every time it fails;

[0051] Permanent Fault Status: The permanent fault status indicates that the PE has failed multiple detections or self-tests and is therefore deemed unusable.

[0052] The shielded state indicates that the PE has been removed from the compute array mapping and no longer receives normal tasks.

[0053] According to some embodiments, the steps by which the health status management section determines whether an anomaly is a transient fault, an intermittent fault, or a permanent fault based on an error threshold, fault count, retest results, and PE historical status include:

[0054] If a PE (Preinstallation Equipment) experiences an error only once and passes the retest, it is determined to be a transient fault.

[0055] If a PE repeatedly fails to produce an error over a period of time, but the retest results are unstable, it is determined to be an intermittent fault.

[0056] If a PE fails to perform a self-test multiple times in a row, or if the same multiply-accumulate path continues to output errors, it is considered a permanent fault.

[0057] According to some embodiments, the PE remapping part queries the location of the faulty PE according to the PE health status table. The PE remapping part combines the backup PE pool and the logical PE to physical PE mapping table to generate a new mapping relationship. When there is an available backup PE or other healthy PE, the PE remapping controller updates the mapping table and issues a new routing configuration to the reconfigurable data routing network, so that the input data, weight data, and part of the result originally sent to the faulty PE are sent to the backup PE or healthy PE. The reconfigurable computing array part continues to execute the inference task according to the updated mapping relationship, realizing faulty PE isolation and backup PE takeover.

[0058] The reconfigurable data routing network is used to reconnect input data, weight data, and some output paths after the PE mapping relationship changes; the reconfigurable data routing network includes a multiplexer array, a crossbar switch, a segmented bus, or a lightweight on-chip network.

[0059] According to some embodiments, the backup PE pool is used to provide redundant computing resources that can take over failed PE tasks, and the backup PEs are arranged in any of the following ways:

[0060] Row-level backup PE arrangement, wherein each row of PE array reserves one or more backup PEs, and when a PE in a row fails, the backup PE in that row takes over;

[0061] Column-level spare PE arrangement, wherein the column-level spare PE arrangement reserves spare PEs for each column of PE array;

[0062] Cluster-level spare PE arrangement, wherein a small number of spare PEs are reserved in each PE cluster;

[0063] A global backup PE pool is arranged by setting up several backup PEs at the edge of the array, which are connected to multiple array areas by the reconfigurable data routing network.

[0064] According to some embodiments, the degraded operation section serves as a backup processing path and is activated when there are insufficient backup PEs, routes cannot be bypassed, or the number of faults exceeds the repair capacity. The degraded operation section reduces the array parallelism, reduces the number of tiles executed simultaneously, reduces the number of parallel output channels, or converts some tasks to serial execution based on the PE health status table and remapping failure results.

[0065] The security checkpoint manager is used to save the key states of the current inference task at network layer boundaries, operator boundaries, or tile boundaries. The key states include the current layer number, the current tile number, the input address, the weight address, the output address, the accumulation state, and the mapping table version.

[0066] Another aspect of this application provides an online fault detection and PE remapping method for NPU arrays, comprising the following steps:

[0067] The NPU receives inference tasks and loads the current model parameters;

[0068] The main controller sets a reliability control strategy based on the application reliability level. The reliability control strategy includes online detection frequency, error threshold, and whether to enable backup PE.

[0069] During the inference execution process, online fault detection is continuously performed;

[0070] If no anomaly is detected, the computing array continues to execute according to the original mapping; if an anomaly is detected in the output of a PE, the fault counter records the error event, and the fault classification unit performs retesting and classification.

[0071] If the fault is transient, the compute block re-execution controller will re-execute the affected tile from the most recent security checkpoint; if the fault is intermittent, the scheduling priority of the faulty PE will be reduced, its computing power usage rights will be retained, and its subsequent running status will be continuously monitored; if the fault is permanent, PE remapping and route reconstruction will be performed.

[0072] After PE remapping and route reconstruction are completed, the system re-executes the affected computation blocks and continues to execute subsequent network layers; if there are insufficient spare PEs, the degraded controller reduces the array parallelism to ensure that the inference task continues to be completed.

[0073] According to some embodiments, the steps of PE remapping and route reconstruction include:

[0074] The online fault detection unit detects operational anomalies in real time and reports PE anomaly events to the health status management unit.

[0075] After receiving an abnormal signal, the health status management section accumulates the corresponding PE error count and determines the fault level. If the PE abnormal event is a permanent fault, it sends the permanent fault PE location information to the PE remapping controller.

[0076] After receiving the fault notification, the PE remapping controller sends a safety switch boundary request to the safety checkpoint manager. The safety checkpoint manager then issues an instruction to the reconfigurable computing array to suspend the operation of the computing blocks currently affected by the fault.

[0077] During the safe window period when the computing block is suspended, the PE remapping controller completes the query of backup PE resources, synchronously updates the logical PE to physical PE mapping table, and issues a new routing configuration instruction to the reconfigurable data routing network.

[0078] The reconfigurable data routing network reconfigures the data path based on the new configuration, shields the data transmission channel of the faulty PE, enables the idle standby PE to take over the computing task, and pushes the path switching completion signal to the reconfigurable computing array section.

[0079] The technical solution disclosed in this application continuously monitors the output results of the PE (Programmer) during inference operation through an online fault detection component. This enables the detection of aging faults, transient disturbances, and permanent failures that occur during operation, rather than relying solely on factory testing. Since the fault detection results are written to the PE health status table, the system can locate specific PEs and record fault history, thus providing a basis for subsequent remapping. Simultaneously, the technical solution disclosed in this application uses a logical PE-to-physical PE mapping table and a PE remapping controller to prevent neural network computation tasks from being fixedly bound to a specific physical PE. When a PE experiences a permanent failure, the system can transfer the task from the original logical PE to a backup PE or other healthy PEs, and synchronously modify the input, weights, and output paths through a reconfigurable data routing network. Therefore, the failure of a single PE will not cause the entire NPU array to fail. The technical solution disclosed in this application prevents fault results from propagating to subsequent network layers through security checkpoints and computation block re-execution mechanisms. For transient faults, the system can recover by re-executing the current computation block; for permanent faults, the system first remaps and then re-executes the affected computation block; in cases where backup PEs are insufficient, the system can enter a degraded operation mode, sacrificing some throughput to ensure the continued availability of inference functionality.

[0080] Compared to traditional NPUs, the technical solution disclosed in this application significantly improves the long-term operating capability of NPUs in high-reliability scenarios such as industrial, automotive, aerospace, and energy IoT applications. Compared to full dual-mode redundancy solutions, the technical solution disclosed in this application reduces hardware overhead and power consumption through sampling detection and backup PE remapping, making it more suitable for edge-side NPU chips. Attached Figure Description

[0081] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 This diagram illustrates an NPU array architecture with online fault detection and PE remapping capabilities according to an example embodiment.

[0083] Figure 2 The diagram illustrates the core structure of PE online fault detection and remapping according to an example embodiment.

[0084] Figure 3 A flowchart illustrating an online fault detection and PE remapping process for an NPU array according to an example embodiment is shown.

[0085] Figure 4 A flowchart illustrating PE remapping and route reconstruction according to an example embodiment is shown. Detailed Implementation

[0086] The embodiments of this application will now be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0087] Those skilled in the art should understand that the following specific embodiments or implementation methods are a series of optimized configurations listed in this application to further explain the specific application content. These configuration methods can be combined or used in conjunction with each other, unless this application explicitly states that some or a specific embodiment or implementation method cannot be associated with or used in conjunction with other embodiments or implementation methods. Furthermore, the following specific embodiments or implementation methods are only considered as optimized configurations and are not intended to limit the scope of protection of this application.

[0088] Example 1:

[0089] Figure 1 This diagram illustrates an NPU array architecture with online fault detection and PE remapping capabilities according to an example embodiment.

[0090] like Figure 1As shown, an NPU array architecture with online fault detection and PE remapping capabilities includes a main controller 101, an on-chip configuration and data bus 102, and functional areas. The functional areas include an online fault detection section 103, a health status management section 104, a PE remapping section 105, a reconfigurable computing array section 106, a fault recovery section 107, and a degraded operation section 108. The main controller 101 distributes inference tasks, model parameters, detection frequencies, error thresholds, remapping strategies, and degraded operation strategies to each functional area via the on-chip configuration and data bus 102, and receives fault and operational statuses returned by each functional area.

[0091] The online fault detection section 103 includes an idle cycle self-test unit, a sentinel test vector generator, a sampling repeat calculation unit, and a result verification and comparison unit. The idle cycle self-test unit performs a self-test when the PE is idle; the sentinel test vector generator inputs preset test data to the target PE; the sampling repeat calculation unit performs repeat calculations on some inference calculations; and the result verification and comparison unit compares the normal calculation results, the self-test results, and the repeat calculation results, and outputs a fault event when an anomaly is detected.

[0092] The health status management section 104 includes an error threshold register, a fault counter, a fault classification unit, and a PE health status table. The error threshold register stores the threshold for determining the fault type; the fault counter records the number of errors for each PE; the fault classification unit determines the fault type based on the number of errors, retest results, and thresholds; and the PE health status table stores the health status, suspected fault status, transient fault status, intermittent fault status, permanent fault status, or masked status for each PE.

[0093] The PE remapping section 105 includes a PE remapping controller, a logical PE-to-physical PE mapping table, a spare PE pool, and a mapping update state machine. The PE remapping controller reads the PE health status table to determine the faulty PEs that need to be replaced; the logical PE-to-physical PE mapping table records the correspondence between computation tasks and actual physical PEs; the spare PE pool provides redundant PEs that can take over the tasks of the faulty PEs; and the mapping update state machine ensures that the mapping table is updated at the safety boundary, avoiding data path confusion during computation.

[0094] The reconfigurable computing array section 106 ensures that input data, weight data, and accumulated results can be transmitted according to the new mapping path after the mapping relationship changes. The reconfigurable computing array section 106 includes a normal PE array, a backup PE, a reconfigurable data routing network, and a partial summation and reduction network. The normal PE array performs the main inference computation; the backup PE takes over the failed PE task after a failure; the reconfigurable data routing network redirects input data, weight data, and intermediate results according to the mapping table; and the partial summation and reduction network performs the accumulation and reduction of the output results of each PE.

[0095] The fault recovery section 107 is used to process the current task after fault detection. If the fault is transient, the current computation block can be re-executed; if the fault is permanent, the faulty PE is isolated first, then remapping is performed, and execution is resumed from the safety checkpoint.

[0096] The fault recovery section 107 includes a security checkpoint manager and a computation block re-execution controller.

[0097] The degraded operation section 108 is used to reduce array parallelism when there is insufficient spare PE. For example, it reduces the number of parallel output channels, reduces the number of tiles running simultaneously, or serializes some computation tasks, thereby ensuring that the inference function continues to run. The degraded operation section 108 includes a degraded operation controller.

[0098] The online fault detection unit 103 is directly connected to the reconfigurable computing array unit 106. During the execution of the inference task, the online fault detection unit 103 collects the PE output results, local accumulation results, or self-test results. The online fault detection unit 103 detects abnormal PEs through sentinel test vector detection, sampling repeated calculation, local result verification, and idle cycle self-test. The online fault detection unit 103 does not directly decide whether to block PEs, but instead sends the fault event such as the abnormal PE number, error type, error occurrence time, and corresponding computing block or tile number to the health status management unit 104.

[0099] The health status management section 104 is used to statistically analyze, retest, and classify fault events. Based on the error threshold, fault count, retest results, and PE historical status, the health status management section 104 determines whether the abnormality is a transient fault, an intermittent fault, or a permanent fault, and writes the judgment result into the PE health status table. The PE health status table also provides status information to the PE remapping section 105, the fault recovery section 107, and the degraded operation section 108, enabling each section to perform subsequent processing based on the same PE health information.

[0100] The PE remapping section 105 queries the location of the faulty PE according to the PE health status table. The PE remapping section 105 generates a new mapping relationship by combining the standby PE pool and the logical PE to physical PE mapping table. When there is an available standby PE or other healthy PE, the PE remapping controller updates the mapping table and issues a new routing configuration to the reconfigurable data routing network, so that the input data, weight data, and part of the result originally sent to the faulty PE are sent to the standby PE or healthy PE. The reconfigurable computing array section 106 continues to execute the inference task according to the updated mapping relationship, realizing the isolation of the faulty PE and the takeover by the standby PE.

[0101] The fault recovery section 107 works in conjunction with the online fault detection section 103, the health status management section 104, and the PE remapping section 105 to handle the current task. If the fault is determined to be a transient fault, the fault recovery section 107 re-executes the affected computational blocks from the nearest network layer boundary, operator boundary, or tile boundary through the security checkpoint manager and the computational block re-execution controller. If the fault is determined to be a permanent fault, the fault recovery section 107 waits for the PE remapping and route reconstruction to be completed before re-executing the affected computational blocks to prevent erroneous results from continuing to propagate to subsequent network layers.

[0102] The degraded operation section 108 serves as a backup processing path, activated when there are insufficient backup PEs, routes cannot be bypassed, or the number of failures exceeds the repair capacity. Based on the PE health status table and remapping failure results, the degraded operation section 108 reduces array parallelism, decreases the number of simultaneously executed tiles, reduces the number of parallel output channels, or converts some tasks to serial execution, thereby ensuring the inference function continues to operate. Thus, even with multiple failed PEs in the array, the NPU can still sacrifice some throughput to continue completing inference tasks.

[0103] The online fault detection section 103 is used to detect PE calculation errors during NPU operation. This embodiment employs a multi-level detection mechanism, rather than relying solely on a single self-test.

[0104] The first detection method is sentinel test vector detection. A sentinel test vector is a pre-designed set of input and weight data, and its correct output is known. The system can send the sentinel test vector to a specified PE or PE cluster during NPU idle periods, task switching intervals, or low-load periods, and compare the output with the standard result. If the results are inconsistent, the corresponding PE is marked as a suspected fault.

[0105] The second detection method is sampling duplicate computation. The system extracts a portion of the computation blocks from the normal inference task, executes them once by the target PE, and simultaneously executes the same computation block by a backup PE or an adjacent healthy PE. The result verification and comparison unit compares the two output results. If the difference exceeds the allowable threshold, the target PE is recorded as abnormal. This method does not require repeating all computations, thus reducing hardware overhead and power consumption.

[0106] The third detection method is local result verification. For calculations such as convolution and matrix multiplication, the system can generate checksums, parity checks, or CRC check values ​​for local output results. If the check value of a certain PE output does not match the prediction range, or is inconsistent with the relationship of adjacent calculation results, an anomaly recording is triggered.

[0107] The fourth detection method is the idle cycle self-test. When certain PEs are temporarily not used in the current layer or the current tile, the idle cycle self-test unit can perform short tests on these PEs to check whether the multipliers, adders, accumulators, registers and input selectors are working properly.

[0108] The above detection methods can be used in combination. For high-reliability scenarios, the detection frequency can be increased; for low-power scenarios, the detection frequency can be reduced or focused detection can be performed only on high-risk PEs.

[0109] The PE health status table records the health status of each PE. Each PE corresponds to one table entry, and the table entry includes at least the following fields: PE location, current status, error count, last error time, retest result, whether it is allowed to participate in the calculation, whether it has been remapped, and the corresponding standby PE identifier.

[0110] The current state of a PE can include the following categories:

[0111] Health status: This PE can participate in the calculation normally.

[0112] Suspected fault status: The PE has encountered an error, but it has not yet been confirmed whether it is a permanent fault.

[0113] Transient fault status: The PE experiences one or a few errors, and returns to normal after retesting.

[0114] Intermittent fault state: The PE experiences errors multiple times at different times, but not every time.

[0115] Permanent Failure Status: The PE fails to perform multiple tests or self-tests and is deemed unusable.

[0116] Masked status: This PE has been removed from the compute array mapping and no longer receives normal tasks.

[0117] The PE health status table can be stored in on-chip SRAM, register array, or dedicated status memory. The main controller can read this table for system diagnostics and maintenance; the PE remapping controller can also directly read this table to generate remapping schemes.

[0118] The fault classification unit determines the PE fault type based on error count, error type, retest results, and thresholds. If a PE experiences only one error and passes the retest, it is classified as a transient fault and can be recovered by re-executing the current computation block. If a PE repeatedly experiences errors over a period of time, but the retest results are unstable, it is classified as an intermittent fault, and the system can reduce its usage priority or observe it. If a PE fails self-test multiple times consecutively, or continuously outputs errors on the same multiply-accumulate path, it is classified as a permanent fault, and the system will disable it and trigger PE remapping. The thresholds for the fault classification unit can be configured by the error threshold register. For example, three consecutive failed tests are classified as a permanent fault; two errors within the same time window are classified as intermittent faults; and a single error that passes the retest is classified as a transient fault.

[0119] The PE remapping controller is used to transfer logical tasks originally assigned to a PE to another physical PE after a permanent failure or high-risk PE is detected. Its core is a logical PE to physical PE mapping table. Neural network operators typically use logical PE numbers, such as logical rows, logical columns, logical channels, or logical computation slots, during compilation or scheduling. During actual execution, the mapping table converts the logical PE numbers into physical PE numbers.

[0120] When a physical PE is determined to have a permanent fault, the PE remapping controller performs the following operations:

[0121] First, read the PE health status table to confirm the location and status of the faulty PE.

[0122] Next, query the backup PE pool to determine if there is an available backup PE. If a backup PE exists, select the backup PE that is closest to the failed PE, has the lowest routing cost, or the least load.

[0123] Then, update the logical PE to physical PE mapping table, changing the logical PE that was originally mapped to the faulty PE to the standby PE.

[0124] Finally, the notification reconfigurable data routing network updates the input data path, weighted path, and partial and output paths, enabling data to bypass the faulty PE and be delivered to the new physical PE.

[0125] If there are not enough spare PEs, the PE remapping controller can choose to reassign the tasks in the row, column or cluster of the failed PE to other healthy PEs to continue running in a way that reduces the degree of parallelism.

[0126] Figure 2 The diagram illustrates the core structure of PE online fault detection and remapping according to an example embodiment.

[0127] The core connectivity relationships between the online fault detection section, health status management section, PE remapping section, and reconfigurable computing array section are as follows: Figure 2 As shown, the online fault detection section includes a sampling and repeating calculation unit, a sentinel test vector generator, and a result verification and comparison unit. The sampling and repeating calculation unit performs repeated calculations on selected PEs or selected computation blocks. The sentinel test vector generator inputs a preset test vector to the target PE, and the outputs of both are sent to the result verification and comparison unit. The result verification and comparison unit also receives calculation results from the reconfigurable data routing network within the reconfigurable computing array section and determines whether there are abnormal PEs by comparing multiple sets of data. The health status management section includes a fault counter, a fault classification unit, and a PE health status table. After the result verification and comparison unit detects an operational anomaly, it sends the abnormal event to the fault counter; the fault counter counts the cumulative number of errors for the corresponding PE; the fault classification unit determines whether the PE is a transient, intermittent, or permanent fault based on the number of errors; the determination result is written to the PE health status table. The PE health status table distinguishes healthy PEs from faulty PEs for the computing array identifier and outputs faulty PE location information to the remapping structure section as the basis for hardware remapping determination.

[0128] A reconfigurable data routing network is used to reconnect input data, weight data, and partial sum / output paths after changes in the PE mapping relationship. This network can be implemented using a multiplexer array, crossbar switch, segmented bus, or lightweight on-chip network. For PE arrays with row-column rules, the reconfigurable data routing network can set a bypass selector in each row or column. When a PE fails, its input data is bypassed to a standby PE or an adjacent healthy PE. For cluster-level PE arrays, a routing selector can be set at the inlet of each PE cluster to distribute data to the corresponding physical PE according to the mapping table. The reconfigurable data routing network must not only redirect input data but also synchronously redirect weight data and partial sum / output paths. Otherwise, even if the input data reaches the standby PE, the weight or accumulation path may still point to the failed PE, leading to calculation errors. Therefore, this invention unifies the input routing, weight routing, and partial sum / output routing under the control of a PE remapping controller.

[0129] A backup PE pool provides redundant computing resources to take over failed PE tasks. Backup PEs can be deployed in several ways:

[0130] The first type is row-level standby PE, which means that each row of the PE array reserves one or more standby PEs. When a PE in that row fails, it is taken over by the standby PE in that row.

[0131] The second type is column-level spare PE, which means that each column of the PE array reserves a spare PE, suitable for pulsating arrays with strong data flow in the column direction.

[0132] The third type is cluster-level spare PE, which means that a small number of spare PEs are reserved in each PE cluster, which is suitable for clustered NPU arrays.

[0133] The fourth type is a global backup PE pool, which involves setting up several backup PEs at the edge of the array and connecting them to multiple array areas via a reconfigurable routing network. This is suitable for scenarios where area is limited but flexible remapping is required.

[0134] Under normal circumstances, the backup PE can be in a low-power standby state or participate in low-priority calculations; when a fault occurs, the backup PE is woken up and takes over the logical position of the faulty PE.

[0135] The safety checkpoint manager stores the critical states of the current inference task at network layer boundaries, operator boundaries, or tile boundaries. These critical states include the current layer number, current tile number, input address, weight address, output address, accumulation state, and mapping table version. When online detection detects an anomaly in a PE output, the system needs to determine if the anomaly has affected the current computation block. If the anomaly occurs within the current tile, the computation block re-execution controller can re-execute the tile from the nearest safety checkpoint. If the PE is determined to be a permanent failure, the PE remapping is performed first, and then the affected tile is re-executed. This prevents erroneous results from propagating to subsequent network layers.

[0136] When the number of faulty physical excavators (PEs) exceeds the number of backup PEs, or when the reconfigurable routing network cannot find suitable alternative paths for some faulty PEs, the degraded operation controller initiates a degraded operation mode. Degraded operation methods include reducing the number of parallel output channels, reducing the number of tiles processed simultaneously, reducing array parallelism, extending computation cycles, or offloading some tasks to healthy PEs for serial execution. Although degraded operation mode reduces throughput, it ensures that the NPU can continue to complete inference tasks after partial hardware failure, making it suitable for industrial control, aerospace, automotive, and long-term unattended equipment.

[0137] Example 2:

[0138] Figure 3 A flowchart illustrating an online fault detection and PE remapping process for an NPU array according to an example embodiment is shown.

[0139] like Figure 3 As shown, a method for online fault detection and PE remapping of a high-reliability NPU array includes the following steps:

[0140] S301, the NPU receives the complete inference task sent down from the upper layer;

[0141] S302, hardware loading of neural network model weight parameters and a pre-configured set of reliability management strategies;

[0142] S303, the system reads the currently effective logical PE to physical PE mapping table, and drives the reconfigurable computing array to perform model inference operations based on the existing computing power allocation relationship;

[0143] S304, during the inference process, online fault detection is started simultaneously, and the output data of all operation PEs are monitored in real time through sampling repeated calculation, sentinel test vector detection, result comparison and other methods.

[0144] S305, determine whether a PE abnormality is detected. If no PE abnormality is detected, jump to S303; otherwise, execute S306.

[0145] S306, record this error event and update the fault counter of the corresponding physical PE;

[0146] S307, initiate the retest process and classify the faults based on the cumulative number of errors and the retest results;

[0147] S308, determine the fault type. If it is a momentary fault, execute S309; ​​if it is an intermittent fault, execute S310; if it is a continuous fault, execute S311.

[0148] S309, the system retrieves the pre-stored security checkpoint data, re-executes the calculation block currently affected by the fault, and returns to the normal inference process after repairing the single calculation error;

[0149] S310, reduce the scheduling priority of the faulty PE, retain its computing power usage rights and continue to monitor its subsequent running status, and return to the normal inference process;

[0150] S311, the physical PE is permanently marked as faulty and updated to the global PE health status table;

[0151] S312, Determine whether there are any idle and available redundant PE resources in the standby PE pool. If there are idle and available redundant PE resources in the standby PE pool, then execute S313; otherwise, execute S314.

[0152] S313, the PE remapping controller updates the mapping table from logical PE to physical PE, and executes S315;

[0153] S314, initiate degraded operation mode, execute S316;

[0154] S315, issue a new routing configuration to the reconfigurable data routing network to switch data flow paths and isolate faulty PEs, then execute S317;

[0155] S316 reduces the overall computing power parallelism and re-splits the scheduling of inference subtasks to reduce computing power requirements, adapts the remaining healthy PE resources to complete the operation, and executes S303.

[0156] S317, re-execute the computation block affected by the fault, then execute S303.

[0157] To prevent faulty PEs from continuing to receive data during the remapping process, this embodiment sets up a mapping update state machine. This state machine ensures that the remapping occurs at a safe boundary, such as a network layer boundary, operator boundary, or tile boundary.

[0158] Figure 4 A flowchart illustrating PE remapping and route reconstruction according to an example embodiment is shown.

[0159] like Figure 4 As shown, the PE remapping and route reconstruction steps include:

[0160] S401, the online fault detection section detects operational anomalies in real time and reports PE abnormal events to the health status management section;

[0161] S402, after receiving an abnormal signal, the health management module performs corresponding PE error count accumulation and fault classification determination;

[0162] S403, if the PE is a permanent fault, send permanent fault PE location information to the PE remapping controller;

[0163] S404, after receiving the fault notification, the PE remapping controller first sends a safety switching boundary request to the safety checkpoint manager;

[0164] S405, the safety checkpoint manager issues an instruction to the reconfigurable computing array section, causing the array to suspend the operation of the computing blocks currently affected by the fault;

[0165] S406, During the safe window period when the computing block is suspended, the PE remapping controller completes the query of the backup PE resource and synchronously updates the mapping table from logical PE to physical PE.

[0166] S407, the PE remapping controller issues a new routing configuration command to the reconfigurable data routing network;

[0167] S408, the reconfigurable data routing network completes the data path reconstruction according to the new configuration, shields the data transmission channel of the faulty PE, enables the idle standby PE to take over the computing task, and pushes the path switching completion signal to the reconfigurable computing array section;

[0168] S409, after receiving the routing switch completion signal, the reconfigurable computing array sends a recalculation request to the security checkpoint manager. The security checkpoint manager retrieves the backup data of the corresponding breakpoint and controls the array to re-execute the previously suspended affected computing blocks from the security checkpoint.

[0169] After the S410 recalculation operation is completed, the system resumes normal operation, and online fault detection is continuously performed synchronously across the entire link.

[0170] Example 3:

[0171] Example 3 is a practical application example of Examples 1 and 2 in engineering:

[0172] (1) Remapping of an industrial edge

[0173] In an industrial edge device, the NPU runs a fault diagnosis model for extended periods. The NPU array consists of multiple PE clusters, each with a reserved spare PE. During system operation, the online fault detection area periodically injects sentinel test vectors into idle PEs, while simultaneously performing sampled repetitive computations on some normal inference computation blocks.

[0174] During a test, the result verification and comparison unit detected that a certain PE (Presumably a PE type) output an error in three consecutive sentinel tests. The fault classification unit determined that the PE was permanently faulty based on the error threshold and wrote its status into the PE health status table. The PE remapping controller queried the standby PEs in the cluster, mapped the logical number of the faulty PE to the standby PE, and controlled the reconfigurable data routing network to bypass the faulty PE.

[0175] After the remapping is complete, the security checkpoint manager re-executes the affected computed blocks from the nearest tile boundary. Finally, the NPU continues to complete the fault diagnosis model inference, and the device does not need to be shut down.

[0176] (2) Re-execution of computational blocks under transient soft errors

[0177] In a vehicle or aerospace application scenario, when the NPU executes a target recognition model, a PE (Pressure Equipment) generates an erroneous output due to a transient disturbance. The sampling and recalculation unit finds that the output of this PE is inconsistent with the recalculation result of the backup PE, so the fault counter records an error. Subsequently, the system performs a retest on this PE, and the retest result is normal. The fault classification unit determines that the error is a transient fault.

[0178] Since this error may affect the output of the current tile, the compute block re-execution controller re-executes the tile from the most recent safety checkpoint. After re-execution, the result is normal, and the PE health status table marks the PE as healthy but records one error history. This embodiment illustrates that the present invention can handle transient failures without masking the PE, avoiding excessive sacrifice of computational resources.

[0179] (3) Degradation operation when the backup PE is insufficient

[0180] In a long-running energy IoT gateway, some physical excavators (PEs) in the NPU array suffer permanent failures due to aging. When the number of failed PEs exceeds the capacity of the backup PE pool, the PE remapping controller is unable to allocate backup PEs to all failed PEs. At this point, the degraded operation controller initiates a degraded mode, changing the original parallel execution of four output channel groups into two channel groups executing in two rounds.

[0181] Although the inference latency increases in this mode, the system can still complete the model inference task and report the fault status to the main controller. This embodiment illustrates that the present invention can maintain functional availability even under severe fault conditions.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An NPU array architecture with online fault detection and PE remapping capabilities, characterized in that, It includes a main controller, an on-chip configuration and data bus, and a functional area, which includes an online fault detection section, a health status management section, a PE remapping section, a reconfigurable computing array section, a fault recovery section, and a degraded operation section. The main controller sends inference tasks, model parameters, detection frequency, error threshold, remapping strategy and degraded operation strategy to the functional area through the on-chip configuration and data bus. The main controller receives the fault status and operation status returned by the functional area through the on-chip configuration and data bus. The online fault detection section is used to detect PE calculation errors during NPU operation. The online fault detection section includes an idle cycle self-test unit, a sentinel test vector generator, a sampling repeat calculation unit, and a result verification and comparison unit. The online fault detection section is directly connected to the reconfigurable computing array section; The health status management section is used to store the health status and fault history of each PE. The health status management section includes a PE health status table, a fault counter, an error threshold register, and a fault classification unit. The PE remapping part is used to modify the computing task allocation relationship after a faulty PE is detected. The PE remapping part includes a logical PE to physical PE mapping table, a PE remapping controller, a spare PE pool, and a mapping update state machine. The reconfigurable computing array section is used to ensure that input data, weight data and accumulation results can be transmitted according to the new mapping path after the mapping relationship changes. The reconfigurable computing array section includes a normal PE array, a backup PE, a reconfigurable data routing network, a weight distribution network and a partial reduction network. The fault recovery section is used to process the current task after fault detection. The fault recovery section includes a security checkpoint manager and a compute block re-execution controller. The degradation operation section is used to reduce the array parallelism when the spare PE is insufficient, and the degradation operation section includes a degradation operation controller; The fault recovery section works in conjunction with the online fault detection section, the health status management section, and the PE remapping section to process the current task. If the fault is determined to be a transient fault, the fault recovery section re-executes the affected computational block from the nearest network layer boundary, operator boundary, or tile boundary through the security checkpoint manager and the computational block re-execution controller. If the fault is determined to be a permanent fault, the fault recovery section waits for the PE remapping and route reconstruction to be completed before re-executing the affected computational block to prevent erroneous results from continuing to propagate to subsequent network layers.

2. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The online fault detection unit collects PE output results, local accumulation results, or self-test results during the execution of the inference task; the online fault detection unit sends fault events to the health status management unit, and the fault events include abnormal PE number, error type, error occurrence time, corresponding calculation block and / or tile number; During NPU operation, the online fault detection section uses any one or any combination of the following four detection methods to detect PE calculation errors: Sentinel test vector detection: The steps of the sentinel test vector detection include the sentinel test vector generator generating a set of input data and weight data as sentinel test vectors, and the correct output results of the input data and the weight data are known; During NPU idle periods, task switching intervals, or low-load periods, the system sends the sentinel test vector into a designated PE or PE cluster to obtain the corresponding output results, and compares the output results with the correct output results. If the comparison results are inconsistent, the corresponding PE is marked as a suspected fault; Sampling repeated calculation: The sampling repeated calculation steps include the system extracting a portion of the calculation blocks from the normal inference task, having the target PE execute it once, and having the backup PE or an adjacent healthy PE execute the same calculation block at the same time; the result verification and comparison unit compares the two output results, and if the difference between the two output results exceeds the allowable threshold, the target PE is recorded as abnormal; Local result verification: The steps of local result verification include the system generating a checksum, parity check or CRC check value for the local output result; if the check value of a certain PE output does not match the prediction range, or the check value of a certain PE output is inconsistent with the relationship of adjacent calculation results, an anomaly record is triggered. Idle Cycle Self-Test: The steps of the idle cycle self-test include performing a short test on certain PEs when they are temporarily not used in the current layer or the current tile. The short test includes checking whether the multiplier, adder, accumulator, register and input selector are normal.

3. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The health status management section performs statistics, retesting, and classification of fault events. Based on error thresholds, fault counts, retesting results, and PE historical status, the health status management section determines whether the anomaly is a transient, intermittent, or permanent fault, and writes the determination result into the PE health status table. The PE health status table simultaneously provides status information to the PE remapping section, the fault recovery section, and the degraded operation section, enabling them to perform subsequent processing based on the same PE health information. The PE health status table is used to record the health status of each PE; each PE corresponds to a table entry, and the table entry includes several fields, including PE location, current status, error count, most recent error time, retest result, whether it is allowed to participate in the calculation, whether it has been remapped, and the corresponding backup PE identifier. The PE health status table is stored in on-chip SRAM, register array, or dedicated status memory; The main controller reads the PE health status table for system diagnosis and maintenance; the PE remapping controller directly reads the PE health status table to generate a remapping scheme.

4. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 3, characterized in that... The current state includes: Health status, which indicates that the PE can participate in the calculation normally; Suspected fault status, which indicates that the PE has encountered an error, but it has not yet been confirmed whether it is a permanent fault; Transient fault state, which indicates that the PE has one or a few errors, and returns to normal after retesting; Intermittent fault state, which indicates that the PE fails multiple times at different times, but not every time it fails; Permanent Fault Status: The permanent fault status indicates that the PE has failed multiple detections or self-tests and is therefore deemed unusable. The shielded state indicates that the PE has been removed from the compute array mapping and no longer receives normal tasks.

5. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The health status management section determines whether an anomaly is a transient, intermittent, or permanent fault based on error thresholds, fault counts, retest results, and historical PE status. The steps include: If a PE (Preinstallation Equipment) experiences an error only once and passes the retest, it is determined to be a transient fault. If a PE repeatedly fails to produce an error over a period of time, but the retest results are unstable, it is determined to be an intermittent fault. If a PE fails to perform a self-test multiple times in a row, or if the same multiply-accumulate path continues to output errors, it is considered a permanent fault.

6. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The PE remapping part queries the location of the faulty PE according to the PE health status table. The PE remapping part combines the backup PE pool and the logical PE to physical PE mapping table to generate a new mapping relationship. When there is an available backup PE or other healthy PE, the PE remapping controller updates the mapping table and issues a new routing configuration to the reconfigurable data routing network, so that the input data, weight data, and part of the result originally sent to the faulty PE are sent to the backup PE or healthy PE. The reconfigurable computing array part continues to execute the inference task according to the updated mapping relationship, realizing the isolation of the faulty PE and the takeover by the backup PE. The reconfigurable data routing network is used to reconnect input data, weight data, and some output paths after the PE mapping relationship changes; the reconfigurable data routing network includes a multiplexer array, a crossbar switch, a segmented bus, or a lightweight on-chip network.

7. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The backup PE pool is used to provide redundant computing resources that can take over the tasks of failed PEs, and the backup PEs are arranged in any of the following ways: Row-level backup PE arrangement, wherein each row of PE array reserves one or more backup PEs, and when a PE in a row fails, the backup PE in that row takes over; Column-level spare PE arrangement, wherein the column-level spare PE arrangement reserves spare PEs for each column of PE array; Cluster-level spare PE arrangement, wherein a small number of spare PEs are reserved in each PE cluster; A global backup PE pool is arranged by setting up several backup PEs at the edge of the array, which are connected to multiple array areas by the reconfigurable data routing network.

8. The NPU array architecture with online fault detection and PE remapping capabilities according to claim 1, characterized in that, The degradation operation section serves as a backup processing path and is activated when there are insufficient backup PEs, routes cannot be bypassed, or the number of faults exceeds the repair capacity. The degradation operation section reduces the array parallelism, decreases the number of tiles executed simultaneously, decreases the number of parallel output channels, or converts some tasks to serial execution based on the PE health status table and remapping failure results. The security checkpoint manager is used to save the key states of the current inference task at network layer boundaries, operator boundaries, or tile boundaries. The key states include the current layer number, the current tile number, the input address, the weight address, the output address, the accumulation state, and the mapping table version.

9. A method for online fault detection and PE remapping of an NPU array, characterized in that, Includes the following steps: The NPU receives inference tasks and loads the current model parameters; The main controller sets a reliability control strategy based on the application reliability level. The reliability control strategy includes online detection frequency, error threshold, and whether to enable backup PE. During the inference execution process, online fault detection is continuously performed; If no anomaly is detected, the computing array continues to execute according to the original mapping; if an anomaly is detected in the output of a PE, the fault counter records the error event, and the fault classification unit performs retesting and classification. If the fault is transient, the compute block re-execution controller will re-execute the affected tile from the most recent security checkpoint; if the fault is intermittent, the scheduling priority of the faulty PE will be reduced, its computing power usage rights will be retained, and its subsequent running status will be continuously monitored; if the fault is permanent, PE remapping and route reconstruction will be performed. After PE remapping and route reconstruction are completed, the system re-executes the affected computation blocks and continues to execute subsequent network layers; if there are insufficient spare PEs, the degraded controller reduces the array parallelism to ensure that the inference task continues to be completed.

10. The online fault detection and PE remapping method for NPU arrays according to claim 9, characterized in that, The steps of PE remapping and route reconstruction include: The online fault detection unit detects operational anomalies in real time and reports PE anomaly events to the health status management unit. After receiving an abnormal signal, the health status management section accumulates the corresponding PE error count and determines the fault level. If the PE abnormal event is a permanent fault, it sends the permanent fault PE location information to the PE remapping controller. After receiving the fault notification, the PE remapping controller sends a safety switch boundary request to the safety checkpoint manager. The safety checkpoint manager then issues an instruction to the reconfigurable computing array to suspend the operation of the computing blocks currently affected by the fault. During the safe window period when the computing block is suspended, the PE remapping controller completes the query of backup PE resources, synchronously updates the logical PE to physical PE mapping table, and issues a new routing configuration instruction to the reconfigurable data routing network. The reconfigurable data routing network reconfigures the data path based on the new configuration, shields the data transmission channel of the faulty PE, enables the idle standby PE to take over the computing task, and pushes the path switching completion signal to the reconfigurable computing array section.

Images