A neural network oriented accelerator, processor

Abstract

The application provides a neural network-oriented accelerator, which comprises a main processing core, an input buffer, a controller, a main systolic array and an output buffer, the input buffer is used for storing feature maps and weight parameters of a neural network, the controller is used for generating operation signals to read the feature maps and the weight parameters from the input buffer, the main systolic array is used for receiving the feature maps and the weight parameters and performing calculation to obtain an output sequence; a sorter is used for reordering each element in the output sequence to select a to-be-detected sequence, a lockstep processing core is used for recalculating the value of each element in the to-be-detected sequence through a lockstep systolic array to obtain a check sequence; a check logic module is used for comparing whether the values of the same elements in the to-be-detected sequence and the check sequence are the same; and an error recovery module is used for recalculating the values of all elements in the to-be-detected sequence that do not pass the check according to a preset recalculation mechanism.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit design, specifically to neural network hardware acceleration technology in the field of integrated circuit design, and more specifically, to an accelerator and processor for neural networks. Background Technology

[0002] With the widespread application of deep learning technology in safety-critical systems, such as perception and decision-making in autonomous vehicles and real-time control of industrial robots, the reliability of neural network processors, as the core of computing, has become the cornerstone of system safety. In these scenarios, hardware may experience transient soft errors due to environmental factors such as neutron impacts and electromagnetic interference, manifesting as random bit flips in memory cells or combinational logic. If such errors are not detected and corrected in a timely manner, they may lead to incorrect neural network inference results, potentially resulting in catastrophic consequences.

[0003] To address the threat of transient soft errors, traditional fault-tolerant designs typically employ spatial redundancy strategies, such as lock-step technology. In the realm of general-purpose processors (CPUs), lock-step technology achieves fault tolerance and error recovery by completely replicating the CPU across two or three cores. Figure 1 As shown, this processor architecture employs dual-core replication technology, comprising an upper main processing core and a lower lockstep processing core. The main processing core and the lockstep processing core execute the same instruction stream synchronously and detect errors through periodic output comparison. If an inconsistency is found, it can be corrected by re-execution. While lockstep technology effectively improves reliability, its drawbacks are also significant: the complete replication of hardware resources (including computing cores, on-chip memory, and interconnects) nearly doubles the chip area, dramatically increases dynamic power consumption, and often results in significant performance loss during error recovery. For resource-constrained and energy-sensitive mobile and edge computing devices, such overhead is often unacceptable.

[0004] Meanwhile, neural networks inherently possess certain fault-tolerant characteristics. Due to their distributed representation, high-dimensional feature mapping, and nonlinear activation functions, errors in individual neurons or connections do not necessarily lead to a significant decrease in the final output accuracy. Furthermore, different hardware modules of a neural network accelerator exhibit varying sensitivities to errors. For example, a single bit flip in the control logic may affect the entire computation process, while a momentary error in a multiplication-accumulation unit in a systolic array may only affect a single output and may be "masked" in subsequent network layer computations. Most existing fault-tolerant schemes fail to systematically utilize this characteristic of neural networks and their hardware implementations, resulting in a lack of targeted redundancy design and the introduction of significant unnecessary overhead.

[0005] In summary, existing technologies still have two shortcomings in ensuring the reliability of neural network processors: First, traditional spatial redundancy strategies such as lockstep achieve fault tolerance by completely replicating the hardware, which brings unbearable area and power consumption overhead and cannot meet the needs of resource-constrained edge computing; Second, existing technologies fail to make full use of the inherent fault tolerance capabilities of neural network algorithms themselves. This lack of targeted equalization protection strategy leads to coarse redundancy granularity and introduces a lot of unnecessary overhead.

[0006] It should be noted that the background information presented here is only for illustrating relevant information about the present invention to aid in understanding the technical solution of the present invention, and does not imply that the relevant information is necessarily prior art. The relevant information was submitted and disclosed together with the present invention, and should not be considered prior art unless there is evidence that the relevant information was disclosed before the filing date of the present invention. Summary of the Invention

[0007] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide an accelerator and a processor for neural networks.

[0008] The objective of this invention is achieved through the following technical solution:

[0009] According to a first aspect of the present invention, an accelerator for neural networks is provided for performing inference computations of neural networks based on an instruction set, wherein the instruction set includes multiple operation instructions, and the accelerator includes a main processing core, a lockstep processing core, a sorter, a checking logic module, and an error recovery module, wherein: the main processing core includes an input buffer, a controller, a main systolic array, and an output buffer, wherein the input buffer is used to store feature maps and weight parameters of the neural network, the controller is used to parse operation instructions to generate operation signals, and read feature maps and weight parameters from the input buffer based on the operation signals and transmit them to the main systolic array, the main systolic array is used to receive feature maps and weight parameters and perform calculations to obtain an output sequence, and transmit the output sequence to the output buffer and the sorter, the output sequence includes multiple elements, each element corresponding to a value, and the output buffer is used to store the output sequence; the sorter is used to sort the output sequence according to a preset reordering mechanism. The elements are reordered, and a preset proportion of the elements at the top of the sorted sequence are selected as the sequence to be detected. The lockstep processing core is equipped with a lockstep pulsation array. The lockstep processing core is used to read the data required to recalculate the sequence to be detected from the input buffer, and recalculate the value of each element in the sequence to be detected through the lockstep pulsation array to obtain a verification sequence. The checking logic module is used to compare whether the values ​​of the same elements in the sequence to be detected and the verification sequence are the same. If they are the same, the element is determined to pass the verification; otherwise, the element is determined to fail the verification. The error recovery module is used to recalculate the values ​​of all elements that failed the verification in the sequence to be detected according to a preset recalculation mechanism, and transmit the recalculated value of each element to the output buffer to replace the value of the corresponding element in the output sequence. The array size of the lockstep pulsation array is smaller than the array size of the main pulsation array, and the ratio of the array sizes of the lockstep pulsation array and the main pulsation array is equal to a preset asymmetric ratio.

[0010] According to some embodiments of the present invention, the preset reordering mechanism is as follows: based on the value of each element in the output sequence, each element in the output sequence is reordered in descending order.

[0011] According to some embodiments of the present invention, the preset recalculation mechanism is as follows: based on the verification result of the sequence to be detected, all elements in the sequence to be detected that have failed verification are determined; based on a preset data reuse factor, with each element that has failed verification as the center, multiple preceding and following elements are selected from the output sequence, and all selected elements are combined with the corresponding elements that have failed verification to form a recalculation subset, wherein each element that has failed verification corresponds to a recalculation subset; based on the recalculation subset corresponding to each element that has failed verification, the data required to recalculate each recalculation subset is read from the input buffer, and the value corresponding to each element in each recalculation subset is recalculated through the lockstep processing core or the main processing core; the value corresponding to each element in each recalculation subset is transmitted to the output buffer to replace the value of the corresponding element in the output sequence.

[0012] According to some embodiments of the present invention, the preset data reuse factor is 16.

[0013] According to some embodiments of the present invention, the preset ratio ranges from 5% to 100%.

[0014] According to some embodiments of the present invention, the preset asymmetric ratio is 1 / 4.

[0015] According to some embodiments of the present invention, the main pulsation array is a two-dimensional pulsation array of 64×64, 32×32, 24×24 or 16×16, and the lockstep pulsation array is a two-dimensional pulsation array of 32×32, 16×16, 12×12 or 8×8.

[0016] According to some embodiments of the present invention, the controller includes an instruction control unit and a data loading and storage unit, wherein: the instruction control unit is used to parse operation instructions to generate operation signals; the data loading and storage unit is used to read feature maps and weight parameters from the input buffer based on the operation signals and transmit them to the main systolic array, and to transmit the output sequence calculated by the main systolic array to the output buffer and the sorter.

[0017] According to a second aspect of the present invention, a processor for neural networks is provided for performing inference computations of neural networks, said processor comprising a plurality of accelerators as described in the first aspect of the present invention.

[0018] Compared with the prior art, the advantages of the present invention are: (1) By selectively copying, the copying of non-critical or already protected redundant modules such as on-chip caches that are protected by ECC (error correction code) is avoided, which directly reduces the area overhead from the source; (2) By importance-aware sorting, the amount of data that needs to be repeatedly calculated and compared is greatly reduced, and the dynamic power consumption and the complexity of the comparison logic are significantly reduced; (3) By a refined local error recovery mechanism, the system pause time after the error occurs is greatly shortened, and the overall execution efficiency is improved. Attached Figure Description

[0019] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0020] Figure 1 This is a schematic diagram of a traditional dual-core lockstep processor architecture;

[0021] Figure 2 This is a schematic diagram of a selective lockstep neural accelerator architecture according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0023] As mentioned in the background section, existing technologies still have two shortcomings in ensuring the reliability of neural network processors: First, traditional spatial redundancy strategies such as lockstep achieve fault tolerance by completely replicating the hardware, which brings unbearable area and power consumption overhead and is difficult to meet the needs of resource-constrained edge computing; Second, existing technologies fail to make full use of the inherent fault tolerance capabilities of neural network algorithms themselves. This lack of targeted equalization protection strategy leads to coarse redundancy granularity and introduces a lot of unnecessary overhead.

[0024] To address the aforementioned issues, the inventors proposed a selective lockstep neural accelerator architecture suitable for wafer-level chips, such as... Figure 2As shown, the architecture includes a full-featured main processing core and a streamlined lockstep processing core. The main processing core has a complete computational systolic array, control logic, data loading path, and storage resources, and is responsible for executing the complete computational task. The lockstep processing core, on the other hand, has been carefully trimmed down, replicating only the control logic and data loading path that have the greatest impact on error propagation. Furthermore, the array size of the computational systolic array inside the lockstep core has been significantly reduced, so the lockstep processing core can only execute local computational tasks. In addition to the main processing core and the lockstep processing core, the architecture also includes a sorter, a checking logic module, and an error recovery module. The sorter is used to identify the subset of key elements (including multiple elements) that contribute the most to the accuracy of the final task from the output sequence (including multiple elements, each corresponding to a value) calculated by the main processing core based on its integrated importance filtering mechanism. The checking logic module is used to compare whether the values ​​of each element in the subset of key elements calculated by the main processing core and the lockstep processing core are consistent. If they are inconsistent, the error recovery process is triggered. The error recovery module is used to quickly locate the range of elements that may be affected based on the location of the element where the error occurred and the data reuse characteristics, and reschedule the calculation task for the range of elements to guide the lockstep processing core or the main processing core to re-execute the calculation to correct the calculation error caused by the transient soft error.

[0025] The inventors proposed a selective lockstep neural accelerator architecture that, through selective replication, avoids copying non-critical or already protected redundant modules such as on-chip caches protected by ECC (Error Correction Code), directly reducing area overhead at the source. Furthermore, by using importance-aware sorting, it significantly reduces the amount of data requiring repeated computation and comparison, substantially lowering dynamic power consumption and the complexity of comparison logic. Even further, through a refined local error recovery mechanism, it greatly shortens system downtime after an error occurs, improving overall execution efficiency. Based on this architecture, this invention can reduce the hardware overhead of fault tolerance mechanisms to a low proportion of traditional methods without sacrificing the reliability of neural network inference, and provides a feasible hardware solution for deploying highly reliable artificial intelligence computing in energy- and cost-sensitive scenarios.

[0026] In summary, this invention provides an accelerator for neural networks, used to perform inference computations of neural networks based on an instruction set. The instruction set includes multiple operation instructions. The accelerator includes a main processing core, a lockstep processing core, a sorter, a checking logic module, and an error recovery module. The main processing core includes an input buffer, a controller, a main systolic array, and an output buffer. The input buffer stores feature maps and weight parameters of the neural network. The controller parses the operation instructions to generate operation signals and reads feature maps and weight parameters from the input buffer based on the operation signals, transmitting them to the main systolic array. The main systolic array receives the feature maps and weight parameters, performs calculations to obtain an output sequence, and transmits the output sequence to the output buffer and the sorter. The output sequence includes multiple elements, each corresponding to a numerical value. The output buffer stores the output sequence. The sorter sorts the output sequence according to a preset reordering mechanism. Each element is reordered, and a preset proportion of elements at the top of the sorted sequence are selected as the sequence to be detected. The lockstep processing core is equipped with a lockstep pulsation array. The lockstep processing core is used to read the data required to recalculate the sequence to be detected from the input buffer, and recalculate the value of each element in the sequence to be detected through the lockstep pulsation array to obtain a verification sequence. The checking logic module is used to compare whether the values ​​of the same elements in the sequence to be detected and the verification sequence are the same. If they are the same, the element is determined to pass the verification; otherwise, the element is determined to fail the verification. The error recovery module is used to recalculate the values ​​of all elements that failed the verification in the sequence to be detected according to a preset recalculation mechanism, and transmit the recalculated value of each element to the output buffer to replace the value of the corresponding element in the output sequence. The array size of the lockstep pulsation array is smaller than the array size of the main pulsation array, and the ratio of the array sizes of the lockstep pulsation array and the main pulsation array is equal to a preset asymmetric ratio.

[0027] To better understand the present invention, the composition and structure of the accelerator will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] I. Main Processing Core

[0029] exist Figure 2 In the accelerator architecture shown, the main processing core includes an input buffer, a controller, a main systolic array, and an output buffer (due to image length limitations). Figure 2 (The main processing core is not marked in the text).

[0030] Specifically, the various components within the main processing core perform inference computation based on control logic and data loading paths. The control logic is not responsible for the actual data computation; rather, it issues instructions, determining where the data comes from, where it goes, and when computation begins. Furthermore, the control logic does not directly correspond to any specific component module but is implicitly embedded within the main processing core's internal state machine, instruction decoder, and scheduler, managing the entire main processing core's operation to ensure the computation process proceeds according to a predetermined rhythm. The data loading path refers to the physical pathways and mechanisms through which data moves from the storage medium (input buffer) to the computation unit (systolic array) and from the computation unit (systolic array) to the storage medium (output buffer), and it follows the commands of the control logic to perform data transfer.

[0031] To better understand this invention, the components of the main processing core are described in detail below.

[0032] 1.1 Input Buffer

[0033] The input buffer is used to store feature maps and weight parameters of the neural network. The feature maps are tensor data obtained by the neural network during inference computation (the feature maps are the original input data when the neural network starts performing inference computation). The input buffer can be a 1.5MB on-chip static random access memory (SRAM).

[0034] 1.2 Controller

[0035] The controller is used to parse operation instructions to generate operation signals, and to read feature maps and weight parameters from the input buffer based on the operation signals and transmit them to the main pulsation array.

[0036] According to one embodiment of the present invention, the controller includes an instruction control unit and a data loading and storage unit, wherein the instruction control unit is used to parse operation instructions to generate operation signals; the data loading and storage unit is used to read feature maps and weight parameters from the input buffer based on the operation signals and transmit them to the main systolic array, and to transmit the output sequence calculated by the main systolic array to the output buffer and the sorter. It should be noted that the instruction control unit is integrated within the controller (not shown in the image due to space limitations). Figure 2 As shown in the figure, the data loading and storage unit is manifested as the physical path and related address logic connecting the "input buffer" and the "pulse array", as well as the "pulse array" and the "output buffer", and the instruction control unit and the data loading and storage unit interact through strict timing and address signals.

[0037] Specifically, the instruction control unit, acting as the master controller, is responsible for parsing neural network operator instructions and generating precise read / write control signals and memory address indices. The data loading and storage unit, acting as the execution unit, addresses and reads feature maps and weight parameters from the input buffer based on the read / write control signals and memory address indices. Based on this, in the inference computation process, the instruction control unit sends read commands to the data loading and storage unit according to scheduling logic. The data loading and storage unit responds to the read commands, reads data from the input buffer, and loads it entirely into the main systolic array to perform conventional inference computation. After the main systolic array completes the inference computation, the instruction control unit then sends write commands to the data loading and storage unit according to scheduling logic, transmitting the output sequence calculated by the main systolic array to the output buffer and the sorter.

[0038] Furthermore, the instruction control unit and the data loading and storage unit also participate in the calculation process of the lockstep processing core. Simply put, after the sorter receives the output sequence calculated by the main pulse array, it sorts the elements in the output sequence and selects the top-ranked elements as important / critical elements. At this time, the instruction control unit sends a read command to the data loading and storage unit according to the scheduling logic. The data loading and storage unit responds to the read command, reads the data required to recalculate the critical elements from the input buffer, and loads it into the lockstep processing core for local inference calculation to obtain the verification sequence. After the lockstep processing core completes the local inference calculation, it passes the verification sequence to the inspection logic module for further processing.

[0039] In addition, the instruction control unit and data loading and storage unit also participate in the fault tolerance verification phase of the logic module and the error recovery phase of the error recovery module. Simply put, during the execution of the fault tolerance verification and error recovery phases, the instruction control unit controls the write-back / release timing of the data flow. It does not immediately allow the data loading and storage unit to release the output sequence (calculation result) calculated by the main systolic array, but instead releases it through a delay module (not shown in the image due to space limitations). Figure 2(As shown in the diagram) A waiting period is introduced until the check logic module completes the comparison between the output sequence and the verification sequence. If the values ​​of all elements in the verification sequence are consistent with the values ​​of the corresponding elements in the output sequence, the data loading and storage unit is allowed to release the output sequence temporarily stored in the output buffer. If the check logic module detects an error, the instruction control unit will calculate all elements affected by the error according to the preset data reuse factor and send a reset and re-fetch signal to the data loading and storage unit. This will cause the data loading and storage unit to immediately clear the dirty data in the pipeline and reread the data required to recalculate all elements affected by the error from the input buffer. The data will then be sent to the main processing core or lockstep processing core to re-execute the local calculation until all recalculated elements pass the verification again. After that, the data loading and storage unit will transmit the data to the output buffer to replace the erroneous elements in the output sequence. Then, the instruction control unit will release the write-back block, and the output sequence temporarily stored in the output buffer will be allowed to be released.

[0040] 1.3 Main Pulsating Array

[0041] The main systolic array receives feature maps and weight parameters, performs calculations to obtain an output sequence, and transmits the output sequence to an output buffer and a sorter. The output sequence includes multiple elements, each corresponding to a numerical value. Each element has an index number to represent a different element; for example, if the output sequence has 1024 elements, the elements are numbered sequentially from 1 to 1024. It should be noted that the main systolic array, as the core computing engine, includes multiple computational units (PEs) responsible for performing multiplication and accumulation operations on neural network layers such as convolutional and fully connected layers.

[0042] The main systolic array serves as the core computing engine, undertaking the inference computation of the neural network. The array size of the main systolic array can be set according to actual needs, and this invention does not impose any special restrictions. For example, the main systolic array can be a 24×24 two-dimensional systolic array, a 16×16 two-dimensional systolic array (high fine granularity, suitable for wafer-level chips with extremely high yield), a 32×32 two-dimensional systolic array (medium throughput), or a 64×64 two-dimensional systolic array (high throughput, suitable for high-performance wafer-level chips).

[0043] 1.4 Output Buffer

[0044] The output buffer is used to store the output sequence. Alternatively, the output buffer can be a 1.5MB on-chip static random access memory (SRAM).

[0045] As can be seen from the foregoing embodiments, the main processing core executes neural network inference calculations according to the instruction flow, generating multiple batches of output sequences (one batch corresponds to one output sequence). Each time an output sequence is generated, the output sequence is temporarily stored in the output buffer and then transmitted to the sorter for reordering.

[0046] II. Sorters

[0047] exist Figure 2 In the accelerator architecture shown, the sorter is used to reorder each element in the output sequence according to a preset reordering mechanism, and select a preset proportion of the elements that are sorted first as the sequence to be detected.

[0048] According to one embodiment of the present invention, the preset reordering mechanism is as follows: based on the value of each element in the output sequence, each element in the output sequence is reordered in descending order.

[0049] According to one embodiment of the present invention, the preset ratio ranges from 5% to 100%. The value of the preset ratio is determined based on actual needs; for example, the preset ratio can be set to 5%, 25%, 30%, or 50%.

[0050] To better understand how the sorter works, let's illustrate with a concrete example. Suppose the output sequence contains 1024 elements. The sorter reorders each element in descending order of its value, and marks the first 25% (1024 × 25% = 256 elements) as important elements to form the sequence to be tested. Therefore, the sequence to be tested contains 256 important elements.

[0051] As can be seen from the foregoing embodiments, the sorter performs a quick sorting of the elements in the output sequence according to the reordering mechanism in order to select the top K elements with the highest importance (K = number of elements in the output sequence × preset ratio, the preset ratio is determined based on actual needs).

[0052] III. Lockstep Processing Core

[0053] exist Figure 2 In the accelerator architecture shown, the lockstep processing core (due to image length limitations, Figure 2 The unlabeled part is used to read the data (partial feature map data and partial weight parameters) required to recalculate the sequence to be detected from the input buffer, and recalculate the value of each element in the sequence to be detected through its own lockstep pulsation array to obtain the verification sequence.

[0054] As described in the foregoing embodiments, the main processing core possesses a complete computational systolic array, control logic, data loading path, and storage resources to perform neural network inference computation. Unlike the main processing core, the lockstep processing core only performs local verification computation. Therefore, in the accelerator architecture proposed in this invention, the lockstep systolic core only copies the control logic and data loading path that have a significant impact on error propagation to support local verification computation. It should be noted that storage resources are not copied because they can be protected by ECC (Error Correction Code) and other methods, eliminating the need for hardware-level duplication. Independent redundant copying would contribute little to improving the overall reliability of the accelerator and would also incur additional area and power consumption overhead. It should also be noted that in wafer-level computation, since data propagates over long distances between multiple computational units (PEs), the data path is highly susceptible to transient soft errors. Considering this impact, this invention ensures that the feature map data and weight parameters are correct before entering the lockstep systolic array in complex wafer-level electromagnetic environments by copying the control logic and data loading path, thereby supporting local verification computation.

[0055] Unlike the main systolic array, the lockstep systolic array, as a selective lockstep computation engine, only performs recalculation verification on important elements in the sequence to be detected. Therefore, the array size of the lockstep systolic array can be set smaller than that of the main systolic array. Furthermore, to match the overall execution efficiency of the accelerator, the ratio of the array sizes of the lockstep systolic array and the main systolic array is equal to a preset asymmetric ratio. It should be noted that systolic arrays of any size are feasible in wafer-level chip environments, but the main systolic array and the lockstep systolic array must maintain a precise throughput ratio to prevent pipeline stalls. The reduction ratio of the lockstep systolic array's size must be balanced with the screening ratio of important elements to ensure that the lockstep systolic array can complete recalculation verification in a timely manner. If the array size is reduced excessively, the utilization rate of the smaller systolic array during recalculation verification will drop significantly, causing the main processing core to frequently enter a suspended state while waiting for recalculation verification results, thus losing the high bandwidth advantage of wafer-level chips.

[0056] According to one embodiment of the present invention, the preset asymmetric ratio is 1 / 4. It should be noted that the asymmetric ratio can be determined according to actual needs, and the present invention does not impose any special limitations.

[0057] For example, if the main pulsation array is a 64×64 two-dimensional pulsation array, then the lockstep pulsation array is a 32×32 two-dimensional pulsation array; if the main pulsation array is a 32×32 two-dimensional pulsation array, then the lockstep pulsation array is a 16×16 two-dimensional pulsation array; if the main pulsation array is a 24×24 two-dimensional pulsation array, then the lockstep pulsation array is a 12×12 two-dimensional pulsation array; if the main pulsation array is a 16×16 two-dimensional pulsation array, then the lockstep pulsation array is an 8×8 two-dimensional pulsation array.

[0058] As can be seen from the foregoing embodiments, there is a computational power asymmetry problem between the main systolic array and the lockstep systolic array. Based on this, this invention proposes an asynchronous recalculation mechanism to achieve local recalculation verification. Simply put, the neural network inference computation does not immediately stop upon the initiation of lockstep verification, but rather achieves asynchronous parallelism between the main processing core's inference and the lockstep processing core's verification through a delayed comparison mechanism. While an output sequence generated by the main processing core is stored in the buffer, important elements selected by the sorter are sent to the lockstep processing core for recalculation. This process is masked within the pipeline of the main processing core's subsequent computations, thus avoiding real-time blocking of the high-bandwidth data stream across the entire wafer. Only when the output sequence needs to be released, and the corresponding local verification task executed by the lockstep processing core has not yet been completed, will the main processing core enter a controlled pause state to ensure data consistency.

[0059] Specifically, the fundamental reason for adopting the asynchronous recalculation mechanism lies in solving the problem of computing power asymmetry between the main processing core and the lockstep processing core. Imagine the main processing core uses a full-size 24x24 two-dimensional systolic array, while the lockstep processing core is only equipped with a 12x12 two-dimensional systolic array, which is only 1 / 4 the size. If forced to perform synchronous computation (i.e., checking each step after computation), the main processing core must significantly reduce its speed to wait for the lockstep processing core, or the lockstep processing core must be increased to the same area as the main processing core (which would violate the design principle of low overhead). Through asynchronous design, the main processing core can run at full speed, while the lockstep processing core only performs "sampling" on selected important elements in the background. As long as the time for the lockstep processing core to complete this sampling does not exceed the time for the main processing core to compute the next batch of data, the overall throughput of the accelerator remains unaffected.

[0060] Although the main processing core runs the neural network's inference computation at full speed, there is no problem with intermediate results being incorrect. For example, suppose the accelerator performs inference computation on multiple layers of a neural network, with the output of the previous layer serving as the input of the next. If Layer N fails during the accelerator's inference computation, the main processing core's computation of Layer N+1 based on the output of Layer N would indeed be an invalid "waste operation." However, probabilistically, transient soft errors are extremely low-frequency events. The accelerator assumes "the current computation is correct" and continues at full speed. Only in the rare cases where an error is detected is the flush mechanism triggered, discarding the intermediate state of Layer N+1 just computed by the main processing core and rolling back to recalculate Layer N. This design trades "recalculation overhead in extremely low probabilities" for "full-speed operation most of the time," making it statistically the most efficient solution. The so-called "masked in the pipeline" refers to achieving "time parallelism" by utilizing the independence of hardware resources. When the main processing core completes the calculation of Layer N and immediately begins the calculation of Layer N+1, the output data of Layer N does not disappear but remains in the output buffer. At this time, an independent lockstep processing core starts in parallel, using its dedicated data loading path to read and verify the important elements of Layer N. Because the workload of the lockstep processing core is reduced (only a portion of the original loss needs to be calculated), although the computing power of the lockstep processing core is only 1 / 4 of that of the main core, the time required for it to complete the verification is usually less than or equal to the time required for the main core to calculate Layer N+1. Therefore, the actions of verifying Layer N and calculating Layer N+1 overlap in physical time. As long as the verification is completed before the data of Layer N is overwritten or released, then for the main processing core, this verification process is "non-existent," thus achieving latency masking. Only when the latency comparison does detect an inconsistency in the calculation results will the main processing core briefly pause and wait until the error is recovered before continuing to execute the inference task.

[0061] As can be seen from the foregoing embodiments, the lockstep processing core provides necessary redundancy protection for the lockstep processing core by copying the control logic and data loading path, thereby supporting the lockstep core to perform local verification calculations. At the same time, it does not copy the storage resources that have been protected by the error correction code, avoiding hardware-level duplication. Thus, while maintaining an extremely high risk-resistant accuracy, it significantly reduces the huge area and power consumption overhead caused by the traditional full lockstep design.

[0062] IV. Check the logic module

[0063] exist Figure 2In the accelerator architecture shown, the inspection logic module is used to compare whether the values ​​of the same elements in the sequence to be detected and the verification sequence are the same. If they are the same, the element is determined to pass the verification; otherwise, the element is determined to fail the verification.

[0064] In this process, the output sequence calculated by the main processing core and the verification sequence recalculated by the lockstep processing core are not compared immediately after calculation. Instead, they are temporarily stored, and the comparison operation is delayed until the output sequence is about to be released. This delay provides scheduling space for the lockstep processing core's calculation, avoiding resource contention and efficiency degradation caused by strict synchronization with the main processing core. Specifically, the output sequence is temporarily stored in the output buffer, while the verification sequence is temporarily stored in a dedicated register group or a small temporary queue associated with the checking logic module inside the accelerator. This temporary storage mechanism ensures that the comparison and verification process does not block the global data flow. Only at critical nodes where data is about to be released will the verification status of the temporary storage result determine whether to release the data normally or trigger an error recovery process.

[0065] It should be noted that the controller locks the write-back / release time of the output sequence by monitoring the task queue status. After the main processing core completes the calculation of specific feature map blocks and stores the output sequence into the output buffer, the controller generates a data release request according to the preset write-back strategy. At this time, the request signal is intercepted, and the checking logic module starts to perform comparison verification. If the verification result shows no error, the checking logic module issues a release permission instruction. If the verification result shows an error, the error recovery process is triggered.

[0066] V. Error Recovery Module

[0067] In the accelerator architecture proposed in this invention, the error recovery module (due to image length limitations, Figure 2 (Not shown in the image) is used to recalculate the values ​​of all elements that failed the verification in the sequence to be detected according to a preset recalculation mechanism, and transmit the recalculated value of each element to the output buffer to replace the value of the corresponding element in the output sequence.

[0068] According to an embodiment of the present invention, the preset recalculation mechanism is as follows: based on the verification result of the sequence to be detected, all elements in the sequence to be detected that have failed verification are determined; based on a preset data reuse factor, taking each element that has failed verification as the center, multiple preceding and following elements are selected from the output sequence, and all selected elements are combined with the corresponding elements that have failed verification to form a recalculation subset, wherein each element that has failed verification corresponds to a recalculation subset; based on the recalculation subset corresponding to each element that has failed verification, the data required to recalculate each recalculation subset is read from the input buffer, and the value corresponding to each element in each recalculation subset is recalculated through the lockstep processing core or the main processing core; the value corresponding to each element in each recalculation subset is transmitted to the output buffer to replace the value of the corresponding element in the output sequence.

[0069] According to one embodiment of the present invention, the preset data reuse factor is 16. The reason for setting the data reuse factor is that during the neural network inference calculation process, if the weights are fixed, the same input data may be repeatedly used to calculate multiple output elements. Therefore, it is necessary to set a data reuse factor to determine other elements that may be contaminated by the same error source. For example, if the Nth element is faulty, the index range of the elements that may be affected is approximately [NR, N+R], where R represents the data reuse factor.

[0070] It should be noted that each recalculated subset also needs to be compared and verified. Only after verification is successful can the values ​​of each element in the recalculated subset be transmitted to the output buffer. The correctness of the recalculation result can be determined by comparing the value of any identical element in the recalculated subset with that in the verification sequence. If they are the same, the verification is successful; otherwise, the verification is unsuccessful.

[0071] It should also be noted that the reason for determining whether the recalculation result is correct by comparing the recalculation subset with the verification sequence is that transient soft errors are low-probability events, and the verification sequence can be considered as the expected correct answer. As long as the value of any identical element in the recalculation subset and the verification sequence is the same, it can be considered that the error recovery module has achieved error correction.

[0072] Specifically, when executing the error recovery process, the error recovery module first suspends any unfinished subsequent computations in the main processing core and clears dirty data from both the main processing core and the lockstep processing core. Then, it reschedules the computation tasks of all elements affected by the error and re-executes these computations through either the lockstep processing core or the main processing core. Because the scope of error recovery is strictly limited, the computation process consumes relatively little time and energy.

[0073] It should be noted that although the accelerator architecture includes an error recovery module, there is still a possibility that the re-executed computation task may fail again. This usually stems from two situations: First, an extremely rare secondary transient soft error may strike the computational logic performing the recovery task again. In this case, re-executing the error recovery process or replacing the accelerator can resolve the issue. Second, the accelerator may have permanent physical damage, i.e., a hard error, causing certain logic gates or wiring to fail to produce correct results at any time. In this case, the accelerator needs to be replaced. Although the selective lockstep mechanism primarily protects against random bit flips caused by factors such as neutron impacts or electromagnetic interference, hardware-level fault injection analysis shows that if the disturbed random bit flips cannot stabilize during the recovery process, the error will still remain in the computational pipeline, causing the recalculated result to not match either the original erroneous result or the expected correct result.

[0074] As can be seen from the foregoing embodiments, the error recovery module can achieve transparent detection and recovery of transient soft errors, and because the error recovery range is strictly limited, the impact on the overall task delay of the accelerator can be minimized.

[0075] Based on the accelerators described in the foregoing embodiments, the present invention also proposes a processor for neural networks, used to perform inference computations of neural networks, the processor including multiple accelerators as described in the foregoing embodiments.

[0076] Specifically, the wafer-level processor proposed in this invention can be composed of thousands of structurally identical accelerators arranged in an array on the surface of a silicon wafer. These accelerators are interconnected through a high-speed on-chip network (NoW) to form a massively parallel computing system. Each accelerator contains a complete computing core (main processing core and lockstep processing core), independent storage medium (input / output buffer), and control logic. Moreover, each accelerator can serve as a computing node with independent fault tolerance, responsible for executing specific operators or hierarchical tasks in a neural network.

[0077] The processor's accelerators can maintain parallel execution while preserving the global execution flow of the deep neural network. This is because the processor uses an on-chip network to construct a clear data flow-driven mechanism for hierarchical computation. The hierarchical structure of the neural network is mapped onto the physical accelerator array within the processor. The computational task of the previous network layer is completed by a group of accelerators. After the computation results are stored in their respective output buffers, they are not directly written to off-chip dynamic random access memory (DRAM). Instead, they are packaged and routed directly through the on-chip network to the input buffer of the accelerator group responsible for computing the next network layer. This design makes the output of the previous network layer physically become the input of the next network layer. Each accelerator's controller monitors the state of its local input buffer and the communication protocol with the on-chip network. Once it receives sufficient data from the upstream accelerator, it automatically starts the computation task of the current network layer. Therefore, although the accelerators operate physically in parallel and independently, the high-speed data transport and data dependencies between the buffers implemented through the on-chip network strictly guarantee the logical order and global data flow between neural network layers.

[0078] It's important to note that a wafer-level processor is not a giant single core that is strictly synchronized under a single clock domain. Instead, it is a distributed system composed of thousands of functionally independent accelerators interconnected via a network on-chip (NoW). Each accelerator has its own independent control logic. When an accelerator detects a transient soft error and triggers an error recovery process, it only suspends its own pipeline and freezes its local program counter. Adjacent accelerators, because they each have their own independent control logic, do not share error interrupt signals from other accelerators. As long as the error interrupt signals from other accelerators do not cause a complete break in the data flow, the accelerators that have not encountered errors will continue to execute according to the predetermined instruction flow without interfering with each other.

[0079] It's also worth noting that within each accelerator, the scope of error recovery is strictly mathematically defined during the error recovery process. The accelerator utilizes a data reuse factor to precisely lock the area requiring rollback and recalculation within a local window before and after the failure point (e.g., only 32 elements), rather than clearing the computational state of the entire neural network layer. This minimally invasive recovery strategy avoids the global pipeline flushing common in traditional architectures, ensuring that the overhead of error handling is borne solely by the accelerator that experienced the error, rather than forcibly resetting thousands of other normally functioning accelerators through a global signal, thus achieving strict spatiotemporal isolation.

[0080] To better understand the processor proposed in this invention, the following explanation uses the inference computation process of a single accelerator in the processor as an example.

[0081] In a wafer-level processor, suppose accelerator number 2048 is executing a convolution block task of a ResNet-18 model, and the main systolic array in this accelerator continuously computes a set of output sequences containing 1024 elements at a frequency of 250MHz. During this time, a sorter extracts the computed elements in real time and marks the largest 25% (i.e., 256 elements) as important elements; the data of these important elements are asynchronously input into a lockstep systolic array for redundant computation, and the lockstep core verifies these key elements with minimal area overhead by delaying the execution rhythm by two cycles.

[0082] If the main systolic array calculates the element with index 500, the value of this element will be incorrect due to the random bit flipping on the wafer surface. Since index 500 is an important element selected by the sorter, this error will be included in the monitoring range. At this time, the calculation will not stop. The accelerator continues to process subsequent elements to maintain the overall throughput of the wafer-level pipeline until all 1024 elements in this batch have been calculated. When the accelerator initiates a write-back request to send the data to the off-chip memory through the wafer-level high-speed interconnect network, the check logic module intervenes to compare and verify. The check logic module compares the 256 pairs of element results of the main systolic array and the lockstep systolic array. It finds that the value of the element at index 500 does not match and immediately intercepts the data release signal of the accelerator.

[0083] Subsequently, the error recovery module dynamically calculates the 32 elements with faulty indices between 484 and 516 based on the data reuse factor. At this point, the accelerator pauses data transmission to the wafer-level bus and immediately clears these 32 potentially damaged elements from the local output buffer. It also drives the lockstep processing core to re-retrieve the corresponding data from the local input buffer for local recalculation. After recalculation, the new values ​​are verified to be correct and filled back into the local output buffer. Finally, the accelerator releases the suspend signal, and the complete 1024 elements are safely written through the wafer-level interconnect.

[0084] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By selectively copying, the copying of non-critical or already protected redundant modules such as on-chip caches that are already protected by ECC (error correction code) is avoided, which directly reduces the area overhead from the source; (2) By importance-aware sorting, the amount of data that needs to be repeatedly calculated and compared is greatly reduced, and the dynamic power consumption and the complexity of the comparison logic are significantly reduced; (3) By a refined local error recovery mechanism, the system pause time after the error occurs is greatly shortened, and the overall execution efficiency is improved.

[0085] This invention can be a system, method, electronic device, computing device, computer-readable medium, and / or computer program product. A computer program product primarily refers to a software product that implements this solution through a computer program.

[0086] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An accelerator for neural networks, used to perform inference computations of neural networks based on an instruction set, wherein, The instruction set includes multiple operation instructions, characterized in that the accelerator includes a main processing core, a lockstep processing core, a sorter, a checking logic module, and an error recovery module, wherein: The main processing core includes an input buffer, a controller, a main systolic array, and an output buffer. The input buffer stores feature maps and weight parameters of the neural network. The controller parses operation instructions to generate operation signals and reads feature maps and weight parameters from the input buffer based on the operation signals and transmits them to the main systolic array. The main systolic array receives feature maps and weight parameters, performs calculations to obtain an output sequence, and transmits the output sequence to the output buffer and a sorter. The output sequence includes multiple elements, each corresponding to a numerical value. The output buffer stores the output sequence. The sorter is used to reorder each element in the output sequence according to a preset reordering mechanism, and select a preset proportion of the elements that are sorted first as the sequence to be detected. The lockstep processing core is equipped with a lockstep pulsation array. The lockstep processing core is used to read the data required to recalculate the sequence to be detected from the input buffer, and to recalculate the value of each element in the sequence to be detected through the lockstep pulsation array to obtain the verification sequence. The inspection logic module is used to compare whether the values ​​of the same elements in the sequence to be detected and the sequence to be verified are the same. If they are the same, the element is determined to pass the verification; otherwise, the element is determined to fail the verification. The error recovery module is used to recalculate the values ​​of all elements that failed the verification in the sequence to be detected according to a preset recalculation mechanism, and transmit the recalculated value of each element to the output buffer to replace the value of the corresponding element in the output sequence. Wherein, the array size of the lockstep pulsation array is smaller than the array size of the main pulsation array, and the ratio of the array sizes of the lockstep pulsation array and the main pulsation array is equal to a preset asymmetric ratio.

2. The accelerator according to claim 1, characterized in that, The preset reordering mechanism is as follows: Based on the numerical value of each element in the output sequence, reorder each element in the output sequence in descending order.

3. The accelerator according to claim 1, characterized in that, The preset recalculation mechanism is as follows: Based on the verification results of the sequence to be detected, identify all elements in the sequence that failed the verification. Based on the preset data reuse factor, multiple preceding and following elements are selected from the output sequence, with each element that fails the verification as the center. All selected elements and their corresponding elements that fail the verification are combined to form a recalculation subset, where each element that fails the verification corresponds to a recalculation subset. Based on the recalculation subset corresponding to each element that fails the verification, the data required to recalculate each recalculation subset is read from the input buffer, and the value corresponding to each element in each recalculation subset is recalculated through the lockstep processing core or the main processing core. The value corresponding to each element in each recalculated reoperator set is transmitted to the output buffer to replace the value of the corresponding element in the output sequence.

4. The accelerator according to claim 3, characterized in that, The preset data reuse factor is 16.

5. The accelerator according to claim 1, characterized in that, The preset ratio ranges from 5% to 100%.

6. The accelerator according to claim 1, characterized in that, The preset asymmetric ratio is 1 / 4.

7. The accelerator according to claim 6, characterized in that, The main pulsation array adopts a two-dimensional pulsation array of 64×64, 32×32, 24×24 or 16×16, and the lockstep pulsation array adopts a two-dimensional pulsation array of 32×32, 16×16, 12×12 or 8×8.

8. The accelerator according to claim 1, characterized in that, The controller includes an instruction control unit and a data loading and storage unit, wherein: The instruction control unit is used to parse operation instructions to generate operation signals; The data loading and storage unit is used to read feature maps and weight parameters from the input buffer based on the operation signal and transmit them to the main pulsation array, and to transmit the output sequence calculated by the main pulsation array to the output buffer and the sorter.

9. A processor for neural networks, used to perform inference calculations in neural networks, characterized in that, The processor includes a plurality of accelerators as described in any one of claims 1-8.

10. A computer device comprising a memory, a processor as claimed in claim 9, and a computer program / instructions stored in the memory, characterized in that, The processor executes the computer program / instructions to perform inference calculations for the neural network.

Images