Method for implementing mapping from at least one input value to at least one output value in fault tolerant manner using processing device with plurality of processing units, and processing device
The method addresses hardware errors in neural network accelerators by assigning criticality information to structures within a processing device, enabling fault-tolerant mapping that maintains functionality and reduces redundancy, thus ensuring safety in critical applications.
Patent Information
- Application Number
- JP2025074959
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-04-29
- Filing Date
- 2025-04-28
- Publication Date
- 2025-11-11
AI Technical Summary
Integrated circuits used for computationally intensive applications like neural networks are susceptible to hardware errors due to transient and permanent faults, which can compromise their functionality, especially in safety-critical applications.
A method for fault-tolerant mapping by assigning criticality information to each structure within a processing device, allowing selective modification or omission of calculations based on error detection and criticality, ensuring minimal error propagation and resource efficiency.
Maintains functional safety in applications like automotive systems by minimizing mapping errors and reducing redundant designs, even in the presence of hardware faults.
Smart Images
Figure 2025168671000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a method for fault-tolerantly implementing a mapping from at least one input value to at least one output value by a processing device having a plurality of processing units, as well as a computing unit, a computer program and a processing device for implementing the same. [Background technology]
[0002] Background of the Invention For example, for computationally intensive computer applications in the field of artificial intelligence or neural networks, dedicated hardware accelerators in the form of integrated circuits can be used. Such hardware accelerators may include multiple processing units that perform computational structures frequently occurring in neural networks, such as basic operations like multiply-add operations. Accelerators may also include multiple easily programmable processing units, such as so-called shader computation cores. High computation speeds are achieved, in particular, by the parallel execution of multiple processing units. To minimize the energy consumption and cost of such hardware accelerators, integrated circuits can be designed with as small a structural size as possible, but this makes them more susceptible to hardware errors. In addition to transient errors, such as bit flips, permanent errors, such as those due to manufacturing process variations and aging, can also occur. Summary of the Invention [Problem to be solved by the invention]
[0003] Disclosure of the Invention According to the invention, a method for fault-tolerantly implementing a mapping from at least one input value to at least one output value by a processing device having a plurality of processing units, as well as a computing unit and a computer program and processing device for implementing the same, are proposed, having the features of the independent claims. Advantageous configurations are the subject of the dependent claims and the following description. [Means for solving the problem]
[0004] In the present invention, a means for mapping at least one input value to at least one output value is used, the mapping being implemented by a processing device having a plurality of processing units and including a plurality of structures with calculation operations that can be evaluated by any one or more of the processing units, each of which is assigned criticality information including information on how critical an error-free evaluation of the structure is to the error-free evaluation of the mapping. If at least one processing unit has an error function, at least one structure is selected based on the criticality information of the structures, where it is determined whether the at least one structure should be modified and evaluated or not evaluated, and a mapping is applied to the input values, where the selected at least one structure is modified and evaluated or not evaluated depending on the previously determined decision. The error function of a processing unit exists, in particular, when the result calculated by the processing unit is incorrect. For example, the calculation or evaluation implemented by the processing unit can be performed using input values whose result or output value is known. The existence of the error function can then be determined from a comparison of the currently determined output value with the known output value.
[0005] According to the present invention, if an error occurs in one or more of the processing units, the mapping can be further implemented by the processing device, so that the mapping error is minimized. The functionality of the processing device can thus be maintained, which is important, for example, in the context of functional safety in safety-critical applications, such as automotive applications. Redundant designs, for example, in which processing devices are duplicated or at least some of the processing units are redundant, can be largely omitted. It should be noted that technical systems, such as vehicles, robots, manufacturing machines, or medical machines, can be controlled depending on the output of the processing units.
[0006] The structure may be any operation or calculation, for example, a relatively basic operation such as multiplication, addition, or multiply-and-accumulate, or a complex operation including multiple basic operations. In a neural network where an image is processed using different color channels, for example, processing one color channel may represent a complex operation.
[0007] The term "mapping" is generally understood as a mapping or function that means to map input values to output values. This mapping is implemented in particular by neural networks, where the structures are weights or multiplications with weights, neurons, layers or channels. For example, these structures may be multiply-accumulate units, where the processing units are configured to perform multiply-accumulate operations.
[0008] According to one configuration, the criticality information of each structure comprises a relevance value, wherein the selection of the at least one structure is performed depending on the relevance value of the structure, in particular, the selected at least one structure has a minimum relevance value and / or a relevance value below a predetermined relevance value threshold, whereby a structure having a higher relevance value is assumed to be more important or more critical to the accuracy of the mapping.
[0009] According to one configuration, the criticality information for each structure includes an error measure indicating the expected average or maximum error of the mapping if the structure is modified and evaluated or not evaluated. The expected or estimated average or maximum error of the mapping can be calculated, for example, before using the mapping or the processing device and / or can be determined using tests. If the mapping is a neural network or generally a machine learning based mapping, training data used for training can be used here, for example. Each structure can be assigned multiple error measures, for example, an error measure for not evaluating the structure and an error measure for evaluating it with a modification.
[0010] According to one configuration, the selection of the at least one structure is performed depending on an error measure of the structure, in particular, the selected at least one structure has a low error measure and / or an error measure below a predetermined error measure threshold, where a low error measure corresponds to a low error of the mapping in the case of no evaluation or a modified evaluation of the structure.
[0011] According to one configuration, an expected global error of the mapping is determined based on an error measure of at least one selected feature, where in particular the expected global error is output and / or transmitted to a functional module that processes the mapping result. This global error can be determined, for example, by summation or by using a statistical method suitable for error estimation or error propagation.
[0012] According to one configuration, each processing unit is configured to evaluate the structure or a part thereof with different precision, to evaluate the structure with reduced precision in case of partial error function of the processing unit, and / or to evaluate the structure two or more times in parallel, each with reduced precision, wherein at least one selected structure is evaluated in a modified manner, i.e., with reduced precision. In case of partial error function of the processing unit, for example, a (selected) structure can be evaluated with reduced precision by a processing unit. In case of complete error function of the processing unit, for example, two (selected) structures can be evaluated with reduced precision by other (fully functional) processing units in parallel.
[0013] According to one configuration, the criticality information for each structure includes precision loss information indicating precision loss of the structure in the case of evaluation with reduced precision (compared to evaluation with full precision). Here, the selection of the at least one structure is performed in particular depending on the precision loss information of the structure. In particular, the selection of the at least one selected structure has low precision loss and / or precision loss below a predetermined precision loss threshold. An error measure of the criticality information can also be determined from the precision loss information and, in some cases, can be used for the selection of the at least one structure.
[0014] According to one configuration, different precisions correspond to different quantizations of values, in particular different bit widths. For example, a processing unit may be configured to perform full-bit width (e.g., 8 bits) or half-bit width (e.g., 4 bits) multiplication or multiply-accumulate operations (i.e., operations in which entries of two input vectors are multiplied pairwise and the products are added), where in the full-bit width, one pair of two binary values having the full bit width (which are multiplied together) is processed, and in the half-bit width, two pairs of two binary values having the half-bit width (which are multiplied together) are processed in parallel. For this, a processing unit may have, for example, four sub-processing units in which half-bit width multiplications are evaluated, where in the case of full-bit width processing, the results of the half-bit width multiplications are appropriately weighted and added. Even if only one of these sub-processing units has an error, half-bit width processing can still be performed.
[0015] According to one configuration, the criticality information of each structure includes correlation information indicating at least one other structure whose result value is strongly correlated with the result value of the structure, in which case, during the application of the mapping to the input values, at least one structure is not evaluated, and estimated result values are determined from the result values of the at least one other structure using the correlation information, and these result values are used instead of the result value of the structure that is not evaluated. Depending on the strength of the correlation, in this configuration, a more or less large error occurs in the estimated result value. It may be assumed that a structure is considered to be strongly correlated only if the absolute value of the correlation (for example, taking a value between -1 and +1) is greater than a predetermined minimum correlation (for example, 0.7, 0.8, or 0.9).
[0016] According to one embodiment, the criticality information of each structure includes resource information indicating which computing power resources are saved and to what extent if the structure is not evaluated or if it is evaluated in a modified manner. Here, the computing power loss caused by the error function of at least one processing unit is determined. In particular, at least one structure is selected so that the saved computing power resources compensate for the computing power loss or compensate within a predetermined tolerance. The resource information indicates, for example, how much computing time is saved in relation to the processing unit.
[0017] According to one configuration, a distribution table is provided for allocating structures to processing units, whereby the distribution table is modified so that at least one selected structure is not evaluated or is evaluated in a modified manner. According to an alternative configuration, a distribution functionality is implemented, whereby structures to be executed are dynamically allocated to processing units taking into account criticality information, whereby the selection of at least one structure is made in particular in response to the application duration of the mapping reaching or exceeding a preset maximum duration. These embodiments effectively enable the implementation of a transition to evaluation of the mapping with modified or no evaluation of the structure in the event of an error function.
[0018] A computing unit according to the invention, for example a system on chip (SoC), a multi-core system from a heterogeneous core system, an accelerator board (e.g. GPCPU, etc.) and / or a chiplet realization, in particular in a control device of a motor vehicle, which implements control functionality using neural networks and includes a processing device with multiple processing units or which controls one of them, in particular programming-wise, is configured to carry out the method according to the invention.
[0019] It is also advantageous to implement the method according to the invention in the form of a computer program or computer program product with program code for performing all method steps, since this leads to particularly low costs, especially if the executing control device is also used for further tasks and therefore remains existing. Finally, a machine-readable memory medium on which the computer program is stored as described above is envisaged. Suitable memory media or data carriers for providing the computer program are, in particular, magnetic, optical and electrical memories, such as hard disks, flash memories, EEPROMs, DVDs, etc. Downloading the program via a computer network (Internet, intranet, etc.) is also possible. Such downloading can take place here either wired or wirelessly (e.g., via a WLAN network, 3G, 4G, 5G, 6G connection, etc.).
[0020] Further advantages and features of the present invention will become apparent from the following description and accompanying drawings.
[0021] The invention is illustrated diagrammatically in the drawings by way of an embodiment and will be explained in the following with reference to these drawings. [Brief explanation of the drawings]
[0022] [Figure 1] FIG. 1 illustrates a processing apparatus having multiple processing units. [Figure 2] 4 is a flowchart illustrating a method for fault-tolerantly implementing a mapping by a processing device having multiple processing units, according to an embodiment of the present invention. [Figure 3] FIG. 1 illustrates an exemplary structure of a neural network. [Figure 4] FIG. 10 illustrates the determination of the error in the case of evaluation of a neuron with reduced precision. DETAILED DESCRIPTION OF THE INVENTION
[0023] Embodiments of the invention 1 shows a processing device 2 having a number of processing units 4. These processing units 4 are, for example, hardware processing units that implement certain calculations in hardware, i.e., by a corresponding arrangement of circuit elements. These processing units 4 may include (relatively simple) calculation cores (for example, so-called shader calculation cores) that implement calculations by executing programs.
[0024] Furthermore, the processing device 2 may have a control unit 6 which controls the processing units 4, i.e. in particular transmits or loads into each processing unit 4 respective input values for each processing unit 4, causes the processing of these input values by the processing unit 4, receives output values of a processing unit 4 from the processing unit 4 and / or stores them and / or transmits them to (other) processing units 4 as input values for subsequent calculations.
[0025] The processing device 2 may generally include any number of processing units 4, in particular hundreds or even more than a thousand processing units. The processing units 4 and optionally the control unit 6 may, for example, be located on a single chip.
[0026] Within a single processing unit or a small number of processing units, cases may arise in which errors occur, in particular hardware errors. Some types of errors may cause a complete failure of the affected processing unit. Other types of errors may lead to a partial failure of the affected processing unit, so that the calculations performed thereon can in particular only be performed with reduced precision (instead of full precision). The method according to the invention achieves that a mapping from at least one input value to at least one output value implemented by the processing device 2, for example in the form of a neural network or generally in the form of a mapping based on machine learning, can still be evaluated or applied even in the event of an error function occurring within one or more processing units, with the error of the mapping resulting from the error function being kept as small as possible.
[0027] 2 shows a flowchart of a method for fault-tolerantly implementing a mapping from at least one input value to at least one output value by a processing device having multiple processing units, according to an embodiment of the present invention. The method can be performed, for example, by a control unit of the processing device.
[0028] In optional step 100, an error identification is performed, in which processing units having an error function in the form of a complete or partial failure are determined. For this purpose, for example, in each processing unit, a calculation implemented by the processing unit is performed or evaluated using input values whose result or output value is known. Then, by comparing the currently determined output value with the known output value, it can be determined whether and what kind of error function is present. This procedure can be performed repeatedly, for example, at predetermined time intervals or at a specific time point (e.g., when the processing device is switched on). Data regarding the corresponding error function indicating the error function of the processing unit can be stored, so that step 100 is only performed optionally; otherwise, previously known data regarding the error function is used, for example, by reading out the stored data.
[0029] Furthermore, the processing device is based on implementing a mapping including a plurality of structures with a calculation operation. These structures can be considered as partial maps or partial functions that at least partially form the mapping. These structures can be evaluated by any one or more processing units, i.e., the evaluation of a structure can be selectively assigned to any one or more processing units. Each of these structures is assigned criticality information, which includes information on how critical an error-free evaluation of the structure is to an error-free evaluation of the mapping. This criticality information can also be considered as information indicating to what extent an erroneous evaluation causes an error in the mapping. The criticality information is determined, in particular, offline.
[0030] In step 110, if at least one processing unit has an error function, at least one structure is selected based on the criticality information of the structure. Here, it is also determined whether the at least one structure should be modified and evaluated or not evaluated. In particular, the at least one structure is selected so that the structure or its evaluation is as little critical as possible to the error-free evaluation of the mapping.
[0031] In step 120, a mapping is applied to the input values, in which at least one selected structure is either evaluated modified (as determined in step 110) or not evaluated. A modified evaluation may in particular consist in that the structure is evaluated with reduced precision (e.g. in case of partial erroneous functioning of the processing unit). If a structure is not evaluated, its result or output value (which may, for example, be required for further calculations) is replaced by a reference value, for example zero, or, if the correlation between the structures is known, by the output value of a structure that is strongly correlated (with the structure not evaluated) or a value derived therefrom using the correlation.
[0032] In general, computations identified as highly critical (recognizable based on the criticality information) are transferred online to processing hardware that is still fully functional, while other, less critical computations are performed on (partially) faulty processing hardware, or alternatively, they are discarded or replaced with mutual information, as described in the previous configuration.
[0033] Furthermore, a method for determining criticality information of structural parts of a neural network evaluated using a computing device (e.g., a hardware accelerator) is described (e.g., when applying a neural network to new data, so-called "inference"). Each computing device may include one or more computing units. The dimensions of these structural parts may range from individual criticalities to larger structural parts (e.g., neurons, filters, or channels). FIG. 3 illustrates an example of a neural network structure, showing the structural parts as neurons 20, channels 22, shape parts 24 (or their recognition parts), filters 26, and layers 28. Starting from a pre-trained (trimmed or complete) neural network, criticality information for each structural part of the neural network is determined. In the following, unless otherwise specified, the present invention will be described based on neurons as structural parts. However, it should be emphasized that the present invention is applicable to both higher-dimensional and lower-dimensional structural parts.
[0034] In the first configuration, the criticality is calculated using a relevance metric, such as the L1 metric (H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, "Pruning filters for efficient convents," arXiv preprint arXiv:1608.08710, 2016), which is used to prune the convents after training. The selected relevance metric is calculated for each structure, in this example, for each neuron. The calculated relevance metric (also called relevance value) is used to rank the structures according to their relevance, i.e., the structure (here, neuron) with the highest L1 value is the most relevant, and vice versa. These or similar relevance metrics (or values) can also be provided using other methods, such as resilience prediction methods from the AI domain (e.g., the method presented in C. Schorn, A. Guntoro and G. Ascheid, “Accurate neuron resilience prediction for a flexible reliability management in neural network accelerators,” in Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2018, pp. 979-984).
[0035] In other configurations, the criticality information can be derived (at least in part) using mutual information of existing structures within each layer. This mutual information can be determined, for example, by calculating cross-correlations or correlations between structures within each layer. For example, (h * w *For a layer with ch) neurons (i.e., h*w neurons for the ch channel), correlations between all neurons are calculated to determine a correlation matrix of size [h*w,ch]. This mutual information is calculated offline. Online, i.e., when using neural networks, the least relevant structures may be removed in the event of a (random) hardware error. Purposefully, rather than setting its output to zero (or any other specified value), the neuron most highly correlated with the missing neuron may be determined, and the output of this highly correlated neuron replaces the output of the missing neuron. Information about the highly correlated (other) structures may be included in the criticality information as correlation information. This replacement by the output of the highly correlated (other) structures leads to relatively little information loss or error.
[0036] In general, an error measure can be determined (e.g., as a measure of information loss) that indicates the average or maximum error that occurs, for example, when a structure output is deleted or replaced with the output of a correlated structure. Such an error measure can be included in the structure criticality information. In an offline analysis, it is also possible to determine the number of structures per layer that can be deleted before a predefined capacity tolerance T is no longer met. If an error measure is included in the structure criticality information, an expected overall error can be determined online from the error measure for each of the structures that are not evaluated or that are evaluated in a modified manner (e.g., replaced with a correlated output or calculated with reduced precision). This overall error should, for example, be below a predetermined maximum overall error threshold.
[0037] In a third configuration, criticality information is determined based on the quantization sensitivity of the neural network. When determining the quantization configuration of the network (i.e., offline during design time), the obtained sensitivity can be used to determine structural parts of the neural network that will result in no information loss or relatively little information loss (relative to other structural parts) when calculated at reduced precision. Corresponding information can be included in the criticality information of the structural parts as precision loss information. Essentially, individual calculations of the network can be ranked according to their relevance / criticality based on their impact on the algorithmic performance of the neural network when implemented at reduced bit width. The criticality ranking can be determined by iterative evaluation of quantization cost functions for layers (e.g., neurons, segments, channels, etc.) where only a portion of the layer is evaluated at reduced bit width.
[0038] Figure 4 shows a diagram of the determination of the error when estimating a neuron with reduced precision. It shows a neuron device 32 consisting of 3x3 neurons. For each neuron 34, 36, 38, the error e that occurs when this neuron is evaluated with reduced precision (compared to evaluation with full precision) is calculated. i , where i∈{1;2;...;n} are calculated. When evaluating a neuron, for example, weighted input values are added, i.e., a multiply-and-accumulate operation is performed. In the case of a reduced-precision evaluation, the multiply-and-accumulate operation is performed, for example, at half-bit width instead of full-bit width. Subsequently, the criticality can be determined based on the error. The error is determined here as the deviation, for example, using the L2 norm (the square of the deviation), between the full-bit width evaluation and the half-bit width evaluation. In Figure 4, neurons evaluated with reduced precision are each indicated by a diagonal line. The other neurons are evaluated with full precision. The error, for example, the L2 norm, is calculated here on the sum of the output values of all neurons (for example, as an L2 vector norm;
number
number
[0039] Furthermore, different precisions or quantizations can already be taken into account when designing a neural network. For example, if y represents the output value when evaluated in full precision (e.g., floating-point arithmetic), then:
number
number
number
[0040] The network configuration can be optimized so that the network capacity is sufficient not only for normal error-free operation, but also when running in a fault-prone mode of operation on (partially) errored processing units. In other words, the mapping can be implemented so that the cost function J, which compares the cost of evaluation with full precision with the cost of evaluation with reduced precision, is optimized (i.e., approaches as close as possible to a desired target value), and in particular minimized.
Claims
1. A method for fault-tolerantly implementing a mapping from at least one input value to at least one output value by a processing device (2) having a plurality of processing units (4), comprising: the map includes a plurality of structures (20, 22, 24, 26, 28) having computational operations that are evaluable by any one or more of the processing units (4); each of the structures (20, 22, 24, 26, 28) is assigned criticality information containing information regarding how critical an error-free evaluation of the structure is to an error-free evaluation of the mapping; If at least one processing unit (4) has an error function, a selection (110) of at least one structure is performed based on the criticality information of the structure; determining whether the at least one structure should be modified and evaluated or not evaluated; The mapping is applied 120 to the input values; A method wherein the at least one selected structure is modified and evaluated or not evaluated.
2. 2. The method of claim 1, wherein the criticality information of each of the structures (20, 22, 24, 26, 28) comprises a relevance value, and the selection (110) of the at least one structure is made depending on the relevance value of the structure, in particular the selected at least one structure has a minimum relevance value and / or a relevance value below a predetermined relevance value threshold.
3. 3. The method of claim 1, wherein the criticality information for each structure comprises an error measure indicating an average or maximum expected error of the mapping if the structure is modified and evaluated or not evaluated.
4. 4. The method of claim 3, wherein the selection (110) of the at least one structure is made depending on the error measure of the structure, in particular the selected at least one structure having a low error measure and / or an error measure below a predetermined error measure threshold.
5. 5. The method according to claim 3 or 4, wherein an expected overall error of the mapping is determined based on the error measure of the at least one selected structure, and in particular the expected overall error is output and / or transmitted to a functional module that processes the result of the mapping.
6. the mapping implements a neural network, and the structural elements (20, 22, 24, 26, 28) are weights or multiplications with weights, neurons (20, 34, 36, 38), layers (28), or channels (24); and / or The method of claim 1 , wherein the structure is a multiply-accumulate unit and the processing unit is configured to perform a multiply-accumulate operation.
7. 7. The method according to claim 1, wherein each processing unit (4) is configured to evaluate the structure (20, 22, 24, 26, 28) or parts thereof with different precision, and each processing unit (4) is configured to evaluate the structure with reduced precision in case of a partial error function of the processing unit and / or to evaluate the structure two or more times in parallel, each with reduced precision, and wherein a selected at least one structure is evaluated with a modified, i.e. reduced, precision.
8. 8. The method of claim 7, wherein the criticality information of each of the structures (20, 22, 24, 26, 28) comprises precision loss information indicative of a precision loss of the structure in case of evaluation with reduced precision, and in particular the selection of the at least one structure is made depending on the precision loss information of the structures (20, 22, 24, 26, 28), in particular the selected at least one structure has a low precision loss and / or a precision loss below a predetermined precision loss threshold.
9. 9. The method of claim 8, wherein the mapping is implemented such that a cost function comparing the cost for evaluation at full precision with the cost for evaluation at reduced precision is optimized, in particular minimized.
10. 10. The method according to any one of claims 7 to 9, wherein different precisions correspond to different quantizations of the values, in particular different bit widths.
11. 11. The method of claim 1, wherein the criticality information for each structure (20, 22, 24, 26, 28) includes correlation information indicating at least one other structure having a result value that is strongly correlated with the result value of the structure in question, and wherein, during application of the mapping to the input values, the at least one structure is not evaluated, and an estimated result value is determined from the result value of the at least one other structure using the correlation information, and this result value is used instead of the result value of the structure that is not evaluated.
12. 12. The method according to claim 1, wherein the criticality information of each of the structural elements (20, 22, 24, 26, 28) comprises resource information indicating which computing power resources are saved and to what extent if the structural element is not evaluated or if it is evaluated in a modified manner, and a computing power loss caused by the erroneous function of the at least one processing unit (4) is determined, and in particular the at least one structural element is selected such that the saved computing power resources compensate for the computing power loss or compensate within a predetermined tolerance.
13. a distribution table is provided for allocating the structures (20, 22, 24, 26, 28) to the processing units (4), and the distribution table is modified so that the selected at least one structure is not evaluated or is evaluated in a modified manner, or 13. The method according to any one of claims 1 to 12, wherein a distribution functionality is implemented, and structures to be evaluated are dynamically assigned to the processing units (4) taking into account the criticality information, and in particular the selection of the at least one structure is made in response to the application duration of the mapping reaching or exceeding a preset maximum duration.
14. 14. Method according to any one of the preceding claims, wherein for each of the processing units (4) it is determined (100) whether the processing unit (4) has an error function.
15. A processing device (2) comprising a plurality of processing units (4) and a control unit (6), said control unit (6) being configured to perform all method steps of the method according to any one of claims 1 to 14.
16. A computing unit configured to perform all the method steps of the method according to any one of claims 1 to 14.
17. 15. A computer program which, when evaluated on a computing unit, causes the computing unit to carry out all the method steps of the method according to any one of claims 1 to 14.
18. 18. A machine-readable memory medium having stored thereon the computer program of claim 17.