Selective deactivation of compute units for artificial neural networks

Description

[0001] The present invention relates to a method for operating artificial neural networks, which provides for a selective and resettable deactivation of hardware components at the runtime of the network, as well as an associated hardware architecture. State of the art

[0002] An artificial neural network (ANN) comprises an input layer, several processing layers, and an output layer. Input variables are read into the ANN at the input layer and processed through the processing layers to the output layer using a processing chain, which is typically parameterized. During the ANN training process, the values ​​for the processing chain parameters are determined that allow the processing chain to optimally map a set of learned values ​​for the input variables to a corresponding set of learned values ​​for the output variables.

[0003] The strength of k-nearest neighbors (KNNs) lies in their ability to process very high-dimensional data, such as high-resolution images, in massively parallel fashion by employing multiple physical processing units in each layer. An example of a hardware architecture for a KNN composed of many processing units is disclosed in GB 2 552 242 A. This document addresses reducing the hardware complexity and power consumption required to implement a KNN. To this end, the computational work in a layer is divided into two sub-steps, and the partial results obtained in each sub-step are combined to obtain the final result.

[0004] From the publication Chakradhar, S., Sankaradas, M., Jakkula, V., & Cadambi, S. (2010, June). A dynamically configurable coprocessor for convolutional neural networks. In Proceedings of the 37th annual international symposium on Computer architecture (pp. 247-257) a CNN (convolutional neural network) compiler is known which automatically translates a highly abstract network specification into a parallel microprogram. Disclosure of the invention

[0005] Within the scope of the invention, a hardware architecture for an artificial neural network (ANN) was developed. The ANN comprises a sequence of an input layer, several processing layers, and an output layer. Each layer maps a set of input variables to a set of output variables. The output variables of the input layer and of each processing layer are input variables of the layer that follows in the sequence. The input variables of the input layer are also the input variables of the ANN as a whole. The output variables of the output layer are also the output variables of the ANN as a whole.

[0006] The hardware architecture comprises a multitude of processing units, with the implementation of each layer distributed across at least two of these units. For example, half of the layer's input variables might be directed to the first processing unit, and the other half to the second. Similarly, in an output layer that performs classification, the responsibility for specific classes—such as the recognition of particular objects or situations—can be distributed among the different processing units. This allows for the parallel processing of highly dimensional data.

[0007] According to the invention, one and the same computing unit contributes to the realization of several successive layers.

[0008] For example, three consecutive layers can be implemented with two parallel processing units. An input signal from the first layer, fed to one of the processing units, then passes through all three layers within that processing unit and is finally output by that processing unit as the output signal of the third layer.

[0009] The term "processing unit" is not limited to the requirement that different processing units must be located on physically separate components that are assembled during the manufacturing of the hardware architecture. Rather, for example, an arrangement of numerous processing units can be integrated into a single application-specific integrated circuit (ASIC). Thus, the term "processing unit" generally refers to a unit, circuit, or assembly that contributes to the implementation of at least one layer of the KNN (Know-Nearest Neighbors) algorithm and can be selectively deactivated.

[0010] At least one resettable shutdown device is provided with which at least one computing unit can be selectively deactivated independently of the input variables supplied to it, such that at least one further computing unit remains activated in all layers to whose realization this computing unit contributes.

[0011] It was recognized that in this way, a dynamic compromise between computational accuracy on the one hand and energy consumption and heat generation of the KNN on the other can be set during runtime. Generally, most calculations in KNN are robust against errors; that is, a certain proportion of errors in these calculations only slightly degrades the quality of the KNN output. Disabling a processing unit results in certain intermediate results being completely missing, which is considered a special case of a computational error.

[0012] The condition that at least one other processing unit remains activated in all layers to whose implementation the deactivated processing unit contributes ensures that the KNN as a whole remains functional. If a layer of the KNN were completely deactivated, the processing chain of the KNN would be interrupted at that point, and the KNN would no longer provide any output.

[0013] The requirement that the shutdown mechanism be resettable ensures that the deactivation of computing units can adapt to the operating situation. For example, it may be most urgent to temporarily deactivate the computing units currently experiencing the highest heat load in order to extend the hardware's lifespan. Similarly, the extent to which deactivating a particular computing unit affects the final result delivered by the KNN as a whole depends on the current operating situation. In contrast, previous "pruning" approaches for removing neurons from the KNN are static: regardless of whether the neurons to be removed are determined before, during, or after the KNN is trained, this determination is binding for all subsequent inference operations of the KNN.

[0014] The fact that the processing unit can be deactivated independently of its input variables ensures, firstly, that deactivation can be enforced by a higher authority. If, for example, the deactivation aims to reduce the heat load on the processing unit, this goal may have the highest priority in order to prevent hardware damage. Secondly, deactivation independent of the input variables, while having the same effect on the computational accuracy of the KNN (Knowledge Neighborhood Network), has a much greater potential to reduce the processing unit's energy consumption than simply setting the input variables to zero.

[0015] The ability to selectively deactivate processing units is particularly beneficial for further miniaturization of hardware architecture. Previously, the empirical observation known as "Dennard's Scaling" held true: reducing transistor sizes in integrated circuits proportionally reduced energy consumption. However, in recent years, transistor size reductions have progressed to areas where energy consumption decreases less than proportionally. Depending on the available cooling options, it may therefore become impossible to operate all transistors continuously at the same time.

[0016] Selective deactivation of processing units is also beneficial even if, in principle, all available processing units could operate simultaneously. Every energy saving also saves costs and extends battery life in battery-powered devices. Furthermore, to increase availability, it can be advantageous to deliberately provide more processing units than are actually required to achieve a specified accuracy. The excess processing units can then, for example, remain deactivated initially and only be activated if other processing units fail.

[0017] In a particularly advantageous embodiment, a control unit is provided for the selective deactivation of computing units. The shutdown device comprises at least one switch that can be actuated by the control unit. This switch can, for example, be a transistor that can be switched electrically. However, the switch can also be, for example, a semiconductor switch or a light-sensitive resistor that can be switched optically.

[0018] In a particularly advantageous embodiment, the switch is arranged such that it can interrupt the supply of a clock signal and / or a supply voltage to the processing unit. Interrupting the supply voltage can completely prevent the processing unit from consuming power. If only the clock signal is interrupted, the majority of the usual energy consumption is still saved. At the same time, this avoids the need to reinitialize the processing unit upon subsequent reactivation, so that the processing unit is available again more quickly.

[0019] In the control unit, the deactivation and subsequent reactivation of processing units can be made dependent on arbitrary conditions. For example, a distribution of the omitted calculations can be specified. Thus, the calculation of every Nth neuron can be omitted, where N is a natural number. Instead of such a homogeneous distribution, however, the deactivations can also be based on an estimate of the importance of the individual calculation results made at the design time of the KNN. Calculations with low relevance are then preferentially deactivated. Studies have shown that the degradation in the output quality of the KNN is less in this case than in the case of a homogeneous distribution.

[0020] In a particularly advantageous embodiment, the control unit receives as input at least one output variable from at least one computing unit, a measure of the switching activity of at least one computing unit, a measure of the energy consumption of at least one computing unit, and / or a measure of the temperature of at least one computing unit.

[0021] In this way, for example, the workload of multiple processing units contributing to the realization of a given layer of the KNN can be evened out over time and / or space. This can reduce hardware aging processes. These aging processes are particularly temperature-dependent. For instance, the electronic properties of semiconductors are progressively degraded by interdiffusion between adjacent regions with different doping concentrations, with the rate of this interdiffusion depending exponentially on the temperature.

[0022] The temperature of the processing unit can be determined in various ways. Besides a temperature sensor, a thermal imaging camera, for example, is particularly suitable, as it can simultaneously record the temperature of many processing units in real time. Another cost-effective option is an indirect determination of the temperature via switching activity. For this, the switching activity is measured over a specific period, and a model is used that predicts the expected temperature of the processing unit from this data.

[0023] The selective shutdown of computing units, on the other hand, does not necessarily require the presence of a control unit. In a further advantageous embodiment, at least one computing unit includes a shutdown device that automatically deactivates the computing unit when a first condition is met, relating to the temperature of the computing unit, and / or to the time course of this temperature, and / or to the time course of the switching activity of the computing unit, and automatically reactivates it when a second condition is met, relating to the temperature of the computing unit, and / or to the time course of this temperature, and / or to the time course of the switching activity of the computing unit. This can then, in particular, eliminate the need for wiring to the central control unit.The KNN then develops its own dynamics with regard to the distribution of the total computing work performed, which depends, among other things, on which computing units are specifically used and to what extent.

[0024] The invention also relates to a method for operating an artificial neural network (ANN). The ANN comprises a sequence of an input layer, several processing layers, and an output layer. Each layer maps a set of input variables to a set of output variables. The output variables of the input layer and of each processing layer are input variables of the layer that follows in the sequence. The input variables of the input layer are also the input variables of the ANN as a whole. The output variables of the output layer are also the output variables of the ANN as a whole.

[0025] The KNN is implemented in a large number of computing units in such a way that the implementation of each layer is divided among at least two of the computing units.

[0026] At least one processing unit is selectively deactivated during the operation of the KNN (Kinky-Nearest Neighbors) upon fulfillment of a predefined deactivation condition, regardless of the input variables supplied to it, such that at least one other processing unit remains activated in all layers to whose implementation this processing unit contributes. The deactivated processing unit is reactivated during the operation of the KNN upon fulfillment of a predefined activation condition.

[0027] Analogous to what was previously described for the hardware architecture, a dynamic compromise between computational accuracy on the one hand and energy consumption and heat generation of the KNN on the other hand is realized at runtime of the KNN.

[0028] In a particularly advantageous embodiment, the disabling condition, and / or the activating condition, includes a measure of the quality of at least one input variable, a measure of the temporal and / or spatial rate of change of at least one input variable, a measure of a quality requirement for at least one output variable, and / or a measure of the relevance of at least one output variable.

[0029] For example, if the input data representing the input variables of the KNN is of high quality, more errors can be tolerated by disabling internal processing steps of the KNN while maintaining the same output quality than with poor input data quality, since the output quality of the DNN can be expected to have a higher baseline level with high input data quality. For example, a good signal-to-noise ratio in a camera image due to good lighting conditions can be considered an indication of high input data quality.

[0030] The quality requirement for output variables can depend, for example, on whether additional information sources are available for the question underlying those variables. If the uncertainty in the KNN output can be compensated for by additional information sources, more processing units can be deactivated. Such redundant information sources could be, for example, other different sensor data processing systems. Likewise, the output quality of the KNN can be improved again if redundant information sources fail by reactivating the previously deactivated processing units.

[0031] The quality requirements for output variables can depend, for example, on the extent to which results from previous time steps, or other contextual information, are available. If the KNN (Kinky-Nearest Neighbors) processes a temporally or spatially coherent stream of input data (e.g., a video sequence), information from the temporal or spatial vicinity of the input data currently being processed can be used to dynamically adjust the deactivation of processing units in specific parts of the network. For instance, if the KNN is tracking an object in a video sequence, certain parts of the neural network are relevant, while others are not currently contributing. The relevant computational steps are determined by the properties of the tracked object (features) and its position in the image.If the features and position of the object were determined by the KNN in time step t, this information can be used in the subsequent time step to deactivate more processing units in parts of the network that process currently irrelevant features or are responsible for currently irrelevant regions of the input image. For the currently irrelevant neurons, it can be assumed that their output value is small or zero anyway, so that deactivating the corresponding processing units does not result in a significant error in the KNN output. The method can be supported by motion prediction of the image information (e.g., by determining the optical flow).

[0032] The relevance of individual computational results within a neural network can be estimated using various methods.

[0033] One example of this is to simulate disabling specific processing units in the KNN (K-nearest Neighbors) for a set of training data and measure the impact on the network's output quality. The simulations can be performed on the actual hardware, in a virtual prototype of the hardware, or in a purely software-based simulation environment. If a significant reduction in output quality is observed across the training data distribution when certain processing steps are deactivated, then these calculations are highly relevant for that specific training dataset.

[0034] The changes in the output values ​​of the DNN when certain computational steps are omitted can also be determined analytically. For example, the gradients of the output values ​​with respect to the intermediate results can be calculated, or the output function of the neural network can be decomposed into the contributions of individual intermediate results.

[0035] In both exemplary approaches, both global relevance values—that is, relevance values ​​resulting from a complete (training) input data distribution—and relevance values ​​of specific groups of input data can be determined. For example, images depicting the same or similar objects, or images where the relevant object is located in a specific region, can be grouped together to determine separate relevance values ​​for these groups. This information can then be used to preferentially deactivate certain parts of the KNN (k-nearest neighbors) algorithm in a given context.

[0036] In a further particularly advantageous embodiment, the deactivation condition and / or the activation condition includes the temperature of at least one processing unit and / or the temperature of at least one heat sink coupled to this processing unit and / or the time course of the switching activity of this processing unit. In this way, as described above, hardware aging processes can be smoothed out. Furthermore, the thermal capacity of a heat sink can be used, for example, to access the performance of more processing units in the event of a short-term increase in demand.

[0037] In a further particularly advantageous embodiment, the KNN (Knowledge Neighborhood Network) serves to recognize objects and / or situations based on input variables. It uses contextual information to determine that the presence of a first subset of the objects and / or situations is less likely than the presence of a second subset. This contextual information can originate from any source. In addition to an external source, it is particularly suitable, for example, to derive the contextual information from the temporal evolution of the input variables themselves, either alternatively or in combination. In this case, deactivating processing units primarily dedicated to recognizing objects and / or situations from the first subset is preferred over deactivating processing units primarily dedicated to recognizing objects and / or situations from the second subset.

[0038] In this way, the energy used in the KNN (Kinetic Near-Nearest Neighbors) system can be focused on detecting those objects and / or situations whose presence is more probable. Systems for detecting objects and / or situations in the vicinity of vehicles, for example, for at least partially automated driving, are designed to detect a very large number of objects or situations. For instance, a vehicle driving at least partially autonomously must be able to react to all currently valid traffic signs in order to drive legally. However, the occurrence of all possible traffic signs and situations cannot be expected at all times. For example, there are neither traffic lights nor pedestrian crossings or level crossings on highways. Similarly, a warning sign indicating that the road leads to a shoreline is not to be expected far from bodies of water.The computing units responsible for detecting such unlikely objects or situations can therefore be preferentially deactivated without compromising the operational safety of the vehicle.

[0039] In a particularly advantageous embodiment, KNN serves to recognize objects and / or situations in the vicinity of a vehicle, with the additional contextual information including the position of the vehicle and / or the road category of a road currently being traveled by the vehicle.

[0040] Deactivating computing units based on this contextual information channels the non-determinism that deactivation inevitably introduces into the KNN into a more orderly path. It can then be plausibly demonstrated, particularly with regard to approval for public road traffic, that the energy and waste heat savings achieved by deactivating computing units do not come at the cost of increased risk.

[0041] If the KNN (Know-Nearest Neighbors) algorithm provides the hardware capability to selectively deactivate computing units, the method can be executed wholly or partially under software control. The corresponding software can then be sold, for example, as an update or upgrade to an existing control unit and is thus a standalone product. Therefore, the invention also relates to a computer program with machine-readable instructions which, when executed on a computer and / or a control unit, cause the computer and / or the control unit to execute the method provided by the invention. Likewise, the invention also relates to a machine-readable data carrier or a downloadable product containing the computer program.

[0042] Further measures improving the invention are described in more detail below, together with a description of preferred embodiments of the invention, with reference to figures. Examples of implementation

[0043] It shows: Figure 1: Exemplary embodiment of the hardware architecture 10 for a KNN 1; Figure 2: Exemplary embodiment of the method 100.

[0044] After Figure 1 The simplified example of KNN 1 comprises an input layer 2, two processing layers 3a and 3b, and an output layer 4. Hardware architecture 10 implements this KNN 1 using eight interconnected processing units 5a-5h. Each processing unit 5a-5h implements three neurons, represented as filled dots. Two of the processing units 5a-5h contribute to the implementation of each of the layers 2, 3a, 3b, and 4.

[0045] Each of the computing units 5a-5h is assigned a shutdown device 6a-6h, designed as a switch in the supply voltage and / or clock signal input, by means of which the respective computing unit can be taken out of service. All shutdown devices 6a-6h are controlled by a central control unit 7.

[0046] The KNN (Knowledge-nearest Neighbors) is designed to detect objects in the vicinity of a vehicle. Figure 1Only four traffic signs are shown as exemplary objects 1a-1d. In a typical driving situation on the motorway, the presence of a traffic sign from the first subset 8a, here a truck overtaking ban 1a or a speed limit 1b, is conceivable. A traffic sign from the second subset 8b, here a sign 1c for a pedestrian crossing or a warning sign 1d for a riverbank, does not, however, regularly occur on a motorway. Therefore, the processing units that are primarily responsible for recognizing these traffic signs 1c and 1d can be deactivated while driving on the motorway without compromising road safety.

[0047] Figure 2Figure 100 shows an embodiment of method 100. According to optional step 105, based on additional contextual information, such as the road category of the currently driven road, the subset 8a of the more likely objects 1a, 1b and the subset 8b of the less likely objects 1c, 1d are determined. According to step 110, it is continuously checked whether the deactivation condition is met. If this is the case (truth value 1), then in step 120, processing units 5a-5h are deactivated. If the subset 8b of less likely objects 1c, 1d has been determined in step 105, then, according to step 125, the deactivation of processing units 5a-5h, which are primarily used for the recognition of objects 1c, 1d from the subset 8b, is preferred.

[0048] In step 130, it is continuously checked whether an activating condition for reactivating the previously deactivated computing units 5a-5h is met. If this is the case (truth value 1), these computing units 5a-5h are reactivated in step 140.

Claims

1. Hardware architecture (10) for an artificial neural network, ANN (1), wherein the ANN (1) comprises a sequence of layers (2, 3a, 3b, 4), wherein each of the layers (2, 3a, 3b, 4) maps its input variable to an output variable by means of a plurality of neurons, and wherein the output variables of the layers (2) are each input variables of the respective following layer (3a, 3b, 4) in the sequence, wherein the hardware architecture (10) comprises a multiplicity of computing units (5a-5h), wherein a realization of each layer (2, 3a, 3b, 4) is divided between at least two of the computing units (5a-5h), wherein, when dividing the realization, the calculation of the plurality of the neurons is divided between at least two computing units, wherein at least one of the computing units has a proportion of the realization on a plurality of layers, wherein, for this realization, an input variable of a first one of the plurality of layers is supplied to the computing unit, passes through all successive layers of the plurality of layers within this computing unit, and is finally output from this computing unit as an output variable of the last one of the plurality of layers, and wherein the hardware architecture (10) comprises at least one resettable shutdown apparatus (6a-6h) that is configured to selectively deactivate at least one computing unit (5a-5h) such that at least one further computing unit (5a-5h) remains activated in all layers (2, 3a, 3b, 4) to the realization of which this computing unit (5a-5h) contributes.

2. Hardware architecture (10) according to Claim 1, wherein a control unit (7) is provided for the selective deactivation of computing units (5a-5h), and wherein the shutdown apparatus (6a-6h) comprises at least one switch that is able to be actuated by the control unit (7).

3. Hardware architecture (10) according to Claim 2, wherein the switch is arranged in such a way that it is able to interrupt the supply of a clock signal and / or of a supply voltage to the computing unit (5a-5h).

4. Hardware architecture (10) according to either of Claims 2 and 3, wherein the control unit (7) contains, as input, at least one output variable leaving at least one computing unit (5a-5h), a measure of the switching activity of at least one computing unit (5a-5h), a measure of the energy consumption of at least one computing unit (5a-5h), and / or a measure of the temperature of at least one computing unit (5a-5h).

5. Hardware architecture (10) according to one of Claims 1 to 4, wherein at least one computing unit (5a-5h) contains a shutdown apparatus (6a-6h) that automatically deactivates the computing unit (5a-5h) if a first condition for the temperature of the computing unit (5a-5h), and / or for the temporal profile of this temperature, and / or for the temporal profile of the switching activity of the computing unit (5a-5h), is met, and automatically reactivates said computing unit if a second condition for the temperature of the computing unit (5a-5h), and / or for the temporal profile of this temperature, and / or for the temporal profile of the switching activity of the computing unit (5a-5h), is met.

6. Method (100) for operating an artificial neural network, ANN (1), wherein the ANN (1) comprises a sequence of layers (2, 3a, 3b, 4), wherein each of the layers (2, 3a, 3b, 4) maps its input variable to an output variable by means of a plurality of neurons and wherein output variables of the layers (2) are each input variables of the respective following layer (3a, 3b, 4) in the sequence, wherein the ANN (1) is realized in a multiplicity of computing units (5a-5h) in such a way that a realization of each layer (2, 3a, 3b, 4) is divided between at least two of the computing units (5a-5h), wherein, when dividing the realization, the calculation of the plurality of the neurons is divided between at least two computing units, wherein at least one of the computing units has a proportion of the realization on a plurality of layers, wherein, for this realization, an input variable of a first one of the plurality of layers is supplied to the computing unit, passes through all successive layers of the plurality of layers within this computing unit, and is finally output from this computing unit as an output variable of the last one of the plurality of layers, wherein, if a predetermined deactivating condition (110) is met during operation of the ANN (1), at least one computing unit (5a-5h) is selectively deactivated (120) in such a way that at least one further computing unit (5a-5h) remains activated in all layers (2, 3a, 3b, 4) to the realization of which this computing unit (5a-5h) contributes. wherein, if a predetermined activating condition (130) is met during operation of the ANN (1), the deactivated computing unit (5a-5h) is reactivated (140).

7. Method (100) according to Claim 6, wherein the deactivating condition (110) and / or the activating condition (130) also contain / contains a measure of the quality of at least one input variable, a measure of the temporal and / or spatial rate of change of at least one input variable, a measure of a quality requirement of at least one output variable, and / or a measure of a relevance of at least one output variable.

8. Method (100) according to either of Claims 6 and 7, wherein the deactivating condition (110) and / or the activating condition (130) also contain / contains the temperature of at least one computing unit (5a-5h), and / or the temperature of at least one heat sink coupled to this computing unit (5a-5h), and / or the temporal profile of the switching activity of this computing unit (5a-5h).

9. Method (100) according to one of Claims 6 to 8, wherein the ANN (1) is used to recognize objects and / or situations (1a-1d) on the basis of input variables, wherein, on the basis of an item of context information, it is ascertained (105) that the presence of a first subset (8a) of the objects and / or situations (1a-1d) is less likely than the presence of a second subset (8b) of the objects and / or situations (1a-1d), and wherein the deactivation (120) of computing units (5a-5h) that are predominantly used to recognize objects and / or situations (1a, 1b) from the first subset (8a) is preferred (125) over the deactivation (120) of computing units (5a-5h) that are predominantly used to recognize objects and / or situations (1c, 1d) from the second subset (8b).

10. Method (100) according to Claim 9, wherein the ANN (1) is used to recognize objects and / or situations (1a-1d) in the surroundings of a vehicle, and wherein the additional context information comprises the position of the vehicle and / or the road category of a road currently being travelled on by the vehicle.

11. Computer program containing machine-readable instructions which, when executed on a computer comprising a plurality of computing units, cause the computer to carry out a method (100) according to one of Claims 6 to 10.