Selective deactivation of computational units of artificial neural networks

Abstract

A hardware architecture (10) for an artificial neural network KNN (1), wherein the KNN (1) comprises a close arrangement of an input layer (2), a plurality of processing layers (3a, 3b) and an output layer (4), wherein each layer (2, 3a, 3b, 4) maps a set of input variables to a set of output variables, and wherein the output variables of the input layer (2) and each processing layer (3a, 3b) are respectively input variables of subsequent layers (3a, 3b, 4) in the close arrangement, wherein the hardware architecture (10) comprises a large number of computers. A computing unit (5a-5h) is provided, wherein the realization of each layer (2, 3a, 3b, 4) is divided between at least two computing units (5a-5h), and at least one resettable disconnection device (6a-6h) is provided, by means of which the at least one computing unit (5a-5h) is selectively deactivated independently of the input variables supplied to the at least one computing unit (5a-5h), so that at least one other computing unit (5a-5h) remains activated in all layers (2, 3a, 3b, 4) realized by the computing unit (5a-5h). A corresponding method (100) for operating an artificial neural network KNN (1), wherein a computing unit (5a-5h) is selectively deactivated (130) and then activated again (140). An associated computer program.

Description

Technical Field

[0001] The invention relates to a method for operating an artificial neural network and to an associated hardware architecture, the method providing for the selective and resettable deactivation of hardware components during the operation of the network. Background Art

[0002] The artificial neural network KNN comprises an input layer, a plurality of processing layers and an output layer. Input variables are read into the KNN at the input layer and processed through the processing layers to the output layer based on a generally parameterized processing chain on the path of the input variables. When training the KNN, the following values ​​are determined for the parameters of the processing chain, with which the processing chain optimally maps a set of learned values ​​of the input variables to an associated set of learned values ​​of the output variables.

[0003] The strength of KNN is that, through the interaction of multiple physical computing units in each layer, KNN can process ultra-high-dimensional data, such as high-resolution images, on a large scale in parallel. GB 2 552 242 A discloses an example of a hardware architecture of KNN consisting of many computing units. The document aims to use this hardware architecture to reduce the hardware cost and current consumption required to implement KNN. To this end, the accumulated computational work in one layer is divided into two sub-steps, and the sub-results obtained in these sub-steps are combined into the final result. Summary of the invention

[0004] In the context of the present invention, a hardware architecture for an artificial neural network KNN is developed. The KNN comprises a close arrangement of an input layer, a plurality of processing layers and an output layer. Each layer maps a set of input variables to a set of output variables. Here, the output variables of the input layer and each processing layer are respectively input variables of subsequent layers in the close arrangement. The input variables of the input layer are also the input variables of the entire KNN. The output variables of the output layer are also the output variables of the entire KNN.

[0005] The hardware architecture comprises a large number of computing units, wherein the implementation of each layer is divided between at least two computing units. Thus, for example, half of the input variables of this layer can only be fed to the first computing unit, and the other half can only be fed to the second computing unit. Similarly, in the output layer for classification, for example, the responsibilities of specific classes, i.e., the recognition of specific objects or situations, can be divided between different computing units. In this way, data with very high dimensionality can be processed in parallel.

[0006] It is also possible that the same computing unit is responsible for implementing a plurality of, for example, consecutive layers. Thus, for example, three consecutive layers can be implemented using two computing units operating in parallel. Then, for example, an input variable of the first layer, which is supplied to one of the computing units, can traverse all three layers within the computing unit and finally be output by the computing unit as an output variable of the third layer.

[0007] The term "computational unit" is not limited to the fact that different computational units must be located on physically separate modules assembled when manufacturing the hardware architecture. Rather, for example, a device consisting of a large number of computational units can be combined in the same application-specific integrated circuit ASIC. Therefore, the term "computational unit" generally refers to a unit, circuit or component that contributes to the implementation of at least one layer of the KNN and can be selectively deactivated.

[0008] At least one resettable disconnection device is provided, by which at least one computing unit can be selectively deactivated independently of input variables supplied to the at least one computing unit, so that at least one other computing unit remains activated in all layers that the computing unit contributes to.

[0009] It has been recognized that in this way a dynamic trade-off between computational accuracy on the one hand and energy consumption and heat release of the KNN on the other hand can be adjusted during the operation of the KNN. Typically, most computations in a KNN are robust to errors, i.e., errors in a certain proportion of these computations only slightly deteriorate the output quality of the KNN. The deactivation of a computational unit leading to the complete loss of a particular intermediate result is considered a special case of a computational error.

[0010] In this case, the condition that at least one other computing unit is activated in all layers that the deactivated computing unit contributes to ensure that the entire KNN can continue to work. If a layer of the KNN is completely deactivated, the processing chain of the KNN will be interrupted at this point and the KNN will no longer provide any output.

[0011] The conditions under which the disconnection device can be reset ensure that the deactivation of the computing units can be adapted to the operating conditions. For example, it may be most urgent to temporarily deactivate each computing unit that is exactly under the strongest thermal load, in order to increase the service life of the hardware. Similarly, the extent to which the deactivation of a specific computing unit affects the final result provided by the entire KNN depends on the current operating conditions. In contrast, the "pruning" schemes used so far to remove neurons from the KNN are static: whether the neurons to be removed are set before, during or after the training of the KNN, this setting is binding for the entire subsequent reasoning run of the KNN.

[0012] The fact that the computational unit can be deactivated independently of the input variables fed to it ensures, on the one hand, that the deactivation can be forced by a superior instance. For example, if the deactivation follows the goal of reducing the thermal load of the computational unit, this goal may have the highest priority to avoid damaging the hardware. On the other hand, deactivation independent of the input variables has a much greater potential to reduce the energy consumption of the computational unit while having the same impact on the calculation accuracy of the KNN, compared to simply setting the input variables to zero.

[0013] The possibility of selectively deactivating computing units is particularly beneficial for the further miniaturization of the hardware architecture. In the past, the empirical observation known as the "Dennard scaling law" worked, namely that reducing the size of transistors in an integrated circuit reduces the energy consumption proportionally. Over the years, the reduction in transistor size has developed to the point where the energy consumption drops less than proportionally. Depending on the specific possibilities available for cooling, it may therefore be possible that it is no longer possible for all existing transistors to be continuously operated simultaneously.

[0014] However, if all existing computing units can in principle be operated simultaneously, it is also sensible to selectively deactivate computing units. Every energy saving can also save costs and extend the battery operating time in battery-operated devices. In order to increase availability, it may also be sensible to purposefully provide more computing units than are actually required to achieve a given accuracy. For example, then, the redundant computing units can initially remain deactivated and only be switched on when other computing units fail.

[0015] In a particularly advantageous design, a control unit is provided for selectively deactivating the computing unit. The disconnection device comprises at least one switch actuable by the control unit. The switch can be, for example, a transistor that can be switched by an electrical path. However, the switch can also be, for example, a semiconductor switch or a photoresistor that can be switched by an optical path.

[0016] In a particularly advantageous embodiment, the switch is arranged such that it can interrupt the supply of a clock signal and / or a supply voltage to the computing unit. An interruption of the supply voltage can completely prevent the consumption of energy by the computing unit. If only the clock signal is interrupted, a large part of the usual energy consumption is still saved. At the same time, it is avoided that the computing unit must be reinitialized when it is subsequently reactivated, so that the computing unit can then be used again more quickly.

[0017] In the control unit, the calculation unit can be deactivated and subsequently reactivated according to any conditions. For example, the distribution of omitted calculations can be predetermined. Thus, for example, the calculation of every Nth neuron can be omitted, where N is a natural number. However, instead of such a uniform distribution, the deactivation can also be performed based on an assessment of the importance of the individual calculation results when designing the KNN. Calculations with less correlation are then preferentially disconnected. Studies have shown that in this case the output quality of the KNN deteriorates less than in the case of uniform distribution.

[0018] In a particularly advantageous embodiment, the control unit receives as input at least one output variable outputted from at least one computing unit, a measure of the switching activity of at least one computing unit, a measure of the energy consumption of at least one computing unit and / or a measure of the temperature of at least one computing unit.

[0019] In this way, for example, the use of a plurality of computing units contributing to a given layer of the KNN can be evenly distributed in time and / or space. Thus, aging processes of the hardware can be reduced. These aging processes are particularly temperature-dependent. Thus, for example, the electronic properties of semiconductors can gradually deteriorate due to interdiffusion between adjacent regions with different doping, wherein the speed of this interdiffusion is exponentially related to the temperature.

[0020] The temperature of the computing unit can be determined in any manner. In addition to temperature sensors, thermal imaging cameras are also suitable for this purpose, which can detect the temperature of many computing units simultaneously in real time. Another cost-effective possibility is to determine the temperature indirectly via the switching activity. For this purpose, the switching activity is measured within a certain period of time and a model is used which predicts the expected temperature of the computing unit from it.

[0021] On the other hand, the selective disconnection of the computing units does not necessarily depend on the presence of a control unit. In another advantageous design, at least one computing unit includes a disconnection device, which automatically deactivates the computing unit when a first condition for the temperature of the computing unit and / or for the time course of the temperature and / or for the time course of the switching activity of the computing unit is met, and automatically activates the computing unit again when a second condition for the temperature of the computing unit and / or for the time course of the temperature and / or for the time course of the switching activity of the computing unit is met. In particular, the wiring expenditure to the central control unit can be saved. The KNN then develops its own dynamics with respect to the distribution of the total computing work, which depends in particular on which computing units are used with what intensity.

[0022] The present invention also relates to a method for running an artificial neural network KNN. The KNN comprises a close arrangement consisting of an input layer, a plurality of processing layers and an output layer. Each layer maps a set of input variables to a set of output variables. Here, the output variables of the input layer and each processing layer are respectively input variables of subsequent layers in the close arrangement. The input variables of the input layer are also the input variables of the entire KNN. The output variables of the output layer are also the output variables of the entire KNN.

[0023] The KNN is implemented with a large number of computing units, such that the implementation of each layer is divided into at least two computing units.

[0024] If a predetermined deactivation condition is met during the operation of the KNN, the at least one computing unit is selectively deactivated independently of the input variables supplied to the at least one computing unit, so that at least one other computing unit remains activated in all layers that the computing unit contributes to the implementation. If a predetermined activation condition is met during the operation of the KNN, the deactivated computing unit is reactivated.

[0025] Similar to what was described previously for the hardware architecture, in this way, a dynamic tradeoff between computational accuracy on the one hand and energy consumption and heat release of the KNN on the other hand is achieved during the operation of the KNN.

[0026] In a particularly advantageous design, the deactivation condition and / or the activation condition comprises a quality measure of at least one input variable, a measure of the temporal and / or spatial rate of change of at least one input variable, a quality requirement measure for at least one output variable, and / or a correlation measure of at least one output variable.

[0027] For example, if it is determined that the input data representing the input variables of the KNN have high quality, more errors can be tolerated by deactivating the internal calculation steps of the KNN while the output quality is the same compared to a case where the input data quality is poor, because the output quality of the KNN can be expected to have a higher basic level in case of high input data quality. For example, a good signal-to-noise ratio of a camera image due to good lighting conditions can be evaluated as a sign of high quality of the input data.

[0028] The quality requirements for the output variables can depend, for example, on whether other information sources are available for the problem based on the output variables. If the uncertainty in the output of the KNN can be compensated again by additional information sources, more computing units can be deactivated. Such redundant information sources can be, for example, other different sensor data processing systems. Likewise, in the event of failure of redundant information sources, the output quality of the KNN can be improved again by reactivating previously deactivated computing units.

[0029] The quality requirements on the output variables can, for example, also depend on the extent to which results from previous time steps or other contextual information are present. For example, if the KNN processes a temporally or spatially continuous input data stream (e.g. a video data sequence), information from the temporal or spatial proximity of the currently processed input data can be used to dynamically adapt the deactivation of calculation units in specific parts of the network. For example, if an object in a video sequence is tracked by the KNN, specific parts of the neural network are relevant to this, while other parts currently do not contribute to this. The relevant calculation steps are determined by the characteristics (features) of the tracked object and the position of the object in the image.

[0030] If the features and position of the object are determined by the KNN in time step t, this information can be used in the following time step to deactivate further computational units in the part of the network that processes currently irrelevant features or is responsible for currently irrelevant regions of the input image. For currently irrelevant neurons it can be assumed that their output values ​​have very small or zero absolute values ​​anyway, so that no significant errors are caused in the output of the KNN by disconnecting the corresponding computational units. The method can be supported by motion prediction of the image information, for example by determining the optical flow.

[0031] The correlation between the individual calculation results within a neural network can be estimated with the help of various methods.

[0032] One exemplary possibility is to simulate the deactivation of specific computational units in the KNN for a set of training data and measure the effect on the output quality of the network. The simulation can be performed on actual hardware, or in a virtual prototype of the hardware, or in a purely software simulation environment. If it is determined that the output quality on the distribution of training data is greatly reduced when deactivating specific computational steps, then these calculations are highly relevant for these training data.

[0033] For example, the output value change of the KNN can also be determined by analysis when a specific calculation step is omitted. For example, the gradient of the output value relative to the intermediate result can be calculated, or the output function of the neural network can be decomposed into the contribution of each intermediate result.

[0034] In two exemplary approaches, both global correlations, i.e. correlations obtained over the complete (training) input data distribution, and correlations for specific groups of input data can be determined. Thus, for example, images showing the same or similar objects or images in which related objects are located in specific areas can be combined in order to determine separate correlation values ​​for these groups. This information can then be used to preferentially deactivate specific parts of the KNN given the context.

[0035] In another particularly advantageous embodiment, the deactivation condition and / or the activation condition comprises the time course of the temperature of at least one computing unit and / or the temperature of at least one heat sink coupled to the computing unit and / or the switching activity of the computing unit. In this way, in particular, aging processes of the hardware can be balanced according to the above description. In addition, for example, the thermal capacity of the heat sink can be used to call up the power of more computing units in the event of a brief increase in demand.

[0036] In another particularly advantageous embodiment, the KNN is used to identify objects and / or conditions based on input variables. Based on context information, a first subset of the objects and / or conditions is determined to be less likely to exist than a second subset of the objects and / or conditions. The context information can come from any source. In addition to external sources, alternatively or in combination, in particular, it is considered to determine the context information from the time course of the input variables themselves. Then, the deactivation of a computing unit that is primarily used to identify objects and / or conditions from the first subset takes precedence over the deactivation of a computing unit that is primarily used to identify objects and / or conditions from the second subset.

[0037] In this way, the energy used in the KNN can be focused on identifying those objects and / or situations that are more likely to be present. In particular, systems for identifying objects and / or situations in the vehicle environment, such as systems for at least partially automated driving, are designed to recognize a very large number of objects or situations. For example, a vehicle that is at least partially automated must be able to react to all currently valid traffic signs in order to drive in compliance with the rules. However, not all possible traffic signs and situations are similarly expected to appear at all times. Thus, for example, there are neither traffic lights nor pedestrian crossings or railway crossings on highways. Similarly, warning signs that the road leads to the shore are not expected to appear in places far from water sources. Therefore, the computing units responsible for identifying such impossible objects or situations can be deactivated preferentially without compromising the operational safety of the vehicle.

[0038] Therefore, in a particularly advantageous embodiment, the KNN is used to recognize objects and / or situations in the vehicle environment, wherein additional context information includes the position of the vehicle and / or the road class of the road on which the vehicle is currently traveling.

[0039] The deactivation of the computing unit controlled based on this context information normalizes the uncertainty that is inevitably introduced into the KNN by the deactivation. It can then be assured, in particular with regard to the authorization of public road traffic, that the energy and heat savings caused by the deactivation of the computing unit do not come at the expense of increased risk.

[0040] If the KNN provides the possibility of selectively deactivating calculation units in hardware, the method can be performed completely or partially under software control. The corresponding software can then be sold, for example, as an update or upgrade to an existing control device and is in this respect an independent product. Therefore, the present invention also relates to a computer program with machine-readable instructions, which, when executed on a computer and / or on a control device, cause the computer and / or the control device to perform the method provided by the present invention.

[0041] Likewise, the invention also relates to a machine-readable data carrier or a download product having the computer program. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Further measures for improving the invention are described in detail below together with the description of preferred exemplary embodiments of the invention based on the drawings.

[0043] Figure 1 An embodiment of a hardware architecture 10 for KNN1 is shown;

[0044] Figure 2 An embodiment of a method 100 is shown. DETAILED DESCRIPTION

[0045] according to Figure 1 , the KNN1 depicted in an exemplary simplified manner comprises an input layer 2, two processing layers 3a and 3b, and an output layer 4. The hardware architecture 10 implements this KNN1 by means of eight mutually connected computing units 5a-5h. Each computing unit 5a-5h implements three neurons shown as solid points. Every two computing units in the computing units 5a-5h contribute to the implementation of each layer in the layers 2, 3a, 3b, 4.

[0046] Each of the computing units 5a-5h is assigned a disconnection device 6a-6h, which is implemented as a switch in the supply voltage and / or the clock signal, by which the corresponding computing unit can be shut down. All disconnection devices 6a-6h are controlled via a central control unit 7.

[0047] KNN is constructed to recognize objects in the vehicle environment. Figure 1Only four traffic signs are shown as exemplary objects 1a-1d. In an exemplary driving situation on a highway, it is conceivable that there are traffic signs from the first subset 8a, here a prohibition on overtaking by trucks 1a or a speed limit 1b. In contrast, traffic signs from the second subset 8b, here a sign 1c for a pedestrian crossing or a sign 1d for warning of a danger on a shore, are not normally present on highways. Therefore, the computing unit that is essentially responsible for recognizing traffic signs 1c, 1d can be deactivated when driving on a highway without losing traffic safety.

[0048] Figure 2 An embodiment of a method 100 is shown. According to optional step 105, a subset 8a of objects 1a, 1b that are more likely to occur and a subset 8b of objects 1c, 1d that are less likely to occur are determined based on additional context information (e.g., the road class of the road currently being traveled). According to step 110, it is continuously checked whether a deactivation condition is met. If the deactivation condition is met (true value 1), the calculation units 5a-5h are deactivated in step 120. If a subset 8b of objects 1c, 1d that are less likely to occur is determined in step 105, then according to step 125, the calculation units 5a-5h that are primarily used to identify objects 1c, 1d from the subset 8b are preferentially deactivated.

[0049] According to step 130 , it is continuously checked whether an activation condition for reactivating previously deactivated computing units 5 a - 5 h is satisfied. If so (true value 1), these computing units 5 a - 5 h are activated again in step 140 .

Claims

1. A hardware architecture for an artificial neural network KNN, wherein the KNN comprises a close arrangement of an input layer, a plurality of processing layers and an output layer, wherein each of the layers maps its input variables to output variables by means of a plurality of neurons, and wherein the output variables of each layer are respectively the input variables of the subsequent layers in said immediate arrangement, The hardware architecture includes a large number of computing units. wherein the implementation of each layer is divided into at least two computing units, wherein when dividing the implementation, the calculation of the plurality of neurons is divided into at least two computing units, wherein at least one of the computing units implements a plurality of layers, and The hardware architecture comprises at least one resettable disconnecting device, which is configured to selectively deactivate at least one computing unit independently of an input variable supplied to the at least one computing unit, so that at least one other computing unit remains activated in all layers in which the at least one computing unit is involved.

2. The hardware architecture according to claim 1, wherein: A control unit is provided for selectively deactivating the computing unit, and wherein the disconnection means comprises at least one switch actuatable by the control unit.

3. The hardware architecture according to claim 2, wherein: The switch is arranged in such a way that it can interrupt the supply of a clock signal and / or a supply voltage to the computing unit.

4. The hardware architecture according to any one of claims 2 to 3, wherein: The control unit receives as input at least one output variable output by at least one computing unit, a measure of the switching activity of at least one computing unit, a measure of the energy consumption of at least one computing unit and / or a measure of the temperature of at least one computing unit.

5. The hardware architecture according to any one of claims 1 to 3, wherein: At least one computing unit includes a disconnection device, which automatically deactivates the computing unit when a first condition for the temperature of the computing unit and / or for the time course of the temperature and / or for the time course of the switching activity of the computing unit is met, and automatically activates the computing unit again when a second condition for the temperature of the computing unit and / or for the time course of the temperature and / or for the time course of the switching activity of the computing unit is met.

6. A method for running an artificial neural network KNN, wherein the KNN comprises a close arrangement of an input layer, a plurality of processing layers and an output layer, wherein each of the layers maps its input variables to output variables by means of a plurality of neurons, and wherein the output variables of each layer are respectively the input variables of the subsequent layers in said immediate arrangement, wherein the KNN is implemented with a large number of computing units, such that the implementation of each layer is divided into at least two computing units, wherein when dividing the implementation, the calculation of the plurality of neurons is divided into at least two computing units, wherein at least one of the computing units implements a plurality of layers, wherein, if a predefined deactivation condition is met during the operation of the KNN, the at least one computing unit is selectively deactivated independently of the input variables supplied to the at least one computing unit, so that at least one other computing unit remains activated in all layers in which the at least one computing unit is involved, In this case, a deactivated computing unit is activated again if a predefined activation condition is met during the KNN operation.

7. The method according to claim 6, wherein: The deactivation condition and / or the activation condition include a quality measure of at least one input variable, a temporal and / or spatial rate of change measure of at least one input variable, a quality requirement measure for at least one output variable, and / or a correlation measure of at least one output variable.

8. The method according to any one of claims 6 to 7, wherein: The deactivation condition and / or the activation condition comprises the temperature of at least one computing unit and / or the temperature of at least one heat sink coupled to the computing unit and / or the time course of a switching activity of the computing unit.

9. The method according to any one of claims 6 to 7, wherein: The KNN is used to identify objects and / or conditions based on input variables, wherein a first subset of the objects and / or conditions is determined to be less likely to exist than a second subset of the objects and / or conditions based on contextual information, so that deactivation of computing units primarily used to identify objects and / or conditions from the first subset takes precedence over deactivation of computing units primarily used to identify objects and / or conditions from the second subset.

10. The method according to claim 9, wherein: The KNN is used to identify objects and / or situations in a vehicle environment, and wherein additional context information includes the location of the vehicle and / or the road class of a road on which the vehicle is currently traveling.

11. A computer program product comprising machine-readable instructions which, when executed on a computer and / or on a control device, cause the computer and / or the control device to execute the method according to any one of claims 6 to 10.

Images