Method, device and computer program for pruning a neural network

Abstract

The present invention relates to a method, a device, and a computer program for pruning a neural network. The method comprises obtaining (130) a neural network. The method comprises determining (140), based on the neural network, the relevance of parameters of the neural network for a prediction of the neural network. The method comprises creating (150), based on the relevance of the parameters of the neural network, at least one mask, wherein the at least one mask indicates whether a change of a parameter or a group of parameters within the scope of pruning the neural network is permitted, prohibited, or restricted. The method comprises pruning (160) the neural network based on the at least one mask. The method comprises providing (180) the pruned neural network.

Description

[0001] The present invention relates to a method, a device and a computer program for pruning a neural network.

[0002] Neural networks (also called neural networks) are a class of models in machine learning and artificial intelligence inspired by the workings of biological brains. They consist of multiple interconnected nodes (also called neurons or units) organized into layers: input layer, hidden layers, and output layer. Each node has specific weights and activation functions that determine how it processes inputs and generates outputs. By training on a large dataset, neural networks adjust their weights to perform tasks such as classification, regression, image recognition, and many others.

[0003] Pruning is a technique used in neural networks to reduce their size and improve efficiency. Pruning aims to identify and remove redundant or unimportant nodes or connections (weights) in the network without significantly impacting accuracy or performance. The purpose of pruning is to reduce computational costs and shrink the model to lighter and faster-running versions, which is particularly useful in resource-constrained environments such as mobile devices or embedded systems, as unused neurons generate runtime and memory costs. Pruning is therefore especially beneficial for non-parallel computers like microcontrollers.

[0004] While it is generally possible to train neural networks from the outset with a smaller number of nodes and weights, it's important to note that smaller neural networks, due to initial conditions and the "dying" of neurons, will not always converge to the same results. Manually resizing larger neural networks by pruning them, on the other hand, is very time-consuming.

[0005] Therefore, there is a need to provide an improved approach that makes it possible to create neural networks that can be used in environments with limited resources.

[0006] This need is met by the subject matter of the independent claim.

[0007] The present invention is based on the understanding that information-theoretic approaches can be used to calculate the relevance of the parameters of a neural network. In particular, a trained neural network can be parameterized with a wide range of input data to determine which parameters contribute significantly and which contribute little to the neural network's predictions. Based on the contribution of the respective parameters, one or more masks are created according to the invention, which control the subsequent pruning of the neural network. By using this approach, the pruning of the neural network can be automated, thereby significantly reducing the effort required. Thus, a neural network can be provided with minimal effort and used in environments with limited resources.

[0008] A first aspect of the present invention relates to a method for pruning a neural network. The method comprises obtaining a neural network. The method comprises determining, based on the neural network, the relevance of parameters of the neural network for making predictions about the neural network. The method comprises creating, based on the relevance of the parameters of the neural network, at least one mask. The at least one mask indicates whether changing a parameter or a group of parameters is permitted, prohibited, or restricted when pruning the neural network. The method comprises pruning the neural network based on the at least one mask. The method comprises providing the pruned neural network.By determining the relevance of the parameters and calculating at least one mask, the pruning of the neural network can be automated, significantly reducing the pruning effort. This allows for the deployment of a neural network with minimal effort, making it suitable for environments with limited resources.

[0009] There are various techniques for pruning a neural network. For example, all weight edges and / or neurons that meet a specific criterion (such as all weight edges whose weight is less than a threshold) can be removed. However, in this case, it is preferable to use an agent trained using reinforcement learning, which is trained to select the next neuron or weight edge to be removed by pruning. Such approaches are described, for example, in Livne, Dor, and Kobi Cohen: "Pops: Policy pruning and shrinking for deep reinforcement learning," Chen, Jianda, Shangyu Chen, and Sinno Jialin Pan: "Storage efficient and dynamic flexible runtime channel pruning via deep reinforcement learning," and Chen, Jianda, Shangyu Chen, and Sinno Jialin Pan: "DECORE: Deep Compression With Reinforcement Learning."In this approach, at least one mask is used to restrict the agent's actions. In other words, pruning can be performed using an agent trained via reinforcement learning, with the pruning of the neural network being restricted by at least one mask. This approach allows the use of information-theoretic approaches in an agent-based pruning procedure and enables pruning of the neural network with minimal impact on its predictive performance.

[0010] In the case of pruning using a reinforcement learning-based agent, the agent is (further) trained on the specific model to be pruned. Accordingly, the process can include training the agent using reinforcement learning. As described previously, the agent is trained to select the part of the neural network that is to be removed.

[0011] Reinforcement learning is based on the specification of a so-called reward function. Accordingly, the agent is trained based on this reward function. A reward function, also called a reward or goal function, is a central component in reinforcement learning (RL). It defines how an agent is rewarded or punished for specific actions in certain states. The reward function indicates how desirable a particular behavior is in a given environment. To this end, the reward function returns a numerical value for a (current) state and an action, indicating how desirable the action is in that state. This value can be either positive (for desired behavior) or negative (for undesired behavior). The agent's goal is to learn a policy that maximizes the expected (long-term) reward.This means that the agent must learn which actions lead to the highest rewards in which states, not only in the short term but also in the long term. The reward function is the primary means by which the agent receives feedback from the environment. This feedback helps the agent to adjust and optimize its decisions. In this case, several aspects are relevant to whether an action by the agent, i.e., the selection of a neuron or a weighting edge, is desirable. Here, the reward function can be based on at least one of the following: the accuracy of a prediction by the pruned neural network (i.e., how good the prediction is after pruning), the achieved compression of the pruned neural network (i.e., by what factor or absolute value could the size of the neural network be reduced), and the explainability of the prediction by the pruned neural network (i.e., how well the prediction can be explained).The accuracy of the prediction can be determined by comparing it to the ground truth (the value the prediction should have). Compression is derived from the ratio or absolute difference between the neurons and / or weight edges of the neural network and its truncated version. Explainability can be quantified using the approach of conformal prediction. The redundancy of the truncated neural network with respect to subtasks can be determined using information theory. These aspects contribute to ensuring that the resulting truncated neural network exhibits the desired properties.

[0012] Furthermore, specifications can be defined that the trimmed network must fulfill. Accordingly, the agent can be trained to trim the neural network in such a way that, after trimming, the neural network has at most a predefined maximum size (i.e., a certain compression must be achieved), at least a predefined inference performance (i.e., a predefined number of inference operations per unit of time, a predefined maximum duration for inference, and / or a maximum number of computations to be performed), and / or at least a predefined prediction quality (i.e., the prediction quality may only suffer to a certain degree). This ensures that the resulting trimmed neural network can be used for its intended purpose.

[0013] A key aspect of the present invention is the calculation of the relevance of the parameters of the neural network. In the context of this disclosure, the neurons, weight edges, and bias values ​​are considered in particular as parameters of the neural network. The relevance of the parameters can preferably be determined using a statistical and / or information-theoretic approach. For example, the approach shown in Fischer, Jonas, Anna Oláh, and Jilles Vreeken: “What's in the box? exploring the inner life of neural networks with robust rules.” can be used to determine the relevance of the parameters. The approach used there is based, in particular, on how frequently the respective neurons are activated. Accordingly, the relevance of parameters of the neural network can be determined based on the activation frequency of neurons in the neural network.The activation frequency can be determined, for example, by using a set of test data (with a large number of input data sets, such as from the training data for the neural network) as input data for the neural network and logging which neurons are activated and how frequently during the processing of the input data sets. By using these information-theoretic approaches, the relevance of the parameters can be determined automatically.

[0014] Generally, pruning is applied to a trained neural network. Accordingly, pruning can also be applied to an existing neural network that has been trained by another party. Alternatively, the process itself can involve training the neural network. This can include training the neural network using training data. The type of training data used depends on the intended purpose of the neural network. In the case of supervised learning, the training data samples (units) include one or more training input values ​​and one or more desired output values. In the case of reinforcement learning, the training data includes input data as well as a reward function, etc.

[0015] In general, when training neural networks, the predictive accuracy of the network increases with the amount of training data used and the better the training data represents the relevant domain. Since training data often does not cover the entire domain and may even be incomplete, data augmentation can be used to supplement or extend the training data. For example, the process can involve supplementing at least part of the training data using a data augmentation technique. This allows incomplete training data to be supplemented, thereby improving the quality of the training and, consequently, the predictive accuracy of the neural network. Alternatively or additionally, the process can involve calculating a set of synthetic training data using a physical or geometric model or a generative machine learning model.This allows training data to be generated that covers aspects of the domain not included in the original training data. This, in turn, can improve the predictive quality of the neural network, particularly for aspects of the domain not covered in the original training data.

[0016] Clipping the neural network reduces its predictive accuracy in most cases. This can be (at least partially) compensated for by further training the model after clipping (so-called fine-tuning). Accordingly, the process can also include training the clipped neural network after it has been clipped.

[0017] The goal of pruning is to reduce the hardware resources required by the neural network for inference. Specifically, pruning can be performed to adapt the neural network's size and / or inference performance to a target hardware architecture. This ensures that the neural network is usable for its intended purpose without requiring costly increases in hardware resources.

[0018] Another aspect of the present invention relates to a device comprising at least one memory, such as a volatile and / or a non-volatile memory, and at least one processor, such as a central processing unit (CPU), a graphics processing unit (GPU), and / or a tensor processing unit (TPU), configured to perform the previously presented method.

[0019] Another aspect of the present invention relates to a computer program comprising instructions which, when the program is executed by a computer or control unit, cause the computer or control unit to execute the above method.

[0020] There are now various ways to advantageously elaborate and further develop the teaching of the present invention. For this purpose, reference should be made, on the one hand, to the claims subordinate to claim 1 and, on the other hand, to the following explanation of preferred embodiments of the invention with reference to the figures. The figures show: Fig. Figure 1a shows a flowchart of a procedure for pruning a neural network; Fig. Figure 1b shows a block diagram of a device for pruning a neural network; Fig. Figure 2 shows the structure of an Optimized Model Generator for Hardware-Specific Applications (OMGEN); and Fig. Figure 3 shows a schematic diagram of a user interface for controlling the Optimized Model Generator for hardware-specific applications.

[0021] This disclosure relates to an optimized model generator for hardware-specific applications (OMGEN). OMGEN enables the generation of explainable and robust neural networks and their improvement or optimization for the specific requirements of a given task and their adaptation to specific target hardware. This leads to a significant improvement in the speed, energy efficiency, and computing power of the generated models. Hereinafter, the terms "model," in the sense of a machine learning model, and "neural network," as an embodiment of the machine learning model, are used synonymously. In particular, this disclosure relates to neural networks, especially deep neural networks, i.e., neural networks with at least one hidden layer.

[0022] The development of neural networks has shown impressive momentum in recent years. However, these systems, originally designed for generic hardware architectures, are increasingly reaching their limits in terms of efficiency and performance. In this context, the creation of hardware-specific neural networks is gaining importance. These specially adapted networks can optimally utilize the capabilities of the underlying hardware, thereby achieving significant improvements in speed, energy efficiency, and computing power. This is particularly useful for deploying AI models on devices with limited resources, such as smartphones, IoT (Internet of Things) devices, or vehicles.

[0023] A particularly useful tool in this context is so-called "pruning." Pruning allows the complexity of neural networks to be reduced by removing superfluous or less important neurons and connections. This results in a leaner, more efficient structure. Pruning can increase the efficiency of networks without significantly compromising performance or accuracy. This has a direct impact on hardware requirements, as the corresponding models are smaller, faster, and more resource-efficient.In this context, reinforcement learning (RL) approaches, among others, have enormous potential (see Livne, Dor, and Kobi Cohen: "Pops: Policy pruning and shrinking for deep reinforcement learning," Chen, Jianda, Shangyu Chen, and Sinno Jialin Pan: "Storage efficient and dynamic flexible runtime channel pruning via deep reinforcement learning," and Chen, Jianda, Shangyu Chen, and Sinno Jialin Pan: "DECORE: Deep Compression With Reinforcement Learning"). These approaches are used to find the best strategy for removing neurons or connections in a network. The RL agent is trained to make decisions about which parts of the network can be removed without significantly impacting performance.

[0024] Furthermore, the explainability of AI models is a crucial factor for their productive use in real-world application scenarios. Explainability is relevant for compliance with legal and ethical guidelines. Depending on the use case, transparent and traceable decision-making processes are necessary to meet regulatory requirements. Explainability also enables more effective debugging and model improvement. When developers can understand how their model arrives at certain results, they can better optimize the model and more easily identify sources of error. OMGEN is described below. OMGEN enables the generation of explainable and robust neural networks and their adaptation to the specific requirements of a given task and target hardware.

[0025] To this end, information-theoretic approaches are combined with modern pruning methods to generate highly specific models. The information-theoretic component is used to identify important parts of the model architecture and protect them in the subsequent pruning step. The main focus of this concept is on optimizing the model architecture with various application- and hardware-specific constraints that the target model should fulfill. This leads to significant improvements in terms of the models' speed, energy efficiency, and computing power.

[0026] Various inputs can be used to apply OMGEN. These include one or more datasets (training data) for the task to be solved (for example, anomaly detection, visual component classification, AD (Automated / Assisted Driving) / ADAS (Advanced Driver Assistance System)). The dataset can consist of real data, synthetic data, or a suitable mixture. This data can then be used both for training the initial model and for subsequent fine-tuning. Optionally, it is also possible to use one or more existing models as the basis for the initial model (via transfer learning). Furthermore, the present concept can use a number of constraints that the final model, i.e., the truncated neural network, must fulfill.This can include, for example, specifications regarding the target hardware (allowed number of floating-point operations per second (FLOPS), memory size, allowed energy consumption), how much the model should be compressed, how specific / generic the final model should be adapted to the task, a desired redundancy for subtasks of the model, explainability, performance, and a final inference time on given hardware.

[0027] The following describes a method and a device based on OMGEN. Accordingly, the method, a corresponding computer program, and the device may have one or more features and / or functions described in the context of OMGEN. It should be noted that the method, the computer program, and the device do not necessarily have all the features and functions described in relation to OMGEN.

[0028] Fig. Figure 1a shows a flowchart of a procedure for pruning a neural network. The procedure includes obtaining a neural network. The procedure includes determining, based on the neural network, the relevance of the neural network's parameters for making a prediction. The procedure includes creating, based on the relevance of the neural network's parameters, at least one mask. The at least one mask indicates whether modifying a parameter or group of parameters is permitted, prohibited, or restricted when pruning the neural network. The procedure includes pruning the neural network based on the at least one mask. The procedure includes providing the pruned neural network. For example, the procedure can be executed by a computer, such as the device shown in Figure 100. Fig. 1b.

[0029] Fig. Figure 1b shows a block diagram of a device 100 for pruning a neural network. The device 100 comprises an optional interface 102, one or more processors 104, and a memory 106, wherein the one or more processors 104 are coupled to the optional interface 102 and the memory 106. The device 100 is configured to perform the method of Fig. 1a. In particular, the one or more processors 104 are configured to provide the functionality of the device 100, for example by executing machine-readable instructions (program code of a computer program), with optional assistance from the interface 102 (for exchanging information, such as receiving training data and / or the neural network and providing the truncated neural network) and the memory 106 (for storing and retrieving information, such as the training data, the constraints and the neural network).

[0030] The method and apparatus 10 of Fig. 1a and Fig. Section 1b will now be explained in Appendix 1 of the OMGEN description. OMGEN comprises up to 10 components, which are described in Fig. 2 shown and described below. Fig. Figure 2 shows an example of the setup of the Optimized Model Generator for Hardware-Specific Applications (OMGEN). The process begins with the optional provision of data (1, 2), for example by an optional data processor and an optional data generator, and the optional training of the model (3), for example by a model trainer, before the importance (relevance) of the individual filters / neurons is determined (4) for example by a relevance calculator and these are masked (5), for example by a mask generator. In the subsequent operation, the user's set limits (restrictions) (6) are optionally fed into the specifications checker (7), and it is checked whether the model meets the requirements. If not, the model is then pruned (trimmed, 8), for example by a model trimmer, and optionally fine-tuned (retrained, 9), for example by a model fine-tuner, and checked again (7).Here, the user optionally receives feedback on the model's performance (10) and can adjust the requirements (6) if necessary. If the model meets the requirements, it can be validated (11), for example by the hardware validator, to determine whether it can be deployed to the selected end devices (12).

[0031] Data Processor 1 is a mechanism for processing the data points used for training. It performs tasks such as splitting the data into training, test, and validation sets, data augmentation (optional), and preprocessing steps to optimize the input data. Accordingly, the process can be described as follows: Fig. 1a also includes preprocessing 110 of training data and / or supplementing 120 of training data.

[0032] Data Generator 2 is an optional component for creating synthetic data points. These can either supplement an existing real-world dataset or function as a separate dataset. Accordingly, the process can be modified by Fig. 1a. Supplementing at least part of the training data using a data supplementation procedure. Supplementing a real dataset allows for optimization of the training and thus the model. The procedure of Fig. 1a can further include the computation of a set of synthetic training data, for example, using a physical or geometric model or a generative machine learning model. By using only synthetic data, a model can be pre-trained to minimize the amount of real data required for subsequent training. The data generator can use various methods to generate synthetic data, such as 3D modeling of objects using Blender (software for generating and animating 3D structures), generative AI methods like diffusion models (such as stable diffusion), or GANs (generative adversarial networks). The use of synthetic data leads to an improvement in both the amount and diversity of data: often, the available real datasets are limited or not diverse enough to train a robust model.Synthetic data can help increase the quantity and variety of training data, which is particularly helpful for rare events or classes. Using synthetic data also leads to cost reduction and increased efficiency: Collecting and annotating real-world data can be very expensive and time-consuming. Synthetic data can be generated more efficiently and cost-effectively, especially when specific scenarios or conditions need to be modeled. Furthermore, using synthetic data leads to improvements in testing and validating models: Synthetic data makes it possible to test and validate models under controlled conditions. Situations or boundary conditions can be specifically simulated to verify how the model performs in these particular cases.The use of synthetic data can also be used to overcome overfitting: If a model is trained exclusively on a limited set of real-world data, there is a risk that it will become too "overfitted" to that specific data. By adding synthetic data that differs from the real-world data, the model's generalizability can be improved. Furthermore, synthetic data can be used to simulate rare events. In some use cases, such as predicting natural disasters or detecting fraud, real-world data for rare events is difficult to obtain. Synthetic data can help to model and better understand such rare events. Additionally, synthetic data can be used to address ethical concerns.Situations that are ethically reprehensible cannot generally be covered by real data, such as an accident involving humans.

[0033] Model Trainer 3 is a mechanism for training an initial model based on the dataset defined by the data processor. Accordingly, the procedure can be described as follows: Fig. 1a To maintain the neural network, training is performed using training data, i.e., the dataset defined by the data processor. This can result in, for example, a highly specialized AI model, which can be fine-tuned (further trained) based on a foundation model and thus optimized for the selected dataset. It is also possible to use one or more existing models as the basis for the initial model to maintain the neural network (see transfer learning). In subsequent steps, the trained model is analyzed and adapted to the specified tasks and the corresponding target hardware. In other words, the subsequent trimming serves to adapt the neural network to a target hardware architecture in terms of size and / or inference performance.

[0034] In Relevance Calculator 4, the relevance (importance) of a model's parameters is calculated in relation to predefined hyperparameters, for example, using statistical approaches or methods from information theory (see Fischer, Jonas, Anna Oláh, and Jilles Vreeken: "What's in the box? exploring the inner life of neural networks with robust rules."). In other words, the relevance of neural network parameters can be determined using a statistical and / or information-theoretic approach, such as based on the activation frequency of neurons in the neural network. This results, for example, in a set of rules governing how parameters are related. Additionally, it is possible to calculate the importance of groups of model parameters, including individual layers in the network, filters (kernels), or blocks in a transformer architecture.

[0035] The calculated relevance from the previous operation is used by mask generator 5 to generate masks for subsequent operations. These serve as regularization and allow, restrict, or prohibit the pruning step from modifying parameters.

[0036] Constraints: Constraints are requirements that can be set externally by the user. The subsequent check (by the Specifications Checker 7) verifies whether the model meets these requirements. These requirements can include, for example, the following: Hardware: This refers to the specifications of the end device to be used. These devices often have limited memory and smaller processors to save costs and energy. Accordingly, the model should be small and performant. Compression: How much should the model be compressed so that it can be used on different types of devices? Explainability: How well should the decisions of a model be comprehensible? In the field of autonomous driving, significantly higher explainability is often required than in other areas of AI application.Redundancy: How many redundant paths should be provided for a decision? This can be important, for example, when 100% reliability is required in quality control, where preferably five checks are performed per component instead of just one, to ensure accuracy. The example parameters are from... Fig. 2 can be represented using the following formula: Optimization function = a⋅hardware+b⋅compression+c⋅explainability+d⋅redundancy where a, b, c, d ∈ [0,1]. This can be used as a basis for the specifications checker.

[0037] The requirements checker 7 defines the model requirements that must be met before deployment or final use. For example, the following metrics can be used: Accuracy (prediction quality) is a crucial metric for many neural models. This metric becomes particularly important during pruning, as pruning often results in a loss of accuracy. Model size is also an important metric when pruning models. Mobile hardware for practical applications often has very limited resources, which is why a smaller model memory footprint is advantageous. Computational requirements and inference time, i.e., the number of operations required to calculate the model (FLOPs / s), are another relevant factor due to the limited computing resources of dedicated hardware.With fewer operations required, the inference time is usually also reduced, which is essential for real-time applications.

[0038] In Model Trimmer 8, pruning is implemented. Pruning is an important component of optimizing neural models, especially when applied to mobile hardware (also known as edge devices, devices located close to the user rather than in the core network). In this section, the requirements (e.g., FLOPS / s, memory capacity, hardware architecture, etc.) of the specifications checker are implemented so that they are met after pruning. For this purpose, an agent-based optimization method can be used, in which the agent is trained, for example, using Shielded Reinforcement Learning (Alshiekh, Mohammed, et al.: "Safe reinforcement learning via shielding"). Accordingly, the procedure can be... Fig. 1a that the pruning 160 is carried out using an agent trained by reinforcement learning, whereby the pruning of the neural network is restricted by at least one mask. The agent is trained to select parts of the model to be removed next. Accordingly, the procedure can include training 165 the agent using reinforcement learning. The agent is trained to select a part of the neural network to be removed. To control the selection, a composite objective function (reward function) is used, which contains the target specifications from the specification checker. Depending on the application, there are a number of factors that can be included in the objective function, such as the performance characteristics of the model (e.g., accuracy), compression strength, explainability of the results, redundancy, etc. Accordingly, in the procedure of Fig. 1a, the agent is trained based on a reward function. The reward function can be based on at least one of the following: the accuracy of a prediction by the truncated neural network, the achieved compression of the truncated neural network, the explainability of the prediction by the truncated neural network, and the redundancy of the truncated neural network with respect to subtasks. Each factor can be used as a weighted penalty / reward as part of the optimization to train the agent. In addition to the objective function, hard constraints can also be used, which the agent must adhere to during the process or which can be used as a termination criterion, for example, a minimum allowed model performance or a maximum allowed size.Accordingly, the agent can be trained to prune the neural network such that, after pruning, the neural network has at most a predefined maximum size, at least a predefined inference performance, and / or at least a predefined prediction quality. These specifications can be defined, in particular, as hyperparameters of the pruning procedure.

[0039] Pruning leads to a reduced model size: Removing unimportant or less important parameters (filter kernels, neurons, or other connections) also reduces the model's size. This reduces not only the memory footprint but also the required computational memory (GPU VRAM (video RAM), RAM (random access memory)). Furthermore, pruning increases computational efficiency: Smaller models require less computation, which also reduces inference time. This makes a model more suitable for real-time applications. Additionally, it improves energy efficiency: Due to the reduced computational effort, the system on which the model is used requires less energy. This is essential for sustainable AI development and use on battery-powered devices.Pruning can also lead to less overfitting, and thus better generalization: Large models often tend to fit the training data too closely, making it difficult to adapt to new and unknown data. Small models tend to generalize better because they contain fewer specific features during training that are tailored to the training data.

[0040] Model Finetuner 9 allows for fine-tuning of the model. Accordingly, the process can be modified from Fig. 1a Training (fine-tuning) 170, after pruning the neural network, includes the pruned neural network. Fine-tuning is the retraining of a pre-trained model. Retraining is particularly advantageous after pruning. During this operation, the user can also view the current performance of the model via the control interface 10. Fine-tuning enables specific adaptation: Through fine-tuning, a pre-trained model can be adapted to a specific use case. Fine-tuning also enables improved performance: By adapting the model to specific features of a data set, accuracy can be improved. In particular, after pruning, there is often a loss of accuracy, which can be restored or further improved through fine-tuning. Fine-tuning also enables transfer learning.Transfer Learning): Unlike specific adaptation, fine-tuning can also be used for transfer learning, i.e., reorienting the model's task. For example, a model previously used to predict animal species can now be adapted for a completely different task, such as predicting car brands.

[0041] The control interface 10 offers the user the possibility to monitor the current status of the model. Fig.Figure 3 shows a schematic diagram of a possible user interface for controlling the Optimized Model Generator for hardware-specific applications. A user can define the parameters of the final model via an input mask. The control interface allows users to query specific parameters such as hardware, accuracy, compression rate, explainability, and redundancy. This enables the user to adjust the requirements. The control interface allows for monitoring, so the user does not have to wait until the process is complete to receive feedback on the current status of the model. Furthermore, the control interface enables dynamic adjustments: Monitoring allows the user to modify their requirements during the process to better achieve a global optimum.The control interface also saves time: it allows the user to skip the pruning and retraining process, giving them earlier insight into the progress.

[0042] Hardware Validator 11 optionally examines the previously trained and pruned model against predefined criteria. This includes both strict requirements imposed by the hardware platform (computing power, memory, etc.) and variable requirements that can be specified by the user (e.g., inference time). Validation for a given platform thus enables compatibility testing: different hardware platforms often have different chip architectures (x86-64, ARM, RISC-V). A fundamental validation of the model's executability can be verified. Furthermore, validation allows for performance optimization: the key difference between hardware platforms for neural models lies in the type of accelerator (CPU, GPU, etc.). A GPU offers significant advantages over a CPU in terms of model inference time.Validation also enables the detection of bottlenecks: The check can help identify potential performance bottlenecks that might occur on the specific hardware, such as processing speed or data transfer times. The hardware validator can also provide information on how well the model scales before or after pruning, depending on the capabilities of the corresponding hardware platform.

[0043] Deployment (or provisioning) 12 describes the transfer of the optimized model (i.e., the trimmed model after fine-tuning) from the development environment to a production or real-time environment. Deploying an optimized model enables real-time applications. This is one of the essential requirements for the final application of neural models, where fast response times are crucial. Deployment also impacts scalability: Successfully deployed models for a specific type of hardware platform are easily scalable to similar platforms in a different (production) environment. Deployment also addresses security and reliability: On the one hand, it enhances security and data protection, as local data no longer necessarily needs to leave the hardware (no cloud required).Furthermore, dedicated hardware is often more stable and reliable, especially in industrial environments. Deployment also impacts cost efficiency: using dedicated hardware often eliminates the need for cloud-based computing resources.

[0044] The presented concept offers advantages in terms of model compression: By removing less important parts, the model size is reduced, resulting in lower memory requirements. The presented concept also offers advantages in terms of efficiency: The described method reduces the network to its essential components without sacrificing performance. This leads to faster inference and saves resources (energy, computing power, and costs). Furthermore, the presented concept offers advantages in terms of lower hardware requirements: The increased efficiency and the reduced model size lower the hardware requirements of the models. This makes it possible to use the models productively with less powerful hardware, thereby saving costs.The presented concept also offers advantages in terms of improved hardware utilization: Since the model can be optimally adapted to a specific target hardware, it can be used better without wasting resources.

[0045] Regarding further advantageous embodiments of the device according to the invention, reference is made to the general part of the description and to the attached claims in order to avoid repetition.

[0046] Finally, it should be expressly pointed out that the exemplary embodiments of the device according to the invention described above serve only to discuss the claimed teaching, but do not limit it to these exemplary embodiments. Reference sign 1 Data processor 2 Data Generator 3 Model Trainers 4 Relevance Calculators 5 Mask Generator 6 restrictions 7 Specification Checkers 8 Model Trimmers 9 model fine-tuners 10 Control surface 11 Hardware Validator 12 Provision 100 Device 102 Interface 104 processor 106 storage 110 Preprocessing training data 120 Adding training data 125 Calculating synthetic training data 130 Maintaining a neural network, for example by training the neural network 140 Determining the relevance of parameters of the neural network 150 Creating at least one mask 160 Trimming the neural network 170 Fine-tuning of the clipped neural network 180 Deploying the truncated neural network

Claims

[1] Methods for pruning a neural network, comprising: Obtaining (130) a neural network; Determine (140), based on the neural network, a relevance of parameters of the neural network for a prediction of the neural network; Creating (150) based on the relevance of the parameters of the neural network, at least one mask, wherein the at least one mask indicates whether changing a parameter or group of parameters is allowed, prohibited or restricted as part of pruning the neural network; Trimming (160) the neural network based on at least one mask; and Providing (180) the clipped neural network. [2] Method according to claim 1, wherein the pruning is carried out by means of an agent trained by reinforcement learning, wherein the pruning of the neural network is restricted by the at least one mask. [3] Method according to claim 2, comprising training (165) the agent by means of reinforcement learning, wherein the agent is trained to select a part of the neural network to be removed. [4] The method according to claim 3, wherein the agent is trained (165) based on a reward function, wherein the reward function is based on at least one of the accuracy of a prediction of the clipped neural network, the compression achieved of the clipped neural network, the explainability of the prediction of the clipped neural network, and the redundancy of the clipped neural network with respect to subtasks, and / or wherein the agent is trained (165) to clip the neural network such that the neural network after clipping has at most a predetermined maximum size, at least a predetermined inference performance, and / or at least a predetermined prediction quality. [5] The method according to any one of claims 1 to 4, wherein the relevance of parameters of the neural network is determined by means of a statistical and / or information-theoretical approach, and / or wherein the relevance of parameters of the neural network is determined based on a frequency of activation of neurons of the neural network. [6] The method according to any one of claims 1 to 5, further comprising training (130) the neural network using training data. [7] The method according to claim 6, comprising supplementing (120) at least part of the training data by means of a data supplementation method and / or computing (125) a set of synthetic training data by means of a physical or geometric model or by means of a generative machine learning model. [8] The method according to any one of claims 1 to 7, further comprising training (170) after pruning the neural network, the pruned neural network, and / or wherein the pruning is performed to adapt the neural network to a target hardware architecture in terms of size and / or inference performance. [9] Device (10) comprising at least one memory (16) and at least one processor (14) configured to perform the method according to any one of claims 1 to 8. [10] Computer program comprising instructions which, when the program is executed by a computer or control unit, cause the computer or control unit to execute the method according to any one of claims 1 to 8.

Images