Method for providing execution of one or more proxy neural networks in resource-constrained device

Abstract

The invention provides a computer-implemented method for providing one or more proxy neural networks for execution on a resource-constrained device, such as an edge device, the method comprising: retrieving a trained initial neural network trained to predict a set of categories for input data; selecting a subset of categories among the set of categories, the subset comprising one or more categories; creating a copy of the initial neural network; a proxy neural network is obtained, the obtaining including retraining a copy of the initial neural network to make a prediction for a subset of categories, where the prediction of the trained initial neural network is used as a ground truth for the retraining.

Description

Technical Field

[0001] The present invention provides a computer-implemented method, system, computer program product, and computer-readable medium for providing execution of one or more agent neural networks on a resource-constrained device, such as an edge device. Background Art

[0002] Today, neural networks are being used to analyze complex data. From evaluating simple mathematical functions to object recognition, anomaly detection, and classification—almost anything can be classified or analyzed with the appropriate network architecture and training. The more features a network can detect and classify, the larger and more complex its architecture can become. The more complex the network, the greater the evaluation workload.

[0003] Typically, this can be solved by adding additional hardware such as a GPU to the system. However, in some areas, such as embedded systems and edge computing, adding additional hardware is not an ideal option due to energy consumption and resource constraints.

[0004] It is therefore an object of the present invention to provide a method that can overcome at least some of the challenges mentioned above and in particular allow the use of neural networks on resource-constrained devices (such as edge devices) even in complex scenarios. Summary of the Invention

[0005] The present invention achieves this object. The present invention provides a method, a system, a computer program product and a computer-readable medium according to the independent claims. Preferred embodiments are specified in the dependent claims.

[0006] The present disclosure provides a computer-implemented method for providing one or more proxy neural networks for execution on a resource-constrained device, such as an edge device, the method comprising: retrieving a trained initial neural network that is tuned to make predictions for a set of input data categories (or in other words, trained to make predictions); selecting a subset of categories in the set of categories, the subset comprising one or more categories; creating a copy of the initial neural network; and obtaining the proxy neural network, the obtaining comprising retraining the copy of the initial neural network to make predictions for the subset of categories, wherein for the retraining, predictions of the trained initial neural network are used as ground truth.

[0007] The initial neural network will also be referred to as a complex neural network. Compared to the proxy neural network, the initial neural network can be a neural network trained in a manner to handle more complex tasks (especially more complex data with more input data categories).

[0008] The proxy neural network described below can also be referred to as a split neural network. This is because, compared to the initial neural network, the proxy neural network is trained to make predictions only for a subset of the categories. Therefore, one can view the proxy neural network as the result of a split of the initial neural network. The proxy neural network can be considered to replace the initial neural network, at least to the extent that it predicts a subset of the categories.

[0009] Proxy neural networks can be thought of as replacements or proxies for the initial neural network. While a single proxy neural network can replace only a subset of the initial neural network's functionality associated with a subset of classes with at least the same accuracy, it is possible to create multiple proxy neural networks that can provide all of the initial neural network's functionality with at least the same accuracy.

[0010] Prior to training, an initial neural network and a proxy neural network can be selected from known neural network architectures. After training, the neural network can be configured to detect anomalies in machine operation (such as anomalies in motor current), and / or determine expected maintenance requirements (such as predicting maintenance intervals for the system and / or its components and / or predicting failure of the system or its components), and / or provide recommended operating parameters for the equipment or system (particularly operating parameters optimized in terms of energy usage).

[0011] According to the present disclosure, a resource-constrained device may be a device whose computing power and / or storage capacity and / or data connection bandwidth is limited, such as a small memory capacity (e.g., less than 100 megabytes or even less), a fixed control loop time (e.g., less than 25 microseconds), or a low data connection bandwidth (e.g., less than 10 kilobytes / second), which is particularly small / low compared to a device configured to execute the trained initial model. In particular, a resource-constrained device may be a dedicated device, e.g., a device specifically configured for a certain application or purpose. For example, a resource-constrained device may be a device whose resources (e.g., computer power and / or storage capacity and / or data connection bandwidth) are insufficient to execute the trained initial model. An example of a resource-constrained device may be an edge device.

[0012] According to the present disclosure, an edge device may be a device that is an interface between an internal network and an external network. Interfacing the network may be an application or purpose of the edge device in a resource-constrained device environment as described above. An edge device may be a device that is configured to connect a network of drives of a machine (e.g., at a manufacturing site) to a network at the manufacturing site. An edge device is typically a computer with low computing power because it is configured for a specific application.

[0013] Executing an agent neural network on an edge device may involve running (also referred to as executing) the trained agent neural network completely locally at the edge device. Input data may be received from other devices at the edge device (such as sensors) and, optionally after preprocessing, input into the neural network.

[0014] Retrieve an initial neural network trained to make predictions for a set of input data categories.

[0015] The initial neural network can be trained based on techniques known in the art for training neural networks. Regardless of how the initial neural network is trained (e.g., which technique is used), the method according to the present disclosure can be used. Therefore, the training of the initial neural network is not specifically described herein. In general, the training method of the present application does not necessarily require a specific training method. It can be selected to obtain a neural network concept (e.g., architecture). As training techniques, for example, backpropagation or reward can be used, which are particularly suitable in this environment.

[0016] Generally speaking, during training, parameters within a neural network (particularly weights) may be adapted so that computations on input data result in output results that are known during training.

[0017] The above applies to both training and retraining.

[0018] An example of the step of selecting a subset of categories within the set of categories is provided further below.The selection may include receiving user input, optionally after presenting potential selections to the user.

[0019] A copy of the initial neural network is retrained to make predictions for the subset of classes, wherein the predictions of the trained initial neural network are used as ground truth for the retraining. Apart from this and the choice of training data, known training techniques can in principle be used.

[0020] In other words, the initial neural network (specifically its predictions) will be used to perform the training of the proxy neural network.

[0021] It is important to note that a copy of the initial neural network trained on a subset of categories means that it is not trained on the other categories in the category set.

[0022] As an example, a neural network can be trained to detect the operating mode of a device or system (such as a drive). For example, the operating modes of a device or system may include idle, paused, normal, and startup modes. Such a network may be referred to as an operating mode detection method. In such an example, the trained initial neural network may classify idle, paused, normal, and startup (i.e., detect each of these modes), or in other words, may be trained for the categories idle, paused, normal, and startup. A copy of the initial neural network may be trained only to detect a subset of the operating modes, for example, only for the normal category and not for the other categories (i.e., idle, paused, normal, and startup). In other words, during training, the network may see idle, paused, and startup modes and, although it has not been trained to distinguish between them, learn that these modes are not normal modes. These modes may appear similar within the data, but may still differ in some facts.

[0023] For simplicity, retraining a copy of the initial neural network may be referred to below as training the proxy neural network.

[0024] It will be understood that, in particular, the copy of the initial neural network can have the same architecture and weights as the trained initial neural network. Retraining will change the weights within the copy. Pruning and tuning can result in changes to the network architecture such that the network architecture is similar, but not necessarily identical, to that of the initial neural network.

[0025] It will be understood from the above that, in other words, the present disclosure may provide a method for automatically splitting a neural network into proxy neural networks, e.g., for resource-efficient inference on an edge device / edge device combination.

[0026] In other words, the present disclosure may provide a method that can automatically split a neural network (e.g., for classification) into smaller subsets of networks (proxy neural networks) so that they can then be executed in a more energy-efficient, resource-friendly manner or individually.

[0027] The methods of the present disclosure allow for overcoming at least some of the aforementioned challenges of existing systems and methods, as will be appreciated below.

[0028] This approach allows data to be analyzed directly as close to the source as possible. This is promising because the entire data stream does not have to be forwarded via networks and interfaces, but only the final (further processed) information.

[0029] When using neural networks, it is often necessary to choose the hardware in a good way or to add additional computing power to the system. Large, complex, and feature-rich neural networks are often executed on specialized hardware such as GPUs or even clusters of GPUs in the cloud.

[0030] In large networks (e.g., Inception neural networks), evaluation occurs using all features almost simultaneously, and there is no way to select or choose exactly a subset of features of the current input. This leads to a scenario where it is not obvious which part of the input is related to which output.

[0031] To add another feature or slightly change one, the entire network has to be retrained and re-evaluated. This makes re-certification necessary, especially in certified products, as the overall results of the network can change drastically.

[0032] Iteratively training the network implies more hardware work, as the entire network is adapted within each iteration.

[0033] This approach may be acceptable for applications where energy consumption or computational resources are not constrained, but for applications where these are important factors, adding bulky and energy-consuming components is generally not a viable solution.

[0034] It will also be understood from the above that the present disclosure can provide a system and method and arrangement, wherein the method can automatically split a complex neural network into a collection of smaller (proxy) networks, the so-called proxy networks can together represent the same (e.g., classification or anomaly detection) capabilities as the original network, but instead of one network detecting everything, there are multiple networks that focus on certain categories. In doing so, the classification results of the original network and the proxy network can remain essentially unchanged, but with the advantage that each proxy focuses on a subset of categories. Users can design and deploy specific combinations of proxy networks to perform a certain task. Accordingly, there is no capacity overhead for detecting categories that are undetectable or need to be detected / detectable in the current application, so no unnecessary reasoning is performed, and the possibility of classifications that are not within the data range (such as false positives) is eliminated. The combination of proxy networks is more resource-efficient (e.g., compared to executing one large network) because the execution of each proxy network can be performed serially, and the absence of overhead also reduces the number of unnecessary calculations.

[0035] According to the present disclosure, obtaining a proxy neural network may include: after retraining the copy of the initial neural network, optimizing the copy of the initial neural network, in particular optimizing to improve performance, reduce size, and / or reduce energy consumption, wherein in particular, the optimization may be performed with the help of neural network optimization techniques (such as pruning, sparsification, and / or hyperparameter tuning).

[0036] In principle, optimization techniques known from the prior art can be used. In the context of the present disclosure, the impact of optimization is greater than in other scenarios, because retraining based only on a subset of categories will yield a high optimization potential, because removing categories will cause several (non-zero) weights of the initial neural network to become zero or close to zero.

[0037] While the optimization is described after retraining a copy of the initial network, it is worth noting that the method can also include additional optimization during training.

[0038] According to the present disclosure, the training data used in retraining a copy of the initial neural network may include training data that is split according to a selection of a subset of classes.

[0039] For example, splitting the training data based on a selection of a subset of classes may mean that the training set may have a subset of classes (perhaps a single class) as positive values, and all other classes combined as negative values.

[0040] Returning to the example of detecting the operating mode above, the selection of the subset of classes could be to select the normal mode as a subset of classes (in this case, the subset of classes consists of a single class). Splitting the dataset based on subset selection could mean that the training set has the class "normal" as positive and all other classes (i.e., idle, paused, started) as negative.

[0041] According to the present disclosure, multiple agent neural networks can be obtained, each agent neural network being trained for a corresponding subset of categories. In particular, the subsets can be mutually exclusive or only partially overlapping. The combined subset of categories of the multiple agent neural networks can include more categories than the corresponding individual subsets, and in particular, the combined subset of categories can include the entire set of categories. In other words, when combined, the multiple agent neural networks can collectively provide more functionality than the corresponding individual agent neural networks. When the combined subset of categories includes the entire set of categories, the agent neural networks can have the full functionality of the initial neural network when combined.

[0042] According to the present disclosure, training of the proxy neural network may be performed until, for each category in the subset of categories, a predetermined prediction accuracy is obtained, in particular, a prediction accuracy that is equal to or within a predetermined tolerance relative to the prediction accuracy of the trained initial neural network.

[0043] That is, training can be performed in such a way that the proxy neural network has at least the same prediction accuracy, or within tolerance, for the function it is supposed to replace as the trained initial neural network. This ensures that splitting and reducing the size of the neural network does not excessively degrade the prediction quality for the subset of classes, and that the corresponding functions are adequately performed.

[0044] According to the present disclosure, a proxy neural network may be obtained taking into account the computational power and / or storage capacity of the system on which the proxy neural network is to be deployed, in particular by relaxing the accuracy requirements for the proxy neural network.

[0045] For example, the systems (such as edge devices) to which the agent neural network is intended to be deployed may have known or assumed limitations on computing power and / or storage capacity. This can make the execution of some neural networks difficult. Through the above features, it is possible to create agent neural networks that do not face such difficulties without trial and error. Relaxing the accuracy requirements of the agent neural network (e.g., allowing a larger predetermined tolerance relative to the prediction accuracy of the trained initial neural network) is one example of reducing the resource requirements of the agent neural network.

[0046] According to the present disclosure, selecting a subset of categories may include receiving user input identifying categories to be selected, and / or receiving user input identifying a number of categories to be selected for the agent neural network, and / or receiving user input identifying a number of agent neural networks to be provided.

[0047] A user interface can be provided with which a user can interact to provide user input. The user can select the number of categories for the agent neural network. The number of categories will affect the computational requirements for executing the agent neural network. For example, the user can input the number based on the expected capabilities of the edge device. Alternatively or additionally, the user can determine the number of agent neural networks to be provided, which can also be based on the expected capabilities of the edge device and / or based on an expected configuration of multiple edge devices.

[0048] According to the present disclosure, one or more automatically created suggestions for selecting a subset of categories may be output to the user, particularly based on the expected deployment of the agent neural network (such as expected input data and / or expected application and / or expected hardware setup).

[0049] Selecting a subset of categories may include receiving user input identifying the subset among the suggestions.

[0050] For example, subsets can be suggested in such a way that the respective subset provides a specific functionality (or group of functionality, e.g., closely related functionality). Thus, a subset can be viewed as providing a basis for creating tools for functionality, while multiple subsets can be viewed as providing a basis for a toolbox or a pool of functionality. For a known intended deployment (which means known required functionality and / or known hardware capabilities), it can be automatically determined which subset(s) are likely to be suitable for the intended deployment (e.g., satisfy functionality and / or hardware requirements).

[0051] Recommendations can be created by identifying one or more subsets that may be suitable for the intended deployment.

[0052] According to the present disclosure, the method may include providing one or more sets of agent neural networks, each set of agent neural networks including two or more of the agent neural networks and capabilities that provide a specific use case or task. This allows for both flexibility, as it is conceivable that multiple agent neural networks can be combined into sets in different ways, and usability, as (meaningful) grouping into sets that provide capabilities that provide a specific use case or task simplifies the selection process between multiple potential agent neural networks.

[0053] It should be understood that the agent neural networks in an agent neural network set can be different from each other, for example, trained for different subsets of categories and / or trained for different tasks. For example, the agent neural network set can be viewed as providing some kind of combination functionality, or can be viewed as a tool for a specific use case or task. Returning to the concept of a toolbox described above, an agent neural network or agent neural network set can be viewed as a toolbox. For example, the method of the present disclosure can include creating an analysis toolbox with multiple small neural network sets (tools) that can be easily combined and applied to a use case (such as analysis within a drive).

[0054] According to the present disclosure, multiple proxy neural networks can be generated / obtained that, together, provide the same functionality as a trained initial neural network. In other words, the entire initial neural network can be reproduced by splitting it into multiple proxy neural networks. For example, this can be achieved by obtaining proxy neural networks such that, for each category in a set of categories, there is at least one proxy neural network trained for that category.

[0055] According to the present disclosure, training an agent neural network can include training the agent neural network to process raw or pre-processed sensor data for determining the state of a component of a physical system (such as a plant). The sensor data can be data obtained by monitoring the physical system. The sensor data can include measurement data obtained by a sensor and / or camera data obtained with the aid of a camera. Before inputting the sensor data into the agent neural network, the sensor data can be pre-processed. By way of example, particularly in the context of machine learning, any suitable data pre-processing technique known in the art can be employed.

[0056] The tasks and functions discussed above may involve making specific predictions based on raw or pre-processed sensor data, in particular determining the state of physical system components. This allows for monitoring of physical systems.

[0057] According to the present disclosure, the method may further include deploying one or more agent neural networks to one or more hardware components (such as edge devices). The agent neural network may be deployed and executed at the edge device. Optionally, the method includes executing the agent neural network at the edge device, particularly for determining the state of the physical system component, for example, as described above.

[0058] The present invention also provides a system comprising a computing system configured to execute and / or control any method of the present disclosure.

[0059] The system according to the present disclosure may further include one or more edge devices, wherein the respective edge devices are configured to execute (particularly execute locally on the edge devices) a subset (particularly a single subset) of the agent neural network.

[0060] The present invention also provides a computer program product comprising instructions, which, when executed by a computer, cause the computer to perform the method of the present disclosure.

[0061] The present invention also provides a computer-readable medium comprising instructions, which, when executed by a computer, cause the computer to perform the method of the present disclosure.

[0062] In the context of this method, the features and advantages described above also apply to the system, computer program product, and computer-readable medium described herein.

[0063] Further features, examples and advantages will become apparent from the detailed description with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In the accompanying drawings,

[0065] Figure 1 An exemplary method according to the present disclosure is illustrated;

[0066] Figure 2 A system according to the present disclosure is illustrated;

[0067] Figure 3 Another exemplary method according to the present disclosure is illustrated. DETAILED DESCRIPTION

[0068] Figure 1 An exemplary computer-implemented method for providing one or more agent neural networks for execution on a resource-constrained device, such as an edge device, according to the present disclosure is illustrated.

[0069] In step S10 , the method includes retrieving a trained initial neural network, the neural network being trained to make predictions for a set of categories of input data.

[0070] In step S11, the method includes selecting a subset of categories from the set of categories, the subset including one or more categories. Selecting the subset of categories may include receiving user input in optional step S11b. In particular, selecting the subset of categories may include receiving user input identifying a category to be selected, and / or receiving user input identifying a number of categories to be selected for the agent neural network, and / or receiving user input identifying a number of agent neural networks to be provided.

[0071] Optionally, the method may include, in optional step S11a, outputting one or more automatically created suggestions for selecting a subset of categories to the user, particularly based on the expected deployment of the agent neural network (such as expected input data and / or expected application and / or expected hardware setup). For example, this step may be performed before step S11b.

[0072] In step S12, the method comprises creating a copy of the initial neural network.

[0073] In step S13, the method includes obtaining a proxy neural network. Optionally, in step S13, a plurality of proxy neural networks are obtained, each of which is trained for a corresponding subset of categories, wherein the combined subset of categories of the plurality of proxy neural networks includes more categories than the corresponding subsets individually include, and in particular, the combined subset of categories includes the entire set of categories. For example, the plurality of proxy neural networks together can provide the same functionality as the trained initial neural network.

[0074] In particular, by relaxing the accuracy requirements on the agent neural network, obtaining the agent neural network can be performed while taking into account the computing power and / or storage capacity of the system (such as an edge device) to which the agent neural network is expected to be deployed.

[0075] Obtaining the proxy neural network comprises, in step S13a, retraining a copy of the initial neural network to make predictions for the subset of classes, wherein for the retraining, the predictions of the trained initial neural network are used as ground truth.

[0076] The training data used when retraining the initial neural network copy includes the training data split according to the selection of the subset of categories.

[0077] Training the agent neural network may include training the agent neural network to process raw or pre-processed sensor data for determining a state of a component of a physical system (such as a plant) (eg, monitoring data obtained by monitoring the component).

[0078] The respective proxy neural network may be trained until a predetermined prediction accuracy is achieved for each category in the category subset, in particular a prediction accuracy that is equal to or within a predetermined tolerance relative to the prediction accuracy of the trained initial neural network.

[0079] Obtaining the proxy neural network may include, after retraining the copy of the initial neural network, optimizing the copy of the initial neural network in optional step S13b, in particular optimizing to improve performance, reduce size, and / or reduce energy consumption. For example, the optimization may be performed using neural network optimization techniques such as pruning, sparsification, and / or hyperparameter tuning.

[0080] The method may include, in optional step S14, providing one or more sets of agent neural networks, each set of agent neural networks including two or more of the agent neural networks; and providing capabilities for specific use cases or tasks.

[0081] In optional step S15, the method may include deploying one or more agent neural networks to one or more hardware components (such as edge devices). A single agent neural network or multiple neural networks (e.g., a set obtained in step S14) may be deployed to corresponding hardware components.

[0082] In optional step S16 , the method may include executing the corresponding deployed agent neural network(s) at hardware component(s), such as edge device(s).

[0083] Figure 2 A system 1 according to the present disclosure is illustrated, comprising a computing system 10 configured to perform the method of the present disclosure, such as, for example, in Figure 1Optionally, the system may further include one or more edge devices 12a, 12b, 12c. The system may further include a data connection 13 for transmitting data, for example, a wireless or wired data connection. Optionally, the system may include one or more sensing devices 14a, 14b, 14c configured to provide sensor data. The sensor data may be data that allows the state of a component 20a, 20b, 20c of the physical system 2 to be determined. For example, the physical system may be a factory, etc.

[0084] exist Figure 2 In order to illustrate the use of the proxy neural network, the initial neural network is also schematically shown and the NN is used I Labeled, and the proxy neural network is NN S1 ,NN S2 ,NN S3 ,NN S4 Mark.

[0085] exist Figure 2 , a trained agent neural network is also schematically illustrated as being deployed to and executed at an edge device.

[0086] Further features, explanations, and advantages associated with the method and system of the present disclosure are summarized below.

[0087] The present disclosure provides a method for automatically splitting a neural network into proxy neural networks (hereinafter also referred to as proxy networks), for example, for resource-efficient inference on a combination of edge devices.

[0088] The method can automatically split a neural network (e.g., for classification) into smaller subsets of networks (proxy networks) so that they can then be executed in a more energy-efficient and resource-friendly manner, or they can be executed individually.

[0089] The method can provide an automated workflow for splitting a neural network into smaller parts that act as proxies for the original network. Each part can then be able to, for example, classify a subset of the previous large neural network.

[0090] Furthermore, each part can be optimized using known neural network optimization mechanisms and techniques (such as pruning, hyperparameter tuning, sparsification) to ensure the most efficient execution.

[0091] To create the agent network, one can use the training data as well as existing classification methods.

[0092] The network architecture of the split neural network can initially be the same as or different from the original network. The architecture of each proxy model can be adapted to enhance classification.

[0093] The training data (e.g., the classes of training data) can be split into several parts, which should also represent the parts of the proxy network in the future. The finer the granularity of the training data split, the finer the granularity at which the proxy network can be applied to specific tasks in the future.

[0094] For training, the training data splits can then be used to train the network splits, and the classification results of the split (proxy) neural network and the original unsplit neural network can be compared.

[0095] When the results of the split / proxy and unsplit networks are within a given increment of accuracy, the training of the proxy network is considered complete.

[0096] During or after training, the proxy network can be optimized, for example, by using pruning, hyperparameter tuning, sparsification, or any other method or technique for neural network optimization to improve the performance, size, or energy consumption of the neural network.

[0097] After the proxy neural network has been created, it can be executed as a replacement for the original large neural network.

[0098] Depending on the current given application or task, not all proxy networks must be executed, but the proxy network that best suits the current application or task can be selected.

[0099] That is, only a subset of the agent network can be deployed to perform a certain task.

[0100] exist Figure 3 An overview of the method according to the present disclosure is illustrated in FIG.

[0101] Figure 3 An exemplary complex neural network capable of classifying features A, B, C, D, etc. is schematically illustrated. The network is split into proxy networks through an automated splitting process, which are then trained, evaluated, and optimized (e.g., through pruning or hyperparameter tuning).

[0102] A collection of smaller neural networks that are smaller than the original complex neural network is obtained, where each network can classify a subset of the original network.

[0103] Next, one or more smaller neural networks can be deployed to edge devices and used for analysis and / or monitoring, etc.

[0104] When it comes to the splitting process, it can be considered as semantic splitting because the network is split based on its semantic features rather than cutting it layer by layer.

[0105] The original neural network will be used to perform the training of the split parts.

[0106] The number of different parts can be selected by the user or automatically. For example, the number of categories of the neural network or the number of categories in the training data can be used.

[0107] The splitting itself does not necessarily occur in any particular direction (such as horizontally or vertically).

[0108] The method of the present disclosure may ignore (ie, not consider) the neural network structure, and may instead only consider the input and output of the original network.

[0109] The neural network structure of the split neural network can be different from, similar to, or even the same as the original neural network.

[0110] At the latest after pruning and hyperparameter optimization have been performed, many neurons and weights that have no impact on the final result of the now very specialized / dedicated split / proxy neural network can be removed.

[0111] When using split proxy networks, the input data can remain the same, but the classification output of each network will only be a subset of the categories of the original neural network.

[0112] Below, an example of a neural network splitting process according to the present disclosure is described in detail:

[0113] 1. Define how many split (proxy) neural networks should be generated.

[0114] 2. Split the training data according to the number chosen in step 1. For example, multiple classes can go into one split or each class can go into a single split.

[0115] 3. Label all training data as X and treat all other data as non-X (depending on step 2)

[0116] 4. Use X and non-X to train the split (proxy) neural network part

[0117] 5. Evaluate the original network of X using the evaluation dataset and the split (proxy) part to check whether the split can classify the input data as well as the original network. Stop the training process when the split is at least as accurate as the original network. Evaluation data is a portion of the data used to train the neural network. It is split into training data for training the neural network and evaluation data for testing the neural network after training. The evaluation results can be used to calculate metrics such as accuracy.

[0118] 6. Repeat with the next tab for the next proxy network.

[0119] 7. Once all split / proxy neural networks have been trained and evaluated, the method can terminate.

[0120] After obtaining the (smaller) split proxy networks, for example using the above method, the method of the present disclosure may optionally include optimizing each proxy network, for example using known optimization techniques (such as pruning, sparsification, hyperparameter tuning, etc.).

[0121] After all agents and optional optimizations have been created, the networks can be deployed. They can be deployed as a single neural network or as a combination of networks.

[0122] As can be understood from the above, the present disclosure can provide a system, method, and arrangement that has a method for automatically splitting a complex neural network into a collection of smaller (proxy) networks, the so-called proxy networks representing the same (e.g., classification or anomaly detection) capabilities of the original network, but rather than having one network to detect everything, there are multiple networks that specialize in certain categories. In doing so, the classification results of the original network and the proxy network remain unchanged, but the advantage is that each proxy specializes in a subset of categories. Users can design and deploy specific combinations of proxy networks to perform a certain task. The ability to detect categories that cannot be detected or need to be detected / detectable in the corresponding current application does not have overhead, so unnecessary reasoning is not performed, and the possibility of classifications that are not within the data range (such as false positives) is eliminated. The combination of proxy networks is more resource-efficient (e.g., compared to executing one large network) because the execution of each proxy network can be completed serially, and the lack of overhead also reduces the number of unnecessary calculations.

[0123] It should be noted that although the above content is described for neural networks for classification purposes, the methods of the present disclosure are also applicable to anomaly detection and / or regression purposes. In some examples, anomaly detection algorithms can be viewed as classification problems, for example, classifying them into normal operating modes and abnormal operating modes.

[0124] According to the present disclosure, the best applicable network split can be selected via a user interface.

[0125] Some use cases and benefits provided by the methods and systems of the present disclosure are described below.

[0126] The method disclosed herein allows for the application of large and complex neural networks on edge devices. In particular, the method allows for the flexible creation, adaptation, and deployment of neural networks for analysis. The need for expert knowledge is reduced or eliminated. No additional hardware is required on the drive, saving hardware resources (and associated costs) and integration time. The method improves resource efficiency due to improved workload and reduced energy consumption, as existing computing power is used rather than integrating new computing power.

[0127] The disclosed method allows the analysis method to be adapted to current needs without requiring retraining or specialized knowledge, thereby saving computing time and energy while simultaneously monitoring or analyzing a device (e.g., its health) or predicting the next maintenance window. In particular, any user may select a network split. For example, a user may select the best applicable network split, e.g., via a user interface. Furthermore, flexibility is high, as different combinations of network splits are conceivable, which may be specific to a particular application or class of applications or be customer-specific.

[0128] With the method of the present disclosure, no additional hardware is required to perform the analysis method, as the proposed method takes care of the resource requirements of the device.

[0129] The methods of the present disclosure may also allow for a “split-on-demand” functionality that can take a large neural network and its training data as input and can perform the splitting described in this disclosure to produce a collection of smaller proxy models for subsequent deployment.

[0130] The disclosed method may also allow for the creation of an analytics toolbox with collections of small neural networks that can be easily combined for use cases such as in-drive analytics.

[0131] The disclosed method may also allow for providing a "split pool" having a comprehensive collection of agent models each used for a dedicated purpose, such as analyzing specific sets and combinations of drive sensor / parameter values ​​or maintenance prediction.

[0132] The disclosed method and / or the resulting proxy network may be used for one or more of the following use cases:

[0133] Run analytics for monitoring and maintenance on edge devices, such as individual drives.

[0134] Neural network analysis can be easily fine-tuned for specific use cases using only the necessary categories.

[0135] Use multiple proxy networks instead of large, complex networks.

[0136] Split complex neural networks with a large number of categories into usable subsets for fine-grained design of category combinations.

[0137] Providing an “on-demand splitting” service that takes a large neural network and data as input and performs the splitting process described above to generate an ensemble of smaller proxy models for subsequent deployment.

[0138] Although the present invention has been described in detail in the drawings and the foregoing description, such description and illustration are to be considered illustrative rather than restrictive. The present invention is not limited to the disclosed embodiments. In view of the foregoing description and the accompanying drawings, it will be apparent to those skilled in the art that various modifications may be made within the scope of the present invention as defined by the claims.

Claims

1. A computer-implemented method for providing one or more proxy neural networks for execution on a resource-constrained device, such as an edge device, the method comprising: Retrieve (S10) the trained initial neural network (NN I ), wherein the initial neural network is trained to make predictions for a set of categories of input data; selecting (S11) a subset of categories from the category set, the subset comprising one or more categories; Creating (S12) a copy of the initial neural network; as well as Obtain (S13) the agent neural network (NN S1 ,NN S2 ,NN S3 ,NN S4 ), said obtaining comprises retraining (S13a) said copy of said initial neural network to make predictions for said subset of categories, wherein for said retraining, the predictions of the trained initial neural network are used as ground truth.

2. The method according to claim 1, wherein obtaining the agent neural network comprises: After retraining the copy of the initial neural network, the copy of the initial neural network is optimized (S13b), in particular optimized to improve performance, reduce size, and / or reduce energy consumption, wherein in particular, the optimization can be performed with the help of neural network optimization techniques such as pruning, sparsification, and / or hyperparameter tuning.

3. The method of any one of the preceding claims, wherein the training data used to retrain the copy of the initial neural network comprises training data split according to the selection of the subset of classes.

4. The method according to claim 1 , wherein a plurality of agent neural networks are obtained, each agent neural network being trained for a respective subset of categories, wherein the subset of categories of the plurality of agent neural networks in combination comprises more categories than the respective subset alone, in particular the subset of categories in combination comprises the entire set of categories.

5. The method according to claim 1 , wherein the training of the proxy neural network is performed until, for each class in the subset of classes, a predetermined prediction accuracy is obtained, in particular a prediction accuracy that is equal to the prediction accuracy of the trained initial neural network or that is within a predetermined tolerance relative to the prediction accuracy of the trained initial neural network.

6. The method according to claim 1 , wherein the proxy neural network is obtained to be executed taking into account the computing power and / or storage capacity of the system to which the proxy neural network is intended to be deployed, in particular by relaxing the accuracy requirements for the proxy neural network.

7. The method of any one of the preceding claims, wherein selecting the subset of categories comprises: receiving user input identifying a category to be selected; and / or receiving user input identifying a number of categories to be selected for the agent neural network; and / or User input is received identifying a number of agent neural networks to be provided.

8. The method according to any of the preceding claims, wherein, in particular based on an expected deployment of the agent neural network, such as expected input data and / or an expected application and / or an expected hardware setup, one or more automatically created suggestions for selecting a subset of categories are output (S11a) to the user.

9. The method according to any of the preceding claims, comprising providing (S14) one or more sets of agent neural networks, each set of agent neural networks comprising two or more of the agent neural networks and providing capabilities for a specific use case or task.

10. The method according to any of the preceding claims, wherein a plurality of proxy neural networks are obtained which together provide the same functionality as the trained initial neural network.

11. The method of any one of the preceding claims, wherein training the agent neural network comprises training the agent neural network to process raw or pre-processed sensor data for determining states of components (20a, 20b, 20c) of a physical system (2), such as a plant.

12. The method according to any one of the preceding claims, further comprising: The one or more agent neural networks are deployed (S15) to one or more hardware components, particularly resource-constrained devices such as edge devices.

13. A system (1) comprising a computing system (10) configured to perform the method according to any one of claims 1 to 12.

14. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 12.

15. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 12.

Images