Method and apparatus for integrating machine learning systems into embedded devices

Abstract

A method is disclosed in which a machine learning system is integrated into a target data processing system. The machine learning system consists of a decision tree whose nodes contain neural networks. First, the hardware resources of the target data processing system are defined. The machine learning system is then trained using training data, with a cost function being optimized to fulfill the specified task of the learning system. The cost function contains an L0 regularization term that penalizes the number of parameters in the learning system. Each node in the decision tree is assigned such an L0 term. Finally, the trained machine learning system is integrated into the target data processing system, which has fewer hardware resources than the training data processing system.

Claims

[1] Method (100) for integrating a machine learning system on a target data processing system (10), wherein the machine learning system comprises at least one decision tree, wherein at least one node of the decision tree comprises at least one neural network, comprising: Defining (101) hardware resources of the target data processing system (10); Training (102) of the machine learning system on provided training data, wherein during training a cost function which characterizes a predetermined task to be learned by the machine learning system is optimized with respect to parameters of the machine learning system, wherein the cost function has at least one L0 regularization term, characterized by , that the L0 regularization term is assigned to the node and the L0 regularization term penalizes a number of parameters of the node depending on the defined hardware resources or defines a minimum or maximum number of parameters of the node; Integrating (103) the trained machine learning system onto the target data processing system (10), which has fewer hardware resources than a data processing system on which the machine learning system was trained. [2] Method according to claim 1, wherein in the step of learning (102) an influence of the L0 regularization term increases with increasing training progress, in particular so that after completion of the learning only one parameter is present per node. [3] Method according to claim 2, wherein the influence of the L0 regularization term is modeled depending on the training progress with an annealing method, in particular deterministic annealing. [4] Method according to one of the preceding claims, wherein the cost function comprises at least one further L0 regularization term which depends on a total number of all parameters of the machine learning system. [5] Method according to one of the preceding claims, wherein the method is used for XGBoost or LightGBM. [6] Device which is arranged to carry out the method according to one of the preceding claims. [7] A computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to claims 1 to 4. [8] A machine-readable storage medium on which the computer program according to claim 7 is stored.

Images