How to select a configuration for a machine learning model

Abstract

This invention provides a method, program, and computer-readable medium for selecting a configuration for a machine learning model to be used for controlling a technical system. [Solution] A method for providing multiple machine learning models involves preparing multiple machine learning models with different configurations and training each model on common training data. Each machine learning model includes multiple continuous layers, and the learnable model parameters included in at least some of these continuous layers are not fitted during the training process. The method also performs training using sensor data of the technical system and corresponding reference truth values, and based on the obtained performance evaluation, selects the configuration of the machine learning model that exhibits the best performance for use in controlling the technical system.

Description

Technical Field

[0001] Prior Art Generally, machine learning models can be used for various tasks in the control of technical systems, such as classifying sensor data (e.g., as an image classifier), determining trajectories (e.g., of a robotic arm, the robot itself, etc.), ensuring safety conditions, identifying anomalies and / or errors, etc. However, in modern times, there are a variety of neural architectures for machine learning models, and each neural architecture can be trained using a plurality of various hyperparameters (such as, for example, learning rate, weight decay, dropout rate, weight coefficients of the error function, etc.). Depending on the technical system and the task to be performed, other neural architectures and / or other hyperparameters of the machine learning model may be more suitable for training the machine learning model (i.e., may have higher performance than other neural architectures, for example).

Background Art

[0002] Therefore, in the framework of neural architecture search (NAS), it is possible to find a neural architecture suitable for a specific task in the control of a technical system. In the framework of hyperparameter optimization, it is possible to find hyperparameters suitable for a specific neural architecture. However, the search space for finding a suitable neural architecture is approximately 10 6 to 10 20The neural architecture may include a number of possibilities. The search space for hyperparameter optimization can be in the realm of approximately two to five discrete or continuous hyperparameters. Training each machine learning model completely (i.e., until convergence) for each neural architecture would require enormous computational resources. For example, as described in the paper "Non-stochastic Best Arm Identification and Hyperparameter Optimization" by K. Jamieson et al. (arXiv:1502.07943, 2015), the computational cost can be reduced by training each machine learning model for each neural architecture for a short period (to obtain a low-accuracy estimate of the potential performance of each machine learning model), and then further training only the promising machine learning models. This is also known as multifidelity performance estimation. However, the computational cost required for exploring neural architectures and / or optimizing hyperparameters remains very high. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] K. Jamieson et al., “Non-stochastic Best Arm Identification and Hyperparameter Optimization”, (arXiv:1502.07943, 2015) [Overview of the project] [Problems that the invention aims to solve]

[0004] Disclosure of the invention This disclosure relates to a method for selecting a configuration for a machine learning model to be used for controlling a technical system. This method makes it possible to select a configuration that involves significantly reduced computational costs. This is achieved, for example, by reducing the memory demand required to train the machine learning model so that multiple machine learning models (from a single graphics processor (GPU)) can be trained simultaneously, thereby significantly reducing the duration (and consequently the computational cost) required to train all the machine learning models. This reduced memory demand is achieved by training only the last layer or a few layers of the machine learning model, rather than training all layers.

[0005] It has been shown that training only the last one or a few layers is sufficient to obtain a (low-precision) estimate of the potential performance of a machine learning model. For example, based on this estimate, training of a promising machine learning model can be continued using a high-fidelity performance evaluation method. [Means for solving the problem]

[0006] Various embodiments include a (computer-implemented) method for selecting a configuration for a machine learning model to be used for controlling a technical system, the method providing (e.g., pre-selecting) a plurality of machine learning models, each of which has a different configuration (e.g., a different model architecture and / or other hyperparameters) from the other machine learning models, and each of which has a plurality of continuum layers, each comprising one or more first continuum layers and one or more second continuum layers (e.g., directly) following the one or more first continuum layers, and the input data of the technical system The present invention relates to a method for training each of a plurality of machine learning models (for a predetermined duration or a predetermined number of training iterations) using (identical) training data having source data and associated reference truth value data, wherein, when training each of the plurality of machine learning models, the learnable model parameters of one or at least one of the plurality of continuous layers are not fitted, and (subsequently) selecting the configuration of the trained machine learning model that has the maximum performance (for example, estimated) as the configuration of the machine learning model to be used for controlling the technical system.

[0007] Various implementation examples are shown below.

[0008] Example 1 is a method for selecting a configuration for a machine learning model to be used for controlling a technical system, as described above.

[0009] Example 2 is configured as described in Example 1, in which case, when training multiple machine learning models, some of the machine learning models are trained simultaneously using one (e.g., graphics) processor.

[0010] Because the memory resources required to train each machine learning model are reduced, multiple machine learning models can be trained simultaneously (i.e., concurrently) with the same amount of memory, which significantly reduces the duration (and consequently the computational cost) required to train all machine learning models.

[0011] In Example 3, the method described in Example 1 or 2 may further optionally include, after training each of the multiple machine learning models, selecting some of the multiple machine learning models that perform better than the others, and further training each of the multiple machine learning models (for a predetermined duration or a predetermined number of training iterations).

[0012] Specifically, this method can follow a multi-fidelity performance evaluation method in which, first, all machine learning models are trained for a relatively short time (for example, by extrapolating their respective learning curves to estimate the future accuracy of each machine learning model) to obtain low-accuracy performance, and then only the machine learning model with the highest accuracy is continued training. This can further reduce computational costs and, consequently, additionally shorten the duration required to train all machine learning models.

[0013] Example 4 is configured as described in any one of Examples 1 to 3, in which case, when training each machine learning model, one or more of the first continuous layers among the multiple continuous layers are not fitted.

[0014] In Example 5, the method described in any one of Examples 1 to 4 may further optionally include quantizing one or more first continuous layers of each machine learning model before training each machine learning model.

[0015] During quantization, the precision of model parameters (e.g., weights, biases, activations, etc.) is reduced (e.g., from 32-bit floating-point numbers to 8-bit integers), which in turn reduces the memory resources required to train the machine learning model.

[0016] Example 6 is configured as described in any one of Examples 1 to 5, wherein one or more second continuous layers among each of the multiple continuous layers have one or more continuous layers, one of which continues directly to the terminal layer of one or more first continuous layers, and in this case, when training each machine learning model, one or more continuous layers are trained using a parameter-efficient fine-tuning technique.

[0017] Example 7 is configured as described in Examples 3 and 6, in which, when additionally training each of the multiple machine learning models, all model parameters of one or more continuous layers are fitted.

[0018] According to Example 6, for example, one or more first continuous layers may not be trained at all, i.e., their model parameters may be frozen, and a first segment of one or more second continuous layers can be trained using a parameter-efficient fine-tuning technique, thereby reducing the computational resources required for training. Then, according to Example 7, this first segment of one or more second continuous layers can be fully trained, i.e., all model parameters of this first segment of one or more second continuous layers are fully trainable. It should be understood that in parameter-efficient fine-tuning techniques, not all model parameters of the machine learning model are trained, but only a very small number of (additional) model parameters are trained.

[0019] Example 8 is configured as described in any one of Examples 1 to 7, in which case the configuration of each machine learning model differs from the configurations of the other machine learning models among the multiple machine learning models with respect to the neural architecture and / or training hyperparameters.

[0020] Specifically, this method can provide neural architecture exploration and / or hyperparameter optimization.

[0021] Example 9 is configured as described in any one of Examples 1 to 8, in which case each machine learning model in the multiple machine learning models is a (e.g., image) classifier.

[0022] Specifically, the machine learning model to be used to control the technical system may be a classifier, such as an image classifier for identifying objects within the surrounding environment of the technical system.

[0023] Example 10 is a method for controlling a technical system, the method comprising capturing sensor data representing the technical system or the surrounding environment of the technical system, inputting the sensor data into a machine learning model of a configuration selected using the method described in any one of Examples 1 to 9 in order to obtain an (associated) output, and using the output to control the technical system.

[0024] Example 11 is a control device configured to implement the method described in Example 10.

[0025] Example 12 is a robot device comprising the control device described in Example 11.

[0026] Example 13 is a data processing unit configured to implement the method described in any one of Examples 1 to 9.

[0027] Example 14 is a computer program comprising instructions for causing a processor to implement the method described in any one of Examples 1 to 10 when the program is executed by the processor.

[0028] Example 15 is a computer-readable medium storing instructions for causing a processor to implement the method described in any one of Examples 1 to 10 when the program is executed by the processor.

[0029] In the drawings, like reference numerals generally refer to the same parts throughout all the various drawings. These drawings are not necessarily to scale; rather, emphasis here is placed on depicting the principles of the invention. In the following specification, various aspects are described with reference to the following drawings.

Brief Description of the Drawings

[0030] [Figure 1] A diagram showing a robot device assembly according to various aspects. [Figure 2]This flowchart shows a method for selecting configurations for machine learning models in various ways. [Figure 3] This figure shows various methods for selecting configurations in various forms. [Figure 4] This figure shows various methods for selecting configurations in various forms. [Modes for carrying out the invention]

[0031] The following detailed description refers to the accompanying drawings, which are provided for the purpose of describing the specific details and embodiments of the present disclosure in which the invention is implementable. Other embodiments are also available without departing from the scope of protection of the present invention, and structural, logical, and electrical modifications are also possible. The various embodiments of the present disclosure are not necessarily mutually exclusive, for some embodiments of the present disclosure can be combined with one or more other embodiments of the present disclosure to form new embodiments.

[0032] Various embodiments will be described in more detail below.

[0033] For the sake of clarity, the following description will exemplify a robotic device in which a robotic arm 120 is controlled using at least one machine learning model as a technical system. It should be understood that this robotic device is an exemplary technical system, and that this technical system may also be any other type of computer-controlled device controlled using at least one machine learning model, such as robots (e.g., manufacturing robots, maintenance robots, household robots, medical robots, etc.), vehicles (e.g., at least partially autonomous vehicles), household appliances, production machinery, personal assistants, access control systems, etc.

[0034] Figure 1 shows various embodiments of a robotic apparatus assembly 100. This robotic apparatus assembly 100 may have a robotic apparatus 101 (abbreviated as "robot"). The robotic apparatus 101 shown in Figure 1 and described exemplified below is an exemplary robotic apparatus for illustrative purposes and may include, for example, an industrial robot in the form of a robotic arm for moving, assembling, or processing workpieces, or for bin picking.

[0035] The robotic device 101 may have a robotic arm 120. This robotic arm 120 may have robotic links 102, 103, 104 and a base (or generally a sill) 105 on which these robotic links 102, 103, 104 are supported. The term "robotic link" refers to the movable parts of the robotic device 101, and the operation of these movable parts enables physical interaction with the surrounding environment, for example, to perform a task or to perform or implement one or more skills.

[0036] For the control of the robotic device 101, the robotic device assembly 100 may have a (robot) control device 106, which is configured to interact with the surrounding environment according to a control program. The last element 104 (as seen from the base 105) of the robotic links 102, 103, 104 is also referred to as the end effector 104. The type of this end effector 104 can determine whether the robotic device 101 is in a normal state capable of performing object grasping operations and / or non-grasping operations. An object grasping operation refers to manipulating an object while it is being grasped, while a non-grasping operation may refer to manipulating an object without it being grasped. Therefore, the robotic device 101 may be capable of object grasping operations if the end effector 104 has at least one grasping tool (which may be a suction device (e.g., a suction head)). In any case, the robotic device 101 may be able to perform non-grasping operations on objects by, for example, sliding the object to change its lateral position and / or orientation (without grasping it).

[0037] Other robot links 102 and 103 (located proximal to the base 105) can form a positioning device, thereby, together with the end effector 104, can form a robot arm 120 (articulated arm) with the end effector 104 at its own end. This robot arm 120 may be a mechanical arm capable of performing functions similar to a human arm (and possibly equipped with a tool at its own end).

[0038] The robot arm 120 may have connecting elements 107, 108, and 109 that connect robot links 102, 103, and 104 to each other and connect to the base 105. The connecting elements 107, 108, and 109 may have one or more joints, each of which can provide relative rotational and / or translational motion (i.e., sliding) to the associated robot links. The motion of the robot links 102, 103, and 104 can be initiated using actuators controlled by the control device 106.

[0039] The term "actuator" can be understood as a component that can act on a mechanism in response to being driven. An actuator can convert instructions (so-called activations) output by the control device 106 into mechanical motion. An actuator, such as an electromechanical converter, can be configured to convert electrical energy into mechanical energy in response to its drive control.

[0040] The term "control device" (also referred to as "control unit") can be understood as any type of logic implementation unit that can execute, for example, software, firmware, or a combination thereof stored in a storage medium, and can provide instructions to, for example, an actuator in this example, including circuits and / or processors. A control device can be configured, for example, to control the operation of a system (in this example, a robot) by program code (e.g., software).

[0041] In this example, the control device 106 may include a computer 110 and a memory 111 that stores code and data for the computer 110 to control the robot device 101 based on these. According to various embodiments, the control device 106 can control the robot device 101 based on a robot control model 112 stored in the memory 111.

[0042] For example, the (grasping) task of the robotic device 101 may be a part retrieval task that involves receiving (e.g., grasping or sucking) one of one or more objects 114 and moving the received object (e.g., to a scanner, another container, etc.). To determine which object to receive and an appropriate gripping position on that object, the control device 106 can use an image of the work area of ​​the robotic device 101 where the object 114 is located.

[0043] As another example, the (non-gripping) task of the robot device 101 may be to slide the object 114 to another position and / or to slide the object 114 so that its orientation changes. In this case, an image of the working area of ​​the robot device 101 where the object 114 is located can be used to determine an appropriate contact position (where the robot device 101 should make contact with the object, but not grip it).

[0044] Images of the surrounding environment of these robotic devices 101 can be supplied by one or more image sensors 113 (for example, mounted on a robotic arm 120, or otherwise mounted, so that the control device 106 can control the field of view of one or more image sensors 113).

[0045] Image sensors used in this specification may include, for example, cameras (e.g., standard cameras, digital cameras, infrared cameras, stereo cameras, etc.), radar sensors, LiDAR sensors, ultrasonic sensors, etc. Therefore, images may be RGB images, RGB-D images, or depth images (also referred to as D images). Depth images as described herein may be any type of image having depth information. Specifically, depth images may include three-dimensional information for one or more objects. Depth images as described herein may include, for example, point clouds provided by LiDAR sensors and / or radar sensors. Depth images may be, for example, images with depth information provided by LiDAR sensors.

[0046] The control device 106 may be configured to control the robot arm 120 based on the output of the robot control model 112 in response to the input of an image into the robot control model 112.

[0047] The robot control model 112 may include one or more machine learning models. In one example, the machine learning model may be an image classifier that classifies images for, for example, object identification and / or anomaly and / or error identification. In another example, the machine learning model may determine a trajectory for moving the robot arm 120 and / or a configuration for receiving (e.g., grasping) the (identified) object. These are merely examples, and it should be understood that these machine learning models can be used for many different (sub)tasks in controlling the robot device 101 and for many more different (sub)tasks in controlling any technical system.

[0048] The following describes various approaches to selecting the configuration of a machine learning model to be used for controlling a technical system.

[0049] The selection of a configuration may be the exploration of a neural architecture and / or the optimization of hyperparameters. Thus, the configuration of a machine learning model used herein can represent the hyperparameters of the neural architecture (i.e., the model architecture) and / or one or more training hyperparameters.

[0050] The hyperparameters of a machine learning model's neural architecture include, for example, the type of machine learning model, the number of (sequential) layers in the model, the layer configuration (e.g., type of computation, number of filters, embedding size, etc.), and the connections between layers (e.g., which layers generate inputs for which other layers).

[0051] The type of machine learning model may indicate, for example, whether it is a reinforcement learning model (e.g., using Q-learning, time-difference (TD), deep adversarial networks, etc.), a transformer model, or a classification model (e.g., a neural network). If the type of machine learning model is a neural network, the type of neural network may indicate, for example, whether it is a convolutional neural network (CNN), a variational autoencoder network (VAE), a sparse autoencoder network (SAE), a recurrent neural network (RNN), a deconvolutional neural network (DNN), a generative adversarial network, a forward-thinking neural network, or a sum-product neural network.

[0052] Training hyperparameters for training a machine learning model include, for example, the learning rate, momentum, weight decay, dropout rate, batch size, and the weight coefficients of the error function.

[0053] Figure 2 shows a flowchart of a (computer-implemented) method 200 for selecting a configuration for a machine learning model to be used to control a technical system, in various forms.

[0054] Method 200 (in 202) provides a plurality of machine learning models, wherein each of the plurality of machine learning models has a different configuration from the other machine learning models, and each of the plurality of machine learning models has a plurality of continuous layers, each including one or more first continuous layers and one or more second continuous layers that (for example, directly) follow the one or more first continuous layers.

[0055] Method 200 (in 204) may include training each of several machine learning models (for a predetermined duration or for a predetermined number of training iterations) using (identical) training data having sensor data of a technical system as input data and associated reference truth value data. When training each of several machine learning models, the learnable model parameters (e.g., all learnable model parameters) of one or at least one of the first continuous layers of each of the multiple continuous layers are unfittable.

[0056] Specifically, one or more of the first continuous layers in each machine learning model can be frozen at least partially.

[0057] Method 200 may (subsequently) (in 206) include selecting a trained machine learning model configuration that has the (estimated) maximum performance as the configuration for the machine learning model to be used for controlling the technical system.

[0058] In the following sections, various aspects of this method 200 will be described in more detail with reference to Figures 3 and 4.

[0059] Figure 3 schematically shows multiple consecutive layers 304 (n=1 to N) of an exemplary machine learning model. Here, the machine learning model may have N (consecutive) layers (where N is an integer greater than or equal to 2). The leading layer 304(1) of the machine learning model can receive the input 302 of the machine learning model, and the terminal layer 304(N) of the machine learning model can output the output 306 of the machine learning model. It should be understood that the layer architecture shown in this example is used for concretization purposes, and these layers can be connected to each other by any method (e.g., using skip connection).

[0060] In various embodiments, NK tip layers (304(1) to 304(NK)) are either not trained, or only a portion of the model parameters of these NK tip layers are trained. That is, one or more first continuous layers 308 may contain NK tip layers, where K is any integer greater than or equal to 1.

[0061] In some embodiments, one or more of the first continuous layers 308 are not trained at all (i.e., not fitted). Specifically, their model parameters (e.g., their weights, biases, etc.) are frozen.

[0062] In other embodiments, only a subset of the learnable model parameters of one or more first continuous layers 308 are trained. Specifically, only some model parameters are frozen. For example, in a convolutional neural network, biases may be adaptable, while weights may be freezeable, or vice versa. For example, in a transformer model, the heads of the attention layer may be trainable (i.e., adaptable), while the query matrix and / or key matrix (of the linear layer) may be freezeable (i.e., unadaptable), or vice versa.

[0063] In both cases, at least some model parameters are not trained, significantly reducing the memory and computational costs during training. For example, in the case of backpropagation, it is not necessary to calculate the gradients (e.g., with respect to weights) for these model parameters.

[0064] If the weights of one or more first continuous layers 308 are not fitted, these first continuous layers 308 retain, for example, their randomly initialized weights, and these weights may function as feature extractors and have the same inductive bias as a fully trained machine learning model. Therefore, even in this case, a (low-precision) estimate can be obtained regarding the performance of a fully trained machine learning model (i.e., the performance that would be obtained if all N layers were trained).

[0065] If one or more consecutive layers 308 are not fitted at all, there is no need to buffer the results calculated by the network nodes (also called intermediate activations) for later backpropagation. Furthermore, there is no need to retain the optimized state in memory due to the frozen model parameters. This significantly reduces memory costs.

[0066] It should be understood that the machine learning models among multiple machine learning models can have different numbers of layers N. Similarly, the values ​​of K for these models can also differ. The accuracy of the estimate improves with increasing K, but it should be understood that computational and memory costs also increase accordingly. In various embodiments, there can be multiple stages of training, where the value of K increases with each stage (and, optionally, only a subset of the machine learning models are further trained). Specifically, the number of frozen layers is adaptable (e.g., reduced in multiple stages during training).

[0067] In various ways, reducing the number of training iterations can further reduce computational costs.

[0068] In various embodiments, the memory and computational costs can be further reduced by quantizing the model parameters (e.g., weights, biases, activations, etc.) of one or more first continuous layers 308. During quantization, their precision is reduced, for example, from 32-bit floating-point numbers to 8-bit integers. As a result, they require less memory, and the computation of the output is accelerated, thus reducing the computational cost. The cost of quantization is a one-time expense and is significantly less than the savings in computational and time costs during subsequent training. For example, the quantization of a layer can be determined using minimum and maximum values ​​and / or by minimizing the propagation quantization error that occurs during computation.

[0069] As described herein, one or more second continuous layers 310 (layers 304(N-K+1) to 304(N)) from a plurality of continuous layers 304 (n=1 to N) can be trained, that is, the model parameters of the second continuous layers 310 can be fitted.

[0070] In various embodiments, one or more front-end layers (starting from 304(N-K+1)) can be trained using a parameter-efficient finetuning method (PEFT). This method requires less computational and memory cost than full training, but if, for example, the number of K is increased in other stages of training and / or these one or more front-end layers are to be trained directly, convergence for these layers can be accelerated (because approximations for model parameters already exist based on PEFT), resulting in a reduction in the overall computational cost of training. An exemplary parameter-efficient finetuning method is Low-Rank Adaptation (LoRA).

[0071] Specifically, this training can be performed as follows: First, the model parameters of one or more first continuous layers 308 are freezeable. Optionally, these model parameters can be quantized. Optionally, one or more leading layers of one or more second continuous layers 310 are representable using a PEFT model. Then, one or more second continuous layers 310 can be trained accordingly (e.g., using gradient backpropagation).

[0072] Following training, the performance of each machine learning model can be determined (e.g., evaluated), and the configuration of the machine learning model with the highest (determined) performance can be selected for use in controlling the technical system. In some embodiments, the machine learning models thus trained can be used directly. In other embodiments, the machine learning models thus trained can be further trained. In yet another embodiment, a new machine learning model can be initialized with the desired configuration and trained on appropriate training data before being used to control the technical system.

[0073] Figure 4 illustrates various scenarios for selecting the configuration of a machine learning model (e.g., exploring neural architectures) for a specific technical system.

[0074] Figures 402, 404, and 406 show the memory cost 408 and computational cost (or duration) 410 required for training, respectively, where the letters represent each machine learning model along with its associated neural architecture.

[0075] Figure 402 illustrates the configuration selection without a multi-fidelity performance evaluation method, where each of the multiple machine learning models is trained to a relatively high degree of accuracy. This requires a very high computational cost.

[0076] Figure 404 illustrates configuration selection using a multi-fidelity performance evaluation method, where each of the multiple machine learning models (A, B, C, and D in this example) is trained to relatively low accuracy for a specific duration, after which only the selected machine learning model (B, C, and D in this example) is further trained.

[0077] Figure 406 illustrates configuration selection combined with partial freezing of layers for each machine learning model using a multi-fidelity performance evaluation method, as described herein. Specifically, this allows multiple machine learning models (e.g., A, B, C, D) to be trained simultaneously (using the same GPU) based on reduced memory resources for training each machine learning model. This reduces the overall cost required for configuration selection and / or allows for a larger number of machine learning models (i.e., a larger search space) that can be trained with the same computational cost.

[0078] A method for controlling a technical system (e.g., a robotic device 101) may include capturing sensor data representing the technical system or its surrounding environment. This sensor data may include, for example, images. This sensor data can be input into a machine learning model of a configuration selected using this method 200, and the technical system can then be controlled based on the output of the machine learning model.

Claims

1. A computer-implemented method (200) for selecting a configuration for a machine learning model to be used for controlling a technical system (100), The above method (200) is, (202) To provide a plurality of machine learning models, wherein each of the plurality of machine learning models has a configuration different from that of the other machine learning models, and each of the plurality of machine learning models has a plurality of continuous layers (304) including one or more first continuous layers (308) and one or more second continuous layers (310) directly following the one or more first continuous layers (308), the leading layer (304(1)) of the first continuous layer (308) receives the input (302) of the machine learning model, and the terminal layer (304(N)) of the second continuous layer (310) outputs the output (306) of the machine learning model. Training each of the plurality of machine learning models (204) using training data comprising sensor data of the technical system (100) as input data and associated reference truth value data, wherein when training each of the plurality of machine learning models, one or more second continuous layers (310) of each of the plurality of continuous layers (304) are fitted, and at least one learnable model parameter of each of the one or more first continuous layers (308) is not fitted (204), Selecting the configuration of the trained machine learning model that has the maximum performance as the configuration for the machine learning model to be used for controlling the technical system (206), Method (200), including.

2. The method according to claim 1 (200), wherein, when training the plurality of machine learning models (204), some of the plurality of machine learning models are trained simultaneously using one processor.

3. The above method further, After training each of the multiple machine learning models (204), select some of the multiple machine learning models that have higher performance than the other machine learning models. This involves additionally training each of the aforementioned multiple machine learning models, The method according to claim 1 or 2 (200), including the method according to claim 1 or 2.

4. The method (200) according to any one of claims 1 to 3, wherein one or more first continuous layers of each machine learning model are quantized before training each machine learning model (204).

5. The method according to any one of claims 1 to 4 (200), wherein, when training each of the machine learning models (304), one or more continuous layers are trained using a parameter-efficient fine-tuning technique.

6. The method according to any one of claims 1 to 5 (200), wherein the configuration of each machine learning model differs from the configuration of other machine learning models among the plurality of machine learning models with respect to the neural architecture and / or training hyperparameters.

7. A method for controlling a technical system (100), The aforementioned method, The capture of sensor data representing the aforementioned technical system (100) or the surrounding environment of the technical system (100), To obtain an output, sensor data is input to a machine learning model of a configuration selected using the method (200) described in any one of claims 1 to 6, The output is used to control the technical system (100), Methods that include...

8. A data processing unit configured to carry out the method described in any one of claims 1 to 7.

9. A computer program that, when executed by a processor, includes instructions for causing the processor to perform the method described in any one of claims 1 to 7.

10. A computer-readable medium which, when executed by a processor, stores instructions causing the processor to perform the method according to any one of claims 1 to 7.

Images