Apparatus and method for exploring a neural architecture in a single step using unlabeled data

Abstract

To provide a computer-implemented method for exploring a neural architecture.SOLUTION: The method includes training a supermodel having the tested architecture based on the labeled training data to update weights, and determining a Pareto-optimal sub-model of the supermodel based on the at least two performance metrics. The determining iteratively performs the steps of computing an output of a reference model for the unlabeled data, the reference model being a largest sub-model of the supermodel, testing a plurality of sub-models from the supermodel, computing respective outputs of the unlabeled data by the sub-models, computing a difference between the output of the reference model and the outputs of the sub-models, and using an optimization algorithm to iteratively test and evaluate the sub-models based on a plurality of objectives.SELECTED DRAWING: None

Description

[Technical Field]

[0001] The present invention relates to a method for exploring neural architectures in a single step ("one-shot"), including a two-stage method, a computer program and a machine-readable storage medium and system. [Background technology]

[0002] Prior art Techniques for neural architecture search (NAS) face the challenge that, compared to traditional supervised learning, available data is not only used for 1) training model weights (training data) and 2) validating the model and estimating its performance on unknown data (validation data), but also for 3) determining the optimal model architecture (search data). Validation data must not be used for search, because this would make it no longer possible to unbiasedly evaluate model performance. Therefore, in existing methods, training data is also used for search.

[0003] In the ideal case, the training data and search data are independent of each other, allowing for unbiased evaluation of architectures (and achieving robustness against potential overfitting to the training data). Techniques such as SPOS (Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun, “Single Path One-Shot Neural Architecture Search with Uniform Sampling”, EECV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58517-4_32) follow this approach by splitting the original training data into two distinct parts: one for training and one for search.

[0004] The challenge with such approaches is the scarcity of data available for training the model, which reduces the quality of the trained model and increases the variance in performance estimates. Therefore, such approaches are only applicable in scenarios where there is sufficient training data available (e.g., more than 50k labeled examples).

[0005] Other technologies, such as BigNAS (Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc Le, “BigNAS: Scaling up Neural Architecture Search with Big Single-Stage Models”, ECCV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58571-6_41) or NASViT (Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, ​​Qiang Liu, Vikas Chandra, “NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet "Training" uses the complete original training data for training the model and also uses part of the training data or the complete training data for architecture exploration, i.e., there is an overlap between the training data and the exploration data. This improves the quality of training, but does not provide an unbiased assessment of performance on unknown data, since the exploration data was used during training. This means that such an approach can also only be used in scenarios where there is sufficient training data available to mitigate the risk of overfitting.

[0006] Finally, attempts have been made to perform NAS without using labeled data, for example, in Chenxi Liu, Piotr Dollar, Kaiming He, Ross Girshick, Alan Yuille, and Saining Xie, "Are Labels Necessary for Neural Architecture Search?", ECCV. 2020. http: / / dx.doi.org / 10.1007 / 978-3-030-58548-8_46. These techniques perform NAS on unlabeled proxy tasks, such as rotation prediction or color modification, and then repurpose the resulting architecture for the original task. While such approaches do not require labeled data for NAS, they also result in a lower quality of the resulting architecture, since the correlation between the architecture's performance on the proxy task and its performance on the actual task may be weaker. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun, “Single Path One-Shot Neural Architecture Search with Uniform Sampling”, EECV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58517-4_32 [Non-patent document 2] Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc Le, “BigNAS: Scaling up Neural Architecture Search with Big Single-Stage Models”, ECCV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58571-6_41 [Non-patent document 3] Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, ​​Qiang Liu, Vikas Chandra, “NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training” [Non-patent document 4] Chenxi Liu, Piotr Dollar, Kaiming He, Ross Girshick, Alan Yuille, Saining Xie, “Are Labels Necessary for Neural Architecture Search?”, ECCV. 2020. http: / / dx.doi.org / 10.1007 / 978-3-030-58548-8_46. Summary of the Invention [Problem to be solved by the invention]

[0008] The present invention aims to solve the problem of one-stage NAS using a limited amount of available labeled data. To maximize the quality of the trained one-shot model (supermodel), the present invention reserves the entire labeled training data for training and uses unlabeled data for architecture search. In particular, it proposes evaluating the performance of a particular subnetwork in the one-shot model by comparing its output on unlabeled data with the output of a reference network. The unlabeled data may be data from the same application, similar data from a similar application, or synthetic data. The selection of the reference network is important because its performance must be higher than that of the subnetwork to be able to generate a qualitatively valuable reference output. Furthermore, its creation should not impose additional training costs on the already computationally intensive NAS procedure. The present invention addresses the above problem by using the largest subnetwork in the one-shot model as the reference network. This is because 1) the largest subnetwork essentially has the best performance of all subnetworks (especially when certain training techniques such as the sandwich rule and in-place distillation are used), and 2) it is already available.

[0009] In summary, the objective of the present invention is to provide a one-shot NAS that uses a small amount of available labeled data, and the present invention effectively solves the above objective by estimating the performance of a sub-network of a one-shot model in the one-shot NAS by comparing the output of this sub-network on unlabeled search data with the output of the largest sub-model in the one-shot model.

[0010] Advantages of the invention The above-mentioned prior art techniques cannot efficiently utilize the limited available training data for one-shot NAS. In contrast, the present invention utilizes the entire labeled training data for model training, thereby improving the quality of the trained supermodel. This reduces the variance in the estimation of the performance of submodels, thereby resulting in better architectures. Furthermore, the present invention maintains independence between the training data and the search data, thereby enabling unbiased estimation of the performance of submodels and resulting in better architectures. Furthermore, by avoiding proxy tasks for NAS and searching actual tasks, the challenges caused by the weak correlation between proxy tasks and actual tasks are alleviated, thereby resulting in better architectures.

[0011] Furthermore, because an already existing submodel (e.g., the largest submodel) is used as a reference model, no additional reference model training and maintenance is required for the present invention, which further reduces computational complexity and development costs. [Means for solving the problem]

[0012] Disclosure of the Invention First, a computer-implemented method for exploring neural architectures, particularly for training a classifier for classifying input signals obtained from a sensor, is proposed. The method begins by training a supermodel having multiple searchable dimensions, where the training is performed by testing architectures within the searchable dimensions, training the supermodel having the tested architecture based on labeled training data, and updating the weights of the supermodel with gradients with respect to the tested model. The supermodel may be a large-scale neural network, where submodels of this network form effective architectures with reduced complexity. Examples of this include fewer channels, fewer levels, smaller embedding size, etc.

[0013] After the training, a step of determining Pareto-optimal sub-models of the supermodel based on at least two performance metrics is performed iteratively.

[0014] Determining the Pareto-optimal submodels begins by computing the outputs of a reference model for unlabeled data, where the reference model is the largest submodel of the supermodel. The performance metrics may be the accuracy of each submodel on a particular computing unit or device and the hardware latency of each submodel. Other hardware-related performance metrics may include latency, flops, power consumption, and / or memory utilization. Preferably, performance metrics for the reference model are not computed, but only model outputs (e.g., classes, object bounding boxes, etc.) are computed, and then only these model outputs are compared to each other.

[0015] Then, multiple sub-models from the supermodel are tested. The sub-models then calculate their own outputs for the unlabeled data, also referred to as propagating the unlabeled data through the sub-models. Differences between the outputs of the reference model and the sub-models are then identified. Finally, an optimization algorithm is applied to iteratively test and evaluate additional sub-models based on multiple objectives, including the difference between the outputs of the reference model and the sub-models and other performance metrics. Examples of other performance metrics include latency, flops, power consumption, and / or memory utilization. By iteratively applying the optimization algorithm, the tested sub-models converge toward a Pareto-optimal sub-model for the above objectives.

[0016] In a further aspect of the invention, it is envisioned that the supermodel, and therefore the submodels, are trained using a classifier to classify input signals. The classifier may be used by receiving sensor signals that include data from sensors, determining input signals that are dependent on the sensor signals, and providing the input signals to the classifier to obtain output signals that characterize the classification of the input signals.

[0017] Such a classifier may then be used to provide an actuator control signal for controlling an actuator, which preferably controls an at least partly autonomous robot and / or manufacturing equipment and / or access control system, comprising all the steps of the method described above, and further comprising determining the actuator control signal in dependence on the output signal.

[0018] The sensors may determine measurements of the environment in the form of sensor signals which may be provided by, for example, digital images, e.g., video, radar, LiDAR, ultrasound, motion, thermal images, or audio signals.

[0019] Preferably, the present invention may be used to classify sensor data, to recognize the presence of objects in the sensor data, or to perform semantic segmentation of the sensor data, e.g. with respect to traffic signs, road surfaces, pedestrians and vehicles.

[0020] The supermodel and submodels may be used to determine a continuous value or multiple continuous values, i.e., perform a regression analysis on elements in the data, such as distance, velocity, acceleration, and tracking of an object, based on relatively simple features (e.g., edges or pixel attributes for an image).

[0021] The invention may be used to calculate control signals for controlling technical systems, e.g. computer-controlled devices, such as robotic systems, vehicles, household appliances, power tools, manufacturing equipment, personal assistants or access control systems.

[0022] Embodiments of the present invention will now be described in more detail with reference to the following drawings. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 illustrates a schematic architecture of a supermodel and examined submodels from the supermodel. [Figure 2] 1 is a schematic flow chart of an embodiment of the present invention. [Figure 3] FIG. 1 illustrates a control system for controlling an at least partially autonomous robot. [Figure 4] FIG. 1 is a diagram showing a control system for controlling a manufacturing device. [Figure 5] FIG. 1 illustrates a control system for controlling the access control system. [Figure 6] FIG. 1 is a diagram illustrating a control system that controls the monitoring system. [Figure 7]FIG. 1 illustrates a control system for controlling an automated personal assistant. [Figure 8] FIG. 1 illustrates a control system for controlling the imaging system. [Figure 9] FIG. 1 illustrates a training system for controlling a classifier. DETAILED DESCRIPTION OF THE INVENTION

[0024] The present invention improves the search procedure of a two-stage neural architecture search. Generally, a two-stage NAS includes the following two stages:

[0025] 1) Training a supermodel. A supermodel is a neural network model in which multiple dimensions of the model are explorable, i.e., the multiple dimensions of the model are not fixed during training of the supermodel. Typical explorable dimensions may be the number of layers in different parts of the network, the number of channels, kernel size, group size in grouped convolutions, embedding dimension, number of attention heads, MLP ratio, etc.

[0026] The supermodel allows for the extraction of submodels that make specific architectural decisions regarding the searchable dimensions. The largest submodel uses the maximum value for each searchable dimension. See Figure 1 for a diagram of supermodel 1 and largest submodel 1a. The training procedure is essentially performed by testing one or more architectures from the one-shot model at each update step, calculating gradients with respect to the selected model, and using this data to update the weights of the supermodel. Known approaches in the prior art, for example, test a single random model. However, we start from the idea that the present invention works particularly well with the sandwich rule of Yu, J. and Huang, TS, "Universally slimmable networks and improved training techniques," ICCV. 2019. https: / / doi.org / 10.48550 / arXiv.1903.05134, which tests two random architectures and the smallest and largest architectures at each update iteration, thereby making the largest model essentially the best model in the search space. Furthermore, techniques such as in-place distillation (known from previous publications) promote similarity between the largest submodel and the smaller submodels.

[0027] After the first phase is completed, the second phase begins.

[0028] 2) Architecture Exploration: In this phase, submodels of the supermodel are identified that are Pareto-optimal with respect to a goal (e.g., accuracy and / or hardware latency and / or other goals). This is essentially done by iteratively selecting and evaluating submodels. It is important to note that this step does not require training of the model, since the trainable weights of the submodels are inherited from the corresponding part of the trainable weights of the supermodel.

[0029] Search efficiency is improved by using optimization algorithms such as evolutionary search. Evaluating the submodels includes, for example, measuring accuracy, latency, etc. (using the search data).

[0030] The process of how to iteratively select and evaluate sub-models to find a "good" (i.e., Pareto optimal) sub-model may be performed by evolutionary optimization, in which new sub-models are created by random mutation of evaluated sub-models (e.g., changing the number of filters or changing the depth in a particular part of the network). That is, the evolutionary optimization continues searching in the neighborhood of the previous best sub-model. A further approach would be to use black-box optimization methods such as Bayesian optimization (BO).

[0031] The main advantage of the present invention is the use of unlabeled data for the architecture exploration phase of the two-stage NAS. This keeps more data available for the first stage, i.e., for training the supermodel, while maintaining independence between the training data and the exploration data. This is achieved by the following steps, as shown by reference numeral 1a in Figure 1, to approximate the application performance (e.g., accuracy: IoU) of the sub-models by calculating the difference between the outputs of these sub-models and the output of the largest sub-model. The largest sub-model may be provided by all maximally explorable dimensions.

[0032] Given a trained supermodel M and unlabeled exploration data D, the output is the following pseudocode: 1) (Optionally) filter the search data D: D'=f filt (D) 2) Largest submodel A big Calculate the output of: y big =M(D,Abig ) 3) For each submodel A: a) Calculate the output of submodel A: y A =M(D',A) b) Output difference

number

[0033] Finally, the difference L can be used as a search goal (eg, minimize L).

[0034] The unlabeled exploration data D may be provided from different potential sources. The unlabeled data may be detected from the same application, which may be the case if only a portion of the available data is labeled. Additionally or alternatively, unlabeled data from similar applications can be collected, for example, data from previous product generations captured with other camera models. Additionally or alternatively, synthetic data generated from the training data distribution can be used as the unlabeled data.

[0035] The choice of unlabeled data may be important because a compromise must be made between the amount of available data and its similarity to the actual application. The amount of data should be sufficient to obtain a low-variance estimate of the sub-model performance. Preferably, a low-variance estimate requires, for example, at least 100 input patterns, with 1,000 to 10,000 being ideal. At the same time, unlabeled data may not represent the actual performance of the sub-model if they are not similar to the application.

[0036] If the domain of the unlabeled data is much larger than the domain of the training data (e.g., the training data are road scenes and the unlabeled data are images from the internet), then the largest output y of submodel 1a big is not used in some situations. In this case, D and thus y big However, if the domains of the training data and the domains of the unlabeled search data are similar, filtering is not necessary. Therefore, f filt There are multiple ways how the search data can be filtered using [D' = D]. A simple filter option can be no filter (D' = D). For image classification, a filter operation can be defined by selecting high confidence images [D' = {d|max(M(d,A big ))>δ,d∈D}].

[0037] In step 3b) above, the difference can be calculated as follows: A first option for the difference is the Kullback-Leibler divergence:

number

number

number

[0038] Figure 2 shows a flowchart 20 of one embodiment of the present invention. The method illustrated in Figure 2 is a deliberate approach for neural architecture search (NAS), a key aspect of machine learning aimed at automating the design of artificial neural network architectures. The method of Figure 2 aims to optimize the process of identifying the most efficient and most effective neural network architecture from a large search space effectively defined within a supermodel (1). The method is divided into several core steps, each contributing to the overall goal of finding a Pareto-optimal submodel that strikes a balance between performance metrics, such as accuracy, and hardware metrics, such as latency.

[0039] The method begins by training (S21) a supermodel 1. This supermodel is used as a basis from which specific architectures (submodels) can be tested and evaluated. Training (S21) can include multiple steps. Training (S21) begins by testing an architecture. Individual architectures are tested within the defined explorable dimensions of the supermodel. These dimensions can include aspects such as the number of layers, layer type (convolutional layer, recurrent layer, etc.), and layer size. In a second step, training is performed using labeled training data. Each tested architecture is trained based on a dataset with labeled training data. After the tested architectures are trained, weight updates of the supermodel are performed. The weights of the supermodel are updated based on gradients calculated relative to the performance of the tested architecture. This step ensures that the supermodel learns from the performance of each architecture, gradually improving the supermodel's ability to generate effective submodels.

[0040] After training (S21), an unlabeled data set is provided (S22). The unlabeled data set may be smaller than the labeled training data set of step S21. These unlabeled data are important for evaluating the performance of the sub-models in a manner independent of costly labeling, and provide insight into the performance of the sub-models.

[0041] Subsequently, the Pareto-optimal sub-models are determined (S23), which are the sub-models that offer the best compromise between at least two performance metrics, such as accuracy and hardware latency, making these sub-models the Pareto-optimal decisions for a particular application.

[0042] Step S23 begins by selecting a reference model. Preferably, the reference model is the largest submodel (1a). While it would be possible to use any other network trained on the same task as the reference model, this would jeopardize one of the main advantages of the present invention: the lack of additional costs for creating a reference model. An output for unlabeled data is calculated based on the reference model. This reference model is required to provide a benchmark. Then, multiple submodels (1b) from the supermodel (1) are tested. Different submodels (1b) are tested, preferably randomly from the supermodel, each representing a different architecture within the search space. The outputs for the submodels are then calculated. Each tested submodel calculates an output for the same unlabeled data. The difference between the output of the reference model and the output of the submodel is then calculated.

number

[0043] After step S23 is completed, the Pareto-optimal sub-models are selected for provision in the target device (S24) based on criteria predefined by the user, for example, for the particular application to be implemented in the target device.

[0044] The sub-model selected according to step S24 may be an image classifier (60), preferably used for different applications as described below.

[0045] 3 shows an embodiment of an actuator with a control system 40. The actuator and its environment together are referred to as the actuator system. A sensor 30 detects the state of the actuator system, preferably at evenly spaced intervals. The sensor 30 may include multiple sensors. Preferably, the sensor 30 is an optical sensor that takes images of the environment. An output signal S of the sensor 30 (or, if the sensor 30 includes multiple sensors, an output signal S for each of these sensors) encoding the detected condition is transmitted to the control system 40.

[0046] In this case, the control system 40 receives a stream of sensor signals S. The control system 40 then calculates a set of actuator control commands A depending on the stream of sensor signals S, which are then transmitted to the actuator units 10, which convert these control commands A into mechanical movements or changes in physical quantities. For example, the actuator units 10 may convert the control commands A into electrical, hydraulic, pneumatic, thermal, magnetic, and / or mechanical movements or changes. Specific, but non-limiting, examples are electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servomechanisms, electromagnets, stepper motors, etc.

[0047] The control system 40 receives the stream of sensor signals S of the sensors 30 at an optional receiving unit. The receiving unit converts the sensor signals S into input signals x. Alternatively, if no receiving unit is provided, the respective sensor signals S may be used directly as input signals x. The input signals x may, for example, be presented as excerpts of the sensor signals S. Alternatively, the sensor signals S may be processed to generate the input signals x. The input signals x include image data corresponding to images captured by the sensors 30. In other words, the input signals x are provided in accordance with the sensor signals S.

[0048] The input signal x is then forwarded to a classifier 60, which may for example be provided by an artificial neural network.

[0049] The classifier 60 is parameterized by parameters φ, which are stored in and provided by the parameter store St1.

[0050] The classifier 60 determines an output signal y from the input signal x. The output signal y includes information that assigns one or more labels to the input signal x. The output signal y is transmitted to an optional transformation unit 80, which transforms the output signal y into control commands A. The actuator control commands A are then transmitted to the actuator units 10 for correspondingly controlling the actuator units 10. Alternatively, the output signal y may be used directly as the control commands A.

[0051] The actuator unit 10 receives the actuator control command A and is correspondingly controlled to perform an action corresponding to the actuator control command A. The actuator unit 10 may include control logic that converts the actuator control command A into a further control command, which is then used to control the actuator 10.

[0052] In further embodiments, the control system 40 may include the sensor 30. In still other embodiments, the control system 40 may alternatively or additionally include the actuator 10.

[0053] Additionally, the control system 40 may include a processor 45 (or multiple processors) and at least one machine-readable storage medium 46 having stored thereon instructions that, when executed, cause the control system 40 to perform a method according to one aspect of the present invention.

[0054] In one preferred embodiment of FIG. 3, the control system 40 is used to control actuators that are at least partially autonomous robots, such as at least partially autonomous vehicle 100 .

[0055] The sensors 30 may include one or more video sensors, one or more radar sensors, one or more ultrasonic sensors, one or more LiDAR sensors, and / or one or more location sensors (e.g., GPS, etc.), some or all of which are preferably, but not necessarily, onboard the vehicle 100.

[0056] Alternatively or additionally, the sensor 30 may include an information system for determining the state of the actuator system. One example of such an information system is a weather information system that determines the current or future state of weather in the environment.

[0057] For example, the classifier 60 can use the input signal x to recognize, for example, an object in the vicinity of the at least partially autonomous robot. The output signal y can include information characterizing where the object is located in the vicinity of the at least partially autonomous robot. A control command A can then be determined according to this information, for example, to avoid a collision with the recognized object.

[0058] The actuator unit 10, which is preferably on board the vehicle 100, may be provided by the brakes, drive system, engine, drive train or steering of the vehicle 100. An actuator control command A can be determined such that the actuator unit (actuator) 10 is controlled so that the vehicle 100 avoids a collision with the recognized object. The recognized object can also be classified according to what the classifier 60 ranks as most likely to be the object, e.g., a pedestrian or a tree, and the actuator control command A can be determined depending on the classification.

[0059] In further embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which can move by, for example, flying, swimming, diving or walking. The mobile robot may in particular be an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. In all the above embodiments, the actuator control command A may be determined such that the drive units and / or steering and / or braking of the mobile robot are controlled so that the mobile robot can avoid collisions with the recognized object.

[0060] In a further embodiment, the at least partially autonomous robot may be provided by a gardening robot (not shown), which uses sensors 30, preferably optical sensors, to identify the state of plants in the environment 20. The actuator units 10 may be nozzles for spraying chemicals. Depending on the recognized type of plants and / or the recognized state, an actuator control command A can be determined to cause the actuator units 10 to spray the plants with an appropriate amount of an appropriate chemical.

[0061] In yet another embodiment, the at least partially autonomous robot may be provided by a home appliance (not shown), such as a washing machine, stove, oven, microwave, or dishwasher. A sensor 30, e.g., an optical sensor, can recognize the state of an object to be processed by the home appliance. For example, if the home appliance is a washing machine, the sensor 30 can detect the state of laundry in the washing machine. An actuator control command A can then be determined depending on the recognized material of the laundry.

[0062] 4 shows an embodiment in which a control system 40 is used to control manufacturing equipment 11 (e.g., a soldering machine, a punching machine, a cutting machine, or a gun drill) of a manufacturing system 200, for example as part of a production line. The control system 40 controls the actuator units 10, which in turn control the manufacturing equipment 11.

[0063] The sensor 30 may, for example, be an optical sensor which detects a characteristic of the manufactured product 12. The classifier 60 can determine the state of the manufactured product 12 from this detected characteristic. The actuator unit 10 which controls the manufacturing device 11 can then be controlled for a subsequent manufacturing step of the manufactured product 12 depending on the determined state of the manufactured product 12. Alternatively, it is conceivable to control the actuator unit 10 during the manufacturing of a subsequent manufactured product 12 depending on the determined state of the manufactured product 12.

[0064] FIG. 5 shows an embodiment in which the control system controls an access control system 300. The access control system may be configured for physical access control. The access control system 300 may include, for example, a door 401. The sensor 30 is configured to recognize a relevant scene to determine whether access should be granted. For example, the sensor 30 may be an optical sensor that provides image or video data, for example to recognize a person's face. The classifier 60 may be configured to interpret the image or video data, for example by matching it with known persons stored in a database, thereby determining the person's identity. Then, depending on the classification of the classifier 60, an actuator control command A can be determined, for example, according to the determined identity. The actuator unit 10 may be a lock that allows or prevents access depending on the actuator control command A. Non-physical and logical access control is also possible.

[0065] FIG. 6 shows an embodiment in which the control system 40 controls the surveillance system 400. This embodiment is largely identical to the embodiment shown in FIG. 5; therefore, only the different aspects will be described in detail. The sensor 30 is configured to recognize the scene under surveillance. The control system does not necessarily control the actuator 10, but rather the display 10a. For example, the classifier 60 can determine a scene classification, e.g., whether the scene detected by the optical sensor 30 is suspicious. The actuator control command A transmitted to the display 10a can then be configured, for example, to cause the display 10a to adapt the content displayed depending on the determined classification, e.g., to highlight objects that have been classified as suspicious by the classifier 60.

[0066] 7 shows an embodiment in which the control system 40 is used to control an automatic personal assistant 250. The sensor 30 may be, for example, an optical sensor for receiving video images of the gestures of the user 249. Alternatively, the sensor 30 may be, for example, an acoustic sensor for receiving voice commands of the user 249.

[0067] The control system 40 then determines an actuator control command A for controlling the automated personal assistant 250. The actuator control command A is determined according to the sensor signal S of the sensor 30. The sensor signal S is transmitted to the control system 40. For example, the classifier 60 may be configured to execute a gesture recognition algorithm, for example, to identify a gesture of the user 249. The control system 40 may then determine an actuator control command A for transmission to the automated personal assistant 250. The control system 40 then transmits the actuator control command A to the automated personal assistant 250.

[0068] Thus, for example, actuator control command A can be determined according to the user gesture recognized by classifier 60. Actuator control command A may then include information that causes automated personal assistant 250 to retrieve information from a database and output the retrieved information in a form suitable for receipt by user 249.

[0069] In a further embodiment, instead of the automated personal assistant 250, it can be envisaged that the control system 40 controls a home appliance (not shown) that is controlled according to the identified user gestures. The home appliance may be a washing machine, a stove, an oven, a microwave oven or a dishwasher.

[0070] 8 illustrates an embodiment of a control system 40 for controlling an imaging system 500, such as an MRT device, an X-ray imaging device, or an ultrasound imaging device. The sensor 30 may be, for example, an imaging sensor. A classifier 60 may then perform a classification of all or part of the detected image. An actuator control command A may then be selected according to this classification, which controls the display 10a. For example, the classifier 60 may interpret a region of the detected image as potentially abnormal. In this case, an actuator control signal A may be determined to cause the display 10a to display the image and highlight the potentially abnormal region.

[0071] An embodiment of a training system 500 is shown in Figure 9. The training device 500 consists of a providing system 51 providing input images from a training data set. The input images are fed to a neural network 52 (e.g. a supermodel) to be trained, which determines output variables therefrom. The output variables and the input images are fed to an evaluator 53, which determines current hyper / parameters therefrom, which are transferred to a parameter memory P where they replace the current parameters. The evaluator 53 may be configured to implement step S21 of the method according to Figure 1.

[0072] The methods performed by training device 500 may be implemented as a computer program stored on machine-readable storage medium 54 and executed by processor 55 .

[0073] The term "computer" includes any device for processing predefined computational instructions, which may exist in the form of software, hardware, or a mixture of software and hardware.

[0074] It is obvious that the method cannot be implemented entirely in the software described above, but may also be implemented in hardware or in a mixed form consisting of software and hardware.

Claims

1. A computer-implemented method (20) for exploring neural architectures, comprising: The method comprises: training (S21) a supermodel (1) having a plurality of explorable dimensions, wherein the training (S21) is performed by testing architectures within the explorable dimensions, training the tested architectures based on labeled training data, and updating weights of the supermodel with training gradients from the tested architectures; determining (S23) a Pareto-optimal sub-model (1b) of said supermodel (1) based on at least two performance indicators; Including, The determining (S23) comprises the following steps: - calculating the output of a reference model for unlabeled data, said reference model being the largest sub-model (1a) of said supermodel (1); inspecting a plurality of sub-models (1b) from said supermodel (1); calculating, by said sub-model (1b), respective outputs of said unlabeled data; calculating the difference between the output of said reference model and the output of said sub-model (1b); using an optimization algorithm to iteratively test and evaluate sub-models based on multiple objectives, said objectives including said differences, said tested architectures of said sub-models, and performance metrics, in particular accuracy and hardware latency; outputting the Pareto-optimal sub-models based on the objectives, preferably based on the performance metrics; The method (20) is carried out by repeatedly performing

2. To train the supermodel, for each training step, at least two architectures are tested, including a minimum and a maximum architecture based on the searchable dimension. The method of claim 1.

3. The unlabeled data is is data obtained from the same or similar applications, or synthetic data generated from a data distribution of the labeled training data; 3. The method according to claim 1 or 2.

4. The optimization algorithm is evolutionary optimization or Bayesian optimization.

4. The method according to any one of claims 1 to 3.

5. The difference is the Kullback-Leibler divergence or the mean square error or the hard label difference.

5. The method according to any one of claims 1 to 4.

6. the unlabeled data has been filtered; the filtering is performed by selecting data points of an unlabeled dataset; The confidence of the supermodel (1) is higher than a predefined threshold.

6. The method according to any one of claims 1 to 5.

7. the supermodel and the submodels are trained to be classifiers (60) for classifying sensor signals; After the training, the classifier (60) is configured to receive a sensor signal (S) comprising data from the sensor (30), determine an input signal (x), in particular an image, that depends on the sensor signal (S), and provide the input signal (x) to the classifier (60) in order to obtain an output signal (y) that characterizes the classification of the input signal (x).

7. The method according to any one of claims 1 to 6.

8. selecting a sub-model from the Pareto-optimal sub-models based on a predefined criterion; Using the selected sub-model, an actuator control signal (A) for controlling an actuator (10) is provided, in particular depending on the output of the sub-model, in particular by determining the actuator control signal (A) depending on the output signal (y).

8. The method according to any one of claims 1 to 7.

9. The actuator (10) controls an at least partially autonomous robot (100), a vehicle, a manufacturing device (200), or an access control system (300). The method of claim 8.

10. A computer program configured to, when executed by a processor (45, 145), cause the computer to carry out all the steps of the method according to any one of claims 1 to 9.

11. A machine-readable storage medium (46, 146) having stored thereon a computer program according to claim 10.

12. A system configured to perform the method of any one of claims 1 to 9.

Images